Introduction

Hi, everyone. I'm Victor. I am a founder of the startup called Serenity GPT, and I'm like,

it would be so great to come to this place, to a church, and to present in front of more than 100 people about the wonderful things we do for the enterprise. And then I started preparing my

talk, and I'm like, there is so much more fun in just talking about bullshit, so there will be no enterprise involved but if you want to talk to me about that by all means come talk to me find me on LinkedIn I would love to to sell to you but I'm not gonna do that today so what is AI good for any ideas come on yes please

conversations right yeah it's good for conversations all right yeah it is good at that but it's also and it doesn't contradict the first two things it's good at it's very good at bullshit in a way as you'll be finding out that's all

What “Bullshit” Means (and Why It Matters)

it's very good at but let's try to define bullshit first this is from a book that I bought just because of its title Amazon suggested it to me randomly a few years ago and it's called Colin bullshit and in the first two chapters

they try to define what bullshit is which is a very elusive concept to define but I think this is a very good definition where bullshit is basically something that is persuading or impressing you without actually any, without caring about what the truth is or how it achieves that purpose.

So it's not led to mislead you, it's not led to lie to you, it's just meant to persuade you in something that is disregarding the truth completely. And we'll be coming back to this when we talk about AI.

What We Mean by AI Today: Generative AI and LLMs

So what is AI? For today's talk AI is just generative AI or gen AI and the meetup started three years ago when chat GPT came out so it's kind of fair to assume that the AI is just gen AI for this audience but then also I'll narrow it down further when I

talk about gen AI today it will be just the large language models and not for example video models, voice models, image models, whatever other modalities there But that's also not really that important since LLMs are the biggest breakthrough that kind of helped move along everything else.

So what are the LLMs? They're these huge neural networks. You could think of them as just

extremely large matrices or tables of numbers. That's all that really defines an LLM and they are trained to predict the next word.

So what that means is you take all of the text that exists, you kind of show the first five words of a sentence, you ask what is the sixth word, then you show the first six words, you ask what is the seventh word, and by doing that you get something that's really amazing.

But what it learns to do is it learns to carry on conversations. You start the conversation, it continues it in a way that sounds very compelling.

How LLMs Work: Next-Word Prediction

Fluency Without Truth: Why Models Sound Confident

1But what is crucial here is that there is no concept of truth, or there is no logic or concept of logic in how AI decides on what to to say to you but it is extremely fluent and compelling at this nonsense from its own point of view so it's incredibly smart and incredibly stupid at the same time the incredibly

smart bit it actually can follow instructions and that's really mind -blowing and I don't think really anyone understands why it's getting to that point where it feels like it's it's almost AGI whatever that means that it's able to reason on its own so you can tell it what to do and it will do it some of the time to various degrees of accuracy but then some things are

extremely frustrating like it will tell you that there are two R's in the word strawberry and you can still reproduce it to this day with some prompting but also when you start digging into it and your prompts become a little bit more complex you realize that it's not following some of your instructions and then there is it's very difficult to get it to actually do

that even though it's it seems so simple to use like i'm telling you to do this why why do i need to say important in capital letters 10 times and people will die if you don't do this to make you do it.

“Sparks of AGI” and the Mystery of Capabilities

There is one paper that was published by Microsoft guys three years ago.

They got early access to GPT -4 and the paper is called Sparks of Artificial General Intelligence from all the experiments with GPT -4 and they muse about basically exactly this, like how can it be so so smart.

So I recommend that paper.

What’s Changed Recently: Better Usage, Not Smarter Models

And one thing to me, I think the last three years there hasn't really been much breakthrough in terms of going from GPT -4 to GPT -5. The models haven't gotten much smarter.

It's just how we use what is already available within GPT -4 of those types of models that has allowed us to do a lot more with large language models.

In particular, the thinking models that came out maybe couple years ago now they let LLMs not just do stuff according to your instructions but also think about how they will do it maybe try doing it see what the output is and then iterate so they add some thinking and in

particularly in the context of bullshit critical thinking that this allows is extremely important so let's let's give this a try and I'll be using them just

Using LLMs for Critical Thinking About the News

Example 1: “AI Used in a Venezuela Raid” and What’s Actually Claimed

some random news articles that I found this one is from Valentine's Day just from a few days ago and I don't mean to be picking on the Guardian specifically it could be really any news outlet and we'll adjust very quickly look at what

it's talking about so US military used anthropic AI model Claude in Venezuela raid report says and I'm not gonna bore you this too much but it basically has a few paragraphs about this AI model how someone mentioned that it was used in the operation then it gives you a paragraph about the raid on Venezuela

from January 3rd and then how Anthropic was the first AI developer to be used in in a war or in the US Department of Defense rather and then how everyone refused to comment on this and the news actually comes from an anonymous source

and blah blah blah if you just read the headlines you're kind of like okay so So the AI is there to get us. And let's see if large language models can help us with critical thinking when we look at the news like this.

So I'm just going to copy -paste this, if I can manage to do this. I'll just do this. Let's see what happens.

And let's go into ChatGPT. let's paste all of that but let's also paste some instructions so what we want to do is this and the instructions would be just this prompt use critical thinking to analyze this article list the main issues you identify in concise

bullet points so let's paste that here and see what happens it will be thinking and we can see what it's well close the chance it wasn't thinking for too long but normally you can see what it's thinking about and it is basically

raising good questions for you to consider so that this is a hard to verify claim where the allegation relies on anonymous Wall Street Journal sources and that everyone declined to confirm and then importantly it's unclear what it means that AI was used.

Was it used just for translation or was it actually used to carry out some military action? And my sense is that it's probably closer to the former if it was used at all and then there is this kind

of circumstantial implication that the way they phrased and the way they positioned them that because Claude was used next to a passage about bombing and 83 people being killed you start thinking about okay yeah he's killing people so

Example 2: Emotional Storytelling vs. Verifiable Reporting

so let's actually do one more exercise just like this so we're gonna pick another particle this time from the observer and this is going to be about January 3rd events so just when when events happen and that's a view from

Caracas the sound of explosions it made me feel glad and this is a story about how someone was relieved that that this that this all happened and okay now I'm

able to copy so I'm making some progress I'm gonna use Claude this time so I'm going to again paste the article but I'm gonna paste the same prompt again and

okay so Claude is not doing too much thinking but it is immediately picking up that this is a single anonymous account and it's basically emotions that are being portrayed and the emotional reactions that is being elicited rather

than relying on some concept of truth or facts there are claims like that this speaks to 80 % of population that are just claims completely unsupported and it yeah so just feels like an emotional story rather than rather than news so my

Two Practical Bullshit Detectors: Emotion and (Mis)Information

top two things to to watch out for when I hear something or when I read the news is things like the appeal to emotion, like this second article, and 1one simple way to use LLMs to try to point it out is just prompted to remove emotionally charged language, prompted to just extract fact -based statements.

The second thing is misinformation or disinformation, basically fact -checking whatever you hear and doubting that whatever people tell you or whatever especially the new styles is actually accurate and the LLMs can be pretty good at that but then if there is no LLM next to me and I need to

figure out if something is bullshit one litmus test is is this something that I would really love to be true and if I actually pause for a second before I say I told you so then chances are that this would be bullshit and it's just that

that people want us, AI actually is very good at telling us exactly what we want to hear, but also news outlets are very good at that as well, or the social media and the echo chambers that are so easily created around the content that we like.

A Pattern to Notice: News as “What Someone Said”

It's just an observation how most of the headlines are about what different people said. These are three headlines from the three articles on January 3rd from The Guardian again. And they all talk about how different people said something.

And if you are just conveying what leaders of certain countries say, then there is not really much news in it and it may change several times a day.

A Better Workflow: Ask an LLM for the Event, the Evidence, and the Spins

So this is just a comment on the general quality of the news and also something that is very easy to dissect with LLMs but so what is the better way the way I actually consume news now is not by looking at any particular news outlet but going to an LLM whatever that is it could be a cloud could be open AI could be perplexity and asking about the event that's ongoing and then asking to well use critical thinking to give me an unbiased view but then also give me the spin to tell me what the left -leaning and the right -leaning views are and let's try that for January 3rd.

Let's go to Claude again, actually let's go to OpenAI so we can see what this is doing, it's browsing different news, it just does more of that and it's done and then it kind of gives you a much more balanced view of what actually happened and it it takes much less time to read it than going through the kind of the feeds that are so popular

now and and then it's trying to think critically as much as it can which often will end up being bullshit as well but at least it provides you another view and it's not necessarily biased any particular way and you can ask it to review its own thinking and sometimes it will find mistakes on it and it will point them out.

Conclusion: AI Can Generate Bullshit—or Help You Call It Out

Right, so AI is very good at generating bullshit but it's also very good at calling bullshit.

So depending on how you use it you will get completely different results and it's really exciting and we are all testing the limits of the possible.

I think that makes it even more important to kind of to figure out the better ways of using AI and to be

excited by it and pushing it further rather than to be disheartened by uses of AI where it doesn't work or where it doesn't serve the purposes that you feel are right.

That's all from me, thank you very much.