AI WORLD HQ
You've probably had this happen. You ask an AI a question, it gives you a confident answer. You push back with a small detail, and suddenly it agrees with you — even though that detail shouldn't have flipped its answer. It's not your imagination. Recent research shows this is a measurable pattern in how today's AI chatbots handle probability.
What "Bayesian" means, in plain English
Researchers describe the problem using a word from probability theory: Bayesian updating — named after Thomas Bayes, a mathematician from the 1700s. The idea is simple. Imagine you're 60% sure it will rain tomorrow. Then you see a forecast calling for clear skies. A logical mind would adjust that 60% downward — maybe to 40%. New evidence, updated belief.
That's what a "Bayesian" thinker does: they shift their confidence up or down based on evidence, in roughly the right amounts. A consistent reasoner never says "I'm 80% sure" and "I'm 30% sure" about the same fact, depending on which order you ask.
Why AI chatbots struggle with this
Large language models (LLMs) are prediction machines — they generate the next most likely word in a sentence, one word at a time. They weren't trained to track a single, internal "confidence level" that updates as the conversation grows. So when you rephrase a question, or add a piece of evidence partway through, the model often answers as if it's seeing the question fresh.
The new research put it bluntly: LLMs are not consistently Bayesian. They can sound more confident than the evidence supports, or flip their answer after new information without good reason. This isn't a bug being patched next week — it's a fundamental trait of how current LLMs work. The same issue affects the major assistants you may already use, like ChatGPT, Claude, and Gemini.
Wrap-up
Today's AI assistants are powerful, but they're not careful reasoners. They don't keep a hidden scoreboard of how confident they are, so they can sound just as certain about two opposite answers in the same conversation. The practical move isn't to distrust them — it's to match your trust to the stakes. Easy stuff, trust away. Anything that matters, double-check.
Your next step today: the next time an AI gives you a confident answer you plan to act on, spend 30 seconds asking it what would change its mind. The reply will tell you a lot — and it's the single best habit you can build with these tools.