The Trap of Not-Knowing

The previous part ended with advice that looked simple: write your own diagnosis first, really write it, and only then go to the model. It’s good advice, and it covers most of your working days. But it carries one silent assumption — that you have a diagnosis to write in the first place.

Think for a moment about the last time you actually put a case to an AI. You don’t check a routine case with the model. The third molar you’ve pulled a hundred times, the irreversible pulpitis with textbook symptoms, the healthy patient who just needs a scaling — nobody takes these in front of the model. You turn to the model exactly when something isn’t sitting right. An extra line in the chart, an image that looks like nothing you’ve seen before, a drug whose name you recognize but whose interaction with today’s work you don’t.

Which means the place you pick up the model and the tool is exactly where you have the least awareness — and that’s natural enough.

When “I Don’t Know” Turns Into “I Know”

An elderly patient is in the chair, with a history of osteoporosis medication, and an unsalvageable molar that needs to come out. Now you face questions you don’t confidently know the answers to: should the drug be stopped, how long before, does stopping it even help, and whose call is it.

You type the question. The answer that comes back is tidy, categorized, complete with risk factors and a specific number for how long to hold the drug. Nothing in the text feels shaky.

Now the most important point of this part: the issue isn’t that the answer is wrong. Much of it may well be right. What actually happened happened somewhere else — five minutes ago you had an open question in front of you, and now you don’t. The feeling of not knowing is gone, but the not-knowing itself is still exactly where it was.

And it was that very feeling of not knowing that was doing the work. It was uncomfortable, and because it was uncomfortable it used to push you to call a colleague, write to the patient’s physician, spend two hours reading a review, or refer the patient outright. The model handed you that answer and, in the same motion, switched off the alarm. The gap wasn’t filled; only its warning light went dark.

Chapter 1 stated this as a constraint: the model is a tool only within the range where you yourself are qualified; outside it, it’s a trap. There, that was a warning. Here you see the mechanism — your need for the model and your capacity to judge it move in opposite directions.

The more you know, the less you need it and the more easily you catch it out; the less you know, the more you need it and the less you can verify what it tells you — and that’s exactly where the trap is set.

Why No Signal Reaches You

When you ask a colleague the same question, you don’t just get their answer. You also get their hesitation. The fact that they say “well, I’ve only seen two cases like this,” or raise an eyebrow, or stop mid-sentence and say let me check on that — these are part of the answer, and you’re unconsciously factoring them in. People’s doubt leaks.

Seeing that doubt makes you aware of the other person’s not-knowing, and that awareness itself helps you find the right path through treatment.

A model’s doubt doesn’t leak. Chapter 2 was about exactly this: a confident tone is separate from correctness, because the fluency of the words and their correctness come out of the same process. A sentence backed by three thousand papers and a sentence backed by nothing are written in the very same tone. So the one signal you used to rely on in the real world is flat here, and it tells you nothing.

Add one more thing that’s easy to overlook: the model doesn’t know how much you know. It hands the same answer about osteonecrosis, at the same tone and the same depth, to a maxillofacial surgeon with twenty years of practice and to a final-year student. There’s no gatekeeper in between to say this question is outside your scope. You are the only gatekeeper, and that is exactly the moment you have the least information to gatekeep with.

The Slower Version of the Same Story

So far this has been about a single conversation. One patient, one question, one decision. But this story also has a month-by-month version that makes no noise, which is exactly why it’s noticed later.

In a study on colonoscopy, the adenoma detection rate in ordinary colonoscopies — the ones performed without AI assistance — dropped from about 28 percent to about 22 percent after a period in which the same endoscopists had worked with an assistive system and then had it taken away. That is, once the tool was taken from them, they were even weaker than before they’d ever gotten used to it.

Two caveats have to be set alongside this right away, because without them the number claims more than it should. First, the study is observational, not a trial; a single study doesn’t settle anything on its own either. Second, and more importantly: that AI system was not at all from the family we’re talking about in this book. It was a diagnostic, specialized AI — the same breed as Chapter 1 — trained for accuracy, and good at its job.

And it’s that second point that makes the argument stronger, not weaker. The erosion didn’t come from the tool getting things wrong; the tool was getting things right. The erosion came from handing off part of the work that had been building and sharpening the person’s own skill. If this happens with an accurate tool, there’s no reason to think it doesn’t happen with an inaccurate one.

The point isn’t to stop using it. The point is that the ordering from the previous part doesn’t solve this one.

There, it was about a single conversation, and the fix was to write your own answer sooner. Here it’s about months, and what slowly shrinks is exactly the thing you were supposed to write first — meaning that with continuous use, your own ability and awareness erode.

So What Can You Do in the Territory of Not-Knowing

The honest answer isn’t to set the model aside. The answer is to change its role in this territory: from something that hands down a verdict to something that hands you a map.

Ask for names, not a verdict. The biggest obstacle where you’re not qualified is that you don’t even know what to look for. You see a white lesion in the mucosa that looks like nothing familiar; asking “what is this” is the worst possible question, because its answer is exactly the thing you cannot judge. But if you ask what the classification of white lesions is, what distinguishes each one, and what you should look for to tell them apart — you get something whose value is different from the value of a verdict.

This map, offered cheaply and quickly by AI, gets verified and takes you out of the chat box, toward a book, a paper, and a person.

If you do want a content-level answer, at least make it traceable. The previous part showed that tying the model to a source doesn’t miraculously raise clinical accuracy. But that same study showed something else that’s exactly useful here: citation and traceability got better. Meaning that when you can’t ask “is this correct,” you can still ask “is this sentence actually in that paper” — and unlike the first question, you know how to answer the second one. Open the paper and look. This isn’t a check on correctness, it’s a check on the honesty of the citation, but in a territory with no other check available, even this one is worth having.

And one line that no technique can get past: when the question is about the very patient in your chair and you don’t know the answer, the right answer is a person, not a model. A referral, a consult, a call to the treating physician. No prompt replaces this, and this whole chapter was really written to arrive at just this sentence.

And From Here On

Five chapters have gone by, and most of what they said was about the boundary: what the model is, why it sounds so confident, how to talk to it, and where it must not sit in clinical work. If by now it feels like the book has spent more time holding you back than opening a path forward, you’re right.

But all these limits had one thing in common: they were in territory where you didn’t hold the raw material and couldn’t judge it. Now let’s move to the territory where you have both — producing scientific and educational content. The text is your own, the source is right in front of you, and every claim can be traced back to it. This is exactly where the model actually adds something, and it happens to be where its more serious techniques live.

One note before the next chapter starts: everywhere we’ve said “the model” so far, it’s as if it were a single thing. It isn’t. Tools differ from each other; some are built for tasks the others are weak at, and part of the skill is knowing which AI model to bring where. Chapter 6 starts right there.