This is article 10 in the series "The Art of Not Telling." It lays out symptoms that go wrong and their remedies, one at a time. Each article is finished once you put down a single file or script. Why that mechanism is needed becomes clear when you read the explanation afterward. The whole picture and the list of articles are in the introduction.
This time it is about the questions that come flying from the AI's side. "Which shall it be, A or B?" "Is my understanding right, that …?" I stopped answering them. Answer, and that answer becomes a premise.
This article is a review as well. I line up again, on the input side, what the 9 articles up to here put down. For anyone who comes into the series here, this is the way in. Even if you have not read the previous 9, the list below still tells you what has been put down.
In article 7 of the previous series, "The Art of Not Listening to the AI's Opinions," I wrote that correcting a drift the AI has made is a job for a mechanism. This time it is not correction but confirmation.
Start by counting just one thing.
It is the number of times, in your most recent session, that the AI asked you "Which shall it be, A or B?" or "Is my understanding right, that …?" Of those, how many would have stopped the work if you had not answered?
CC BY 4.0
Answer, and the answer becomes a premise
Anthropic's prompting guidance, broken down by model, writes this about how often to seek confirmation. Let the model make the everyday judgments, and seek confirmation only when a difference in reading changes the substance of the work.
This is not "do not ask." It is only that a line has been drawn.
And what happens when you answer on the near side of that line is what I measured in article 7. An answer goes over as a premise. Say "go with A" and B does not get looked for. Say "yes, that is right" and that place does not get checked again.
What I understood fits in one sentence.
Answering a question is handing the answer over as a premise. So instead of answering, you put the places where no answer is needed down in advance.
There is nothing to put down. What is already down is the 9 up to here
There is nothing new to build in this article. The questions went down because 9 articles' worth of mechanisms stand in front of them. Lined up, they come out like this.
| Article | What was put down | Where it stands (the question you get asked without it) |
|---|---|---|
| 1 | The automated test that requires a request to run to be backed by machinery | "Shall I verify it?" |
| 2 | The requires in the catalog of missions (bind only what gets handed over) | "Which order shall I go in?" |
| 3 | The form that has the screen gather the details of a bug | "Tell me the steps to reproduce" |
| 4 | The plan documents that have been written, in themselves (precedent built up) | "Shall I put up a plan first?" |
| 5 | The ratchet on the count of emphasis | "Is this an important rule?" |
| 6 | The guide that writes the shape of the output out in full (the one exception) | "Which format shall I put it out in?" |
| 7 | The closing sentence, pinned to a question | "Is this reading all right?" |
| 8 | Detecting a prohibition with nothing to replace it | "Is this not allowed?" |
| 9 | The structure that requires a marker at the other end of a name | "This file is like this, right?" |
The right-hand column is the questions I get away without answering. Every one of them has been replaced: not "I answer" but "a program answers," or "there is no need to ask."
The patterns I found in the previous 2 series came to 5. Translate them onto the input side and they land just as they are.
| The pattern (previous 2 series) | What it means on the input side |
|---|---|
| A rule works only when it is read, and it thins out the more you add | An answer to a question is the same. It works only in that 1 session |
| Stop the AI and it goes looking | Answer, and it stops looking (measured in article 7) |
| You cannot measure whether something was read. You look only at whether it is there | A "yes" to "do you understand?" measures nothing |
| A policy has its effect decided by the moment it is slipped in | An answer to a question is slipped in at that moment and no other. It does not stay for the next session |
| A self-report leans toward "done." A report only tells you that it stopped | A "yes" to "is this all right?" leans toward done as well |
What had happened to the entries the previous series said nobody was looking at
In article 8 of the previous series I wrote this: an entry marked "Machine check: none" drifts away from the implementation in silence, for as much as nobody is looking at it.
I had thought they were left hanging in the air just like that. I counted in article 8, and it was not so. The 5 where I could not write what to do instead had moved onto automated tests afterward. Instead of drifting away, they had gone across to the automated tests. ⚠️ What the previous series counted was the list of rejections, 17 of them, and this is the behavior rules, 43 of them. The lists are different. The same thing had been happening somewhere else too, and that is all it says: the breakdown over there has not moved. I did not plan it that way. Moving 1 across every time I could not write it is how it came out.
⚠️ Telling them apart: what the 9th of the previous series deals with is different
The previous series, "The Art of Not Listening to the AI's Opinions," put down 8 mechanisms for not weighing the AI's proposals up twice, and its finale took "the art of not choosing from the options" as a 9th. That one is about when the AI lines up the options for you: the judgment side. This one is about when the AI throws a question at you, on the input side, and it is a different thing.
And that one states it flatly.
It came about on its own, as the result of putting 8 down. I cannot explain how it was placed, so I cannot hand it over as an art either. What I can hand over is only the 8.
⚠️ This article is the same. I will not be explaining how it gets handed over. What the list above lines up is only the 8 this series put down (a different 8 from the previous series' 8, and the numbers being the same is a coincidence).
What I stopped answering
"Let us go with A." "Yes, that is right." "That part is fine." It is not that the questions went down. The number of times I am asked is not much different from before. What changed is that most of the questions now close on the program's side, and that only the ones that do not close come through to me.
Caveat: there are questions you should answer
One, there are questions you should answer. As that guidance at the top says, I answer when a difference in reading changes the substance of the work. "We are not building this feature," "we do not use this word": a program cannot decide those. This article is not "say nothing at all"; it is about where the line gets drawn.
Two, the list of 8 is the shape it takes on my machine. You do not have to put down the same 8. What I want you to look at is not the contents but the order: when you are asked, you look first at whether some mechanism can take the question on.
How to verify: when you are asked, go looking for what takes it on
Prerequisites
- There is nothing to put down
- You have a list of the mechanisms you have already put down (something like the table in this article. Write the ones on your machine out on paper or in a note)
- There is 1 occasion coming up where the AI asks you for confirmation
Time required: 5 minutes (once, on the spot)
Steps
- When the AI asks you for confirmation, you stop your hand before answering ("Which shall it be, A or B?" "Is my understanding right, that …?")
- You sort that question into 2. There is 1 criterion for deciding: does a difference in reading change the substance of the work? The ones that change it ("we are not building this feature," "we do not use this word") are the questions you answer. The ones that do not (how to proceed, the format, anything that works out either way) are the questions a mechanism ought to be taking on
- If it is a question a mechanism ought to take on, you do not answer: you go and look at that mechanism. Going to look is the real point. Answer, and you will never know which of the next "What to look at" it was
What to look at (not pass or fail: it splits on what you find when you go and look)
- The mechanism is there, but it has stopped — you have found a mechanism that is not running. It means you now know where to fix it
- There is no mechanism to take that question on — you answer. The first time it is fine to answer. And then you record it: when the same question comes a second time, that is when you put down 1 mechanism to take it on
Cleanup
- None needed
⚠️ Putting it down the second time is the real point. As in the previous 2 series, a mechanism put down ahead of the need does not settle in.
Next time it is "The art of not having the AI review." I do not have the AI review. Even so, the gaps get found before I find them.
Series: The Art of Not Telling
- ← Previous: 9. The art of not letting the AI think
- → Next: 11. The art of not having the AI review
- All articles: Introduction
End of CC BY 4.0