The Art of Not Criticizing the AI

CC BY 4.0

There were 16 prohibitions, and 15 of them have "instead" written in them

I counted. The behavior rules I hand to the 🤖AI come to 43. One rule to a file, and what gets loaded at the start of a session is only the index. File names and a one-line summary line up there, and the body is read when it is opened (exactly as measured in article 1 of "The Art of Not Listening to the 🤖AI's Opinions").

Before I counted, I took it that what was lined up there were prohibitions in the negative. In fact it was not so.

15 of the 16 that begin in the negative had a continuation in the positive written underneath. Here is one of them.

description: do not swallow errors. do not let a feature run on incomplete data; raise a clear error.

When the data is not in the expected format, it must not be made to work by falling back.

**Why:** the user cannot notice the problem / the error does not reach the developer / it half
        works on incomplete data and makes the experience worse

**How to apply:**
- If the data you expect is not there, notify with console.error plus a message the user can see
- "Show an error" is always better than "show nothing"

The top half is the prohibition, the bottom half is what to do instead. Anthropic's prompting guidance points the same way: an example of the shape you want works better than an instruction about what you do not want. The list on my machine had already gone that way, 15 times over.

The 1 I could not write had moved into an automated test

That leaves 1. Open the rule that is in the negative and yet has no "what to do instead," and the body is a single line. The file name of an automated test. Because the alternative could not be written, it became an automated test. Across all 43 it is the same: of the 6 that carry no "what to do instead," 5 say "moved to a check." They all take this shape.

description: any non-GET fetch requires X-CSRF-Token. moved to a check

**A check is watching this**: test/reference/memory_rule_csrf_token_test.rb

What the practical part of the addendum to "The Art of Not Reading" (the article that used a hook to harden the declaration made before taking a screenshot) said was this: a prohibition with nothing to replace it breeds an abandonment of verification. Hand over the prohibition alone, and nobody can confirm whether it was kept. Instead of abandoning it, the list on my machine handed what could not be written over to the automated tests.

What I understood fits in one sentence.

Criticism can be rewritten into the shape you want. What could not be rewritten was never something to guard with words: it is something to move into an automated test.

I counted 6, but 1 of them was only a difference of format

Let me be honest. After counting the "6" above with a command, I opened them up. 1 of them was wrong.

That file has no section called **How to apply:**. What it has instead are 2 headings, "the area only E2E can hold" and "the area a unit test finishes on its own," and the whole thing is a table of which one to write in. From beginning to end its contents are "what to do instead," and only the format was different.

What a command can count goes as far as the name of a section, never what is inside it. Rules with neither a replacement nor an automated test came to not 6 but none at all.

The mechanism: put a prohibition with nothing to replace it in front of you the moment the save is done

What you put down is not 1 automated test but 1 section added to an audit you already have. On my machine, when the 🤖AI saves a behavior rule, the diff is put in front of it the moment the save is done (that is what article 7 of "The Art of Not Reading" put down). I added 1 thing to it.

# .claude/hooks/audit_memory_write.py (a shortened version of the code on my machine)
PROHIBITION = re.compile(r"禁止|するな|してはならない|使わない|出さない|べきではない|しないこと")
REPLACEMENT = "**How to apply"

# ⚠️ look at the whole file after the save, not the diff. How to apply is sometimes
#    written in an earlier session, and looking only at the diff misreads it as absent
unreplaced = sorted({
    os.path.basename(path)
    for path, _ in changes
    if PROHIBITION.search(read(path) or "") and REPLACEMENT not in (read(path) or "")
})

if unreplaced:
    parts += [
        f"⚠️ **prohibition with no replacement**: {', '.join(unreplaced)}",
        "   there is a prohibition, but no `**How to apply:**` (what to do instead).",
        "   if you cannot write one, move it to a check. a prohibition nobody verifies is not kept.",
    ]

It takes no issue with the prohibition itself. All it looks at is whether what to do instead is there in the same file.

This comes out after the save, not before it. It is as article 8 of "The Art of Not Reading" put it: whether a policy works is decided not by its content but by the moment it is inserted. Put it before, and the warning turns into the question of how to word it so that it goes through.

The first version of this audit did in fact run before the save, and the 🤖AI could walk straight past it by rephrasing a few words. After the save, the top priority is already finished, so the same point reads as a question of whether to fix it.

The same sentence becomes a different thing on where you put it alone.

flowchart LR
  P["Inserted before the save"] --> P1["The top priority is getting the save through<br/>the policy becomes a thing to beat to get the save through"] --> P2["Rephrase a few words and walk past"]
  Q["Inserted after the save"] --> Q1["The save is already finished<br/>the policy reads as a criterion for judging"] --> Q2["It turns into whether to fix it"]

What I stopped saying

"No, not like that." "That is wrong." "Please do not do X." It is not that I notice less. I find just as many gaps as before. What changed is that what I write went from the negative to a positive example, and that when I cannot write a positive example, I stopped writing it in words.

Caveat: it is looking at only 7 words

One, it sees only the wording. It is a regular expression of 7 words, and rewriting it as "X is not desirable" slips past (article 1 and article 5 have the same structure, and this is the 3rd time). This time another hole came out as well: it sees only the name of a section (the 1 case above).

Two, 43 is on the many side. On a small project 5 will do. This article is not about how many there are. It is about whether you stop at the negative when you write your 1st.

Three, a positive example is not necessarily right. Write "do this instead" and the 🤖AI does exactly that (article 2).

What you write has to stay inside the places you really know. Where you do not know, all you write is the constraints and the way to check.

How to verify: save nothing but a prohibition and confirm that it is put in front of you

Prerequisites

• An audit that puts the diff in front of you the moment a save is done (a hook corresponding to PostToolUse) is already running
• You have finished adding to it the 1 section that looks for a prohibition with nothing to replace it
• Try this somewhere disposable, not in the real store of your rules. Behavior rules are shared by several sessions, so a trial write reaches the session next to yours

Time required: 10 minutes

Steps

1. You have the 🤖AI write and save 1 behavior rule with nothing in it but a prohibition (a single line, "do not create X." No "what to do instead" attached). Be sure to put in a word in the negative (prohibited / do not / must not). The audit is looking at that wording
2. You have the 🤖AI add 1 line of "what to do instead" to that same rule and save it again

Pass conditions (all of them have to hold)

• In step 1, a warning comes out after the save is done
• The warning in step 1 carries the point about a prohibition with nothing to replace it, along with the name of the file
• In step 2, the warning no longer comes out

If it does not pass

• It stopped before the save — the position is wrong. Put it before, and the warning turns into the question of how to word it so that it goes through, and the 🤖AI rephrases a few words and walks straight past
• The warning is still there in step 2 — the name of the heading the audit is looking for and the name of the heading the 🤖AI wrote do not match

Cleanup

• Delete the rule files you made in steps 1 and 2

Next time it is "The art of not letting the 🤖AI think." I do not let the 🤖AI think. Even so, the 🤖AI's answers stay right.

Series: The Art of Not Telling

• ← Previous: 7. The art of not interrogating the 🤖AI
• → Next: 9. The art of not letting the 🤖AI think
• All articles: Introduction

End of CC BY 4.0