---
title: "The Art of Not Reading #1: The Art of Not Reading Test Output"
author: garplab
publisher: TypingTube
license: CC BY 4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_scope: 「CC BY 4.0」の印から始まる節（仕組み・検証手順・コード）。印の無い本文は著作権を留保
canonical: https://typing-tube.net/articles/en/cd93e0c5cac832
series: "バイブコーディングにおける読まない技術"
language: en
---


> This is article 1 in the series "The Art of Not Reading." Each article is finished once you put down a single file or script. The whole picture and the list of articles are in the [introduction](https://typing-tube.net/articles/en/34b02627c718fa).

This time it is test output. Every time the AI runs the tests, tens of thousands of lines flow in — how much of it do you actually need?

Start by checking one thing. In your most recent session with an AI coding agent, find the output from the place where the tests ran. How many lines is it? Is there anyone who read that output to the end?

Probably nobody did. The AI is another matter, though. **An AI cannot skim past tool output.** Every line that arrives goes into the context, becomes material for the next response, and gets billed. Run the tests 5 times while fixing things and it goes in 5 times. The more the context fills up, the more the quality of the responses drops, so the structure is this: **the more you run the tests, the less able you are to fix them.**

In the production project on my own machine, the full test output came to 26,147 lines (2.1MB). It is [typingtube](https://typingtube.net), a web service for practicing typing along with music videos on YouTube, which I run on my own. Running 13,950 tests produces that much. This article puts down one mechanism: a wrapper (a script that wraps the original command and runs it on your behalf) that **turns it into 29 lines**.

What I understood fits into one sentence.

> **The AI puts all of the test output into its context. So change the output side, into a form where only the failures and the counts arrive.**

## The mechanism: turn the way out into a single door

Replace the entry point for running tests with a single wrapper. The full text goes to a log; only a summary goes to the conversation. This is a reduced version of the real thing (the example is Rails / Minitest, but you can use it for anything by swapping the lines you `grep` for whatever your own test runner prints).

```bash
#!/bin/bash
# scripts/test.sh — keeps the whole output in a log, returns only a summary to the conversation
log=tmp/test_last.log
mkdir -p tmp
bin/rails test "$@" >"$log" 2>&1
status=$?

echo "--- summary (full log: ${log}) ---"
grep -E "^Finished in " "$log" | tail -1
grep -E "^[0-9]+ runs, " "$log" | tail -1   # the line with the counts. ⚠️ never cut this one (see below)

failures=$(grep -cE "^(Failure|Error):" "$log" || true)
if [ "${failures:-0}" -gt 0 ]; then
  echo "--- ${failures} failure(s)/error(s) (first excerpt) ---"
  grep -A 3 -E "^(Failure|Error):" "$log" | head -40
fi
exit $status
```

This is all that comes out.

```
--- summary (full log: tmp/test_last.log) ---
Finished in 277.851362s, 50.2067 runs/s, 318.7712 assertions/s.
13950 runs, 88571 assertions, 0 failures, 0 errors, 228 skips
```

My real summary is 29 lines long, and what this article puts down is the minimal form: its core 3 lines. The remaining lines — a list of the areas that were never run, and so on — get added in later articles of this series.

What matters is that **not one line has been added to the instructions given to the AI**. You could write "summarize the output before you read it" in your instruction file, but as the [introduction](https://typing-tube.net/articles/en/34b02627c718fa) showed, that is a solution on the side that dilutes as you add to it. I changed the entrance, not the instructions. Rules can be broken; an entrance is the only way through.

Placing it is not enough, though — the AI will not go through this entrance on its own. A standard command like `rails test` is a word burned into the AI by its training. Your `scripts/test.sh` is a word it is seeing for the first time today. Even if you write "use the wrapper" in your instruction file, what surfaces the moment it goes to run the tests is the standard command it has seen hundreds of millions of times. One line in an instruction file has to compete with a habit and a priority ingrained in the AI, and it usually loses. So we block off the standard command instead.

On Claude Code, adding one hook that stops a bare `rails test` and points to the wrapper makes the entrance genuinely single. A hook is a script that cuts in immediately before a command runs, and you register it in `.claude/settings.json` like this.

```json
{
  "hooks": {
    "PreToolUse": [{
      "matcher": "Bash",
      "hooks": [{
        "type": "command",
        "command": "$CLAUDE_PROJECT_DIR/.claude/hooks/use_test_wrapper.sh"
      }]
    }]
  }
}
```

All it does is "if the command is a bare `rails test`, print how to use the wrapper and finish without letting it run."

```bash
#!/bin/bash
# .claude/hooks/use_test_wrapper.sh
command=$(cat | python3 -c 'import json,sys; print(json.load(sys.stdin).get("tool_input",{}).get("command",""))')
case "$command" in
  *scripts/test.sh*|*SKIP_TEST_GUARD=*) exit 0 ;;   # let the wrapper through, and the explicit escape hatch
esac
if printf '%s' "$command" | grep -qE '(^|[;&|[:space:]])(bin/)?rails[[:space:]]+test([[:space:]]|$)'; then
  echo "Run tests with scripts/test.sh <same args> (full output stays in tmp/test_last.log)" >&2
  exit 2   # exit 2 = do not let it run; make the AI read the guidance on stderr
fi
exit 0
```

The AI cannot move with the bare command, and gets to go on only once it has read the guidance. Unlike a human who notices and cuts into the conversation with "use the wrapper," the guidance from a hook arrives inside the flow of the work. The AI rereads it on the spot and resumes without having its own priorities knocked over — that no human instruction is inserted is itself part of why the mechanism works. This shape shows up again and again through the series.

## What I stopped reading

The raw output of the tests. Neither the human nor the AI reads it. It is not thrown away, though. The whole text stays in `tmp/test_last.log` every time, and the first line of the summary says where that is. Only when a cause really has to be investigated with the full text do I open it.

## Caveat: anomalies other than failures stop being visible

The blind spot of this mechanism is that **anomalies that do not take the shape of a failure** get hidden. A surge in skips, warnings, a drop in the number of tests run — even when a whole file of tests stops being loaded, it reads "0 failures" as long as everything left passes.

That is why the line with the counts is the one thing never cut from the summary. The `13950 runs, ... 228 skips` in the run above is it. If runs that were 13,950 last time have dropped to 9,000, something is happening even with no failures at all. Picking up anomalies from the numbers alone, without reading, is what article 9 (the art of not reading handoffs) puts together.

## How to verify: confirm once that failures do arrive

**Prerequisites**

- The wrapper from this article (`scripts/test.sh`) is in place, and the hook that stops the bare test command is registered
- You have 1 or more tests, and they run on your machine

**Time required**: 5 minutes

**Steps**

1. **You** deliberately break 1 test. Open 1 test file that finishes quickly and change a single character of an expected value (`assert_equal "abc", ...` becomes `"abd"`). ⚠️ You put it back in the cleanup, so **note down the file name and the line number**
2. **Ask the AI** to "run the tests." Do not invoke the wrapper yourself — ask the AI, because what you want to confirm is what arrives when the AI goes through the door

**Pass conditions** (the summary that comes back has to satisfy all of them)

- **The location of the full log** is there (a single line like `tmp/test_last.log`)
- **The name of the test you broke, and the gap between expected and actual**, are there (the lines matching `Expected: "abc"` and `Actual: "abd"`. The file name you noted in step 1 shows up here)
- **The count line** has `failures` at **1 or more** (a line like `13950 runs, 1 failures`)

**If it does not pass**

- **It said `0 failures`** —— the test you broke never ran in the first place. Check that the file you picked in step 1 is inside the target of the run, then redo step 2
- **It failed, but no test name or gap is there** —— the summary is not picking up enough lines. Widen the range of the `grep` in the wrapper

**Cleanup**

- Put the single character back and have the AI run it once more. You are done when the count line returns to `0 failures`
- ⚠️ Forget to put it back and everything you do after this keeps failing

What you confirmed is only the "a failure always arrives" side of it. If you begin operating without seeing this, you will be using it without being able to tell whether a quiet summary means "there are no failures" or "the wiring never delivers failures."

---

This article dealt only with how output is delivered. **When and how many tests to run** (re-running, running in parallel, selectively running heavy E2E) is a different family of problems, so it gets a section of its own in the "tests" addendum after the main series concludes.

Next time, the art of not reading memory. **The notes the AI keeps appending after every failure — I do not read them back.** And they have never grown into a problem.

---

**Series: The Art of Not Reading**

- ← Previous: [Introduction: I Barely Read What the AI Outputs Anymore](https://typing-tube.net/articles/en/34b02627c718fa)
- → Next: [2. The art of not reading memory](https://typing-tube.net/articles/384477896fec22)
- All articles: [Introduction: I Barely Read What the AI Outputs Anymore](https://typing-tube.net/articles/en/34b02627c718fa)