[ { "id": "exit-0-because-of-the-pipe", "problem": "My command clearly failed but $? is 0", "symptom": "You pipe a tool into something to shorten its output, and the shell reports success even though the tool refused to run:\n\n```\n$ mytool --verify | tail -40\nERROR: refusing to publish, index is empty\n$ echo $?\n0\n```\n\nThe misleading part: the error text is right there on screen, so it does not look like the exit code is lying. People go hunting for a bug in the tool's exit-code handling, or add `set -e` — which changes nothing, because nothing in the pipeline failed as far as the shell is concerned.", "cause": "A shell pipeline reports the exit status of its last command, not the first. `mytool` returned 2; `tail` read its input happily and returned 0, and 0 is what `$?` gets. `set -e` does not help for the same reason — the pipeline as a whole succeeded. Any wrapper that reads `$?` after a pipe is reading a fact about `tail`.", "fix": "Pick one:\n\n```bash\nset -o pipefail # bash/zsh: pipeline fails if ANY stage fails\nrc=${PIPESTATUS[0]} # bash: read the first stage's status explicitly\nmytool --verify > out.txt; rc=$?; tail -40 out.txt # or just don't pipe\n```\n\nTo tell it worked: run the failing command through the pipe again and check `echo $?` is now non-zero. Note `pipefail` is not POSIX — `dash`/`sh` will reject it (see the `Illegal option -o pipefail` entry).\n\nThe general rule: do not pipe the command whose exit status is the thing you came for. If you are capturing status and trimming output in the same line, you are doing one of them by accident.", "isWorkaround": false, "appliesTo": "bash, zsh, and any CI step or task runner that reads `$?` after a pipe." }, { "id": "runner-exit-0-over-failed-child-build", "problem": "npm run build says it passed, but the build output is stale", "symptom": "A task runner or background job reports `exit code 0` and the notification says the build succeeded. The build actually died earlier — the real builder exited `127`, and that number appears nowhere in the runner's summary. The output directory still holds yesterday's artifact, so the site or bundle looks plausible on a quick check.", "cause": "The wrapper script spawns the real builder as a child process. Under `npm run`, a shell wrapper, or a background runner, two exit codes exist and they disagree — the wrapper's own status and the child's. The runner reports the wrapper's, which is 0 because the wrapper itself ran to completion. The meaningless one arrives first and loudest.", "fix": "Never accept a runner's exit code as evidence about a wrapped build. Two things, both cheap:\n\n1. Make the wrapper propagate the child's status (`exec` the builder, or capture and `exit $rc`) and print the child's own status in the log.\n2. Verify the artifact, not the report — assert the manifest exists by name and carries the expected item count.\n\nStep 2 is the one that catches the case step 1 misses. A build that clears its output directory before failing leaves an absent manifest, and that absence is the real signal — a stale-but-present artifact is what makes this survive a casual look.\n\nYou will know it works when you deliberately break the build (rename an entry point) and the runner goes red instead of green.", "isWorkaround": false, "appliesTo": "Any `npm run` / Make / background-runner wrapper around a build. Observed with a builder exiting 127 while the runner reported 0." }, { "id": "precommit-dies-on-import-not-test", "problem": "git commit fails on every branch and blames an import I didn't touch", "symptom": "`git commit` fails repo-wide with exit 2, on branches that have nothing to do with the change. The output names a module import, not a test:\n\n```\nE ModuleNotFoundError: No module named 'websockets'\n!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!\n```\n\nThe misleading symptom: CI is green. So everyone concludes the hook is broken, or blames whichever open PR they happen to be looking at. Three people diagnosed this independently within two hours and each blamed a different PR.", "cause": "A merged change added test files that import packages at module scope without adding those packages to the pre-commit hook's `uv run --isolated --with …` list. Undeclared imports kill pytest during collection, before a single test runs — so the gate dies on every branch, and the error names the import rather than any test.\n\nCI stayed green because the CI workflow installs those packages from its own separate list. The thing reporting health was not the thing that was broken. Two dependency lists that must agree, and nothing ever diffs them.", "fix": "Run the hook's own command against a clean checkout of the merge target — not CI, not the suite:\n\n```bash\nuv run --isolated --with pytest --with pytest tests/\n```\n\nThen add the missing wheel to the hook's list. To confirm: the command above must reach `collected N items` instead of `Interrupted: 1 error during collection`.\n\nPrevention, since this recurs: any PR that adds a module-scope import under `tests/` must add the wheel to the hook's list in the same commit. And when a gate and CI disagree, do not average them — ask which one actually executes the thing you are about to change, and run that one.", "isWorkaround": false, "appliesTo": "Python repos with a pytest pre-commit hook using an isolated dependency list separate from CI's." }, { "id": "pr-diff-stat-lies-on-stale-branch", "problem": "The PR says +1306 / −0 but merging it would delete a file", "symptom": "A pull request advertises a large number of additions and zero deletions. It reads as purely additive, so it gets approved on inspection. Against the current target branch the same change is `+301 / −1159` — a near-total revert of a file that had grown while the branch sat.\n\nAll four open PRs in one repo were mis-stated the same way; two would have deleted a 1,200-line file outright.", "cause": "The diff stat GitHub shows is computed against the merge-base, not the current tip of the target branch. On a branch that is weeks behind, the merge-base describes a world that no longer exists, so every file the target has gained since looks unchanged. The count is arithmetically correct and answers a question nobody asked.", "fix": "Diff two dots, locally:\n\n```bash\ngit fetch origin\ngit diff origin/main.. --stat # what merging would really do\n```\n\n`origin/main...` (three dots) is merge-base-relative and reproduces the illusion — and the GitHub compare API is three-dot only, so no amount of asking the API will give you the true answer. If you cannot check out locally, compare file contents by ref (`git show origin/main:path` vs `git show :path`).\n\nYou will know which one you are looking at by the numbers moving: if two-dot and three-dot stats differ, the branch is stale and the PR page is showing you the wrong one.", "isWorkaround": false, "appliesTo": "Git, and the GitHub PR \"Files changed\" stat. Worst on branches more than a few dozen commits behind." }, { "id": "one-file-permanently-modified", "problem": "One file shows as modified in every checkout and nothing clears it", "symptom": "`git status` lists the same file as modified, forever. `git checkout -- `, `git restore`, `git stash`, and `git reset --hard` all appear to succeed and the file is dirty again immediately. `git diff` may show nothing useful, or the whole file as changed.\n\nThe misleading path: people conclude the working tree or the repo is corrupt, or that an editor is rewriting the file on save.", "cause": "The committed blob has mixed line terminators while `.gitattributes` declares something like `text eol=crlf` for that path. On checkout the smudge filter writes pure CRLF; on comparison the clean filter normalises back — and neither can ever reproduce a blob that is partly LF and partly CRLF. The working copy therefore cannot equal the index, by construction, so no reset can converge.", "fix": "Confirm the diagnosis first — an `i/mixed` index side is the tell:\n\n```bash\ngit ls-files --eol -- \n# i/mixed w/crlf attr/text eol=crlf <-- this\n```\n\nThe only real fix is to re-commit the file normalised (rewrite it with consistent terminators and commit that). After the fix, `git ls-files --eol` should show `i/lf` (or `i/crlf`) and `git status` goes quiet.\n\nDo not mask it with `git update-index --assume-unchanged`. That flag makes the path invisible to every tool that reads `git status` as evidence — including guards written later by someone who has no idea the flag was set. The noise is annoying; the mask is a trap.", "isWorkaround": false, "appliesTo": "Git repos using `.gitattributes` with `text eol=crlf`, typically mixed Windows/Linux checkouts." }, { "id": "rebase-blocked-by-file-i-never-touched", "problem": "Rebase refuses because of a file my branch never touched", "symptom": "A branch cannot be rebased, checked out over, or `reset --hard`-ed. Every git operation refuses in turn, naming one specific file — and that file has nothing to do with the branch's changes and was never modified on it:\n\n```\nerror: Your local changes to the following files would be overwritten by checkout:\n tools/run_service.ps1\nPlease commit your changes or stash them before you switch branches.\n```\n\nThe misleading part: the error points at the file, so the instinct is to fix the file on this branch. Stashing and committing both fail to help.", "cause": "The file is a mixed-line-terminator blob under `text eol=crlf` (see the permanently-modified-file entry), so it is permanently dirty and blocks every operation that wants a clean tree. The repair for it already landed on the default branch — but every branch that diverged before that repair still carries the broken blob. The error names the file rather than the missing fix, so nothing points at the actual cause.", "fix": "Find the repair, to confirm it exists and postdates your branch:\n\n```bash\ngit log origin/main -- \n```\n\nThen rebuild the branch on top of current main rather than moving main under it:\n\n```bash\ngit checkout -f -q --detach origin/main\ngit switch -q -c rb/\ngit cherry-pick \n```\n\nThe cherry-picks apply cleanly because the broken blob is no longer in play. You will know it worked when the rebuilt branch rebases and checks out without naming the file at all.\n\nThe general rule: when a rebase is blocked by a file the branch never touched, look for a fix that landed after the branch diverged.", "isWorkaround": false, "appliesTo": "Git. Presents as an unrelated dirty file rather than as a merge conflict." }, { "id": "gitignore-swallowed-my-source-file", "problem": "git add did nothing and git status says the tree is clean — but my new file is right there", "symptom": "You write a new module, `git add -A`, and nothing is staged. `git status` reports a clean tree. `git commit` succeeds with a smaller file list than you intended, and the whole thing reads as \"everything is committed\".\n\nNo output anywhere contradicts it. The only hint is a line buried in `git add`'s output if you passed the path explicitly:\n\n```\nThe following paths are ignored by one of your .gitignore files:\ntools/speaking_token.py\nhint: Use -f if you really want to add them.\n```\n\nWith `git add -A` you do not even get that. The file exists on exactly one disk, and a fix ships untested because its test file was swallowed too.", "cause": "A credential-hygiene rule in `.gitignore` — `*_token*`, or `*secret*`, or `*credentials*` — was written for token files and also matches any source file whose name contains that substring: `speaking_token.py`, `tests/test_speaking_token.py`, `secrets_index.py`. Ignoring is designed to be silent, so the absence of a warning is not evidence of anything.", "fix": "Enumerate what is actually invisible — this is the one question that already means what you want:\n\n```bash\ngit ls-files --others --ignored --exclude-standard -z -- tools tests\n```\n\n`--others --ignored` is precisely \"on disk, untracked, and ignored\". Then un-ignore by explicit full path:\n\n```gitignore\n!tools/speaking_token.py\n!tests/test_speaking_token.py\n```\n\nNever weaken the pattern and never un-ignore with a wildcard — `!*_token*.py` re-opens the door for a real credential file that happens to end in `.py`.\n\nDo not build the guard out of `git check-ignore`. That answers \"does a pattern match this path\", which is a different question: `.gitignore` has no effect on a file that is already tracked, so a `check-ignore` sweep names a pile of committed files and cries wolf. A guard that cries wolf gets switched off, and then the real case walks through.", "isWorkaround": false, "appliesTo": "Git. Any repo with substring globs like `*_token*` / `*secret*` in `.gitignore`." }, { "id": "commit-status-pending-forever", "problem": "GitHub commit status says pending and never goes green", "symptom": "A PR is approved, `mergeStateStatus` says `CLEAN`, and the commit status API says pending:\n\n```bash\n$ gh api repos/OWNER/REPO/commits/SHA/status -q .state\npending\n```\n\nYou wait for it to go green. It never will. The misleading part is that `pending` is exactly what a genuinely-running job reports, so waiting looks like the correct response.", "cause": "`statuses=0`. The repo reports CI through check-runs, not through the legacy commit-status API, so that commit has no statuses at all. The combined state of an empty set is defined as `pending`, so the endpoint reports pending forever on any repo that does not use the legacy API. `pending` from zero inputs and `pending` from a running job are the same string and different facts.", "fix": "Ask the endpoint that has the data:\n\n```bash\ngh api repos/OWNER/REPO/commits/SHA/check-runs \\\n -q '.check_runs[] | \"\\(.name): \\(.status)/\\(.conclusion)\"'\n# test: completed/success\n```\n\nAnd take `mergeStateStatus: CLEAN` at face value — that is GitHub's own statement that every required check is satisfied.\n\nTo tell which case you are in before believing any rolled-up status: check the count it rolled up.\n\n```bash\ngh api repos/OWNER/REPO/commits/SHA/status -q '.statuses | length' # 0 = meaningless\n```\n\nThe general rule: an aggregate over an empty set is not a measurement. Same shape as \"0 of 0 tests failed\", an all-green dashboard with no probes registered, and a health check whose target list is empty.", "isWorkaround": false, "appliesTo": "GitHub REST API / `gh` CLI, on repos using GitHub Actions check-runs (i.e. most of them)." }, { "id": "illegal-option-pipefail", "problem": "Script runs fine locally, dies in CI with 'Illegal option -o pipefail'", "symptom": "Every local run passes. The CI runner fails immediately:\n\n```\n./scripts/check.sh: 3: set: Illegal option -o pipefail\n```\n\nThe misleading part: the script's shebang says `#!/usr/bin/env bash`, and you tested it. Locally the test harness invoked it as `sh script.sh` and it worked — so the wrong interpreter was indistinguishable from the right one on your machine.", "cause": "The script is bash, but it is being invoked with `sh`. On a Windows git-bash box `sh` IS bash, so `sh script.sh` behaves identically to `bash script.sh`. On the Linux runner `sh` is `dash`, which has no `pipefail`. The bug was always there; only one of the two machines can express it.", "fix": "Invoke it with the interpreter it declares — either run the script directly (`./scripts/check.sh`, with the exec bit set) or call `bash script.sh` explicitly in the harness.\n\nThen watch out for the second trap, because the obvious fix is also wrong on Windows. A bare `bash` resolves through the Windows PATH to WSL's bash, which cannot see `D:/...` at all and reports:\n\n```\nbash: /d/projects/scripts/check.sh: No such file or directory\n```\n\n— for a file that is plainly present. Two interpreters with the same name, one unable to reach the repository, and the error names the file rather than the interpreter. Two defences:\n\n- Resolve the interpreter as the sibling of one you already know works: `/bin/bash` beside `/bin/sh`, by absolute path.\n- Pass paths as POSIX (`Path(...).as_posix()`). A shell handed `D:\\a\\b` eats the backslashes and reports `D:ab` missing.\n\nConfirm by running the harness with the interpreter pinned and seeing `set -o pipefail` accepted on both machines.", "isWorkaround": false, "appliesTo": "Mixed Windows (git-bash / WSL) and Linux-CI setups. `pipefail` is a bash/zsh extension, not POSIX." }, { "id": "pgrep-f-matches-its-own-shell", "problem": "pgrep says 3 processes are running and there are none (or pkill kills my own session)", "symptom": "Two failures, same mechanism, both silent. Counting:\n\n```bash\npgrep -fc \"venv/bin/python -m tools.worker.run_once\"\n3\n```\n\n— when the true answer is 0. And killing:\n\n```bash\npkill -f \"supervise-job\"\n```\n\n— which kills the diagnosing session itself, producing an empty result that looks exactly like a command that ran and found nothing. Hit five times in one night, in both directions. An inflated count reads as \"processes are running\"; an empty output reads as \"nothing matched\". Neither announces that the query included itself.", "cause": "`pgrep -f` and `pkill -f` search full command lines, and the command line of the process asking the question is in that search space. Run over SSH, the pattern appears in the argv of the `bash -c` that invokes it and in the SSH command line carrying it — so the query counts itself twice over.", "fix": "Put every process operation inside a script and invoke it by name, so the invoking argv contains nothing that describes a process:\n\n```bash\n# /usr/local/bin/workerctl — the pattern lives only in here\ncase \"$1\" in\n count) pgrep -fc \"venv/bin/python -m tools.worker.run_once\" ;;\n stop) pkill -f \"venv/bin/python -m tools.worker.run_once\" ;;\nesac\n```\n\n```bash\nworkerctl count # not: pgrep -f \"\"\n```\n\nNo shell running that script can contain the pattern. This must cover the launcher too — a cleanup script is still defeated if the command that calls it spells out the process it is about to start. That is exactly how the fourth and fifth instances happened, after the first three were understood.\n\nThe bracket trick (`pgrep -f \"[w]orker.run_once\"`) only helps when a human typed the pattern literally; it does not survive being passed through a variable, and it does nothing for `pkill`.\n\nTo tell it worked: run the counter with no workers running and confirm it returns 0, then start one and confirm it returns 1.", "isWorkaround": false, "appliesTo": "Linux/macOS `pgrep -f` / `pkill -f`, especially when invoked over SSH or from `bash -c`." }, { "id": "git-help-opened-hundreds-of-browser-tabs", "problem": "Something is opening dozens of browser tabs on my Windows desktop", "symptom": "Browser tabs for `git-interpret-trailers.html` (or any other git subcommand's docs) appear on the desktop, dozens of them, ~30 MB each, until memory runs out. Nothing in any log says anything.\n\nThe misleading part is that every in-process signal is healthy: the offending command exits 0, stdout is empty, stderr is suppressed, and the process's own memory and CPU are unchanged. The only detector was a human sitting at the machine watching it fill up.", "cause": "A script probed for a subcommand's availability with a fallback:\n\n```bash\ngit interpret-trailers --version >/dev/null 2>&1 || git interpret-trailers --help >/dev/null 2>&1\n```\n\nTwo things compose into the bug. `--version` exits 129 on that subcommand, so the `||` branch runs every time. And on Windows, `git --help` does not print text — it launches the HTML documentation in the default browser. `>/dev/null 2>&1` suppressed a stream that was never the point: the side effect landed outside the process, where no redirection, exit code or resource measurement can see it.", "fix": "Delete the probe. Ask the thing to do the work rather than asking whether it could — the real operation reports its own failure, so the capability check was never needed:\n\n```bash\nif ! git interpret-trailers --parse < msg.txt > out.txt 2>err.txt; then\n # handle absence here\nfi\n```\n\nIf you must probe, never use `--help` or `git help ` on Windows, and treat any `--help` in any tool as a possible viewer launch.\n\nTo confirm: run the script on the Windows box with the browser closed and check that no window opens. A clean exit code proves nothing here — that was the whole problem.", "isWorkaround": false, "appliesTo": "Git for Windows. `git --help` and `git help ` open the bundled HTML docs in a browser rather than printing to the terminal." }, { "id": "gcloud-file-is-not-a-database", "problem": "gcloud crashed (DatabaseError): file is not a database — and reinstalling only fixes it until tomorrow", "symptom": "Every credentialed command dies:\n\n```\nERROR: gcloud crashed (DatabaseError): file is not a database\n```\n\nYou reinstall, it works, you declare it fixed, and it is broken again the next morning. Two misleading conclusions follow: that the SDK install is flaky, and — worse — that \"the credential vault is read-only on this machine\", which is the belief that gets built around instead of debugged. In one case that belief stood for ~3 months, with a hand-written `--secret=` shim built on top of it.", "cause": "The corrupt files are in the config directory, not the install directory: `%APPDATA%\\gcloud\\credentials.db` was zero-filled and `access_tokens.db` malformed. An installer replaces the program; it never replaces the program's state. So every reinstall genuinely repaired the only thing it touches, and bought a few hours of false confidence.\n\nA zero-filled SQLite file is not a corruption mystery — it is a write that died partway. Both volumes on that box were under 10% free.", "fix": "When a reinstall does not hold, stop reinstalling and go diff the state directory.\n\n```bash\n# 1. Is it actually a database? A real SQLite file starts with these 16 bytes.\nhead -c 16 \"$APPDATA/gcloud/credentials.db\" | xxd # expect: SQLite format 3\n\n# 2. Ask SQLite itself\nsqlite3 \"$APPDATA/gcloud/credentials.db\" \"pragma integrity_check;\"\n\n# 3. Check free disk on every volume the tool writes to\n```\n\nThen remove the corrupt files and re-authenticate (`gcloud auth login`), after fixing the disk — otherwise the next write dies the same way and you are back here tomorrow. File timestamps on the state directory tell you how long it has really been broken; that number is usually the shock.\n\nThe adjacent trap that let this survive: application-default credentials kept working throughout, so every REST call against the same service succeeded while every CLI call died. Whoever tested after a reinstall happened to test the layer that was never broken. \"Auth works\" is not a claim — \"auth works via the CLI\" is. Verify the layer the caller actually uses.", "isWorkaround": false, "appliesTo": "Google Cloud SDK on Windows (`%APPDATA%\\gcloud`); the same shape applies to any CLI keeping SQLite state outside its install directory." }, { "id": "regex-escape-became-a-backspace-byte", "problem": "My regex looks right in the diff and matches nothing at runtime", "symptom": "A pattern reads correctly in the editor, in `git diff`, and in code review — and matches zero things when it runs. The zero then gets reported as a finding: \"no work verbs in these utterances\", \"no AI content on these channels\". Exit 0 throughout, no error anywhere.\n\nThe file looks like this to every human who opens it:\n\n```python\nWORK_VERB = re.compile(r\"\\b(finish|build|deploy)\\b\")\n```\n\nThree separate breakages of the same pattern in one session, each from a different layer of escaping — and the second attempt broke it in the opposite direction from the first, which is the tell that the method was wrong rather than the count.", "cause": "The escaping crossed three layers — shell heredoc → Python string literal → file contents — and each layer consumed one level:\n\n```\nattempt what reached the file what the engine saw\n----------------------------- ------------------------- ---------------------------\nheredoc -> python -> file \\b collapsed to \\x08 a literal BACKSPACE byte\nthe fix for that \\b written verbatim a literal backslash then b\nbuilt with chr(92) + \"b\" \\b a word boundary\n```\n\nThe backspace byte is the dangerous one: `0x08` is invisible in an editor, invisible in `git diff`, and invisible in review. Everyone who looked at it saw `\\b`. It also parses perfectly — so a syntax check, a lint pass and an import all succeed.", "fix": "Never accept a pattern because the source text looks right. Assert against the compiled object, with one string that must match and one that must not:\n\n```python\nimport importlib, sys; sys.path.insert(0, \".\")\nmod = importlib.reload(importlib.import_module(\"mypkg.intent\"))\nassert mod.WORK_VERB.search(\"can you finish the corpus\"), \"work verb not seen\"\nassert not mod.WORK_VERB.search(\"can you see something\"), \"perception read as work\"\n```\n\nBoth ends are required. Asserting only the positive passes on a pattern that matches everything; asserting only the negative passes on one that matches nothing — and matching nothing is exactly the failure being hunted.\n\nWhen you are generating code through layers of escaping, build the escape from `chr(92)` so no heredoc or outer string can eat it, and parse the file before writing it so a collapsed escape becomes a refusal with the file untouched, rather than a broken module found by the next command.\n\n⚠ Honest caveat: this rule is correct and remembering it does not work. The session that wrote it hit the same escape four more times the same day, hours after documenting it. A rule you must recall at the moment of typing is documentation, not a control — put the check in a helper the writing path calls, or it will not fire.\n\nGeneralises past regexes: any artifact built by string manipulation must be verified by the thing that will consume it, not by reading what was written.", "isWorkaround": false, "appliesTo": "Any codegen crossing shell heredoc / string-literal / file boundaries — which includes most LLM-driven patch scripts. Python examples; the mechanism is language-independent." }, { "id": "pip-install-target-did-not-upgrade", "problem": "install script says success, old version is still on disk", "symptom": "A deploy script pins a package version, re-runs over an existing deployment, prints its own verification line and exits 0:\n\n```\nInstalling collected packages: mylib\nSuccessfully installed mylib-2.4.1\nlisten OK\n```\n\nThe old version is still there. Runtime behaviour then drifts with nothing in the commit log to blame — especially bad if your code reaches into that package's private internals, where a version change alters behaviour without changing any API you call.", "cause": "Two parts.\n\n1. `pip install --target ` does not replace an existing package directory without `--upgrade`. It finds the directory present and leaves it alone.\n2. The verification step then imported the package and, finding one, declared success. Importing proves a package exists. It does not prove it is the one you shipped.", "fix": "Add `--upgrade` (or clear the target first), and — this is the half that matters — assert the resolved version rather than merely importing something:\n\n```python\nimport importlib.metadata as md\nresolved = md.version(\"mylib\")\nassert resolved == PINNED, f\"expected {PINNED}, got {resolved}\"\n```\n\nAn assertion that can fail is the point. To prove the check works, pin a version you know is not installed and watch the install script go red.\n\nThe general shape: deployment claims are the purest form of this class — the thing reporting success is the thing that was supposed to do the work.", "isWorkaround": false, "appliesTo": "pip with `--target` (vendored/bundled deployments). `--upgrade` is required for replacement; `pip install -t` alone is not idempotent in the way it appears." }, { "id": "coding-agent-push-reverted-merged-work", "problem": "The coding agent pushed 'your requested changes' and silently deleted work that was already on main", "symptom": "You post a review comment on a PR. Fourteen minutes later the agent pushes a commit described as making the requested changes. The requested change is there. So are 35 files and 3,621 deletions, and among them a security fix that merged to `main` that morning, plus a generated data file, plus the tests for the fixes the review asked for.\n\nNone of it appears in the diff as a deletion. It appears as the branch's own content. The preview deploy goes green. The PR page says its requested changes were made.\n\nFourteen minutes is the tell, and it is the whole finding in one number: it is faster than anyone looks.", "cause": "Measured, and it is narrower and worse than \"the agent decided to revert things\". Compare the trees:\n\n```bash\ngit rev-parse ^{tree} # -> \ngit rev-parse ^{tree} # -> \n```\n\nIdentical. It made zero edits. It re-pushed its stale snapshot verbatim, under its own original commit message — about something unrelated to the review.\n\nThe sequence is not read review → decide → edit → push. It is:\n\n1. A message arrives at a session whose state reads `COMPLETED`. `COMPLETED` is not terminal — sessions return to `IN_PROGRESS` on being messaged.\n2. The revived session runs to its completion step.\n3. The completion step publishes whatever the working tree contains — and that tree is a depth-1 clone with no network in either direction (`fetch` and `push` both exit 128), so it cannot compute a merge base or see one commit of what has landed since.\n\nSo the blast radius is the whole tree, always, and it is not proportional to what the message asked for. An empty message would have done the same damage. The trigger is contact, not content.", "fix": "On any repo where an automated agent has write access to PR branches:\n\n1. Never merge on a green check after an agent has pushed. One command:\n ```bash\n git diff --stat main | tail -1\n ```\n A deletion count in the thousands on a nine-file PR is the tell.\n2. Compare trees to tell a response from a republication. Equal trees mean nothing was done and everything was overwritten.\n3. Rebuild a stale branch before reviewing it, not after — and before sending anything to a parked session with write access, establish how old its tree is. If the branch is behind, rebuild it first or do not make contact.\n4. Re-verify anything you merged that day if an agent has since pushed to a stale branch. The revert is silent and reads as the branch's own content.\n\nSwitching the agent to act only on explicit mention removes the trigger, but that is an account-level setting and therefore the repo owner's decision, not something to change mid-task.", "isWorkaround": false, "appliesTo": "Hosted coding agents that hold a parked session with its own shallow checkout and push to PR branches. Two independent incidents measured, 2026-09; the second produced a PR deleting ~1,500 lines." }, { "id": "delegated-agent-returned-no-edits", "problem": "The delegated coding agent ran for its whole budget and changed nothing", "symptom": "Two dispatches against different PRs in different worktrees both run to exhaustion and return:\n\n```json\n{\"files_changed\": [], \"notes\": \"Still no tracked source edits\"}\n```\n\nThen die with `rc=4294967295`. The obvious reading — the one everyone takes — is that the model failed or is not capable enough. Runs had been coming back empty across sessions for weeks and every one was filed that way, so nobody looked at what the model had been asked to do.", "cause": "It did exactly what it was told. Both worktrees contain a `CLAUDE.md` whose managed block says \"MUST run impact analysis before editing any symbol\" and \"MUST run `detect_changes()` before committing\". The agent read that, took it as binding, and spent its entire run building a code index over a 138,000-symbol repository. Its own transcript says so plainly, and reports accurately that it has not edited anything yet.\n\nThe file even carried an override saying those gates were conditional on an MCP server being loaded — but the override sat BELOW the generated block, and a worker reading top-down has already accepted the mandate by the time it arrives.\n\nA config file IS part of the prompt the moment an agent can read it. A delegated worker inherits every instruction file in the directory you hand it, including ones addressed to a different agent, in a different harness, with a different toolset.", "fix": "Before blaming a worker that produced nothing, read its transcript — it will usually say, in plain words, what it spent the time on.\n\nThen fix the dispatch, not the model. State explicitly which in-repo instructions do not apply, by name:\n\nThis repo's `CLAUDE.md` describes tooling you do not have. Ignore its impact-analysis and pre-commit gates. Do not run any command that builds, indexes, installs or downloads.\n\nAnd order the file so the override cannot be missed: put conditional exemptions above the generated mandate block, not below it.\n\nYou will know it worked when the next dispatch returns a non-empty `files_changed` within a fraction of the time budget.", "isWorkaround": false, "appliesTo": "Any delegated coding agent (Codex, Jules, Claude Code, Aider…) launched into a worktree containing agent instruction files written for a different harness." }, { "id": "untracked-stale-tool-shadows-main", "problem": "The tool produces garbage on this branch and works fine on main", "symptom": "A repo tool produces a report that is damning and specific — findings against a file the PR never touched, every real finding marked UNKNOWN, and 447,915 characters truncated. You spend an hour fixing the tool.\n\nThe tool does not have the bug. The only hint is one line among the noise:\n\n```\n$ git status\n?? tools/review.py\n```", "cause": "The tool landed on `main` (and was fixed twice more) after your working branch was cut. On this branch the file is therefore untracked — so `python tools/review.py` runs a copy that predates both fixes, while the version on `main` already contains the fix for the exact symptom you are chasing.\n\nTwo failure modes stack, and the second costs more than the first:\n\n1. An untracked file is invisible to every check that reasons about the branch. `git status` reports it, but as noise among genuine scratch files, and nothing distinguishes \"a file I made\" from \"an older copy of a tool that landed on main since\".\n2. A measurement taken with the wrong build is still a real measurement. The 447,915 truncated characters were real. The misattributed findings were real. Every observation was true of the thing that ran, which makes the diagnosis feel well-evidenced at every step — and points at the wrong code.", "fix": "Before diagnosing a tool from its output, establish which copy ran:\n\n```bash\ngit log --oneline -1 -- tools/review.py # on your branch\ngit log --oneline -1 origin/main -- tools/review.py # on main\n```\n\nEnumerate the whole class rather than the one file you noticed — there are usually several:\n\n```bash\ngit ls-files --others --exclude-standard | while read -r f; do\n git cat-file -e \"origin/main:$f\" 2>/dev/null && echo \"SHADOW: $f\"\ndone\n```\n\n\"Delete the untracked copies\" is the wrong reflex. One of five shadows found this way was larger than main's version — unmerged work, not a stale copy. Size does not tell you which is which; the symbol set does:\n\n```bash\ncomm -23 <(grep -E '^(def |class )' local.py | sort) \\\n <(git show origin/main:path.py | grep -E '^(def |class )' | sort)\n```\n\nFor any repo tool with an estate-wide job, run it from a clean worktree on `main`:\n\n```bash\ngit worktree add ../.wt-main origin/main\npython ../.wt-main/tools/review.py ...\n```\n\nRe-running from main's copy is the confirmation: here it went from 227 seconds and 447,915 truncated characters to 75 seconds and zero truncation, with all three real findings cleared.", "isWorkaround": false, "appliesTo": "Git repos where tooling lives in-tree and branches are long-lived. Five shadowed tools found on one branch." }, { "id": "search-always-returns-the-same-big-file", "problem": "My retrieval tool returns plausible results and never the right one", "symptom": "You ask a RAG or keyword search over your own notes and get results back — ten files, plausible names, good-looking scores. The answer is in none of them. The same large file is the #1 hit for 15 of 20 unrelated questions.\n\nThe misleading part is that it looks like it is working every single time. 12 of 14 measured failures returned rows. A zero result invites a second search; a confident non-answer ends one. That is why this survived months of daily use.", "cause": "The score is `occurrences × tier weight`, with no stopword removal. A 136 KB append-only log file has unboundedly many occurrences of `what`, `is` and `the`, so it outranks the 6.9 KB file that actually answers the question. The ranking is measuring file size.", "fix": "First, diagnose it in two minutes — this is the probe:\n\nRun 20 unrelated questions and count how often the same file is ranked first. More than two or three, and your ranking is measuring length, not relevance.\n\nThen fix the scorer: strip stopwords before scoring, and normalise by document length (BM25 does both, and is a dozen lines if you do not want a dependency).\n\nScore whether the ANSWER came back, not whether ROWS came back. Build the evaluation the other way round: locate the answer string in the corpus first by direct grep, then ask the question in a person's words and assert the answer text appears in the top hit's own printed snippet. \"Did it return something\" would have scored every one of those 14 failures as a success — which is exactly how this survived.\n\nOne more thing to check while you are there: assert that the top hit's printed excerpt contains the answer. A separate bug in the same tool showed the first N matching lines rather than the densest, and in a curated store the first match is almost always the frontmatter header — returning the right file and displaying the wrong part of it scores as a hit in every metric and reads as a miss to the human.", "isWorkaround": false, "appliesTo": "Hand-rolled lexical retrieval over a personal/notes corpus. Measured: 21 of 22 questions had an answer present; the tool returned it for 8 (36%)." }, { "id": "attributeerror-after-nine-passing-tests", "problem": "AttributeError on the first real input, with a full green test suite", "symptom": "Nine tests pass. The first time real data reaches the code:\n\n```\nAttributeError: 'Utterance' object has no attribute 'raw'\n```\n\nThe attribute does not exist and never did — `Utterance.__slots__` never contained `raw`. The misleading part is the green suite: it is the thing that stops anyone suspecting the seam, because the feature reading `.raw` is named in the tests and covered by them.", "cause": "Every test drove the consumer directly, constructing its own arguments:\n\n```python\nconversation.heard(speaker=\"a\", text=\"ask x to reindex\", raw=\"\", ...)\n```\n\nThe real caller passes an `Utterance` object. The object and its caller had never been introduced. Both halves are independently correct — the function handles a `raw` argument properly, and the object never carries one. Nothing in either unit is wrong. The defect lives exactly in the gap the unit tests were written to avoid touching.\n\nA test that constructs its own inputs is testing the function's contract with itself, not with the system, and it will pass forever while the seam it stands on does not exist.", "fix": "Write one test that starts where the data starts — feed the raw input the system actually produces (bytes, PCM, the HTTP request) and assert on the far end:\n\n```python\nutt = hear(real_pcm_bytes)\nassert hasattr(utt, \"raw\") # the seam\nassert conversation.heard(utt) # the consumer, given the real object\n```\n\nThat one test is worth more than the nine that hand-build the argument.\n\nWhere a boundary genuinely cannot be crossed in the test environment (an SDK that will not import in CI), a structural assertion that the two names still match is better than nothing and costs three lines:\n\n```python\nassert \"raw\" in Utterance.__slots__\n```\n\nIf entering at the top is awkward, that awkwardness is the finding: it means the stages are not separable, and the gap between them is where the bug is.", "isWorkaround": false, "appliesTo": "Any codebase where unit tests construct their inputs as literals instead of taking them from the producing layer. Python example; the mechanism is language-independent." }, { "id": "voice-agent-drops-the-word-no", "problem": "My voice agent ignores 'no' and 'stop' but hears everything else fine", "symptom": "Long instructions are transcribed and acted on perfectly. Short ones vanish. A 0.7-second \"No.\" — the word that stops something about to happen — is discarded silently, with a counter reading:\n\n```\nrejected=hallucination=1\n```\n\nand nothing about which word. Because the drop is silent and the long utterances work, the system reads as \"mostly working\" rather than as \"deaf to refusals\".", "cause": "Whisper does not return `\"\"` for audio containing no speech. It returns confident, fluent text — \"Thank you.\", \"You\", \"Okay\" — so listeners carry a denylist of known no-speech outputs. To stop that denylist eating a person who genuinely said \"okay\", it gets gated on duration: below ~1.5s the word is presumed invented.\n\nThe bias is structural, not incidental. Every word that halts an action is short — no, stop, wait, cancel, don't. Confirmations, instructions and explanations are long. A length filter is a refusal filter wearing other clothes.\n\nDuration cannot do this job at any setting: a bare \"Thank you.\" takes about a second to say, so a bar high enough to exclude room noise also excludes the real phrase — and a half-second \"No.\" is perfectly audible and far under any such bar. Four review rounds argued about this guard and every one argued about audibility while using duration as a stand-in for it.", "fix": "Stop filtering on duration and filter on loudness, which the code is almost certainly already computing in its silence check and throwing away. Measured through a real room: speech RMS 2826 against a dead-air floor of 30 — two orders of magnitude, never consulted.\n\nTo tell it worked: say a short, quiet \"no\" into the room and assert it reaches the intent layer, and play two seconds of room tone and assert it does not.\n\n⚠ This is a workaround, not a finished fix, and the gap is worth knowing before you copy it. The loudness bar was calibrated on TTS played into the channel, not on a human microphone at conversational distance. A quiet \"no\" spoken from across the room may still fall under it, and nobody has measured where the real threshold should sit. The cost asymmetry says the bar is still wrong in the dangerous direction: a false \"no\" costs one aborted action; a missed \"no\" costs an action someone tried to stop and could not. The words that halt things may need exempting from the loudness bar entirely rather than merely given a generous one. Re-measure against your own users' voices before trusting any number here.", "isWorkaround": true, "appliesTo": "Whisper-based voice listeners with a hallucination denylist, measured 2026-09-01. The loudness threshold is uncalibrated for real microphones — see the caveat in the fix." }, { "id": "safety-classifier-stops-a-reply", "problem": "A response stops mid-way with a safety-classifier message", "symptom": "An assistant reply in a Cowork or dispatch session is cut off part-way and replaced with a notice that a safety classifier stopped the message. It is not a tool error and nothing crashed. Any tool calls that reply had not yet made do not run, so the work simply stops there.\n\nIt tends to happen while the assistant is relaying results, for example listing repositories, not when you ask for anything unusual. That is what makes it look like a product bug.", "cause": "The trigger is text that came back from a tool, not anything you typed and not what the assistant set out to do. In the case recorded on 2026-09-26, an inventory of GitHub repositories (`gh repo list`) returned a repository whose description names an offensive-security technique. The reply was relaying that listing, description included, and the classifier stopped it as that text passed through.\n\nTwo things are worth knowing about where such text comes from. The descriptions involved belonged to forks, and a fork copies its description from the upstream project, so you did not write it and may never have read it. And the same estate had two such forks, not one; an inventory of 1,288 repositories was checked for this entry.\n\nThe classifier runs outside the session and cannot be switched off from inside it. What was observed is that it reacts to the text in the reply: the same session continued once that text was no longer being relayed. How it decides internally is not documented here, because it was not measured.", "fix": "Nothing in the product is broken, and neither you nor the assistant did anything wrong. In order of preference:\n\n1. **Remove the trigger at its source.** Edit the description on the repository that carries it, for example `gh repo edit OWNER/REPO --description \"...\"`. On a fork this changes only your copy, not the upstream project. Archiving or deleting a fork you no longer need also works. Search your inventory for other descriptions of the same kind while you are there.\n2. **Let a flagged reply reroute instead of stopping.** If your setup offers automatic model switching when a message is flagged, turning it on lets the reply continue on another model. This is how the 2026-09-26 session was unblocked. It is a workaround: the trigger text is still there.\n3. **Do not relay untrusted free text.** When an assistant reports tool results such as repository lists, have it give names and metadata and leave out free-form description strings. This is the durable habit, and it also keeps third-party text out of replies for other reasons.", "isWorkaround": false, "appliesTo": "Assistant sessions that relay tool-returned text into replies, recorded 2026-09-26 with a GitHub repository inventory. The classifier's internal rules were not measured; the entry describes observed behaviour only." } ]