For AI agentsA Markdown index of these docs is at /llms.txt. Every page is also Markdown: add .md to its address.

Agent Rules / Agent workflow

What should an agent report when it can't verify an edit?

Rule AR-30

Write down what you couldn't verify in the final message.

Every edit session ends with some things checked and some not. The final message is the only place the next reader learns which is which. Say what you ran, what it showed, and what you did not check, with file and line. A report that says "done" and hides the second list moves the risk to whoever trusts it.

Why it matters

An agent changes send_invoice(doc) to take a locale. It updates five calls, runs pyright (0 errors) and the invoice tests (passed). The final message:

Done. Added locale to send_invoice and updated all callers. Tests pass.

Three things in that message are wrong or empty. "All callers" is five direct calls (AR-27). The dynamic call at mailer.py:88 was never read. "Tests pass" doesn't say which tests, and none reach mailer.py:88. The reader has no way to know any of this, and the first sign of it is an exception in production.

A list of callers that doesn't say whether it is the full list is a guess with good formatting. A report that doesn't say what wasn't checked has the same flaw.

The steps

  1. Keep a running list during the work. Every time you skip something, read something without running it, or hit a limit of a tool, add a line with file:line.
  2. Sort it into three groups at the end:
  • Checked: the command and its result.
  • Read, not run: you looked at it and believe it's right, nothing executed it.
  • Not checked: you didn't look, a tool said it couldn't resolve it, or no test reaches it.
  1. State numbers with their bound. "5 direct calls updated, 1 dynamic site read, 0 tests reach it."
  2. Name the commands. pyright exit 0, pytest tests/test_invoice.py 12 passed. A reader can repeat that; they can't repeat "tests pass".
  3. Say what would close each open item. One line: "run the nightly import", "ask the owner of the consuming service".
  4. Keep the opening line true. Lead with what's finished and what isn't, in that order of strength: what works first, then the open items.
  5. Don't bury it. Open items go in the message itself, not only in a log the reader has to find.

Worked example

The same session, reported two ways (illustrative):

Before: Done. Added locale to send_invoice and updated all callers. Tests pass. After: send_invoice(doc, locale="en") is in place, with 5 direct calls updated (find-references and rg -n -w agree: billing/, cli/, tests/). Checked: pyright, 0 errors (exit 0). pytest tests/test_invoice.py, 12 passed. Read, not run: mailer.py:88 builds the name with getattr(mailer, f"send_{doc_type}"). For invoices it passes one argument, so it takes the default locale. Not checked: no test reaches mailer.py:88. Callers outside this repo (the consuming billing-worker) were not searched. To close: run the nightly invoice job on staging; ask billing-worker's owner about send_invoice.

The second message is longer and the reader still finishes it in under a minute.

Rule block for AGENTS.md / CLAUDE.md

markdown## Final message - State what was changed, then what you checked, with the exact commands and results. - List what you read but did not run, and what you did not check, each with file and line. - Give counts with their bounds ("5 direct calls, 1 dynamic site read"), not "all callers". - Say which tools could not resolve something, and what would close each open item. - If a check could not run (missing tool, no tests, no network), say so. Do not report it as passing. - Do not write "done" or "all tests pass" if any item above is open.

Edge cases

  • Nothing is open. Say what you checked and that you found no open items, for example "pyright exit 0, 41 tests passed, rg and find-references agree". A clean report with its evidence is stronger than an unqualified one.
  • Too many open items to list. Group by reason ("14 string-keyed lookups in export/, same pattern") and give the search that produces them.
  • Open items you introduced (a default value that hides a missed caller, a skipped test). Name them as your own choices.
  • Another agent or a person reads the message next. Write for someone who has not seen the session. Include paths, not "the file above".
  • Pull request descriptions. Put the same three groups there. A description that only lists the diff leaves review to rediscover the risk.

What LLMGRAPH shows

LLMGRAPH labels each caller certain or open; a getattr dispatch that is split across two lines is not listed, so name it in your 'not checked' group yourself.

References

  • Pyright exit codes: https://github.com/microsoft/pyright/blob/main/docs/command-line.md
  • pytest exit codes (0 passed, 1 failed, 5 no tests collected): https://docs.pytest.org/en/stable/reference/exit-codes.html
  • LSP 3.17 specification (find references, rename): https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/

Changelog

  • 2026-10-30: draft 1.
  • 2026-10-30: copy-gate wording fixes
  • AR-25 What to check before changing a function signature?
  • AR-26 How does an agent know an edit is finished?
  • AR-27 Is a list of callers ever the full list?
  • AR-29 What should a coding agent run before finishing?

Last verified

For agentsthis page as Markdown/agent-readable/ar-30.mdllms.txt