For AI agentsA Markdown index of these docs is at /llms.txt. Every page is also Markdown: add .md to its address.

Agent Rules / Agent workflow

What should a coding agent run before finishing?

Rule AR-29

Run the type checker and the affected tests before saying \"done\".

Two commands catch most of what a signature or rename edit breaks: the type checker over the whole project, and the tests that cover the code you edited. An edit that was never checked is a draft. Run both, read the exit codes, and put the commands and results in the final message.

Why it matters

An agent changes a return type from str to str | None in a lookup helper and reads the three callers in the file it edited. They all handle None. The task closes. A fourth caller in another package does lookup(key).lower(). pyright reports "lower" is not a known attribute of "None" in that file, and pytest would fail the one test that reaches it. Neither was run, so neither said anything.

The two checks cover different ground:

  • The type checker sees every call site it can resolve, including files you didn't open and paths no test takes.
  • Tests run real behaviour: values, side effects, and the code paths the types can't describe.

Neither is the full picture (AR-27 applies to both). Together they shrink the open set to something you can name.

The steps

  1. Find the project's own commands first. Read AGENTS.md, CONTRIBUTING.md, pyproject.toml, package.json scripts, the CI config. Use those over the defaults below.
  2. Type-check the whole project. ``bash pyright # exit 0: no errors, 1: errors, per the pyright docs mypy -p mypackage # or: mypy src/ (directories are checked recursively) npx tsc --noEmit # checks types, writes no output files
  3. Run the affected tests, then the wider suite if time allows. ``bash pytest tests/test_money.py # one file pytest tests/test_money.py::test_format_amount # one test pytest -k "format_amount" # by keyword npm test # runs the "test" script from package.json
  4. Read the exit code, not only the text. pytest exits 0 when all tests were collected and passed, 1 when some failed, 5 when no tests were collected. Exit 5 means your selection matched nothing, which is not a pass.
  5. Run linters the project already uses (ruff check, eslint .) if they are part of CI.
  6. Fix or report. A failure you didn't cause is still a fact for the final message. Don't edit tests to match new behaviour unless the task says the behaviour should change.
  7. Write the commands and their results into the final message (AR-30).

Worked example

Illustrative session, not a captured run:

$ pyright src/app/handlers.py:77:20 - error: "lower" is not a known attribute of "None" 1 error, 0 warnings (exit 1) fix: handlers.py:77 -> handle None from lookup() $ pyright 0 errors, 0 warnings (exit 0) $ pytest tests/test_lookup.py tests/test_handlers.py -q ... 18 passed (exit 0) $ pytest tests/test_exports.py -q no tests ran (exit 5) <- wrong path; re-run with tests/export/

The exit 5 line is the kind of thing that turns into "tests pass" in a rushed summary. It doesn't pass; it didn't run.

Rule block for AGENTS.md / CLAUDE.md

markdown## Checks before "done" - Use the commands this repo defines (see CONTRIBUTING.md, package.json scripts, CI config). Defaults if none: `pyright` or `mypy -p <pkg>`; `npx tsc --noEmit`; `pytest <paths>`; `npm test`. - Run the type checker over the whole project, not only the files you edited. - Run the tests that cover the code you edited. An empty selection (pytest exit 5) is not a pass. - Do not change a test to make it pass unless the task changes the behaviour it describes. - Put each command and its result (pass, fail, exit code) in the final message. - If a check could not run, say which one and why.

Edge cases

  • No type checker configured. Say so; run the tests and linters, and rely more on AR-25 and AR-26 lists.
  • Slow suites. Run the affected tests first and the full suite once, last. Report which you ran.
  • Flaky or already-failing tests. Run the failing test on the base commit (git stash or a clean checkout) to see whether you caused it, and report either way.
  • Tests that mock the changed function. A mocked call doesn't use the new signature. A passing test there says little about the real callers; check the type checker output for them.
  • Partial typing. Calls through untyped values aren't checked. Strict settings on the files agents edit (AR-17) increase what the checker sees.
  • JavaScript without TypeScript. Without tsc there is no type check; npx tsc --noEmit --allowJs --checkJs is one option if the project enables it. Verified 2026-10-30 (https://www.typescriptlang.org/tsconfig/): checkJs "works in tandem with" allowJs, and enabling it reports errors in JavaScript files, the equivalent of // @ts-check at the top of every included JavaScript file; allowJs defaults to false unless checkJs is set.

What LLMGRAPH shows

The stop check lists each broken call by file and line, marked certain or not.

References

  • Pyright command line (usage, exit codes 0 to 4): https://github.com/microsoft/pyright/blob/main/docs/command-line.md
  • mypy command line (files, -m, -p, --strict): https://mypy.readthedocs.io/en/stable/command_line.html
  • TSConfig reference, noEmit: https://www.typescriptlang.org/tsconfig/#noEmit
  • pytest, how to invoke (node IDs, -k): https://docs.pytest.org/en/stable/how-to/usage.html
  • pytest exit codes: https://docs.pytest.org/en/stable/reference/exit-codes.html
  • npm test: https://docs.npmjs.com/cli/v10/commands/npm-test
  • TSConfig allowJs, checkJs: https://www.typescriptlang.org/tsconfig/ (verified 2026-10-30)

Changelog

  • 2026-10-30: draft 1.
  • 2026-10-30: docs checks resolved
  • 2026-10-30: copy-gate wording fixes
  • AR-26 How does an agent know an edit is finished?
  • AR-30 What should an agent report when it can't verify an edit?
  • AR-17 Which type checker settings make code easier for agents?

Last verified

For agentsthis page as Markdown/agent-readable/ar-29.mdllms.txt