# What should a coding agent run before finishing? > Rule AR-29: Run the type checker and the affected tests before saying \"done\". Two commands catch most of what a signature or rename edit breaks: the type checker over the whole project, and the tests that cover the code you edited. An edit that was never checked is a draft. Run both, read the exit codes, and put the commands and results in the final message. ## Why it matters An agent changes a return type from `str` to `str | None` in a lookup helper and reads the three callers in the file it edited. They all handle `None`. The task closes. A fourth caller in another package does `lookup(key).lower()`. `pyright` reports `"lower" is not a known attribute of "None"` in that file, and `pytest` would fail the one test that reaches it. Neither was run, so neither said anything. The two checks cover different ground: - The type checker sees every call site it can resolve, including files you didn't open and paths no test takes. - Tests run real behaviour: values, side effects, and the code paths the types can't describe. Neither is the full picture (AR-27 applies to both). Together they shrink the open set to something you can name. ## The steps 1. **Find the project's own commands first.** Read `AGENTS.md`, `CONTRIBUTING.md`, `pyproject.toml`, `package.json` scripts, the CI config. Use those over the defaults below. 2. **Type-check the whole project.** ```bash pyright # exit 0: no errors, 1: errors, per the pyright docs mypy -p mypackage # or: mypy src/ (directories are checked recursively) npx tsc --noEmit # checks types, writes no output files ``` 3. **Run the affected tests**, then the wider suite if time allows. ```bash pytest tests/test_money.py # one file pytest tests/test_money.py::test_format_amount # one test pytest -k "format_amount" # by keyword npm test # runs the "test" script from package.json ``` 4. **Read the exit code, not only the text.** pytest exits 0 when all tests were collected and passed, 1 when some failed, 5 when no tests were collected. Exit 5 means your selection matched nothing, which is not a pass. 5. **Run linters the project already uses** (`ruff check`, `eslint .`) if they are part of CI. 6. **Fix or report.** A failure you didn't cause is still a fact for the final message. Don't edit tests to match new behaviour unless the task says the behaviour should change. 7. **Write the commands and their results into the final message** (AR-30). ## Worked example Illustrative session, not a captured run: ```text $ pyright src/app/handlers.py:77:20 - error: "lower" is not a known attribute of "None" 1 error, 0 warnings (exit 1) fix: handlers.py:77 -> handle None from lookup() $ pyright 0 errors, 0 warnings (exit 0) $ pytest tests/test_lookup.py tests/test_handlers.py -q ... 18 passed (exit 0) $ pytest tests/test_exports.py -q no tests ran (exit 5) <- wrong path; re-run with tests/export/ ``` The exit 5 line is the kind of thing that turns into "tests pass" in a rushed summary. It doesn't pass; it didn't run. ## Rule block for AGENTS.md / CLAUDE.md ```markdown ## Checks before "done" - Use the commands this repo defines (see CONTRIBUTING.md, package.json scripts, CI config). Defaults if none: `pyright` or `mypy -p `; `npx tsc --noEmit`; `pytest `; `npm test`. - Run the type checker over the whole project, not only the files you edited. - Run the tests that cover the code you edited. An empty selection (pytest exit 5) is not a pass. - Do not change a test to make it pass unless the task changes the behaviour it describes. - Put each command and its result (pass, fail, exit code) in the final message. - If a check could not run, say which one and why. ``` ## Edge cases - **No type checker configured.** Say so; run the tests and linters, and rely more on AR-25 and AR-26 lists. - **Slow suites.** Run the affected tests first and the full suite once, last. Report which you ran. - **Flaky or already-failing tests.** Run the failing test on the base commit (`git stash` or a clean checkout) to see whether you caused it, and report either way. - **Tests that mock the changed function.** A mocked call doesn't use the new signature. A passing test there says little about the real callers; check the type checker output for them. - **Partial typing.** Calls through untyped values aren't checked. Strict settings on the files agents edit (AR-17) increase what the checker sees. - **JavaScript without TypeScript.** Without `tsc` there is no type check; `npx tsc --noEmit --allowJs --checkJs` is one option if the project enables it. Verified 2026-10-30 (https://www.typescriptlang.org/tsconfig/): `checkJs` "works in tandem with" `allowJs`, and enabling it reports errors in JavaScript files, the equivalent of `// @ts-check` at the top of every included JavaScript file; `allowJs` defaults to false unless `checkJs` is set. ## What LLMGRAPH shows The stop check lists each broken call by file and line, marked certain or not. ## References - Pyright command line (usage, exit codes 0 to 4): https://github.com/microsoft/pyright/blob/main/docs/command-line.md - mypy command line (files, `-m`, `-p`, `--strict`): https://mypy.readthedocs.io/en/stable/command_line.html - TSConfig reference, `noEmit`: https://www.typescriptlang.org/tsconfig/#noEmit - pytest, how to invoke (node IDs, `-k`): https://docs.pytest.org/en/stable/how-to/usage.html - pytest exit codes: https://docs.pytest.org/en/stable/reference/exit-codes.html - npm test: https://docs.npmjs.com/cli/v10/commands/npm-test - TSConfig `allowJs`, `checkJs`: https://www.typescriptlang.org/tsconfig/ (verified 2026-10-30) ## Changelog - 2026-10-30: draft 1. - 2026-10-30: docs checks resolved - 2026-10-30: copy-gate wording fixes ## Related - [AR-26: How does an agent know an edit is finished?](https://beta.llmgraph.in/agent-readable/ar-26.md) - [AR-30: What should an agent report when it can't verify an edit?](https://beta.llmgraph.in/agent-readable/ar-30.md) - [AR-17: Which type checker settings make code easier for agents?](https://beta.llmgraph.in/agent-readable/ar-17.md) Last verified 2026-10-30