Verification (verify.yml)

Otto automatically verifies generated code — running lint, build, and test commands — before opening a pull request. You can customize this behavior with a .otto/verify.yml file in your repository root.

Quick Start

In most cases, you don't need a verify.yml file at all. Otto auto-detects your project type and runs the appropriate commands. Add one only when you need to customize the defaults.

# .otto/verify.yml
max_retries: 3
runtime:
  node: '20'
verify:
  - pnpm run lint
  - pnpm run build

Configuration Reference

Field	Type	Default	Description
`enabled`	boolean	`true`	Set to `false` to skip verification entirely
`max_retries`	number	`2`	How many times the fix agent retries on failure (0–5)
`runtime`	map	(auto-detected)	Runtime tool versions to install (e.g., `node: "20"`)
`install`	string	(auto-detected)	Dependency install commands
`verify`	string	(auto-detected)	Verification commands (lint, build, test, etc.)
`timeout.install_seconds`	number	`300`	Timeout per install command (seconds)
`timeout.verify_seconds`	number	`180`	Timeout per verify command (seconds)

Full Example

# .otto/verify.yml

# Disable verification entirely (not recommended)
# enabled: false

# Number of times the fix agent will retry on failure (0–5)
max_retries: 3

# Runtime versions (installed via mise)
runtime:
  node: '20'
  python: '3.12'

# Dependency installation (run sequentially, fail-fast)
install:
  - pnpm install --frozen-lockfile
  - pip install -e .

# Verification commands (run sequentially, all results collected)
verify:
  - pnpm run lint
  - pnpm run build
  - pnpm run test
  - pytest

# Per-command timeouts
timeout:
  install_seconds: 600
  verify_seconds: 300

Auto-Detection

When install and verify are omitted, Otto detects your project type and generates commands automatically.

Project Type	Detected By	Install Command	Verify Commands
Node.js (pnpm)	`pnpm-lock.yaml`	`pnpm install --frozen-lockfile`	`pnpm run lint`, `pnpm run build`, `pnpm run test`
Node.js (yarn)	`yarn.lock`	`yarn install --frozen-lockfile`	`yarn run lint`, `yarn run build`, `yarn run test`
Node.js (npm)	`package-lock.json`	`npm ci`	`npm run lint`, `npm run build`, `npm run test`
.NET	`.csproj` or `.sln`	`dotnet restore`	`dotnet build --no-restore`, `dotnet test --no-build`
Go	`go.mod`	`go mod download`	`go vet ./...`, `go build ./...`
Python	`pyproject.toml`	`pip install -e .`	`ruff check .`, `pytest`
Rust	`Cargo.toml`	(none)	`cargo check`, `cargo clippy`
Makefile	`Makefile`	(none)	`make lint`, `make build`, `make test`

For Node.js projects, only scripts that exist in your package.json are included. If your package.json has no test script, pnpm run test won't be added.

For Makefile projects, only targets that exist in your Makefile are included.

Runtime Version Detection

Otto also auto-detects runtime versions when the runtime field is omitted:

Node.js — checked in order:

.node-version file
.nvmrc file
.tool-versions file (nodejs entry)
package.json engines.node field (extracts major version)

.NET — reads global.json sdk.version field

Other runtimes (Go, Python, Rust) — defaults to latest when auto-detected

Runtimes are installed via mise, so any tool mise supports can be specified in the runtime map (e.g., java: "21", ruby: "3.3").

Partial Configuration

You can override specific fields while relying on auto-detection for the rest. If you provide runtime, max_retries, or timeout but omit install and verify, Otto will auto-detect the commands and merge your overrides.

# Auto-detects everything else
max_retries: 4

# Auto-detects install and verify commands
runtime:
  node: '22'

# Auto-detects commands, extends timeout
timeout:
  verify_seconds: 600

How Verification Works

After the code agent finishes making changes, Otto runs verification before opening a PR:

Install runtimes — Installs each tool version from runtime via mise
Run install commands — Runs install commands sequentially; stops immediately if any fail
Run verify commands — Runs verify commands sequentially; collects all results (does not stop on first failure)
Retry on failure — If any verify command fails, a fix agent receives the errors and attempts to correct the code, then verification re-runs (up to max_retries times)
Report in PR — Results are included in the pull request body

PR Result Examples

When all checks pass:

### Verification

✅ All checks passed (3 commands, 1 attempt)

When checks fail after retries:

### Verification

> **Warning**: Verification failed after 3 attempts

- `pnpm run lint` (exit code 1)
- `pnpm run build` (exit code 1)

Best Practices

Start with auto-detection. Otto's defaults work well for most projects. Only create a verify.yml when you need to customize behavior.

Keep verify commands fast. Verification runs on every agent task. Long-running test suites slow down the feedback loop. Consider running only fast checks (lint, build, unit tests) and leaving integration tests to your CI pipeline.

Use enabled: false sparingly. Disabling verification means the agent's code won't be checked before a PR is opened. Use this only for repositories where verification isn't applicable.

Set appropriate timeouts. The defaults (5 minutes for install, 3 minutes per verify command) work for most projects. Increase them for large monorepos or slow build steps.

Increase retries for complex projects. If your codebase has strict linting or complex build requirements, bumping max_retries to 3–4 gives the fix agent more chances to correct issues.