Skip to main content

Testing and fixtures

botctl uses recorded fixtures to keep classifier behavior stable as Claude and Codex UI text changes.

Corpus layout​

Fixture cases live under fixtures/cases/*.

These checked-in fixture files are intentionally separate from machine-local runtime artifacts under <state-root>/artifacts. state.db holds control-plane state; fixture corpora, capture logs, and replay inputs stay as regular files.

Each case directory is a self-contained scenario, usually containing:

  • expected.txt — expected state plus recap metadata
  • frame.txt — the captured pane text used by the classifier
  • control-mode.log — raw control-mode lines when the case was recorded
  • metadata.txt — record-time summary for debugging and review

Some older cases only contain expected.txt and frame.txt; replay still reads those.

Recording a fixture​

Use record-fixture when you have a live session to capture:

botctl record-fixture --session demo --pane %19 --case new_case --output-dir fixtures/cases

Recording stores the frame, expected state, and supporting evidence in a new case directory.

For Codex CLI panes, fixtures can also be created from direct tmux capture-pane snapshots when control-mode recording is not appropriate. Preserve enough screen text to explain the Codex prompt or dialog that produced the expected state.

Replaying a fixture​

Replay verifies that the classifier still matches the recorded expectation:

botctl replay --path fixtures/cases/new_case

The replay test suite also walks every directory under fixtures/cases/*.

Classification checks​

Use the direct classifier command for a single frame or fixture file:

botctl classify --path fixtures/cases/chat_ready/frame.txt

Regression workflow​

  1. add or update the fixture case
  2. update the classifier if needed
  3. update guarded workflows if the state contract changed
  4. run cargo test

The most important regression test is tests/replay.rs, which ensures every fixture case still classifies as expected.

Test basics​

cargo test

Related docs: command-reference.mdx, workflows.mdx, automation.mdx, serve-mode.mdx.