All docs
How Assertions Work
How Studio checks each replay step, and how to control how strict each check is.
How replay checking works
When you replay a session, Studio runs each recorded action against the live server and compares the result to what was recorded. If anything doesn’t match, that step fails.
There are two things checked per step:
- Result - what the server returned (tool output, resource contents, whether a widget matched, etc.)
- Call counts - how many times each tool, resource, or widget was invoked
Assertion modes
Not every field should be checked the same way. Studio lets you set a mode per field to control how strict the comparison is.
| Mode | Behavior |
|---|---|
exact | Must match exactly. Default for most fields. |
shape | Structure and types must match, but values can change. Use when content varies but the shape stays the same. |
flaky | Ignores UUIDs, timestamps, and other known non-deterministic values. Everything else is checked exactly. |
ignore | Always passes. Use for fields you don’t care about. |
When to use each
- Tool returns a booking confirmation with a generated ID: use
flakyso the ID is skipped but everything else is checked. - Tool returns different data each run but always the same structure: use
shape. - A field changes every time and you don’t need to verify it: use
ignore. - Everything should be identical: leave it as
exact.
Questions, bugs, or feedback?
Reach the team at [email protected] . Bug reports, feature requests, and questions are all welcome.