All docs

How Assertions Work

How Studio checks each replay step, and how to control how strict each check is.

How replay checking works

When you replay a session, Studio runs each recorded action against the live server and compares the result to what was recorded. If anything doesn’t match, that step fails.

There are two things checked per step:

  • Result - what the server returned (tool output, resource contents, whether a widget matched, etc.)
  • Call counts - how many times each tool, resource, or widget was invoked

Assertion modes

Not every field should be checked the same way. Studio lets you set a mode per field to control how strict the comparison is.

ModeBehavior
exactMust match exactly. Default for most fields.
shapeStructure and types must match, but values can change. Use when content varies but the shape stays the same.
flakyIgnores UUIDs, timestamps, and other known non-deterministic values. Everything else is checked exactly.
ignoreAlways passes. Use for fields you don’t care about.

When to use each

  • Tool returns a booking confirmation with a generated ID: use flaky so the ID is skipped but everything else is checked.
  • Tool returns different data each run but always the same structure: use shape.
  • A field changes every time and you don’t need to verify it: use ignore.
  • Everything should be identical: leave it as exact.

Questions, bugs, or feedback?

Reach the team at [email protected] . Bug reports, feature requests, and questions are all welcome.