Concrete scenarios
Describe the situation, input, and behavior to protect so a test case represents a deliberate customer-facing scenario rather than a vague instruction.
Mibo turns agent expectations into test cases you can run and inspect.
Mibo helps teams describe a real agent scenario, add semantic and procedural assertions, run selected cases deliberately, and evaluate applicable traces when the evidence is available.
A test plan is hard to act on when expectations stay in prose.
Teams need to turn a real customer situation into a concrete input, an expected behavior, and evidence they can review. Mibo keeps those parts together without treating one scenario as a claim about every interaction.
Semantic expectations
Write criteria for meaning, safety, completeness, or alignment in plain language. The evaluator scores the response and the available context against what you described.
Procedural evidence
Check observable facts such as node or tool calls, arguments, status, schema, timing, or token usage when the active response or trace includes the needed fields.
From a real expectation to evidence you can investigate
Build a focused case, choose the checks that express its behavior, and use the result to decide what to inspect next. Active runs and passive traces remain distinct signals.
01
Describe the scenario
Write the customer situation and the input that should exercise it. Include the expected outcome and any important edge or safety condition.
02
Add assertions
Combine semantic criteria with procedural assertions for the response and the execution evidence your agent can expose. Set thresholds and expected values deliberately.
03
Run and evaluate
Run the selected case against a connected Agent, then evaluate applicable canonical traces from real interactions asynchronously when your system sends them.
A single case can connect a customer scenario to its evidence
Semantic assertions
Describe what a useful, safe, complete, or grounded response should mean. The score is evidence for review and depends on the response and context available to the evaluator.
Procedural assertions
Verify deterministic facts such as a required tool call, an argument, a response field, an HTTP status, a schema, a time limit, or token usage. Missing trace fields surface as missing instrumentation.
Frequently asked questions
What is an AI agent test case?
It is a defined scenario with an input, expected behavior, and assertions that Mibo can evaluate. Assertions can describe response meaning or check observable execution facts such as tool calls, arguments, status, schema, timing, or tokens.
How are test cases different from general AI agent evaluation?
A test case is the concrete scenario and assertion definition. Mibo evaluates its configured checks during a deliberate active run and can reuse applicable active test cases when your system sends traces from real interactions.
Can Mibo run a selected test case before release?
Yes. Active testing sends the selected input to a connected Agent and evaluates the response and available execution evidence. The result is evidence for that scenario, not proof about unobserved cases.
Can a test case be evaluated against a real trace?
Yes, when the test is Active and eligible for passive evaluation and your system sends a canonical trace after the interaction. Mibo classifies applicable scenarios and evaluates them asynchronously without calling the Agent again.
What if a test assertion needs data that the trace does not contain?
Mibo reports missing instrumentation for supported checks that cannot find the required trace field instead of silently treating the assertion as passed or failed. Add the relevant fields if that evidence matters to your workflow.
Do AI agent test cases guarantee complete coverage?
No. The signal is bounded by the scenarios and assertions you configure, the inputs and traces available, classification, instrumentation, and the breadth of interactions you exercise or observe. Mibo does not claim guaranteed quality or automatic remediation.
Keep exploring
Create tests with Mibo's Test Architect
Use the maintained documentation to define scenarios, inputs, assertions, and passive behavior.
Read test creation docs →Assertion reference
Review the supported semantic and procedural checks and the trace evidence each one can use.
Read the assertion docs →AI agent testing
Define scenarios, run active checks before release, and reuse them for passive production evaluation.
Explore AI agent testing →AI agent evaluation
Understand how Mibo evaluates configured behavior beyond a response code.
Explore evaluation →AI agent test coverage
See which conditional scenarios real traffic matches while keeping observed coverage distinct from assertion results.
Explore test coverage →How to test AI agents
Follow a practical workflow for scenarios, assertions, active runs, passive traces, and interpretation limits.
Read the guide →See Mibo in action
Run the canonical live reliability check with a synthetic customer request.
Run a live check →