Skip to main content
Log in Get started

Check what your agent does in sensitive situations.

A reassuring answer can hide an unsafe tool call. Check the response and the actions recorded in the trace.

Get started freeExplore test cases

A safe answer can hide an unsafe action

A support agent might say it sent a request for review while its trace shows a restricted tool call. Check both before deciding what happened.

Customer-facing scenarios

Define the safety-related situation, input, and behavior you expect so a test case reflects a concrete release concern rather than generic safety advice.

Semantic behavior

Write criteria for a response to refuse, clarify, disclose uncertainty, or stay within an approved scope. The evaluator scores the response and context available to it.

Recorded actions

Inspect recorded facts such as required or forbidden tool calls and their arguments when the trace contains the necessary instrumented evidence.

Test the authorization boundary

Define who can authorize the action, then test a request that lacks that authorization.

  1. Name the forbidden action

    Choose a sensitive operation and state the authorization needed. Include requests that are ambiguous, unauthorized, or influenced by untrusted content.

  2. Separate words from actions

    Check what the response claims and what the tools did. A refusal in the answer cannot compensate for a recorded forbidden mutation.

  3. Review evidence and severity

    Inspect every forbidden action and confirm trace completeness. Your team decides escalation and remediation; a passing check does not establish overall safety.

Review safety checks alongside the trace

Semantic safety checks

Write what the response should say or avoid. An AI-powered check scores it against your criterion using the available context.

Procedural safety checks

Check whether a forbidden tool was called or an allowed tool received the required arguments. Confirm the relevant actions are captured; missing spans limit the conclusion.

Frequently asked questions

What is AI agent safety testing?

It is the deliberate testing of safety-related behavior in a defined AI agent scenario. Mibo lets teams combine semantic criteria about the response with procedural checks on supported execution evidence.

How does Mibo support safety testing before release?

Run your agent with the chosen scenario, then send the recorded execution over OTLP for Mibo to evaluate. Review the available evidence before deciding on a release.

Can Mibo inspect safety behavior after real interactions?

After an interaction, send its recorded execution over OTLP. Mibo evaluates applicable configured checks without calling the agent again.

What safety-related checks can teams configure?

Checks can assess response meaning and captured execution facts such as tool calls and arguments. Available evidence depends on your instrumentation.

What if a safety check needs data that the trace does not contain?

The outcome depends on the check. Some checks report missing instrumentation; others evaluate the missing value against their configured condition. Inspect the required fields and the check’s behavior before interpreting the result.

Does AI agent safety testing guarantee that an agent is safe?

No. Configured checks provide bounded evidence, not a safety guarantee or automatic remediation. The signal is limited by your scenarios, checks, inputs and traces, classification, instrumentation, and the breadth of interactions you exercise or observe.

Keep exploring