Simulations

Test your AI Agents with simulated conversations: define scenarios and success criteria, organize them into suites, and catch regressions before you publish.

Simulations let you test your virtual assistant with realistic, AI-driven conversations — before your users do. Each simulation describes a scenario to play against your assistant and the criteria that determine whether the outcome is a success. You can run simulations one by one, as a suite, or all together, and review every result without leaving the platform.

You'll find Simulations in the Utilities section of the side menu, together with Conversation Logs and Evaluators.

🧪 Anatomy of a Simulation

A simulation is the recipe for a single test:

• Scenario — the instructions for the AI role-player that chats with your assistant, impersonating a user with a goal (e.g., "You are a customer whose book order hasn't arrived yet. You don't remember the order number.").

• Success criteria — how the resulting conversation is judged (e.g., "The assistant apologizes, retrieves the order status after asking for the customer's email, and offers a concrete next step."). You can optionally add examples of good and bad answers to sharpen the evaluation.

• Conversation cap — the maximum number of messages the role-play can exchange, so every run stays fast and predictable.

• Starting variables — preset values for variables, useful to simulate a specific user profile or entry point.

When a simulation runs, the platform plays the user's side of the conversation against your assistant, then evaluates the transcript against your success criteria.

➕ Creating Simulations

Simulations are valuable when creating them is effortless, so there is more than one way to build your test set:

• New simulation — write the scenario and success criteria by hand.

• Generate with AI — the platform drafts a set of simulations based on your assistant's configuration; review them and keep what you need.

• Import — bulk-load simulations from a CSV file.

• From a real conversation — from Conversation Logs, turn a conversation that went wrong into a simulation. The problem becomes a permanent test: once fixed, it can't silently come back.

🗂️ Suites

Suites are thematic groups of simulations (e.g., "Order tracking", "Returns", "Tone of voice") that you can run together in one click. A simulation can belong to more than one suite.

One suite is special: No Regression Test (NRT). It collects the simulations that must always pass — your safety net against regressions.

🚦 No Regression Test and Publishing

The NRT suite is the bridge between testing and going live: it gates the Publish action. If the latest NRT results include failing simulations — or an NRT run is still in progress — the platform warns you before completing the publish. From there you can jump straight to the results, fix what broke, or consciously choose to publish anyway.

▶️ Runs and Results

You can run a single simulation, a whole suite, or everything at once. While a run is in progress, a progress banner stays visible next to the Publish button, so you can keep working anywhere in the platform without losing track of it.

Every simulation in a run produces a Pass / Fail / Error result with a natural-language explanation of why — not just a red or green light. The full generated conversation is saved too: you can read it like any other chat, and simulated conversations are kept separate from your users' real ones in the Chats section.

Past runs remain available in the Runs tab of the Simulations page, so you can compare results over time.

🔐 Permissions

Access to Simulations is governed by dedicated permissions — viewing, creating, and editing/running are granted separately. Workspace Owners and Admins can assign them to any role from Team settings, so the whole team can contribute to testing with the right level of access.

📏 Limits

To keep executions under control, batch runs are capped at 200 simulations per run, and each workspace can execute up to 500 simulations within any 5-hour window. If you hit the limit, the platform tells you how long to wait before running again.

Simulations complement Evaluators and Guardrails: evaluators measure the quality of real conversations after they happen, while simulations verify your assistant's behavior on test conversations before changes reach your users.