From e6992d01df66356aff74dc8b6a5ad65ae7550659 Mon Sep 17 00:00:00 2001 From: Him188 Date: Sat, 17 Oct 2026 20:57:49 +0900 Subject: [PATCH] docs(agents): Simulation test pricing A run that ends in Pass or Fail is charged a platform fee per agent reply plus the agent's, the simulated user's and the judge's tokens. The basic models bill on their tokens here. The result card shows the total and the billing page itemizes it. Co-Authored-By: Claude Opus 5.5 --- agents/pricing.mdx | 42 +++++++++++++++++++++++++++---------- agents/test/agent-tests.mdx | 16 +++++++++----- 2 files changed, 42 insertions(+), 16 deletions(-) diff --git a/agents/pricing.mdx b/agents/pricing.mdx index cd6e59f3..eeb5c0a1 100644 --- a/agents/pricing.mdx +++ b/agents/pricing.mdx @@ -1,6 +1,6 @@ --- title: "Pricing" -description: "Rates for conversations, LLM usage, telephony, and transfers, plus the trial balance and billing" +description: "Rates for conversations, LLM usage, telephony, transfers, and simulation tests, plus the trial balance and billing" icon: "credit-card" --- @@ -57,6 +57,25 @@ What a minute costs on such a model therefore depends on your agent: how long th After a conversation, each message in the dashboard's conversation view shows its model requests and token cost. A [custom LLM](/agents/build/custom-llm) endpoint carries no LLM charge. +### Simulation tests + +A [Simulation test](/agents/test/agent-tests#simulation-tests) has the agent talk to a simulated user in text, then an LLM judge scores the conversation. There is no conversation rate. Each run that ends in **Pass** or **Fail** is charged: + +- A platform fee of \$0.003 per agent reply. A [fixed first message](/agents/build/configuration#first-message) is spoken as written and is not counted, a generated one is. +- The tokens of the agent, the simulated user, and the judge, each at the rate of the model it ran on. An agent on a [custom LLM](/agents/build/custom-llm) endpoint carries no charge for its own tokens. + +A run that ends in **Error** is not charged, and every repeat of a test is a run of its own. Single Turn and Tool tests are free. + +With no conversation rate to cover them, the basic models are billed on their tokens in Simulation tests. Prices are in USD per 1 million tokens: + +| Model | Input | Cached input | Output | +| --------------------- | ------ | ------------ | ------ | +| Gemini 3.5 Flash Lite | \$0.30 | \$0.03 | \$2.50 | +| GPT-5.6 Luna | \$0.20 | \$0.02 | \$1.20 | +| Gemma 4 26B | \$0.10 | \$0.05 | \$0.30 | + +The other models are billed at the same rates as in conversations. The result of each run shows what it was charged, and the [Agents billing page](https://fish.audio/app/agents/billing) itemizes the platform fee and each participant's tokens. + ### Outbound destinations The standard phone surcharge applies to inbound calls and most outbound calls. Outbound calls to the destinations below replace it with a rate that depends on the number dialed: @@ -75,26 +94,27 @@ The countries purchased numbers can call are listed under [allowed destinations] Every account starts with a one-time **\$5 trial balance**, about 60 minutes of conversation at the base rate. It pays for real usage at the normal rates: -- Every charge draws on it first: web sessions, phone calls and their surcharges, transferred call time, number rental, and LLM tokens on token-billed models. Only what the trial balance does not cover is charged to your API credit. -- A session can start on any model, including token-billed ones, as long as the trial balance or your API credit is above zero. +- Every charge draws on it first: web sessions, phone calls and their surcharges, transferred call time, number rental, LLM tokens on token-billed models, and Simulation test runs. Only what the trial balance does not cover is charged to your API credit. +- A session can start on any model, including token-billed ones, as long as the trial balance or your API credit is above zero. The same goes for running Simulation tests. - [Preview calls](/agents/test/preview-calls) in the Builder are free on every LLM, including token-billed ones, and never draw on the trial balance. - The [Agents billing page](https://fish.audio/app/agents/billing) shows how much of the trial balance is left, and the Agents home page shows it as a usage meter. ## Billing -Usage is charged to your **API credit**, the same prepaid balance as the Fish Audio speech APIs. Sessions are charged when they end, and number rental is charged daily. +Usage is charged to your **API credit**, the same prepaid balance as the Fish Audio speech APIs. Sessions are charged when they end, Simulation test runs when they finish, and number rental is charged daily. Every charge is listed on the [Agents billing page](https://fish.audio/app/agents/billing). ## Examples -| Scenario | Breakdown | Total | -| ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------ | -| Web conversation, 4 min 30 s, basic LLM | Conversation 4.5 min × \$0.06 = \$0.27 | \$0.27 | -| Inbound phone call, 6 min, basic LLM | Conversation 6 min × \$0.06 = \$0.36
Phone surcharge 6 min × \$0.01 = \$0.06 | \$0.42 | -| Outbound call, 10 min, Japanese mobile number | Conversation 10 min × \$0.06 = \$0.60
Japan mobile surcharge 10 min × \$0.22 = \$2.20 | \$2.80 | -| Inbound call, 3 min with the agent, then 5 min with a person after a warm transfer | Conversation 3 min × \$0.06 = \$0.18
Phone surcharge 3 min × \$0.01 = \$0.03
Warm transfer 5 min × \$0.025 = \$0.125 | \$0.335 | -| Web conversation, 5 min, token-billed LLM | Conversation 5 min × \$0.06 = \$0.30
LLM usage: the session's tokens at the LLM's rates | \$0.30 + LLM usage | +| Scenario | Breakdown | Total | +| ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------- | +| Web conversation, 4 min 30 s, basic LLM | Conversation 4.5 min × \$0.06 = \$0.27 | \$0.27 | +| Inbound phone call, 6 min, basic LLM | Conversation 6 min × \$0.06 = \$0.36
Phone surcharge 6 min × \$0.01 = \$0.06 | \$0.42 | +| Outbound call, 10 min, Japanese mobile number | Conversation 10 min × \$0.06 = \$0.60
Japan mobile surcharge 10 min × \$0.22 = \$2.20 | \$2.80 | +| Inbound call, 3 min with the agent, then 5 min with a person after a warm transfer | Conversation 3 min × \$0.06 = \$0.18
Phone surcharge 3 min × \$0.01 = \$0.03
Warm transfer 5 min × \$0.025 = \$0.125 | \$0.335 | +| Web conversation, 5 min, token-billed LLM | Conversation 5 min × \$0.06 = \$0.30
LLM usage: the session's tokens at the LLM's rates | \$0.30 + LLM usage | +| Simulation test run with 6 agent replies | Platform fee 6 × \$0.003 = \$0.018
LLM usage: the agent's, the simulated user's, and the judge's tokens | \$0.018 + LLM usage | ## Going further diff --git a/agents/test/agent-tests.mdx b/agents/test/agent-tests.mdx index 6d1d4651..b8a47307 100644 --- a/agents/test/agent-tests.mdx +++ b/agents/test/agent-tests.mdx @@ -277,11 +277,11 @@ Open the test and switch to its **Run** tab. Pick an agent that has access, then The verdict card shows **Pass** or **Fail** with the time the run took, followed by details for the test type: -| Type | What you see | -| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Single Turn** | The full reply the agent generated and why the judge passed or failed it. | -| **Tool** | Which tool the agent called, if any, against what the test expected. | -| **Simulation** | The judge's summary, every success condition with its verdict and rationale, every assertion with its outcome, who ended the conversation, after how many turns and on which channel, the LLM cost of the agent, the simulated user, and the judge with their token breakdown, and the full transcript with each tool call marked by where its answer came from: **Mock** for an entry in the test, **Tool mock** for the tool's own mock response, **Real** when it reached the real endpoint, **No mock** or **No matching mock** when it failed for lack of one. | +| Type | What you see | +| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Single Turn** | The full reply the agent generated and why the judge passed or failed it. | +| **Tool** | Which tool the agent called, if any, against what the test expected. | +| **Simulation** | The judge's summary, every success condition with its verdict and rationale, every assertion with its outcome, who ended the conversation, after how many turns and on which channel, what the run was charged and whether the trial balance paid it, and the full transcript with each tool call marked by where its answer came from: **Mock** for an entry in the test, **Tool mock** for the tool's own mock response, **Real** when it reached the real endpoint, **No mock** or **No matching mock** when it failed for lack of one. | A Simulation run with an **Unknown** condition fails and carries a **Needs review** badge. If a run can't complete, the card shows **Error** with a message that says whether the agent caused it, for example its custom LLM endpoint stopped responding, or the platform did, for example our language model did not answer. A conversation that broke off or hit its time limit is never judged, so it can't pass by accident. The transcript is still shown below the error, as it is when the conversation finished but could not be judged. @@ -352,6 +352,12 @@ jobs: Simulation tests take minutes, so give the job a timeout that covers your longest scenarios. To tolerate occasional failures of repeated Simulation tests, gate on `pass_rate` instead, for example `jq -e '.pass_rate >= 0.8'`. +## Billing + +Single Turn and Tool tests are free. Each Simulation run that ends in **Pass** or **Fail** is charged a platform fee of $0.003 per agent reply, plus the LLM tokens of the agent, the simulated user, and the judge. A fixed first message is not counted as a reply, a generated one is. Runs that end in **Error** are not charged, and every repeat is a run of its own. The basic models are billed on their tokens in Simulation tests, see [Pricing](/agents/pricing#simulation-tests) for the rates. + +Runs draw on the trial balance first, then your API credit, and each one is itemized on the [Agents billing page](https://fish.audio/app/agents/billing): the reply fee, then the tokens of each participant by model and token category with their rates. The test name on each line opens that run's result. Over the API, a run's `usage.cost_usd` is what it was charged, `0` for a run that is not charged, and two fields show what it adds up to: `usage.platform_fee` gives the rate per reply and the fee, and `usage.llm` has one entry per participant and model with its amount and the tokens and cost of each token category (input, cached input, cache writes, output). The charge is usually settled as soon as the run ends, and until then `usage.cost_usd` and `usage.platform_fee` are `null` and `usage.llm` is empty. Running a Simulation test needs the trial balance or your API credit above zero: otherwise the Builder and [Run Tests](/api-reference/endpoint/agent/run-tests) refuse it with a 402, and a queued run that finds the balance used up ends in **Error** without being charged. + ## Limits | Field | Limit |