API Reference

Automated testing API

Test an agent with simulated customers, compare its versions and read how each conversation scored.

A test scenario is a situation and the simulated customers who bring it: each customer has a name, a goal, a personality and the objections they raise. A test run runs a scenario: every customer holds a conversation with the agent version you pick (by default the deployed one), or with up to four versions to compare them, and the evaluator scores each conversation on the agent's goals and competencies. Nothing reaches a real customer and nothing is booked. A workspace's first 50 finished test conversations are free; each one after that uses credits at its plan's test conversation price. Keys need testing:read or testing:write.

The usual flow: create a scenario with POST /v1/agents/:id/test-scenarios, start a run with POST /v1/test-scenarios/:id/runs, then poll GET /v1/test-runs/:id until its status is completed, partial or failed: it carries each version's results, the winner of a comparison and every conversation with its transcript and scores. To try a single message first, send it with POST /v1/agents/:id/test-chat.

Changes run as the key's creator

Creating, editing and running tests apply Automated testing's own rules as the member who created the API key: their role must allow it (owners, admins and builders; operators can also start runs), and a key whose creator has left the workspace is refused. A reseller key reads, but does not change, a client's tests.

Endpoints

GET/v1/agents/:id/test-scenariosList an agent's test scenarios POST/v1/agents/:id/test-scenariosCreate a test scenario GET/v1/test-scenarios/:idGet a test scenario PATCH/v1/test-scenarios/:idUpdate a test scenario DELETE/v1/test-scenarios/:idDelete a test scenario POST/v1/test-scenarios/:id/runsRun a test scenario GET/v1/agents/:id/test-runsList an agent's test runs GET/v1/test-runs/:idGet a test run POST/v1/test-runs/:id/evaluateScore a test run again POST/v1/agents/:id/test-chatTry the agent in a test chat

List an agent's test scenarios

GET

/v1/agents/:id/test-scenarios

Lists the agent's test scenarios, newest first. A scenario is a situation and the simulated customers who bring it; its goals and competencies are what a run scores when it names none.

Scope: testing:read

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Query parameters

Name Type Description

limit

integer

Results per page. 1–100.

Default: 20

offset

integer

Results to skip. At least 0.

Default: 0

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-scenarios?limit=10" \
  -H "Authorization: Bearer YOUR_API_KEY"

A page of scenarios.

200

Response

{
  "success": true,
  "data": [
    {
      "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
      "agent_id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Price objection",
      "description": "A customer asks how much a cleaning costs and pushes back on the price.",
      "channel": "messages",
      "max_turns": 12,
      "conversations_per_customer": 1,
      "pass_threshold": 70,
      "required_keywords": [],
      "customers": [
        {
          "name": "Ana",
          "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
          "personality": "Direct, a little impatient",
          "background": "Has not been to a dentist in two years.",
          "language": "es-MX",
          "objections": [
            "It is cheaper across the street."
          ],
          "pains": [
            "Sensitive gums"
          ],
          "gains": [
            "An appointment this week"
          ],
          "demographics": {}
        }
      ],
      "goals": [
        {
          "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
          "weight": 1
        }
      ],
      "competencies": [
        {
          "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
          "weight": 1
        }
      ],
      "created_at": "2026-10-12T17:40:11.402911+00:00",
      "updated_at": "2026-10-12T17:40:11.402911+00:00"
    }
  ],
  "pagination": {
    "total": 1,
    "limit": 10,
    "offset": 0,
    "has_more": false
  }
}

Errors

Status	Code	When
400	validation_error	A query parameter is invalid (`details.errors` lists each one).
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Create a test scenario

POST

/v1/agents/:id/test-scenarios

Writes a test scenario for the agent, with the dashboard's rules: only this agent's goals and competencies can be linked, an archived agent takes no new scenario, and the scenario is saved with its links at once or not at all. A new scenario runs on messages, one conversation per customer, unless you say otherwise.

Scope: testing:writeWorkspace keys only · acts as the key's creator

The change runs Automated testing's own rules as the member who created the key: the key needs the scope and that member a role that writes tests (owner, admin or builder). A key whose creator has left the workspace, and a reseller key, are refused.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Request body (JSON)

Name	Type	Description
`name` Required	`string`	The scenario's name. 1–200 characters.
`description`	`string \| null`	The situation the customers bring. Up to 5000 characters.
`customers`	`object[]`	The simulated customers who bring the situation. A run needs at least one. 0–50 items.
`customers[].name` Required	`string`	What the customer is called. Up to 200 characters.
`customers[].goal` Required	`string`	What they want from the conversation: the simulated customer pursues it. Up to 2000 characters.
`customers[].personality`	`string`	How they talk, e.g. `direct, a little impatient`. Up to 200 characters.
`customers[].background`	`string \| null`	Who they are and their situation. Up to 4000 characters.
`customers[].language`	`string \| null`	The language they write in (`es-MX`); default the agent's. Up to 35 characters.
`customers[].objections`	`string[]`	What they push back with. 0–20 items.
`customers[].pains`	`string[]`	Their problems. 0–20 items.
`customers[].gains`	`string[]`	What would win them. 0–20 items.
`customers[].demographics`	`object`	Any other facts about them, kept as sent.
`max_turns`	`integer`	The most turns a conversation takes (default 15). 1–30.
`conversations_per_customer`	`integer`	Conversations each customer has with each version in a run. 1–5. Default: `1`
`channel`	`string`	Where the conversations happen. Calls can't be tested yet: a run of a `calls` scenario is refused. One of: messages, calls. Default: `messages`
`pass_threshold`	`number`	The overall score a conversation needs to pass. 1–100. Default: `70`
`required_keywords`	`string[]`	Words the agent must say. 0–50 items.
`goals`	`object[]`	The agent's goals a run scores when it names none. 0–50 items.
`goals[].goal_id` Required	`uuid`	A goal of the agent (`GET /v1/goals?agent_id=`).
`goals[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`
`competencies`	`object[]`	The agent's competencies a run scores when it names none. 0–50 items.
`competencies[].competency_id` Required	`uuid`	A competency of the agent.
`competencies[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-scenarios" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Price objection",
  "description": "A customer asks how much a cleaning costs and pushes back on the price.",
  "max_turns": 12,
  "customers": [
    {
      "name": "Ana",
      "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
      "personality": "Direct, a little impatient",
      "language": "es-MX",
      "objections": [
        "It is cheaper across the street."
      ]
    }
  ],
  "goals": [
    {
      "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735"
    }
  ],
  "competencies": [
    {
      "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee"
    }
  ]
}'

The scenario.

201

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Price objection",
    "description": "A customer asks how much a cleaning costs and pushes back on the price.",
    "channel": "messages",
    "max_turns": 12,
    "conversations_per_customer": 1,
    "pass_threshold": 70,
    "required_keywords": [],
    "customers": [
      {
        "name": "Ana",
        "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
        "personality": "Direct, a little impatient",
        "background": "Has not been to a dentist in two years.",
        "language": "es-MX",
        "objections": [
          "It is cheaper across the street."
        ],
        "pains": [
          "Sensitive gums"
        ],
        "gains": [
          "An appointment this week"
        ],
        "demographics": {}
      }
    ],
    "goals": [
      {
        "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "weight": 1
      }
    ],
    "competencies": [
      {
        "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "weight": 1
      }
    ],
    "created_at": "2026-10-12T17:40:11.402911+00:00",
    "updated_at": "2026-10-12T17:40:11.402911+00:00"
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	A goal or competency is not this agent's.
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).
409	conflict	The agent is archived, so it can't be tested (`details.reason` is `AGENT_ARCHIVED`).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Get a test scenario

GET

/v1/test-scenarios/:id

Returns one test scenario with its simulated customers and its goal and competency links.

Scope: testing:read

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY"

The scenario.

200

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Price objection",
    "description": "A customer asks how much a cleaning costs and pushes back on the price.",
    "channel": "messages",
    "max_turns": 12,
    "conversations_per_customer": 1,
    "pass_threshold": 70,
    "required_keywords": [],
    "customers": [
      {
        "name": "Ana",
        "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
        "personality": "Direct, a little impatient",
        "background": "Has not been to a dentist in two years.",
        "language": "es-MX",
        "objections": [
          "It is cheaper across the street."
        ],
        "pains": [
          "Sensitive gums"
        ],
        "gains": [
          "An appointment this week"
        ],
        "demographics": {}
      }
    ],
    "goals": [
      {
        "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "weight": 1
      }
    ],
    "competencies": [
      {
        "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "weight": 1
      }
    ],
    "created_at": "2026-10-12T17:40:11.402911+00:00",
    "updated_at": "2026-10-12T17:40:11.402911+00:00"
  }
}

Errors

Status	Code	When
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Update a test scenario

PATCH

/v1/test-scenarios/:id

Changes the fields you send. customers, goals and competencies each replace the whole list; the scenario and its links are saved at once or not at all. Runs that already ran keep the customers they met.

Scope: testing:writeCan wait for approvalWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Request body (JSON)

Name	Type	Description
`name`	`string`	The scenario's name. 1–200 characters.
`description`	`string \| null`	The situation the customers bring. Up to 5000 characters.
`customers`	`object[]`	The simulated customers (replaces the whole list: send every customer to keep). 0–50 items.
`customers[].name` Required	`string`	What the customer is called. Up to 200 characters.
`customers[].goal` Required	`string`	What they want from the conversation: the simulated customer pursues it. Up to 2000 characters.
`customers[].personality`	`string`	How they talk, e.g. `direct, a little impatient`. Up to 200 characters.
`customers[].background`	`string \| null`	Who they are and their situation. Up to 4000 characters.
`customers[].language`	`string \| null`	The language they write in (`es-MX`); default the agent's. Up to 35 characters.
`customers[].objections`	`string[]`	What they push back with. 0–20 items.
`customers[].pains`	`string[]`	Their problems. 0–20 items.
`customers[].gains`	`string[]`	What would win them. 0–20 items.
`customers[].demographics`	`object`	Any other facts about them, kept as sent.
`max_turns`	`integer`	The most turns a conversation takes. 1–30.
`conversations_per_customer`	`integer`	Conversations each customer has with each version in a run. 1–5.
`channel`	`string`	Where the conversations happen. Calls can't be tested yet: a run of a `calls` scenario is refused. One of: messages, calls.
`pass_threshold`	`number`	The overall score a conversation needs to pass. 1–100.
`required_keywords`	`string[]`	Words the agent must say. 0–50 items.
`goals`	`object[]`	The agent's goals a run scores when it names none (replaces the list). 0–50 items.
`goals[].goal_id` Required	`uuid`	A goal of the agent (`GET /v1/goals?agent_id=`).
`goals[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`
`competencies`	`object[]`	The agent's competencies a run scores when it names none (replaces the list). 0–50 items.
`competencies[].competency_id` Required	`uuid`	A competency of the agent.
`competencies[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`

Example request

curl

curl -X PATCH "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "max_turns": 15,
  "conversations_per_customer": 2
}'

The updated scenario.

200

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Price objection",
    "description": "A customer asks how much a cleaning costs and pushes back on the price.",
    "channel": "messages",
    "max_turns": 15,
    "conversations_per_customer": 2,
    "pass_threshold": 70,
    "required_keywords": [],
    "customers": [
      {
        "name": "Ana",
        "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
        "personality": "Direct, a little impatient",
        "background": "Has not been to a dentist in two years.",
        "language": "es-MX",
        "objections": [
          "It is cheaper across the street."
        ],
        "pains": [
          "Sensitive gums"
        ],
        "gains": [
          "An appointment this week"
        ],
        "demographics": {}
      }
    ],
    "goals": [
      {
        "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "weight": 1
      }
    ],
    "competencies": [
      {
        "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "weight": 1
      }
    ],
    "created_at": "2026-10-12T17:40:11.402911+00:00",
    "updated_at": "2026-10-12T17:52:40.551827+00:00"
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	No field to change was sent, or a goal or competency is not this agent's.
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Delete a test scenario

DELETE

/v1/test-scenarios/:id

Deletes the scenario and its simulated customers. The runs it ran stay readable with GET /v1/test-runs/:id, without their scenario.

Scope: testing:writeCan wait for approvalWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Example request

curl

curl -X DELETE "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY"

The scenario was deleted.

200

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "deleted": true
  }
}

Errors

Status	Code	When
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Run a test scenario

POST

/v1/test-scenarios/:id/runs

Starts a test run of the scenario: every simulated customer holds a conversation with the agent version (by default the deployed one), or with each of up to four versions to compare them on the same customers, and the evaluator scores each conversation on the goals and competencies you name (by default the scenario's). Nothing reaches a real customer and nothing is booked. A workspace's first 50 finished test conversations are free; each one after that uses credits at its plan's test conversation price, charged when it finishes. Answers the run with its conversations pending; follow it with GET /v1/test-runs/:id.

Scope: testing:writeSupports Idempotency-KeyWorkspace keys only · acts as the key's creator

A run takes a few minutes: each conversation runs and is scored in the background. Poll GET /v1/test-runs/:id until status is completed, partial or failed.

The run starts as the member who created the key: owners, admins, builders and operators run tests. A key whose creator has left the workspace, and a reseller key, are refused.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Request body (JSON)

Name	Type	Description
`agent_version_ids`	`uuid[]`	The agent versions to test, each once (`GET /v1/agents/:id` lists them). Two to four compare them: the run then names a winner. Default: the deployed version. 1–4 items.
`conversations_per_customer`	`integer`	Conversations each customer has with each version. Default: the scenario's. 1–5.
`channel_id`	`uuid`	One of the agent's messaging channels: the run applies its settings (scheduling, language). Default: the agent's oldest messaging channel.
`goal_ids`	`uuid[]`	The agent's goals to score. With `competency_ids`, at least one in all; both absent: the scenario's. 0–50 items.
`competency_ids`	`uuid[]`	The agent's competencies to score. 0–50 items.
`rerun_of`	`uuid`	An earlier run of this scenario: this run meets the same customers it met.

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000/runs" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Idempotency-Key: 11111111-2222-4333-8444-555555555555" \
  -H "Content-Type: application/json" \
  -d '{
  "agent_version_ids": [
    "2ea87934-6e03-4db7-80f2-ac753b724c82",
    "c5c0810c-4614-4f0d-927d-62e587dad45e"
  ],
  "goal_ids": [
    "7b7b8def-a340-4bfd-b164-779fa5843735"
  ]
}'

The run, started.

201

Response

{
  "success": true,
  "data": {
    "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
    "status": "running",
    "error": null,
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": {
      "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
      "name": "Price objection",
      "description": "A customer asks how much a cleaning costs and pushes back on the price."
    },
    "variants": [
      {
        "key": "A",
        "agent_version_id": "2ea87934-6e03-4db7-80f2-ac753b724c82",
        "version_label": "v1.3"
      }
    ],
    "results": [
      {
        "variant": "A",
        "conversations": 3,
        "open": 3,
        "completed": 0,
        "failed": 0,
        "passed": 0,
        "preferred": null,
        "average_turns": null
      }
    ],
    "winner": null,
    "rubric": [
      {
        "id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "kind": "goal",
        "name": "Book qualified meeting"
      },
      {
        "id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "kind": "competency",
        "name": "Objection handling"
      }
    ],
    "conversations": [
      {
        "id": "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f",
        "variant": "A",
        "customer": {
          "id": "1e2f3a4b-5c6d-4e7f-8a9b-0c1d2e3f4a5c",
          "name": "Ana"
        },
        "status": "pending",
        "error": null,
        "passed": null,
        "summary": null,
        "goals": [],
        "competencies": [],
        "transcript": []
      }
    ],
    "created_at": "2026-10-12T17:41:03.118204+00:00",
    "completed_at": null,
    "updated_at": "2026-10-12T17:41:03.118204+00:00"
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	The scenario is a `calls` scenario, has no customer, would run more than 100 conversations or has nothing to score; or a goal, competency or channel is not the agent's.
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).
404	not_found	An agent version is not the agent's, or `rerun_of` is not a run of this scenario.
409	conflict	The agent is archived, so it can't be tested (`details.reason` is `AGENT_ARCHIVED`).
409	conflict	`agent_version_ids` is absent and the agent has no deployed version.
500	internal_error	The run could not start; it is kept as failed with the reason (`details.test_run_id`).

Plus the errors every request can return (401, 403, 429, 500): Errors.

List an agent's test runs

GET

/v1/agents/:id/test-runs

Lists the agent's test runs, newest first, with each run's status and how many of its conversations are open, finished, failed and passed.

Scope: testing:read

status: running while a conversation is open; completed when all finished, partial when some failed, failed when every one failed or the run could not start (error says why).

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Query parameters

Name	Type	Description
`limit`	`integer`	Results per page. 1–100. Default: `20`
`offset`	`integer`	Results to skip. At least 0. Default: `0`
`scenario_id`	`uuid`	Only the runs of this scenario.
`status`	`string`	Only runs in this status. One of: pending, running, completed, partial, failed, cancelled.

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-runs?status=completed&limit=10" \
  -H "Authorization: Bearer YOUR_API_KEY"

A page of runs.

200

Response

{
  "success": true,
  "data": [
    {
      "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
      "name": "Price objection - Oct 12, 2026, 11:41 AM",
      "status": "completed",
      "scenario": {
        "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
        "name": "Price objection"
      },
      "variants": [
        {
          "key": "A",
          "agent_version_id": "2ea87934-6e03-4db7-80f2-ac753b724c82",
          "version_label": "v1.3"
        }
      ],
      "conversations": {
        "total": 3,
        "open": 0,
        "completed": 3,
        "failed": 0,
        "passed": 2
      },
      "error": null,
      "created_at": "2026-10-12T17:41:03.118204+00:00",
      "completed_at": "2026-10-12T17:43:52.007311+00:00"
    }
  ],
  "pagination": {
    "total": 1,
    "limit": 10,
    "offset": 0,
    "has_more": false
  }
}

Errors

Status	Code	When
400	validation_error	A query parameter is invalid (`details.errors` lists each one).
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Get a test run

GET

/v1/test-runs/:id

Returns one test run as Automated testing shows it: each version's results, the winner of a comparison (the most conversations passed, the evaluator's head-to-head breaking a tie), the goals and competencies scored, and every simulated conversation with its transcript, the evaluator's summary and its verdict on each goal and competency.

Scope: testing:read

A conversation's status is pending, running or evaluating while it is open, then completed or failed (error says why). passed is null until it is scored.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test run id.

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/test-runs/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY"

The run.

200

Response

{
  "success": true,
  "data": {
    "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
    "status": "completed",
    "error": null,
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": {
      "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
      "name": "Price objection",
      "description": "A customer asks how much a cleaning costs and pushes back on the price."
    },
    "variants": [
      {
        "key": "A",
        "agent_version_id": "2ea87934-6e03-4db7-80f2-ac753b724c82",
        "version_label": "v1.3"
      }
    ],
    "results": [
      {
        "variant": "A",
        "conversations": 3,
        "open": 0,
        "completed": 3,
        "failed": 0,
        "passed": 2,
        "preferred": null,
        "average_turns": 5.3
      }
    ],
    "winner": null,
    "rubric": [
      {
        "id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "kind": "goal",
        "name": "Book qualified meeting"
      },
      {
        "id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "kind": "competency",
        "name": "Objection handling"
      }
    ],
    "conversations": [
      {
        "id": "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f",
        "variant": "A",
        "customer": {
          "id": "1e2f3a4b-5c6d-4e7f-8a9b-0c1d2e3f4a5c",
          "name": "Ana"
        },
        "status": "completed",
        "error": null,
        "passed": true,
        "summary": "The agent gave the price, answered the objection with the included check-up and booked Thursday at 10:00.",
        "goals": [
          {
            "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
            "title": "Book qualified meeting",
            "met": true,
            "justification": "The customer accepted Thursday at 10:00."
          }
        ],
        "competencies": [
          {
            "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
            "name": "Objection handling",
            "level": "proficient",
            "passed": true,
            "justification": "It acknowledged the price concern and explained what the cleaning includes."
          }
        ],
        "transcript": [
          {
            "role": "customer",
            "text": "Hola, ¿cuánto cuesta una limpieza dental?",
            "tools": []
          },
          {
            "role": "agent",
            "text": "¡Hola! La limpieza cuesta 750 MXN e incluye revisión. ¿Te gustaría agendar esta semana?",
            "tools": []
          }
        ]
      }
    ],
    "created_at": "2026-10-12T17:41:03.118204+00:00",
    "completed_at": "2026-10-12T17:43:52.007311+00:00",
    "updated_at": "2026-10-12T17:43:52.007311+00:00"
  }
}

Errors

Status	Code	When
404	not_found	No test run with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Score a test run again

POST

/v1/test-runs/:id/evaluate

Scores the run's finished conversations again (all of them, or the ones you name) with the goals and competencies as they are now, for example after rewording a goal. Each conversation goes back to evaluating; its previous scores are replaced once the new scoring is queued, and kept when it can't be.

Scope: testing:writeCan wait for approvalWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test run id.

Request body (JSON)

Name	Type	Description
`conversation_ids`	`uuid[]`	Only these conversations of the run. Default: every finished one. 1–100 items.

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/test-runs/550e8400-e29b-41d4-a716-446655440000/evaluate" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "conversation_ids": [
    "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f"
  ]
}'

The conversations being scored again; skipped lists the ones that couldn't be, with the reason.

200

Response

{
  "success": true,
  "data": {
    "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
    "rescoring": [
      "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f"
    ],
    "skipped": []
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test run with this id in your workspace (ids of other workspaces answer the same).
404	not_found	A conversation is not part of this run (`details.conversation_ids`).
409	conflict	The run has no finished conversation, or none could be scored again (`details.skipped` says why for each).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Try the agent in a test chat

POST

/v1/agents/:id/test-chat

One turn of Try it yourself: you write as a customer and the agent version answers through its real engine (its prompt, knowledge, tools and model), with the tools it used. Nothing is sent or booked and no credit is used; each reply counts against the workspace's free test replies of the day. Send the chat_id of an answer to continue that chat.

Scope: testing:writeWorkspace keys only · acts as the key's creator

A reply can take up to 90 seconds when the agent uses tools. Tools that would change something (a booking) are simulated (simulated: true).

The turn runs as the member who created the key (owners, admins and builders try agents); the chat appears in the dashboard's Try it yourself history.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Request body (JSON)

Name	Type	Description
`message` Required	`string`	What the customer writes. 1–4096 characters.
`agent_version_id`	`uuid`	The version that answers, any version of the agent. Default: the chat's, else the deployed one.
`compare_agent_version_id`	`uuid \| null`	Another version that answers the same messages, side by side (`reply.compare`).
`chat_id`	`uuid`	The `chat_id` of an earlier answer: the turn continues that chat, with its versions.
`channel_id`	`uuid \| null`	One of the agent's channels whose settings apply (scheduling, language). Default: chosen for you; a chat keeps its channel.

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-chat" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "message": "Hola, ¿tienen citas el jueves?"
}'

The agent's reply.

200

Response

{
  "success": true,
  "data": {
    "chat_id": "2b3c4d5e-6f70-4a8b-9c0d-1e2f3a4b5c6d",
    "channel_id": "9f19ef32-eaef-452c-bebc-14d9759cdb64",
    "message": {
      "id": "3c4d5e6f-7a8b-4c9d-8e0f-1a2b3c4d5e6f",
      "text": "Hola, ¿tienen citas el jueves?",
      "at": "2026-10-12T18:02:11.204+00:00"
    },
    "reply": {
      "id": "5e6f7a8b-9c0d-4e1f-8a2b-3c4d5e6f7a8b",
      "text": "¡Claro! El jueves tengo libre a las 10:00 y a las 16:30. ¿Cuál prefieres?",
      "at": "2026-10-12T18:02:14.871+00:00",
      "model": null,
      "latency_ms": 3612,
      "tools": [
        {
          "tool": "check_availability",
          "simulated": false,
          "summary": "2 free times on Thursday"
        }
      ],
      "compare": null
    },
    "replies_left_today": 287
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	The engine refused the turn (a clinic agent with no messaging channel to take settings from: `details.reason` `ENGINE_REFUSED`).
403	forbidden	Automated testing isn't available in this workspace, or the member who created the key can't test (owners, admins and builders).
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).
404	not_found	The version, the chat or the channel is not the agent's.
409	conflict	The agent is archived, so it can't be tested (`details.reason` is `AGENT_ARCHIVED`).
409	conflict	The chat tested other versions, is full (160 messages) or runs on another channel (`details.reason`), or the agent has no deployed version to default to.
429	rate_limited	The workspace used today's free test replies, or sent too many in a minute (`details.reason`: `TEST_REPLIES_LIMIT` or `TEST_REPLIES_BURST`).
503	service_unavailable	The agent's engine did not answer (`details.reason` `ENGINE_FAILED`), or test replies can't be counted right now.

Plus the errors every request can return (401, 403, 429, 500): Errors.

PreviousAgenda NextKnowledge API

API Reference

Automated testing API

Test an agent with simulated customers, compare its versions and read how each conversation scored.

Changes run as the key's creator

Endpoints

List an agent's test scenarios

GET

/v1/agents/:id/test-scenarios

Lists the agent's test scenarios, newest first. A scenario is a situation and the simulated customers who bring it; its goals and competencies are what a run scores when it names none.

Scope: testing:read

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Query parameters

Name Type Description

limit

integer

Results per page. 1–100.

Default: 20

offset

integer

Results to skip. At least 0.

Default: 0

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-scenarios?limit=10" \
  -H "Authorization: Bearer YOUR_API_KEY"

A page of scenarios.

200

Response

{
  "success": true,
  "data": [
    {
      "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
      "agent_id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Price objection",
      "description": "A customer asks how much a cleaning costs and pushes back on the price.",
      "channel": "messages",
      "max_turns": 12,
      "conversations_per_customer": 1,
      "pass_threshold": 70,
      "required_keywords": [],
      "customers": [
        {
          "name": "Ana",
          "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
          "personality": "Direct, a little impatient",
          "background": "Has not been to a dentist in two years.",
          "language": "es-MX",
          "objections": [
            "It is cheaper across the street."
          ],
          "pains": [
            "Sensitive gums"
          ],
          "gains": [
            "An appointment this week"
          ],
          "demographics": {}
        }
      ],
      "goals": [
        {
          "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
          "weight": 1
        }
      ],
      "competencies": [
        {
          "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
          "weight": 1
        }
      ],
      "created_at": "2026-10-12T17:40:11.402911+00:00",
      "updated_at": "2026-10-12T17:40:11.402911+00:00"
    }
  ],
  "pagination": {
    "total": 1,
    "limit": 10,
    "offset": 0,
    "has_more": false
  }
}

Errors

Status	Code	When
400	validation_error	A query parameter is invalid (`details.errors` lists each one).
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Create a test scenario

POST

/v1/agents/:id/test-scenarios

Scope: testing:writeWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Request body (JSON)

Name	Type	Description
`name` Required	`string`	The scenario's name. 1–200 characters.
`description`	`string \| null`	The situation the customers bring. Up to 5000 characters.
`customers`	`object[]`	The simulated customers who bring the situation. A run needs at least one. 0–50 items.
`customers[].name` Required	`string`	What the customer is called. Up to 200 characters.
`customers[].goal` Required	`string`	What they want from the conversation: the simulated customer pursues it. Up to 2000 characters.
`customers[].personality`	`string`	How they talk, e.g. `direct, a little impatient`. Up to 200 characters.
`customers[].background`	`string \| null`	Who they are and their situation. Up to 4000 characters.
`customers[].language`	`string \| null`	The language they write in (`es-MX`); default the agent's. Up to 35 characters.
`customers[].objections`	`string[]`	What they push back with. 0–20 items.
`customers[].pains`	`string[]`	Their problems. 0–20 items.
`customers[].gains`	`string[]`	What would win them. 0–20 items.
`customers[].demographics`	`object`	Any other facts about them, kept as sent.
`max_turns`	`integer`	The most turns a conversation takes (default 15). 1–30.
`conversations_per_customer`	`integer`	Conversations each customer has with each version in a run. 1–5. Default: `1`
`channel`	`string`	Where the conversations happen. Calls can't be tested yet: a run of a `calls` scenario is refused. One of: messages, calls. Default: `messages`
`pass_threshold`	`number`	The overall score a conversation needs to pass. 1–100. Default: `70`
`required_keywords`	`string[]`	Words the agent must say. 0–50 items.
`goals`	`object[]`	The agent's goals a run scores when it names none. 0–50 items.
`goals[].goal_id` Required	`uuid`	A goal of the agent (`GET /v1/goals?agent_id=`).
`goals[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`
`competencies`	`object[]`	The agent's competencies a run scores when it names none. 0–50 items.
`competencies[].competency_id` Required	`uuid`	A competency of the agent.
`competencies[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-scenarios" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Price objection",
  "description": "A customer asks how much a cleaning costs and pushes back on the price.",
  "max_turns": 12,
  "customers": [
    {
      "name": "Ana",
      "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
      "personality": "Direct, a little impatient",
      "language": "es-MX",
      "objections": [
        "It is cheaper across the street."
      ]
    }
  ],
  "goals": [
    {
      "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735"
    }
  ],
  "competencies": [
    {
      "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee"
    }
  ]
}'

The scenario.

201

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Price objection",
    "description": "A customer asks how much a cleaning costs and pushes back on the price.",
    "channel": "messages",
    "max_turns": 12,
    "conversations_per_customer": 1,
    "pass_threshold": 70,
    "required_keywords": [],
    "customers": [
      {
        "name": "Ana",
        "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
        "personality": "Direct, a little impatient",
        "background": "Has not been to a dentist in two years.",
        "language": "es-MX",
        "objections": [
          "It is cheaper across the street."
        ],
        "pains": [
          "Sensitive gums"
        ],
        "gains": [
          "An appointment this week"
        ],
        "demographics": {}
      }
    ],
    "goals": [
      {
        "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "weight": 1
      }
    ],
    "competencies": [
      {
        "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "weight": 1
      }
    ],
    "created_at": "2026-10-12T17:40:11.402911+00:00",
    "updated_at": "2026-10-12T17:40:11.402911+00:00"
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	A goal or competency is not this agent's.
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).
409	conflict	The agent is archived, so it can't be tested (`details.reason` is `AGENT_ARCHIVED`).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Get a test scenario

GET

/v1/test-scenarios/:id

Returns one test scenario with its simulated customers and its goal and competency links.

Scope: testing:read

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY"

The scenario.

200

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Price objection",
    "description": "A customer asks how much a cleaning costs and pushes back on the price.",
    "channel": "messages",
    "max_turns": 12,
    "conversations_per_customer": 1,
    "pass_threshold": 70,
    "required_keywords": [],
    "customers": [
      {
        "name": "Ana",
        "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
        "personality": "Direct, a little impatient",
        "background": "Has not been to a dentist in two years.",
        "language": "es-MX",
        "objections": [
          "It is cheaper across the street."
        ],
        "pains": [
          "Sensitive gums"
        ],
        "gains": [
          "An appointment this week"
        ],
        "demographics": {}
      }
    ],
    "goals": [
      {
        "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "weight": 1
      }
    ],
    "competencies": [
      {
        "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "weight": 1
      }
    ],
    "created_at": "2026-10-12T17:40:11.402911+00:00",
    "updated_at": "2026-10-12T17:40:11.402911+00:00"
  }
}

Errors

Status	Code	When
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Update a test scenario

PATCH

/v1/test-scenarios/:id

Scope: testing:writeCan wait for approvalWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Request body (JSON)

Name	Type	Description
`name`	`string`	The scenario's name. 1–200 characters.
`description`	`string \| null`	The situation the customers bring. Up to 5000 characters.
`customers`	`object[]`	The simulated customers (replaces the whole list: send every customer to keep). 0–50 items.
`customers[].name` Required	`string`	What the customer is called. Up to 200 characters.
`customers[].goal` Required	`string`	What they want from the conversation: the simulated customer pursues it. Up to 2000 characters.
`customers[].personality`	`string`	How they talk, e.g. `direct, a little impatient`. Up to 200 characters.
`customers[].background`	`string \| null`	Who they are and their situation. Up to 4000 characters.
`customers[].language`	`string \| null`	The language they write in (`es-MX`); default the agent's. Up to 35 characters.
`customers[].objections`	`string[]`	What they push back with. 0–20 items.
`customers[].pains`	`string[]`	Their problems. 0–20 items.
`customers[].gains`	`string[]`	What would win them. 0–20 items.
`customers[].demographics`	`object`	Any other facts about them, kept as sent.
`max_turns`	`integer`	The most turns a conversation takes. 1–30.
`conversations_per_customer`	`integer`	Conversations each customer has with each version in a run. 1–5.
`channel`	`string`	Where the conversations happen. Calls can't be tested yet: a run of a `calls` scenario is refused. One of: messages, calls.
`pass_threshold`	`number`	The overall score a conversation needs to pass. 1–100.
`required_keywords`	`string[]`	Words the agent must say. 0–50 items.
`goals`	`object[]`	The agent's goals a run scores when it names none (replaces the list). 0–50 items.
`goals[].goal_id` Required	`uuid`	A goal of the agent (`GET /v1/goals?agent_id=`).
`goals[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`
`competencies`	`object[]`	The agent's competencies a run scores when it names none (replaces the list). 0–50 items.
`competencies[].competency_id` Required	`uuid`	A competency of the agent.
`competencies[].weight`	`number`	How much the item counts, above 0 and at most 10. 0–10. Default: `1`

Example request

curl

curl -X PATCH "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "max_turns": 15,
  "conversations_per_customer": 2
}'

The updated scenario.

200

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Price objection",
    "description": "A customer asks how much a cleaning costs and pushes back on the price.",
    "channel": "messages",
    "max_turns": 15,
    "conversations_per_customer": 2,
    "pass_threshold": 70,
    "required_keywords": [],
    "customers": [
      {
        "name": "Ana",
        "goal": "Find out the price of a dental cleaning and book only if it is under 800 MXN.",
        "personality": "Direct, a little impatient",
        "background": "Has not been to a dentist in two years.",
        "language": "es-MX",
        "objections": [
          "It is cheaper across the street."
        ],
        "pains": [
          "Sensitive gums"
        ],
        "gains": [
          "An appointment this week"
        ],
        "demographics": {}
      }
    ],
    "goals": [
      {
        "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "weight": 1
      }
    ],
    "competencies": [
      {
        "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "weight": 1
      }
    ],
    "created_at": "2026-10-12T17:40:11.402911+00:00",
    "updated_at": "2026-10-12T17:52:40.551827+00:00"
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	No field to change was sent, or a goal or competency is not this agent's.
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Delete a test scenario

DELETE

/v1/test-scenarios/:id

Deletes the scenario and its simulated customers. The runs it ran stay readable with GET /v1/test-runs/:id, without their scenario.

Scope: testing:writeCan wait for approvalWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Example request

curl

curl -X DELETE "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY"

The scenario was deleted.

200

Response

{
  "success": true,
  "data": {
    "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
    "deleted": true
  }
}

Errors

Status	Code	When
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Run a test scenario

POST

/v1/test-scenarios/:id/runs

Scope: testing:writeSupports Idempotency-KeyWorkspace keys only · acts as the key's creator

A run takes a few minutes: each conversation runs and is scored in the background. Poll GET /v1/test-runs/:id until status is completed, partial or failed.

The run starts as the member who created the key: owners, admins, builders and operators run tests. A key whose creator has left the workspace, and a reseller key, are refused.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test scenario id.

Request body (JSON)

Name	Type	Description
`agent_version_ids`	`uuid[]`	The agent versions to test, each once (`GET /v1/agents/:id` lists them). Two to four compare them: the run then names a winner. Default: the deployed version. 1–4 items.
`conversations_per_customer`	`integer`	Conversations each customer has with each version. Default: the scenario's. 1–5.
`channel_id`	`uuid`	One of the agent's messaging channels: the run applies its settings (scheduling, language). Default: the agent's oldest messaging channel.
`goal_ids`	`uuid[]`	The agent's goals to score. With `competency_ids`, at least one in all; both absent: the scenario's. 0–50 items.
`competency_ids`	`uuid[]`	The agent's competencies to score. 0–50 items.
`rerun_of`	`uuid`	An earlier run of this scenario: this run meets the same customers it met.

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/test-scenarios/550e8400-e29b-41d4-a716-446655440000/runs" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Idempotency-Key: 11111111-2222-4333-8444-555555555555" \
  -H "Content-Type: application/json" \
  -d '{
  "agent_version_ids": [
    "2ea87934-6e03-4db7-80f2-ac753b724c82",
    "c5c0810c-4614-4f0d-927d-62e587dad45e"
  ],
  "goal_ids": [
    "7b7b8def-a340-4bfd-b164-779fa5843735"
  ]
}'

The run, started.

201

Response

{
  "success": true,
  "data": {
    "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
    "status": "running",
    "error": null,
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": {
      "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
      "name": "Price objection",
      "description": "A customer asks how much a cleaning costs and pushes back on the price."
    },
    "variants": [
      {
        "key": "A",
        "agent_version_id": "2ea87934-6e03-4db7-80f2-ac753b724c82",
        "version_label": "v1.3"
      }
    ],
    "results": [
      {
        "variant": "A",
        "conversations": 3,
        "open": 3,
        "completed": 0,
        "failed": 0,
        "passed": 0,
        "preferred": null,
        "average_turns": null
      }
    ],
    "winner": null,
    "rubric": [
      {
        "id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "kind": "goal",
        "name": "Book qualified meeting"
      },
      {
        "id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "kind": "competency",
        "name": "Objection handling"
      }
    ],
    "conversations": [
      {
        "id": "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f",
        "variant": "A",
        "customer": {
          "id": "1e2f3a4b-5c6d-4e7f-8a9b-0c1d2e3f4a5c",
          "name": "Ana"
        },
        "status": "pending",
        "error": null,
        "passed": null,
        "summary": null,
        "goals": [],
        "competencies": [],
        "transcript": []
      }
    ],
    "created_at": "2026-10-12T17:41:03.118204+00:00",
    "completed_at": null,
    "updated_at": "2026-10-12T17:41:03.118204+00:00"
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	The scenario is a `calls` scenario, has no customer, would run more than 100 conversations or has nothing to score; or a goal, competency or channel is not the agent's.
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test scenario with this id in your workspace (ids of other workspaces answer the same).
404	not_found	An agent version is not the agent's, or `rerun_of` is not a run of this scenario.
409	conflict	The agent is archived, so it can't be tested (`details.reason` is `AGENT_ARCHIVED`).
409	conflict	`agent_version_ids` is absent and the agent has no deployed version.
500	internal_error	The run could not start; it is kept as failed with the reason (`details.test_run_id`).

Plus the errors every request can return (401, 403, 429, 500): Errors.

List an agent's test runs

GET

/v1/agents/:id/test-runs

Lists the agent's test runs, newest first, with each run's status and how many of its conversations are open, finished, failed and passed.

Scope: testing:read

status: running while a conversation is open; completed when all finished, partial when some failed, failed when every one failed or the run could not start (error says why).

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Query parameters

Name	Type	Description
`limit`	`integer`	Results per page. 1–100. Default: `20`
`offset`	`integer`	Results to skip. At least 0. Default: `0`
`scenario_id`	`uuid`	Only the runs of this scenario.
`status`	`string`	Only runs in this status. One of: pending, running, completed, partial, failed, cancelled.

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-runs?status=completed&limit=10" \
  -H "Authorization: Bearer YOUR_API_KEY"

A page of runs.

200

Response

{
  "success": true,
  "data": [
    {
      "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
      "name": "Price objection - Oct 12, 2026, 11:41 AM",
      "status": "completed",
      "scenario": {
        "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
        "name": "Price objection"
      },
      "variants": [
        {
          "key": "A",
          "agent_version_id": "2ea87934-6e03-4db7-80f2-ac753b724c82",
          "version_label": "v1.3"
        }
      ],
      "conversations": {
        "total": 3,
        "open": 0,
        "completed": 3,
        "failed": 0,
        "passed": 2
      },
      "error": null,
      "created_at": "2026-10-12T17:41:03.118204+00:00",
      "completed_at": "2026-10-12T17:43:52.007311+00:00"
    }
  ],
  "pagination": {
    "total": 1,
    "limit": 10,
    "offset": 0,
    "has_more": false
  }
}

Errors

Status	Code	When
400	validation_error	A query parameter is invalid (`details.errors` lists each one).
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Get a test run

GET

/v1/test-runs/:id

Scope: testing:read

A conversation's status is pending, running or evaluating while it is open, then completed or failed (error says why). passed is null until it is scored.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test run id.

Example request

curl

curl -X GET "https://www.nutalk.ai/v1/test-runs/550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer YOUR_API_KEY"

The run.

200

Response

{
  "success": true,
  "data": {
    "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
    "status": "completed",
    "error": null,
    "agent_id": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": {
      "id": "4f1c2b3a-5d6e-4f70-8a9b-0c1d2e3f4a5b",
      "name": "Price objection",
      "description": "A customer asks how much a cleaning costs and pushes back on the price."
    },
    "variants": [
      {
        "key": "A",
        "agent_version_id": "2ea87934-6e03-4db7-80f2-ac753b724c82",
        "version_label": "v1.3"
      }
    ],
    "results": [
      {
        "variant": "A",
        "conversations": 3,
        "open": 0,
        "completed": 3,
        "failed": 0,
        "passed": 2,
        "preferred": null,
        "average_turns": 5.3
      }
    ],
    "winner": null,
    "rubric": [
      {
        "id": "7b7b8def-a340-4bfd-b164-779fa5843735",
        "kind": "goal",
        "name": "Book qualified meeting"
      },
      {
        "id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
        "kind": "competency",
        "name": "Objection handling"
      }
    ],
    "conversations": [
      {
        "id": "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f",
        "variant": "A",
        "customer": {
          "id": "1e2f3a4b-5c6d-4e7f-8a9b-0c1d2e3f4a5c",
          "name": "Ana"
        },
        "status": "completed",
        "error": null,
        "passed": true,
        "summary": "The agent gave the price, answered the objection with the included check-up and booked Thursday at 10:00.",
        "goals": [
          {
            "goal_id": "7b7b8def-a340-4bfd-b164-779fa5843735",
            "title": "Book qualified meeting",
            "met": true,
            "justification": "The customer accepted Thursday at 10:00."
          }
        ],
        "competencies": [
          {
            "competency_id": "b97659d1-b147-4831-86d0-3e36b6bd3fee",
            "name": "Objection handling",
            "level": "proficient",
            "passed": true,
            "justification": "It acknowledged the price concern and explained what the cleaning includes."
          }
        ],
        "transcript": [
          {
            "role": "customer",
            "text": "Hola, ¿cuánto cuesta una limpieza dental?",
            "tools": []
          },
          {
            "role": "agent",
            "text": "¡Hola! La limpieza cuesta 750 MXN e incluye revisión. ¿Te gustaría agendar esta semana?",
            "tools": []
          }
        ]
      }
    ],
    "created_at": "2026-10-12T17:41:03.118204+00:00",
    "completed_at": "2026-10-12T17:43:52.007311+00:00",
    "updated_at": "2026-10-12T17:43:52.007311+00:00"
  }
}

Errors

Status	Code	When
404	not_found	No test run with this id in your workspace (ids of other workspaces answer the same).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Score a test run again

POST

/v1/test-runs/:id/evaluate

Scope: testing:writeCan wait for approvalWorkspace keys only · acts as the key's creator

Path parameters

Name	Type	Description
`id` Required	`uuid`	The test run id.

Request body (JSON)

Name	Type	Description
`conversation_ids`	`uuid[]`	Only these conversations of the run. Default: every finished one. 1–100 items.

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/test-runs/550e8400-e29b-41d4-a716-446655440000/evaluate" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "conversation_ids": [
    "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f"
  ]
}'

The conversations being scored again; skipped lists the ones that couldn't be, with the reason.

200

Response

{
  "success": true,
  "data": {
    "id": "6a7b8c9d-0e1f-4a2b-9c3d-4e5f6a7b8c9d",
    "rescoring": [
      "8c9d0e1f-2a3b-4c4d-8e5f-6a7b8c9d0e1f"
    ],
    "skipped": []
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
403	forbidden	The member who created the key has a role that can't do this or has left the workspace, or the key is a reseller key.
404	not_found	No test run with this id in your workspace (ids of other workspaces answer the same).
404	not_found	A conversation is not part of this run (`details.conversation_ids`).
409	conflict	The run has no finished conversation, or none could be scored again (`details.skipped` says why for each).

Plus the errors every request can return (401, 403, 429, 500): Errors.

Try the agent in a test chat

POST

/v1/agents/:id/test-chat

Scope: testing:writeWorkspace keys only · acts as the key's creator

A reply can take up to 90 seconds when the agent uses tools. Tools that would change something (a booking) are simulated (simulated: true).

The turn runs as the member who created the key (owners, admins and builders try agents); the chat appears in the dashboard's Try it yourself history.

Path parameters

Name	Type	Description
`id` Required	`uuid`	The agent id.

Request body (JSON)

Name	Type	Description
`message` Required	`string`	What the customer writes. 1–4096 characters.
`agent_version_id`	`uuid`	The version that answers, any version of the agent. Default: the chat's, else the deployed one.
`compare_agent_version_id`	`uuid \| null`	Another version that answers the same messages, side by side (`reply.compare`).
`chat_id`	`uuid`	The `chat_id` of an earlier answer: the turn continues that chat, with its versions.
`channel_id`	`uuid \| null`	One of the agent's channels whose settings apply (scheduling, language). Default: chosen for you; a chat keeps its channel.

Example request

curl

curl -X POST "https://www.nutalk.ai/v1/agents/550e8400-e29b-41d4-a716-446655440000/test-chat" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "message": "Hola, ¿tienen citas el jueves?"
}'

The agent's reply.

200

Response

{
  "success": true,
  "data": {
    "chat_id": "2b3c4d5e-6f70-4a8b-9c0d-1e2f3a4b5c6d",
    "channel_id": "9f19ef32-eaef-452c-bebc-14d9759cdb64",
    "message": {
      "id": "3c4d5e6f-7a8b-4c9d-8e0f-1a2b3c4d5e6f",
      "text": "Hola, ¿tienen citas el jueves?",
      "at": "2026-10-12T18:02:11.204+00:00"
    },
    "reply": {
      "id": "5e6f7a8b-9c0d-4e1f-8a2b-3c4d5e6f7a8b",
      "text": "¡Claro! El jueves tengo libre a las 10:00 y a las 16:30. ¿Cuál prefieres?",
      "at": "2026-10-12T18:02:14.871+00:00",
      "model": null,
      "latency_ms": 3612,
      "tools": [
        {
          "tool": "check_availability",
          "simulated": false,
          "summary": "2 free times on Thursday"
        }
      ],
      "compare": null
    },
    "replies_left_today": 287
  }
}

Errors

Status	Code	When
400	validation_error	The body is not JSON, or a field is missing or invalid (`details.errors` lists each field).
400	validation_error	The engine refused the turn (a clinic agent with no messaging channel to take settings from: `details.reason` `ENGINE_REFUSED`).
403	forbidden	Automated testing isn't available in this workspace, or the member who created the key can't test (owners, admins and builders).
404	not_found	No agent with this id in your workspace (ids of other workspaces answer the same).
404	not_found	The version, the chat or the channel is not the agent's.
409	conflict	The agent is archived, so it can't be tested (`details.reason` is `AGENT_ARCHIVED`).
409	conflict	The chat tested other versions, is full (160 messages) or runs on another channel (`details.reason`), or the agent has no deployed version to default to.
429	rate_limited	The workspace used today's free test replies, or sent too many in a minute (`details.reason`: `TEST_REPLIES_LIMIT` or `TEST_REPLIES_BURST`).
503	service_unavailable	The agent's engine did not answer (`details.reason` `ENGINE_FAILED`), or test replies can't be counted right now.

Plus the errors every request can return (401, 403, 429, 500): Errors.

PreviousAgenda NextKnowledge API