Get the failed checks of an eval run and the calls to read for them

curl --request GET \
  --url https://voice-api.onclarity.com/v1/projects/{projectId}/agents/{agentId}/eval-submissions/{id}/improve-candidates \
  --header 'Authorization: Bearer <token>'

import requests

url = "https://voice-api.onclarity.com/v1/projects/{projectId}/agents/{agentId}/eval-submissions/{id}/improve-candidates"

headers = {"Authorization": "Bearer <token>"}

response = requests.get(url, headers=headers)

print(response.text)

const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

fetch('https://voice-api.onclarity.com/v1/projects/{projectId}/agents/{agentId}/eval-submissions/{id}/improve-candidates', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "submission_id": "sub_01J9",
  "submission_name": "Booking regression 2026-10-22 14:00",
  "state": "completed",
  "n": 20,
  "verdicts_before": null,
  "tested_version": {
    "id": "ver_789",
    "number": 12,
    "pinned_draft": false
  },
  "live_version": {
    "id": "ver_789",
    "number": 12
  },
  "draft_version": null,
  "draft_changed_since_run": null,
  "failed_checks": [
    {
      "test_case_id": "tc_01",
      "name": "Quotes the trip total",
      "source": "scenario",
      "scenario_names": [
        "Roaming quote"
      ],
      "failed": 2,
      "passed": 0,
      "applicable": 2,
      "failed_call_ids": [
        "call_01",
        "call_02"
      ]
    }
  ],
  "calls_with_failures": 2,
  "usable_calls": 3,
  "calls": [
    {
      "call_id": "call_01",
      "conversation_id": "conv_01",
      "scenario_id": "sc_01",
      "scenario_name": "Roaming quote",
      "silence_mode": "off",
      "run_index": 0,
      "failed": 1,
      "failed_test_case_ids": [
        "tc_01"
      ],
      "picked_for": "coverage"
    },
    {
      "call_id": "call_02",
      "conversation_id": "conv_02",
      "scenario_id": "sc_01",
      "scenario_name": "Roaming quote",
      "silence_mode": "off",
      "run_index": 1,
      "failed": 1,
      "failed_test_case_ids": [
        "tc_01"
      ],
      "picked_for": "most_failures"
    }
  ],
  "coverage": {
    "covered": 1,
    "total": 1
  },
  "uncovered_test_case_ids": [],
  "unusable_calls": [
    {
      "call_id": "call_04",
      "conversation_id": null,
      "scenario_id": "sc_02",
      "scenario_name": "Late payment",
      "reason": "not_connected",
      "hangup_reason": "eval_tester_never_joined"
    }
  ],
  "excluded_verdicts": {
    "error": 0,
    "not_sure": 1,
    "not_relevant": 4
  },
  "pending_calls": 0
}

{
  "error": "Invalid or revoked token"
}

{
  "error": "Eval submission not found"
}

{
  "error": "<string>",
  "code": "<string>",
  "message": "<string>",
  "path": "<string>",
  "reason": "model",
  "tier": "<string>",
  "provider": "<string>",
  "model": "<string>",
  "issues": [
    {
      "path": "<string>",
      "code": "<string>",
      "message": "<string>"
    }
  ],
  "details": [
    "<unknown>"
  ]
}

Runs

Get the failed checks of an eval run and the calls to read for them

Returns what failed in an eval run, counted by check, and picks up to n of the calls with failures to read: first one call for each failed check, then the calls with the most failures. “Improve from evals” shows these numbers and reads these calls.

Only passed and failed verdicts count. error, not_sure and not_relevant verdicts are never failures; they are counted in excluded_verdicts. Calls that did not connect, were not judged or have no conversation are not read; they are listed in unusable_calls, each with a reason code. New codes may be added: treat an unknown one like not_connected.

On a paused run verdicts keep arriving. Pass verdicts_before to count only the verdicts judged up to that moment, so two reads return the same calls.

GET

projects

{projectId}

agents

{agentId}

eval-submissions

{id}

improve-candidates

Get the failed checks of an eval run and the calls to read for them

curl --request GET \
  --url https://voice-api.onclarity.com/v1/projects/{projectId}/agents/{agentId}/eval-submissions/{id}/improve-candidates \
  --header 'Authorization: Bearer <token>'

import requests

url = "https://voice-api.onclarity.com/v1/projects/{projectId}/agents/{agentId}/eval-submissions/{id}/improve-candidates"

headers = {"Authorization": "Bearer <token>"}

response = requests.get(url, headers=headers)

print(response.text)

const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

fetch('https://voice-api.onclarity.com/v1/projects/{projectId}/agents/{agentId}/eval-submissions/{id}/improve-candidates', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "submission_id": "sub_01J9",
  "submission_name": "Booking regression 2026-10-22 14:00",
  "state": "completed",
  "n": 20,
  "verdicts_before": null,
  "tested_version": {
    "id": "ver_789",
    "number": 12,
    "pinned_draft": false
  },
  "live_version": {
    "id": "ver_789",
    "number": 12
  },
  "draft_version": null,
  "draft_changed_since_run": null,
  "failed_checks": [
    {
      "test_case_id": "tc_01",
      "name": "Quotes the trip total",
      "source": "scenario",
      "scenario_names": [
        "Roaming quote"
      ],
      "failed": 2,
      "passed": 0,
      "applicable": 2,
      "failed_call_ids": [
        "call_01",
        "call_02"
      ]
    }
  ],
  "calls_with_failures": 2,
  "usable_calls": 3,
  "calls": [
    {
      "call_id": "call_01",
      "conversation_id": "conv_01",
      "scenario_id": "sc_01",
      "scenario_name": "Roaming quote",
      "silence_mode": "off",
      "run_index": 0,
      "failed": 1,
      "failed_test_case_ids": [
        "tc_01"
      ],
      "picked_for": "coverage"
    },
    {
      "call_id": "call_02",
      "conversation_id": "conv_02",
      "scenario_id": "sc_01",
      "scenario_name": "Roaming quote",
      "silence_mode": "off",
      "run_index": 1,
      "failed": 1,
      "failed_test_case_ids": [
        "tc_01"
      ],
      "picked_for": "most_failures"
    }
  ],
  "coverage": {
    "covered": 1,
    "total": 1
  },
  "uncovered_test_case_ids": [],
  "unusable_calls": [
    {
      "call_id": "call_04",
      "conversation_id": null,
      "scenario_id": "sc_02",
      "scenario_name": "Late payment",
      "reason": "not_connected",
      "hangup_reason": "eval_tester_never_joined"
    }
  ],
  "excluded_verdicts": {
    "error": 0,
    "not_sure": 1,
    "not_relevant": 4
  },
  "pending_calls": 0
}

{
  "error": "Invalid or revoked token"
}

{
  "error": "Eval submission not found"
}

{
  "error": "<string>",
  "code": "<string>",
  "message": "<string>",
  "path": "<string>",
  "reason": "model",
  "tier": "<string>",
  "provider": "<string>",
  "model": "<string>",
  "issues": [
    {
      "path": "<string>",
      "code": "<string>",
      "message": "<string>"
    }
  ],
  "details": [
    "<unknown>"
  ]
}

Authorizations

Authorization

string

header

required

Your organization API key (it starts with anek_org_). The same key works on the Voice API and the Support API. See Authentication.

Path Parameters

projectId

string

required

Project id

agentId

string

required

Agent id

string

required

Eval submission id

Query Parameters

integer

default:20

How many calls with failures to pick.

Required range: 1 <= x <= 200

verdicts_before

string<date-time>

Count only verdicts judged at or before this moment (ISO 8601).

Response

The failed checks and the calls picked

submission_id

string

required

submission_name

string

required

state

enum<string>

required

The run's state, as the status endpoint reports it.

Available options:

not_started,

in_progress,

paused,

failed,

completed

integer

required

Required range: 1 <= x <= 200

verdicts_before

string<date-time> | null

required

tested_version

object

required

The version the run tested. id is null when that draft was discarded.

Show child attributes

live_version

object | null

required

The agent's published version now; null when it has none.

Show child attributes

draft_version

object | null

required

The agent's draft now; null when it has none.

Show child attributes

draft_changed_since_run

boolean | null

required

For a run on a draft, whether that draft was edited after the run; null otherwise.

failed_checks

object[]

required

Checks that failed in at least one call, most failures first.

Show child attributes

calls_with_failures

integer

required

Required range: x >= 0

usable_calls

integer

required

Calls with a conversation and at least one verdict other than error.

Required range: x >= 0

calls

object[]

required

The calls picked, in the order they were picked.

Maximum array length: 200

Show child attributes

coverage

object

required

Show child attributes

uncovered_test_case_ids

string[]

required

Failed checks that none of the picked calls fails.

unusable_calls

object[]

required

Show child attributes

excluded_verdicts

object

required

Show child attributes

pending_calls

integer

required

Calls not finished or not judged yet.

Required range: x >= 0