> ## Documentation Index > Fetch the complete documentation index at: https://docs.support.onclarity.com/llms.txt > Use this file to discover all available pages before exploring further. # Create an eval judge > Creates a judge. The prompt template must contain the placeholders the judge fills for each test case; `GET …/eval-judges/templates` returns ready-made templates to start from. ## OpenAPI ````yaml api-reference/voice-openapi.json POST /v1/projects/{projectId}/eval-judges openapi: 3.0.3 info: title: Clarity Voice API version: '1.0' description: >- Voice agents and their tools, outbound calls, conversations, campaigns, telephony, and evals. servers: - url: https://voice-api.onclarity.com security: - organizationApiKey: [] tags: - name: Projects description: >- The projects your organization owns. Every other resource lives in a project. - name: Agents description: Create, configure, version and publish voice agents. - name: Outbound calls description: Have a published agent phone a number now. - name: Conversations description: >- One record per call session, with its calls, recordings, transcripts and artifacts. - name: Calls description: The calls of a project, with their status, direction, timing and outcome. - name: Campaigns description: Outbound calling campaigns that phone a list of numbers on a schedule. - name: Eval suites description: >- Reusable sets of eval scenarios and test cases to measure an agent against. - name: Eval judges description: The LLM judges that score eval calls against your test cases. - name: Eval scenarios description: The simulated-caller scenarios of your eval suites. - name: Eval test cases description: The checks a judge applies to each eval call. - name: Eval submissions description: Run an eval suite against an agent and read the verdicts. - name: Phone numbers description: >- The phone numbers of a project, where their calls come from, and how agents call out from them. - name: Inbound SIP trunks description: Bring calls from your own SIP infrastructure to Clarity agents. - name: Outbound SIP trunks description: >- The SIP endpoints Clarity dials through for outbound calls from your numbers. - name: Tools description: The tools agents call during a conversation, and which agents use them. paths: /v1/projects/{projectId}/eval-judges: parameters: - $ref: '#/components/parameters/ProjectIdPath' post: tags: - Eval judges summary: Create an eval judge description: >- Creates a judge. The prompt template must contain the placeholders the judge fills for each test case; `GET …/eval-judges/templates` returns ready-made templates to start from. operationId: createEvalJudge requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/EvalJudgeCreateRequest' example: name: Default judge description: Full template, temperature 0 model: gemini-2.5-flash prompt_template: |- Test case: {name} When relevant: {when_relevant} Success: {success_criteria} Failure: {failure_criteria} Transcript: {conversation_text} temperature: 0 responses: '201': description: The created judge content: application/json: schema: type: object required: - judge properties: judge: $ref: '#/components/schemas/EvalJudge' example: judge: id: 2b3c4d5e-6f7a-4b8c-9d0e-1f2a3b4c5d6e project_id: 4f3a9b2c-1d2e-4f5a-8b6c-7d8e9f0a1b2c name: Default judge description: Full template, temperature 0 model: gemini-2.5-flash prompt_template: |- Test case: {name} When relevant: {when_relevant} Success: {success_criteria} Failure: {failure_criteria} Transcript: {conversation_text} temperature: 0 created_by: org-token:42 created_at: '2026-10-22T10:00:00.000Z' updated_at: '2026-10-22T10:00:00.000Z' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': description: A judge with this name already exists in the project content: application/json: schema: $ref: '#/components/schemas/Error' example: error: >- An eval judge named "Default judge" already exists in this project '422': $ref: '#/components/responses/ValidationFailed' components: parameters: ProjectIdPath: name: projectId in: path required: true schema: type: string description: Project id schemas: EvalJudgeCreateRequest: type: object required: - name - model - prompt_template properties: name: type: string minLength: 1 maxLength: 256 description: Unique within the project. description: type: string nullable: true model: type: string minLength: 1 description: >- The judge model. Must be a model the platform's judge supports (for example `gemini-2.5-flash`); anything else is refused with 422. prompt_template: type: string description: >- The judge prompt. Must contain the placeholders `{name}`, `{when_relevant}`, `{success_criteria}`, `{failure_criteria}` and `{conversation_text}`. Start from `GET …/eval-judges/templates`. temperature: type: number minimum: 0 maximum: 2 default: 0 EvalJudge: type: object description: >- An LLM judge configuration: the model, temperature and prompt template used to decide each test case of an eval call. required: - id - project_id - name - model - prompt_template - temperature properties: id: type: string project_id: type: string name: type: string maxLength: 256 description: type: string nullable: true model: type: string example: gemini-2.5-flash prompt_template: type: string temperature: type: number minimum: 0 maximum: 2 created_by: type: string nullable: true created_at: type: string format: date-time nullable: true updated_at: type: string format: date-time nullable: true Error: type: object required: - error additionalProperties: true properties: error: type: string description: Error message code: type: string description: >- A stable machine-readable reason, present on the refusals a client may branch on. Among them: `PUBLISH_CAPABILITY_REQUIRED` (403, the credential may not change what is live), `AGENT_NOT_PUBLISHED` (409, the agent has no published version yet), `AGENT_TYPE_NOT_DISPATCHABLE` (422, a Voice Assist agent named where an agent must place a call or answer one itself — a campaign, an outbound caller ID, a dispatched call, an eval; a number's inbound dispatch rule accepts one; the body carries `agent_type` and `agent_id`), `ASSIST_FLEET_NOT_CONFIGURED` (422 on a dispatch rule naming a Voice Assist agent in a project whose LiveKit has no Voice Assist listener), `NOT_APPLICABLE_TO_AGENT_TYPE` (422, a voice-only setting written on a Voice Assist agent), `AGENT_TYPE_BOUNDARY` (409), `AGENT_TYPE_NOT_SUPPORTED` (400), `INVALID_CUSTOMER_TIMEZONE` (400, `customer_timezone` is not a time zone name; the body carries the value), `VOICE_ASSISTANT_REQUIRED` (403, the organization is not licensed for Voice Assist), `ORGANIZATION_UNAVAILABLE` (503), `REGION_PROFILE_CONFLICT` (503), `LLM_SERVICE_TIER_UNAVAILABLE` (422, an agent's processing speed is not offered by its model or not enabled for the organization; `reason` says which) and `AUTH_UPSTREAM_UNAVAILABLE` (503 on any endpoint, with `Retry-After`: the credential could not be checked right now; it was not rejected and the request was not carried out). ValidationError: type: object description: >- A validation failure. `error` is a human-readable summary; `issues` (request validation) or `details` (agent configuration validation) locate each problem. A refused processing speed answers with `error` and `code` both `LLM_SERVICE_TIER_UNAVAILABLE` and the fields `path`, `reason`, `tier`, `provider` and `model`. properties: error: type: string code: type: string description: >- A machine-readable code, on the failures that carry one: `LLM_SERVICE_TIER_UNAVAILABLE` for a processing speed the agent cannot use. message: type: string path: type: string description: >- With `LLM_SERVICE_TIER_UNAVAILABLE`: the refused field, `agent_model_config.llm_service_tier`. reason: type: string enum: - model - organization description: >- With `LLM_SERVICE_TIER_UNAVAILABLE`: `model` when the language model does not offer the speed (or the value is not a speed at all), `organization` when your organization does not have it. tier: type: string description: With `LLM_SERVICE_TIER_UNAVAILABLE`, the refused value, as text. provider: type: string description: >- With `LLM_SERVICE_TIER_UNAVAILABLE`, the agent's language model provider. model: type: string description: With `LLM_SERVICE_TIER_UNAVAILABLE`, the agent's language model. issues: type: array items: type: object properties: path: type: string description: Dotted path of the offending field code: type: string message: type: string details: type: array items: {} responses: Unauthorized: description: >- No credential was sent, or the API key is invalid or revoked. A key that could not be checked at all answers 503 `AUTH_UPSTREAM_UNAVAILABLE` instead (see Errors). content: application/json: schema: $ref: '#/components/schemas/Error' example: error: Invalid or revoked token Forbidden: description: >- The credential carries no access to this project, or the operation needs administrative rights content: application/json: schema: $ref: '#/components/schemas/Error' NotFound: description: No such resource on this project content: application/json: schema: $ref: '#/components/schemas/Error' ValidationFailed: description: >- The request body or query string failed validation. `error` lists each problem prefixed with the field it concerns; `issues` carries the same information per field. content: application/json: schema: $ref: '#/components/schemas/ValidationError' example: error: 'name: String must contain at least 1 character(s)' issues: - path: name code: too_small message: String must contain at least 1 character(s) minimum: 1 securitySchemes: organizationApiKey: type: http scheme: bearer description: >- Your organization API key (it starts with `anek_org_`). The same key works on the Voice API and the Support API. See [Authentication](/authentication). ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.