> ## Documentation Index > Fetch the complete documentation index at: https://docs.simforge.ai/llms.txt > Use this file to discover all available pages before exploring further. # Get an evaluation ## OpenAPI ````yaml /api-reference/openapi.json get /api/v1/evaluations/{id} openapi: 3.1.0 info: title: SimForge Cloud API version: '2026-10-24' description: >- The public SimForge API: what `simforge --cloud`, `simforge login` and the Python client call. Authenticate with a bearer token: the access token `simforge login` stores, an AI agent's access token (its recipe is `/auth.md`), or an `sfk_` API key. A `/api/v1` 401 carries `WWW-Authenticate: Bearer resource_metadata="/.well-known/oauth-protected-resource"`. `/api/v1` errors are an `ErrorEnvelope`; `/api/cli` and `/agent` errors are an OAuth error object. Versions: send `SimForge-Version: ` (2026-10-01, 2026-10-21, 2026-10-22, 2026-10-23, 2026-10-24) and every `/api/v1` response is in that version's shape and names it in the same header; this document describes 2026-10-24. A request without the header gets 2026-10-01, where an error's hint is `detail.hint` and `GET /api/v1/evaluations` names its rows `jobs`. A header that is not a date, or a date before 2026-10-01, is `400 bad_request`. Every response also carries `SimForge-Cli-Minimum`: the oldest `simforge` CLI the server serves. Retries: from 2026-10-21, send a new `Idempotency-Key` (1-255 characters) with every POST, PUT, PATCH and DELETE, and the same key when retrying that request. The first request's answer is kept for 24 hours and a repeat gets it again with `Idempotent-Replayed: true`; the key with any other request, or while the first is still running, is `409 idempotency_conflict` (`retryable` while running). A 5xx answer is not kept. Creating an API key refuses a key (its secret is shown once). license: name: Apache-2.0 identifier: Apache-2.0 servers: - url: https://dev.simforge.ai description: Development (dev.simforge.ai) security: [] tags: - name: Sign-in description: >- `simforge login`: OAuth 2.0 device and browser sign-in for the CLI, the session behind a token. AI agents sign in through auth.md (`/auth.md`): register, a person approves, then the claim and JWT-bearer grants. - name: Account description: The principal, its workspaces, API keys and usage, and system status. - name: Runs description: >- Cloud runs (`--cloud`): renders, simulations, Recast log ingestion and transfers, their events and artifacts. - name: Models description: Customer fine-tunes and their workspace-owned model weights. - name: Datasets description: >- Datasets, their files as signed links (`simforge.dataset-files/v1`, `simforge datasets pull`), and copies out of shared datasets (SIMLOG-25k). - name: Maps and vehicles description: >- Maps the workspace can use, the vehicles a scenario can bind and the asset gallery. - name: Authoring description: >- Hosted scenario authoring: drafts, transactions, validation and map discovery. - name: Distribution description: >- The runtime and map pack agreement and signed downloads (`simforge runtime install`, `simforge maps pull`). - name: Telemetry description: Runtime usage telemetry, a condition of the runtime terms. - name: Benchmark description: Renderer benchmark results and scene packs (`simforge bench`). - name: Challenge description: >- The driving challenge: team, registry login and submissions (`simforge challenge`). paths: /api/v1/evaluations/{id}: get: tags: - Runs summary: Get an evaluation operationId: getEvaluation parameters: - in: path name: id required: true schema: type: string responses: '200': description: Evaluation data. content: application/json: schema: $ref: '#/components/schemas/Evaluation' '400': description: The request failed. content: application/json: schema: $ref: '#/components/schemas/ErrorEnvelope' '401': description: The request failed. content: application/json: schema: $ref: '#/components/schemas/ErrorEnvelope' '403': description: The request failed. content: application/json: schema: $ref: '#/components/schemas/ErrorEnvelope' '404': description: The request failed. content: application/json: schema: $ref: '#/components/schemas/ErrorEnvelope' security: - simforgeToken: - runs:read components: schemas: Evaluation: type: object properties: id: type: string pattern: ^(run|cmpj)_ kind: type: string description: >- Legacy: alpamayo.openloop for the default model or a workspace alias of it, policy.openloop for every other model (before API version 2026-10-24: the lane it ran on, the default model's family lane or policy.openloop). Name the model by model.modelVersionId. idempotencyKey: type: - string - 'null' description: >- Caller-supplied job identity; qualification namespaces are excluded from evaluation suites. status: type: string enum: - queued - dispatching - dispatch_unknown - running - cancelling - succeeded - partial - failed - cancelled workspaceId: type: string description: >- The workspace it ran in; absent on an evaluation published to every workspace (`published`). published: type: boolean enum: - true description: >- An evaluation SimForge published to every workspace (`GET /api/v1/evaluations?published=true`): read-only, without its workspace, origin, submitter, estimate or cost. origin: type: string enum: - browser - desktop - cli - api description: >- Where it was launched from: Studio (browser), the desktop app, the simforge CLI (cli) or an API key (api). model: type: object properties: family: type: string description: Its model family (architecture and upstream weights). revision: type: string description: The upstream weights revision (40 hex) its family pins. quant: type: string description: Its quantization. modelVersionId: type: - string - 'null' description: >- The registered model version it ran; absent or null on an evaluation from before cloud runs that names none. name: type: string description: >- Its name, as `GET /api/v1/evaluation-options` lists an offered model. checkpointDigest: type: - string - 'null' description: >- The sha256 of the checkpoint it runs; a model job runs only beside a model host serving exactly this checkpoint. aliasOf: type: - string - 'null' description: >- The offered model this workspace's own version copies (the same family, quantization and checkpoint); null for an offered model and any other version. owner: type: string description: >- Whose model it is: `simforge` (a model SimForge offers) or `workspace` (this workspace's own version, an alias of an offered model included). An open string: a reader keeps a value it does not know. required: - family - revision - quant description: >- What it ran: the identity block (`ModelIdentity`); an evaluation from before cloud runs may lack modelVersionId, name, checkpointDigest, aliasOf and owner. params: type: object additionalProperties: {} description: >- The runner's job parameters (its instants, render source and sampling). modelParams: type: object additionalProperties: anyOf: - type: number - type: string description: >- The model's parameters it runs with, by name, defaults filled in (what params sent, else each one's default). customParams: type: boolean description: >- True when a parameter whose comparability is custom (a sampling setting, a navigation instruction) was off its default: compare it only with runs of the same params. estimateCents: type: object properties: lowCents: type: number minimum: 0 highCents: type: number minimum: 0 basis: type: string enum: - measured - unbenchmarked - per_scenario required: - lowCents - highCents - basis reservedCents: type: number settledCents: type: - number - 'null' scored: type: - boolean - 'null' result: type: - object - 'null' properties: metrics: type: object additionalProperties: {} artifacts: type: array items: type: object properties: role: type: string artifactId: type: string mediaType: type: string bytes: type: number minimum: 0 sha256: type: string pattern: ^[a-f0-9]{64}$ required: - role - artifactId - mediaType - bytes - sha256 required: - metrics - artifacts additionalProperties: {} required: - id - kind - status - model - params - scored - result additionalProperties: {} ErrorEnvelope: type: object properties: error: type: object properties: code: type: string enum: - bad_request - validation_failed - unauthorized - insufficient_scope - workspace_required - not_found - conflict - gone - idempotency_conflict - lease_lost - budget_exceeded - max_cost_required - idempotency_key_required - insufficient_credits - checkout_closed - cloud_not_open - email_unverified - terms_acceptance_required - evaluation_launch_disabled - evaluation_admissions_paused - fine_tunes_not_enabled - model_name_reserved - model_name_invalid - model_name_taken - model_quota_exceeded - checkpoint_too_large - checkpoint_format_unsupported - model_family_not_fine_tunable - checkpoint_file_unexpected - checkpoint_is_offered_model - model_version_exists - param_unknown - param_invalid - param_override_not_allowed - upload_incomplete - upload_mismatch - model_in_use - rate_limited - internal_error reason: type: string hint: type: string param: type: string retryable: type: boolean detail: {} path: type: string requestId: type: string pattern: ^req_[0-9A-HJKMNP-TV-Z]{26}$ required: - code - reason - retryable - requestId required: - error securitySchemes: simforgeToken: type: http scheme: bearer description: >- A `sfc_at_…` access token from `simforge login`, or an `sfk_…` API key. An operation's security lists the scopes the credential must hold. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.