simforge evaluate

Reference for simforge evaluate: usage, arguments and flags, generated from the CLI.

Generated from the CLI’s own help (CLI 0.2.0-rc.5). Global flags and exit codes are on the command reference.

evaluate

The reject filters (and an optional intent rubric) over a trace; exit 2 when it rejects. With —cloud: test a model by letting it drive your saved scenarios: simforge evaluate —cloud —model MODEL —dataset NAME [—scenarios A,B] [—yes] [—wait]. It is priced first and starts only once you confirm the total (—yes when nothing can prompt); no TRACE is read

simforge evaluate [OPTIONS] <TRACE>

Runs on: local runtime, or hosted with —cloud.

Argument / flag	Notes	What
<trace>	required	The trace (.trace.json or .trace.json.gz, any released format)
--filter MODE		Reject filters: critical | negative-control | all
--trivial-ttc S		The trivially-safe TTC threshold, seconds
--reject-collisions		Reject any collision
--motion-audit-version MOTION_AUDIT_VERSION	one of declared-speed-v1, pose-motion-v2	The motion-plausibility audit to run. Default: declared-speed-v1
--rubric FILE		An intent rubric (IntentRubric JSON) evaluated alongside criticality
--blind-review-out FILE		Write the context-blind review packet here (needs —rubric)
--traffic-law MAP_DIR		Measure red-light and stop-bar compliance on this installed map directory (map.xodr, derived/odr/locations-v2.json.gz); reported as trafficLaw
--traffic-law-actor ACTOR	repeatable	An actor to measure (repeatable; default: the trace’s metric subject)
--model MODEL		The model that drives, by name (case-insensitive) or id: one of the models the host offers now (run without it to list them). Required
--dataset DATASET		Your dataset of saved scenarios, by name (case-insensitive) or id. Required
--scenarios SCENARIOS	repeatable	Only these scenarios of the dataset, by name or id (comma-separated). Default: all of them
--param NAME=VALUE	repeatable	One of the model’s parameters, as NAME=VALUE (repeatable): the options’ params of the model, typed as listed (e.g. —param seed=7). Absent ones take their defaults
--yes	one of true, false	Start at the quoted price without asking; required when no one can answer the prompt
--wait	one of true, false	Follow the run until it finishes