simforge open-loop

Reference for simforge open-loop: usage, arguments and flags, generated from the CLI.

Generated from the CLI’s own help (CLI 0.2.0-rc.5). Global flags and exit codes are on the command reference.

open-loop artifacts

A download grant ({url, sha256, bytes, …}) for one artifact of the evaluation’s result

simforge open-loop artifacts [OPTIONS] <EVALUATION_ID> <ARTIFACT_ID>

Argument / flag	Notes	What
<evaluation_id>	required	The evaluation (run_..., or cmpj_... from before cloud runs)
<artifact_id>	required	The artifact, by its id in the result manifest

open-loop list

List the workspace’s evaluations, whichever model ran them

simforge open-loop list [OPTIONS]

Argument / flag	Notes	What
--limit LIMIT	default 20	At most this many items on the page
--cursor CURSOR		The next page: the nextCursor of the page before it
--status STATUS		Only evaluations in these statuses (comma-separated)
--model MODEL		Only one model’s evaluations, by name (case-insensitive) or id: a model the host lists (GET /api/v1/evaluation-options), or a model version of your own by id. Default: every model’s

open-loop result

The evaluation’s result manifest: scored metrics, artifacts, model/render/source pins and compute cost

simforge open-loop result [OPTIONS] <EVALUATION_ID>

Argument / flag	Notes	What
<evaluation_id>	required	The evaluation (run_..., or cmpj_... from before cloud runs)

open-loop run

Score a succeeded cloud scenario render with a driving model the host offers for open loop, named by —model (requires —cloud, —model, —max-cost and —idempotency-key, also with —dry-run). The help’s models lists the host’s open-loop models

simforge open-loop run [OPTIONS] --max-cost <CENTS> <RENDER_RUN_ID>

Argument / flag	Notes	What
<render_run_id>	required	The succeeded cloud scenario render run to evaluate
--prediction-interval-s SECONDS	default 0.5	Seconds between scored prediction instants across the whole render; an exact multiple of 0.1 (0.5 = 2 Hz)
--model MODEL		The model that predicts, by name (case-insensitive) or id: one of the models the host offers for open loop now (GET /api/v1/evaluation-options, each model’s loops.openLoop; run without it to list them). Required
--seed SEED		Model sampling seed (the model’s seed parameter). Default: the model’s
--samples N		Trajectory samples drawn by the model (its numTrajSamples parameter). Default: the model’s; a model with a fixed count accepts only that
--param NAME=VALUE	repeatable	One of the model’s parameters, as NAME=VALUE (repeatable): the options’ params of the model, typed as listed (e.g. —param temperature=0.8). Absent ones take their defaults
--max-cost CENTS	required	Explicit maximum cloud cost in cents; required, also with —dry-run
--wait		Wait for the evaluation to finish

open-loop show

One evaluation: status, cost and attempts

simforge open-loop show [OPTIONS] <EVALUATION_ID>

Argument / flag	Notes	What
<evaluation_id>	required	The evaluation (run_..., or cmpj_... from before cloud runs)

open-loop watch

Follow an evaluation until it succeeds, is partial, fails or is cancelled

simforge open-loop watch [OPTIONS] <EVALUATION_ID>

Argument / flag	Notes	What
<evaluation_id>	required	The evaluation (run_..., or cmpj_... from before cloud runs)