simforge bench

Reference for simforge bench: usage, arguments and flags, generated from the CLI.

Generated from the CLI’s own help (CLI 0.2.0-rc.5). Global flags and exit codes are on the command reference.

bench

Benchmark the renderer on this machine: the Helios Score, per-factor sub-scores, frame correctness

simforge bench [OPTIONS]
       simforge bench <COMMAND>

Runs on: local runtime.

Argument / flag	Notes	What
--suite SUITE	one of quick, full; default quick	Which scenes to run
--scenes ID	repeatable	Only these scene ids (comma-separated), from the suite’s pack
--set KEY=VALUE	repeatable	Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu N		The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer’s choice (the first discrete GPU)
--out FILE		Write the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIR		A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIR		Keep every sampled frame and the session logs in this directory
--download-only		Download and verify the suite’s inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busy		Run even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDS	default 300	Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quiet		No progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIR		Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIR		Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets

bench compare

Compare two results: per-scene time deltas against the noise of both runs, and frame identity

simforge bench compare [OPTIONS] <A.json> <B.json>

Runs on: local runtime.

Argument / flag	Notes	What
<a.json>	required	The baseline result
<b.json>	required	The candidate result
--suite SUITE	one of quick, full; default quick	Which scenes to run
--scenes ID	repeatable	Only these scene ids (comma-separated), from the suite’s pack
--set KEY=VALUE	repeatable	Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu N		The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer’s choice (the first discrete GPU)
--out FILE		Write the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIR		A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIR		Keep every sampled frame and the session logs in this directory
--download-only		Download and verify the suite’s inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busy		Run even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDS	default 300	Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quiet		No progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIR		Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIR		Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets

bench run

Run the benchmark (the default when no subcommand is given)

simforge bench run [OPTIONS]

Runs on: local runtime.

Argument / flag	Notes	What
--suite SUITE	one of quick, full; default quick	Which scenes to run
--scenes ID	repeatable	Only these scene ids (comma-separated), from the suite’s pack
--set KEY=VALUE	repeatable	Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu N		The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer’s choice (the first discrete GPU)
--out FILE		Write the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIR		A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIR		Keep every sampled frame and the session logs in this directory
--download-only		Download and verify the suite’s inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busy		Run even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDS	default 300	Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quiet		No progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIR		Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIR		Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets
--suite SUITE	one of quick, full; default quick	Which scenes to run
--scenes ID	repeatable	Only these scene ids (comma-separated), from the suite’s pack
--set KEY=VALUE	repeatable	Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu N		The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer’s choice (the first discrete GPU)
--out FILE		Write the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIR		A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIR		Keep every sampled frame and the session logs in this directory
--download-only		Download and verify the suite’s inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busy		Run even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDS	default 300	Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quiet		No progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIR		Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIR		Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets