Preview and render

Render a scenario as a 2D top-down video, a schematic view or photoreal 3D sensor data, on your machine or in the cloud.

One renderer, the Helios Engine, in three modes:

Mode	What you get	Runs on
2d	Top-down map video of the scenario	CPU
schematic	A clean schematic view through the rig’s cameras	GPU
3d	Photoreal camera images and sensor passes	GPU

Pick a sensor rig

Schematic and 3D renders look through a rig: the cameras on the vehicle. Without --rig they use the basic-dash-camera rig on the ego, a front camera at the preset’s capture (1280x720 at 24 fps for the default showcase); write your own to choose others.

simforge rig init --preset basic-dash-camera --out rig.json
simforge rig show rig.json

rig init writes the cameras at the --profile capture: showcase (1280x720 at 24 fps) unless you pass --profile training (512x384 at 20 fps). The basic dash camera’s camera is dashcam-front. Other presets include alpamayo-2cam, alpamayo-6cam, tesla-hw3 and waymo-5th-gen; simforge rig init --help lists them all, and the vehicle classes.

Preview a frame

simforge preview frame s.json --mode 2d --at 5 --out plan.png
simforge preview frame s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --out frame.png
simforge preview clip s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --seconds 3 --out clip.mp4

Render

simforge render s.json --mode 2d --out out-2d
simforge render s.json --mode schematic --rig rig.json --out out-schematic
simforge render s.json --mode 3d --rig rig.json --out out-3d

--out is a folder that must not exist or be empty. 3D renders take --preset showcase or --preset training (the dataset look at a fixed cost per frame); without --preset, a 3D render uses showcase, locally and with --cloud. They also take --passes to choose among rgb, id, depth and semantic.

GPU memory and texture tiers

GPU memory depends on the source texture tier, the material texture limit and the camera rig. The local CLI still selects a full-chain source (uastc-full) on GPUs with at least 16 GiB, or a preprocessed 512 px source (bc7-512) below that, and records the choice in localDefaults. --textures overrides the source tier. On Apple silicon the budget is three quarters of system memory.

Material textures now have a default maximum dimension of 1024 px, including map, vehicle, pedestrian and prop textures. This is an intentional fine-detail softening to reduce GPU memory; it does not change camera resolution, geometry or sensor cadence. The limit is an upper bound, not an upscale: a 512 px source stays 512 px, and normal mip planning may choose smaller textures.

Base-color textures whose alpha controls cutouts or transparency keep their normal mip selection instead of the size cap, preserving fence openings and foliage coverage. The exception uses actual source alpha; an opaque texture is not exempt just because its material is labelled MASK or BLEND.

Environment plates keep their authored resolution: reducing the star map to 1024 px removed fine night-sky detail. The material cap does not resize sky/moon plates, render targets or sensor buffers.

Set render.set.textureMaxSize to 1024, 2048, 4096 or "full" in an authored scenario, or pass --set textureMaxSize=... to the CLI. "full" removes this limit while preserving normal mip/residency planning. To recover full source detail on a machine that otherwise selects the 512 px tier, set both:

simforge render s.json --mode 3d --textures uastc-full --set textureMaxSize=full --out out-full

The historical tier measurements below have this material limit disabled (textureMaxSize=full); they are not measurements of the new 1024 px default.

GPU memory	What fits (measured peak)
8 GiB	512 px textures on every public map, up to six 1920x1080 cameras (4.5 GiB)
12 GiB	512 px textures everywhere; full textures on every public map except san-ramon-phase-1, one 1280x720 camera (8.1 GiB)
16 GiB	Full textures on every public map, up to six 1920x1080 cameras (11.9 GiB)

Peak GPU memory of one 3D render (showcase preset, one 1280x720 camera unless stated):

Map	Full textures	512 px textures
san-ramon-phase-1	11.3 GiB	3.8 GiB
san-ramon-phase-1, six 1920x1080 cameras	11.9 GiB	4.5 GiB
di-rosa-sf	8.1 GiB	2.8 GiB
richmond-field-station	7.3 GiB	2.1 GiB
belmont-research-center	7.2 GiB	2.3 GiB
yale-street	6.5 GiB	2.3 GiB
el-camino-road	3.5 GiB	1.9 GiB
easterbrook-discovery-school	2.0 GiB	1.8 GiB

The training preset needs about the same; a schematic render needs 0.2 GiB. Measured on an RTX 5080 (driver 595.84). The machine-readable table is sdk/contracts/runtime-support.v1.json in the toolkit.

Output format

A 3D render writes:

Path	Format
render.json	The render’s summary, also printed by the command: preset, rig, texture tier, map and actor-pack identities, gates, timings, videos, warnings
results.json	Every captured file with its sensor, tick, pass, byte count and sha256
<sensor>/<tick:08>.rgb.png	The camera image, RGBA8 PNG, sRGB, with --video off or without ffmpeg (a video render encodes the frames into video/<sensor>.mp4 as it draws them)
<sensor>/<tick:08>.id.png	Instance ids, RGBA8 PNG: id = R + 256·G + 65536·B, 0 = sky or background
legend.json	Instance id to the static map mesh it is
<sensor>/<tick:08>.semantic.png	Semantic class in the B channel of an RGBA8 PNG, CARLA CityScapes ids: 0 unlabeled, 1 building, 2 fence, 4 pedestrian, 5 pole, 7 road, 9 vegetation, 10 vehicle, 12 traffic sign, 18 traffic light
<sensor>/<tick:08>.depth.f32.bin	Raw little-endian float32, width x height, row-major, reverse-Z: distance along the camera axis in metres = nearM / value (nearM is the camera’s near plane, 0.05 m by default); 0 = nothing hit
observed-frames.jsonl	One line per frame: every actor’s position, rotation and visibility as rendered
parity.json	How closely the rendered poses match the simulated trace
video/<sensor>.mp4	The RGB frames as H.264 video (needs ffmpeg; --video off skips it)
job/	The exact inputs the renderer ran with (timeline, scene state, texture plan)

Ticks count from the clip start at the rig’s frame rate. Lidar sources write <sensor>/<tick:08>.ply and radar sources .csv. A 2D render writes render.json and video/topdown.mp4. Traces are identical on every machine; pixels are identical run to run on the same GPU model and driver, and differ slightly between GPUs.

Without ffmpeg, a 2D render that requests video preserves its images under frames/topdown/<index:08>.png and reports that directory in frame.directory; it warns that no video was encoded. --video off remains a render-and-measure operation without retained frames.

Before you render

Check the GPU with simforge doctor and the runtime’s render modes with simforge runtime status. A render that cannot run on this machine fails with a named error; it never falls back to another device or to the cloud. A scene that does not fit in GPU memory fails with native_gpu_out_of_memory: use --textures bc7-512, fewer or smaller cameras, or --cloud. A 3D render whose GPU memory cannot be measured warns gpu_memory_unmeasured; --vram-budget sets the budget explicitly.

Render in the cloud

The same command runs on SimForge’s machines with --cloud and a cost cap in cents:

simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 200 --wait --download

Cloud video downloads include only the encoded videos and final metadata (render.json, results.json, manifest.json, observed-frames.jsonl, parity.json when present, and runtime-info.json). A cloud render draws only the RGB camera pass; its captured per-frame RGB, lidar and radar files are intermediate outputs and are not uploaded. A requested still preview retains its final RGB images instead of videos. The cloud results.json inventory lists only published files, so every listed capture is downloadable. The local renderer’s diagnostic job/ directory is not exported in full; its authoritative job/timeline.json is included.

See Run in the cloud for pricing, caps and following a run.