Custom sensor rigs

Bring your own cameras, lidar and radar: the rig file, the vehicle frame and rotation conventions, render profiles, and how to use one rig locally, in the cloud and across scenarios.

A sensor rig is a JSON file that lists what a render captures: the cameras, lidars and radars, where each one sits on the vehicle, which way it looks and what it records. The format is simforge.render-rig/v1. A rig is not part of a scenario: you pass it to each render, so one rig works with every scenario, and one scenario can be rendered through many rigs.

Schematic and 3D renders look through a rig. Without --rig they use the built-in basic-dash-camera on the scenario’s ego. 2D renders are a top-down map view and take no rig.

The quick path

simforge rig init --preset basic-dash-camera --class car --out rig.json
simforge rig validate rig.json
simforge rig show rig.json
simforge render s.json --mode 3d --rig rig.json --out out-3d

rig init writes a built-in rig fitted to a vehicle class’s default box, which is the easiest starting point to edit. rig validate checks the file exactly as a render reads it and exits 2 with the problem. rig show prints what the rig captures: each sensor’s mount, the direction it faces, its field of view, and how much of the horizon the cameras cover together, with any gaps. Every field is in the rig file reference, and three complete, rendered rigs are in example rigs.

How a rig attaches to the vehicle

Every scenario names an ego: the vehicle that carries the sensors (a car, truck, bus, van, motorcycle or drone). A render mounts every source of the rig on that ego, whatever actorId the file names, and says so with the warning render_rig_rehosted when it moves one. So actorId: "ego" in a rig works for any scenario. A scenario without an ego is refused (render_ego_missing).

Mounts are rigid: a sensor moves with the vehicle body, including its pitch and roll over bumps and on slopes, and it never sees the vehicle that carries it. The one exception is a drone camera with mountFrame: "gimbal", whose rotation follows the drone’s stabilized gimbal while its position rides the body.

The vehicle frame

Positions are metres in the vehicle’s own frame. The origin is the centre of the vehicle’s box at ground level.

Axis	Points	Notes
+X	forward, the direction of travel	the front bumper of a 4.8 m car is at x = 2.4
+Y	up	the roof of a 1.5 m tall car is at y = 1.5
+Z	left	the left side of a 1.9 m wide car is at z = 0.95

 Top view (looking down)                 Side view (from the left)

            +X forward                              +Y up
                ^                                     ^
                |                                     |      o-> camera, looking +X
         +------+------+                        +-----+----------+
         |      |      |                        |     |          |
 +Z <----+------o      |                        |     |          |
 left    |   vehicle   |                     ---+-----o----------+---> +X forward
         |    box      |                        ground (y = 0); o = origin
         +-------------+
            o = origin (box centre, at ground level)

X forward, Y up and Z left make a left-handed frame. Keep that in mind when you bring mounts from a right-handed convention: see converting calibrations below.

Note. Lidar point clouds (PLY) use a different frame: +Z is to the right (CARLA parity). See sensor outputs.

Default boxes, used to fit the built-in rigs: car 4.8 x 1.9 x 1.5 m, van 5.5 x 2.0 x 2.2 m, truck 9.5 x 2.5 x 3.5 m, drone 1.0 x 1.0 x 0.45 m (length x width x height). A scenario actor with its own dimensions keeps its own box, and your rig’s positions are used as written.

Rotations

A sensor’s rotation is three angles in radians, all optional and 0 by default. At zero rotation a sensor looks straight ahead along +X, level, with its image upright.

Field	Turns the sensor	Positive means	Range
yawRad	about the vehicle’s up axis	to the left (counter-clockwise seen from above)	-pi to pi
pitchRad	about the sensor’s sideways axis	nose up	-pi/2 to pi/2
rollRad	about the sensor’s forward axis	right side down	-pi to pi

They apply in that order: yaw first, then pitch about the turned sensor, then roll about its own line of sight. There is no quaternion or matrix input. Worked examples:

Sensor	yawRad	pitchRad	rollRad
Forward, level	0	0	0
Forward, tilted 5 degrees down	0	-0.0873	0
Left-facing (+90 degrees)	1.5708	0	0
Right-facing (-90 degrees)	-1.5708	0	0
Front-left at 60 degrees	1.0472	0	0
Rear-facing	3.14159	0	0
Straight down (a mapping camera)	0	-1.5708	0

A rear camera’s yaw must stay inside the range: write 3.14159 or -3.14159, not 3.1416, which rig validate refuses (transform.rotation is outside yaw/roll [-pi, pi], pitch [-pi/2, pi/2]).

Converting a calibration you already have

Many calibrations use a right-handed vehicle frame with x forward, y left and z up (FLU), or x forward, y right and z down (FRD). To move a mount into a SimForge rig:

Your frame	SimForge x	SimForge y	SimForge z	yaw	pitch	roll
FLU (x fwd, y left, z up), origin at ground under the box centre	x	z	y	yaw	-pitch (FLU pitch is nose down)	roll
FRD (x fwd, y right, z down), origin at ground under the box centre	x	-z	-y	-yaw	pitch	roll

If your origin is somewhere else, such as the rear axle, subtract that point’s position in the SimForge frame first: for a rear axle 1.4 m behind the box centre, x = x_axle - 1.4. Check every converted rig with simforge rig show: its facing column names the side each sensor looks at (front, front-left, left, rear-left and so on), which catches a flipped sign at once.

Render profiles

The --preset flag picks how a 3D render looks; the rig picks what it captures.

Profile	Look	Built-in rigs record
showcase	photoreal, the default for a 3D render (local and --cloud)	1280x720 at 24 fps
training	the dataset look at a fixed cost per frame	512x384 at 20 fps
schematic	a clean schematic view (--mode schematic)	1920x1080 at 30 fps

Without --rig, the default dash camera gets the profile’s capture size and rate. rig init writes its cameras at a profile’s capture (--profile, default showcase): fit a rig to the profile you render with, such as --profile training for a training render. A rig file keeps its own: each camera renders at its own width, height and fps whatever the profile. Give every camera of a rig the same fps; cameras at different rates are scheduled on the union of their frame times.

Use a rig

Locally, with the runtime installed:

simforge rig validate rig.json
simforge preview frame s.json --mode 3d --rig rig.json --camera cam-front --at 5 --out frame.png
simforge render s.json --mode 3d --rig rig.json --out out-3d
simforge render s.json --mode 3d --rig rig.json --preset training --passes rgb,depth --out out-train

preview frame renders one camera at one moment, which is the fastest way to check a mount; --camera takes a camera’s outputName. --passes chooses the camera passes (rgb, id, depth, semantic; all four by default).

In the cloud, the same file goes with the request:

simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --dry-run
simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 100 --wait --download

A cloud render sends the rig as part of the run (the rig of a render run in the Cloud API) and renders it with the same runtime. It returns each sensor’s video plus render.json and results.json, not the per-frame images, depth or point clouds: render locally when you need those. See sensor outputs.

In Studio, the Render panel’s sensor step offers the built-in rigs, a single front camera and the rig this scenario last rendered with. Studio has no rig upload yet: to render your own rig file, use the CLI, locally or with --cloud.

Reuse a rig across scenarios

A rig names no map, scenario or actor of its own, so keep one file per sensor setup and pass it to every render:

for s in scenarios/*.json; do
  simforge render "$s" --mode 3d --rig rigs/surround.rig.json --out "out/$(basename "$s" .json)"
done

Positions are absolute metres on the vehicle, so a rig written for a car sits in the same place on a truck: the cameras are not moved to the truck’s roof. Write one rig per vehicle class you render, or start each from simforge rig init --preset <preset> --class <class>, which fits a built-in rig to the class’s box.

One built-in rig is calibrated rather than fitted: qwen-drive-3cam, Qwen-Drive’s three native 120° f-theta cameras as measured on one car (1920x1080 at 10 fps). simforge rig init --preset qwen-drive-3cam --out rig.json writes it as it is, for --class car only and without --profile. Render it with --set lens.distortion=0: the f-theta lens is the image, so the renderer’s own lens post-warp must be off.

What a rig cannot describe yet

These are not supported today. A field for any of them is refused as unknown:

• Pinhole intrinsics as fx, fy, cx, cy, or a shifted principal point: a pinhole camera is horizontalFovDeg with square pixels and the principal point at the image centre.
• Brown-Conrady (OpenCV radial-tangential), Kannala-Brandt or equidistant fisheye distortion: the only lens model is the f-theta polynomial (cameraModel).
• Exposure, shutter speed, rolling shutter, motion blur per camera, noise or lens flare settings.
• IMU, GNSS, ultrasonic, thermal or event cameras. The vehicle’s pose at every frame is in observed-frames.jsonl.
• Separate depth, semantic or instance camera sources: those are passes of an RGB camera (--passes); a source with modality depth, semantic or instance validates but the renderer refuses it (native_modality_unsupported).
• A lidar whose vertical field is not symmetric about its mount (for example +15 to -25 degrees), lidar noise or dropout, and radar noise or clutter.