A camera entry says where frames come from and what the bridge is allowed to produce from them. At most 50 cameras live in one file.
cameras:
front:
source:
kind: ros
topic: /camera/image_raw
type: sensor_msgs/msg/Image
width: 1280
height: 720
fps: 15
bitrate_kbps: 2000
snapshot_interval_seconds: 5
description: Forward-facing camera on the mast.
| field | type | bounds | absent means |
|---|---|---|---|
source.kind |
ros, rtsp, mjpeg or v4l2 |
— | required |
width |
integer | 1 to 7680 pixels; even with a hardware encoder |
required |
height |
integer | 1 to 4320 pixels; even with a hardware encoder |
required |
fps |
integer | 1 to 60 |
required |
bitrate_kbps |
integer | 1 to 50000 |
required |
snapshot_interval_seconds |
integer | 1 to 3600 |
required |
description |
text | 1 to 2000 characters | offered as description: null |
role |
a role key, or a list of keys | a key, or 1 to 50 keys; see Roles & Access | the first rule of access.rules that fits, then the organisation’s default for read; a camera that does not read a ROS topic fits no rule, so only the default applies |
kind fixes which other fields the source may carry, so an impossible camera
is unrepresentable rather than merely invalid. There is no way to write an
RTSP camera with a ROS topic.
kind |
fields |
|---|---|
ros |
topic (ROS graph name), type (sensor_msgs/msg/Image, sensor_msgs/msg/CompressedImage or foxglove_msgs/msg/CompressedVideo) |
rtsp |
url (rtsp:// or rtsps://, at most 2048 characters), optional transport (tcp or udp, default tcp), optional passthrough (true or false, default false), optional credentials |
mjpeg |
url (http:// or https://, at most 2048 characters), optional credentials |
v4l2 |
device (a path under /dev/, at most 128 characters) |
What the four numbers bound
width, height, fps and bitrate_kbps are the resolution and load the
bridge produces before sending, not the camera’s own. Together they are
your control over the robot’s bandwidth, which is why they live in the
configuration and not in a viewer’s request: a viewer must never be able to
make a robot send more.
fps is a ceiling and not a clock, so a camera delivering ten frames a second
stays at ten. bitrate_kbps bounds the live encoding only. Raising the
resolution or the frame rate against a fixed bitrate buys blur, not detail.
A snapshot can arrive smaller than width and height. Its JPEG has a
byte ceiling, and the bridge gives up quality first and then resolution to fit
it, reporting the size it actually encoded.
A passthrough camera is the exception: the bridge
forwards the source’s own stream, so its bandwidth is set at the camera.
width and height must equal the stream’s, and fps and bitrate_kbps do
not bound its live video (below).
With a hardware encoder, any live.encoder
but software, width and height must both be even, a passthrough camera’s
too: the live video is H.264, which takes no odd size. A configuration that
names such an encoder and has an odd size is refused; what the robot does when
the encoder comes from its ROS parameter is on that page.
What the bridge does not do twice
From bridge 6.4.0 the bridge skips the work the robot has already done or has a chip for.
A JPEG that already fits is the snapshot. When the source delivers JPEG in
exactly the configured width and height, and the picture fits the byte
ceiling, the bridge sends those bytes unchanged. It decodes nothing and
encodes nothing. That holds for a ros source of type
sensor_msgs/msg/CompressedImage and for an mjpeg source, with or without a
live viewer.
Any other frame takes the path above: decoded, resized, encoded. That includes
a JPEG of another size, one over the byte ceiling, and the kinds not every
browser shows, such as CMYK or 12-bit. The bridge reads what a frame is from
its bytes, not from the format a ROS message claims.
To have a camera node’s JPEG passed through, set width and height to what
the node sends, or the node to what you set.
An MJPEG stream is decoded only when somebody needs the pixels. That is while the camera is watched live, and for a snapshot that cannot be passed through.
Live video can use the robot’s encoder chip. That is one setting for the
whole robot, live.encoder,
and it is off until you choose it.
A source that already delivers H.264 can go without the encoder. A ROS
topic that carries H.264 is passed through as it is. An
RTSP camera is passed through when its source says passthrough: true, and
decoded and encoded again without it. live.encoder does not reach a camera
that is passed through.
Snapshots always run
snapshot_interval_seconds is how often a still frame is captured, and it
runs whether or not anyone is watching. The cloud holds the one frame and
serves every reader from it, however many poll.
Live streaming is the opposite: the cloud counts viewers, the first one starts
the stream and the last one ends it. In
low-bandwidth mode a running
stream is re-encoded at camera_bitrate_kbps or ended outright, and a new one
is refused with the code low_bandwidth. A passthrough stream is always
ended.
By default live video travels in RTP packets of at most 1200 bytes, which
fits a path with an MTU of 1280, the size NetBird and Tailscale set. The
number is a setting of the robot, not of a camera, and needs bridge 6.3.0:
live.max_packet_bytes.
The four sources
ros— the only source the bridge subscribes to rather than opens, so it needs no URL, no device and nobody to authenticate to. A topic that carries H.264 is passed through.rtsp— a network camera the robot itself can reach.udploses frames on a congested link and loses them silently, which is why an omittedtransportmeanstcp.mjpeg— one JPEG after another over HTTP. There is no transport to choose, and credentials travel as HTTP Basic. The bridge decodes it only while somebody watches, or for a snapshot it cannot pass through.v4l2— a capture device attached to the robot, such as a USB camera. A/dev/v4l/by-id/...symlink survives a reboot that renumbers/dev/video0.
The scheme restrictions and the /dev/ requirement are not a formality.
The bridge opens these values with libraries that would as happily serve
file: or an ordinary video file, so without the restriction a configuration
document would be arbitrary file access on the robot. The bridge re-derives
the same constraints itself rather than trusting the wire, and a device may
hold no .. segment and may not end in a slash.
H.264 passthrough
A source that already delivers H.264 reaches live viewers without the bridge decoding or encoding video. The bridge forwards the access units it receives. That takes the picture out of the bridge’s hands: what the source sends is what a viewer gets.
| source | passed through |
|---|---|
ros with type: foxglove_msgs/msg/CompressedVideo |
always |
ros with type: sensor_msgs/msg/CompressedImage |
when a message’s format contains h264, in any case; a topic that carries JPEG stays what it was |
rtsp |
with passthrough: true; without it the stream is decoded and encoded again, as before |
cameras:
yard:
source:
kind: rtsp
url: rtsp://cam-1.plant.local/stream1
passthrough: true
width: 1920
height: 1080
fps: 25
bitrate_kbps: 4000
snapshot_interval_seconds: 5
Passthrough needs bridge 6.4.0. An older bridge decodes an RTSP
stream whatever the key says, and reports a CompressedVideo camera as a type
it does not take.
The bridge does not depend on foxglove_msgs. A robot without the package
reports unknown type for the one camera that names it, and runs on.
What the stream has to bring
- H.264 as an Annex-B byte stream. Baseline, Main and High are forwarded alike. High is shown to play in Chrome. Elsewhere it is established practice that browsers play a camera’s stream forwarded unchanged, as long as it has no B-frames. Any other codec is refused.
- One picture per ROS message. Each message carries one whole access unit. A publisher that sends a picture’s slices in separate messages gets each slice forwarded as a frame of its own.
- No B-frames. A browser plays frames in the order they arrive. The bridge finds a B-frame by its slice type, also when the first one appears mid-stream; a stream that is running then ends.
- A keyframe every one to two seconds, with SPS and PPS. A new viewer starts at the source’s next keyframe and sees nothing until then, so the keyframe interval set at the camera is the longest a viewer waits. Neither RTSP nor a ROS topic can ask the source for a keyframe. The bridge replays no old ones either: everybody already watching would see the picture jump back. A stream with no keyframe for ten seconds is refused.
- The configured size.
widthandheightmust equal the stream’s own, which the bridge reads from the stream. If they differ, the camera reports an error naming both sizes and publishes nothing.
Each refusal is a camera error under the code source_error. The console
shows such a camera’s source as not answering; the sentence that names the
reason is in the bridge’s log on the robot.
The bridge writes that sentence each time the reason changes, and an unchanged reason again at most once a minute. After eight such lines in a minute for one camera, one more says that the reason keeps changing.
The camera recovers by itself once the stream is one the bridge can forward.
What the four numbers mean here
| field | for a passthrough camera |
|---|---|
width, height |
must equal the stream’s size |
fps |
not enforced for live video; bounds how often a snapshot is taken |
bitrate_kbps |
not enforced; it is what the track declares to the media server |
The bridge cannot drop or shrink frames of a stream it does not encode. Frame rate and bitrate are set at the camera.
Snapshots, pauses and a weak link
A snapshot is the newest keyframe: when one is due, the bridge decodes that one frame and nothing else. A snapshot is therefore at most one keyframe interval old.
While nobody watches, the bridge discards what arrives and keeps only the newest keyframe. A viewer who comes back starts at a keyframe, like a new one.
In low-bandwidth mode a running
passthrough stream ends with the code low_bandwidth, under camera: reduce
as under stop. There is no bitrate to reduce.
live.max_packet_bytes holds for a
passthrough stream as for any other: the bridge still cuts the packets.
live.encoder does not apply, since nothing is encoded.
Credentials
source:
kind: rtsp
url: rtsp://cam-1.plant.local/stream1
credentials:
username: ops
password: "..."
Both fields are 1 to 128 characters and both are optional. For MJPEG the
bridge sends a real Authorization: Basic header and leaves the URL
untouched. RTSP offers no such channel, so there the name and password go
inside a connect URL built fresh for that one call and never written back into
the stored document.
Credentials may also sit in the URL itself, as
rtsp://user:pass@host/stream. Where both are present, the explicit block
wins.
There is no secret store behind this. A password written here is in every published version, and those are immutable. It cannot be removed from history and cannot be rotated without republishing, and anyone who may read a robot's configuration history may read every camera password that was ever in it.
One rule bounds that: the publish audit event carries the version number and never the document body.
Plain http:// is permitted for an MJPEG camera, because these usually sit on
the robot’s own network. Basic credentials on such a URL then travel in the
clear, which is a choice this format lets you make and does not pretend is
free.
A camera slug shares one namespace with every datapoint, action, service and publisher in the file. The grammar and the reserved names are on the overview.