The bridge sends live video as RTP packets. This section says how large one may be and what encodes the video.
It holds for the whole robot: the packet size is a property of the transport and the encoder is part of the computer, and neither belongs to one camera. A key here overrides the bridge’s ROS parameter of the same name; an absent key leaves the parameter alone.
live:
max_packet_bytes: 1200
encoder: software
max_packet_bytes is bridge 6.3.0 and up, encoder 6.4.0 and up. An
older bridge ignores what it does not know: before 6.3.0 it sends packets of
up to 1326 bytes, which a path with an MTU of 1280 does not carry, and before
6.4.0 it encodes on the CPU whatever this section says.
The keys
| Key | Default | Bounds | Meaning |
|---|---|---|---|
max_packet_bytes |
1200 | integer, 576 to 1350 |
The largest RTP packet the bridge sends. |
encoder |
software |
software, auto, nvenc, vaapi or v4l2 |
What encodes live video. |
The packet size
The number counts the packet before encryption: fixed header, header extensions and payload. Retransmissions are included.
It is not an MTU. The MTU belongs to the path to each viewer, and the robot does not know that path. On the wire a packet is up to about 80 bytes longer than the setting: the header extensions the media server may add, the SRTP tag, UDP, IP and, through a relay, TURN.
A change holds from the next frame, for running streams too. Nobody reconnects.
Why 1200
It is what browsers send. With those 80 bytes on top it fits a path of 1280, the smallest MTU IPv6 guarantees and the one NetBird and Tailscale set.
A packet that does not fit is not trimmed on the way. It is dropped, and the viewer gets a track that never shows a picture: the short last packet of each frame arrives, every full one is lost.
When to lower it
Lower it when a viewer’s path carries less than 1280 bytes, which is what a tunnel inside a tunnel leaves: each one takes its own header. Take the smallest MTU on the path and subtract 80.
Smaller packets mean more of them, each with its own headers, so go as low as the path needs and no lower.
The ceiling is 1350 because the bridge’s SRTP library protects no packet above 1356 bytes. A larger setting would not be a larger packet. It would be no video.
Trying a value on the robot
The key exists as a ROS parameter on the bridge node, so a value can be tried without a publish:
ros2 param set /fleetless_bridge live.max_packet_bytes 1000
A value outside the bounds is refused by the set, with the rule, and the parameter keeps what it had. In this file the editor marks it as you type and publish refuses it, so it never reaches the robot.
The key in this section wins for as long as it is there. Publish a configuration without it and the parameter is in force again; without either, the bridge sends 1200.
The encoder
By default the bridge encodes live video on the CPU. A robot with an encoder chip can use that instead.
encoder |
What encodes | GStreamer element, tried in this order |
|---|---|---|
software |
the CPU: VP8 or H.264, whichever the media server picks | none |
nvenc |
NVIDIA: a Jetson, then a desktop card | nvv4l2h264enc, then nvh264enc |
vaapi |
Intel and AMD graphics | vah264enc, then vah264lpenc, then vaapih264enc |
v4l2 |
a V4L2 encoder, as on a Raspberry Pi 4 | v4l2h264enc |
auto |
the first of nvenc, vaapi and v4l2 that starts |
as above |
software is what the bridge did before the setting existed, so an upgrade
changes no robot. Every other value sends H.264 and nothing else, so a
viewer’s browser has to decode H.264. Google Chrome does, and is the browser
this was shown in; a Chromium built without proprietary codecs does not, and
shows no picture.
vah264lpenc is the low-power encoder of the same driver. Intel Arc and
Meteor Lake encode H.264 only there, so on them the first chain finds no
element and the second one does.
A Raspberry Pi 5 and a Jetson Orin Nano have no hardware encoder. With any
value but software they end on software, as described below.
The setting applies at the next live start. A stream that is running keeps the encoder it has. A camera whose source already delivers H.264 and is passed through is not encoded at all, so this key does not reach it.
Even sizes only
With any value but software, every camera’s width and height must be
even. H.264 takes no odd size, in a hardware encoder or in software, and a
camera with one would get no picture. That holds for a passed-through camera
too: it is not encoded, but its stream is H.264 of the configured size.
So it is refused before it gets that far:
- In this file. A configuration that names such an encoder here and has a
camera of odd size is refused, with a finding at the camera’s
widthorheight. The editor shows it among its findings, and publish refuses it. - On the robot. When the encoder comes from the bridge’s ROS parameter
instead, the cloud cannot see it. The bridge then refuses the live start of
a camera it encodes with
live_unavailableand a message naming the setting and the size. It logs that once per camera for a setting and a size, and again after either changes. Its snapshots go on, and the robot’s other cameras start as before. A passed-through camera meets its own check instead: its stream cannot have an odd size, so it never matches the configured one.
Either way the fix is an even size, or encoder: software. A camera that
becomes even in a new configuration starts at the next attempt; nothing has to
be restarted.
What it needs on the robot
The hardware encoders are reached through GStreamer, and the bridge package
does not depend on it: a robot that stays on software installs nothing.
Every other value needs these packages:
sudo apt install python3-gi gir1.2-gstreamer-1.0 gir1.2-gst-plugins-base-1.0 \
gstreamer1.0-plugins-base gstreamer1.0-plugins-bad
and the plugin that holds the encoder element:
| Element | Comes with |
|---|---|
nvv4l2h264enc |
L4T, on a Jetson |
nvh264enc, vah264enc, vah264lpenc |
gstreamer1.0-plugins-bad |
vaapih264enc |
gstreamer1.0-vaapi |
v4l2h264enc |
gstreamer1.0-plugins-good |
GStreamer is loaded with the first hardware session after the bridge starts.
When its plugin registry is missing or stale, GStreamer then runs its own
plugin scanner, gst-plugin-scanner, as a child process, and starts it again
for the rest of the scan if a plugin crashes it. That delays that one
picture: by 0.2 s for 250 plugins on our development machine.
GStreamer writes the registry to the bridge user’s cache directory:
$XDG_CACHE_HOME/gstreamer-1.0/, which is ~/.cache/gstreamer-1.0/ unless
that variable is set.
Its libraries may write short-lived files of their own besides. One is known:
GStreamer’s orc library creates orcexec.*, maps it and deletes it at once,
in $XDG_RUNTIME_DIR, else in $HOME, else in $TMPDIR or /tmp. The
bridge’s own process does so at the first session whose chain converts the
picture in videoconvert, as every vaapi chain does, on every supported ROS
distribution; with liborc 0.4.42 the scanner creates one as well.
The scan happens at the first hardware start, again after an update of a
GStreamer package, and at every start of a bridge whose user cannot write the
cache directory, unless GST_REGISTRY names a file it can write. Nothing the cloud
sends reaches the scanner’s command line: the binary and its arguments are
GStreamer’s. A robot on software loads no GStreamer and starts no such
process.
When the encoder does not start
A hardware encoder that fails costs no stream. If GStreamer is missing, the element is not there, the device cannot be opened, or the encoder returns nothing within a second for a frame, that live session is encoded in software H.264 instead. Nothing is renegotiated, and the stream goes on.
The session’s first picture waits for the attempt: up to a second for an encoder that returns nothing, up to five for a device that does not open.
The bridge logs one warning with the setting, the element chain it tried and the reason:
live.encoder=nvenc: no hardware encoder could be used (nvenc/jetson cannot be
built: GStreamer has no element 'nvvidconv'; nvenc/desktop cannot be built:
GStreamer has no element 'nvh264enc'). This session is encoded in software
H.264. Until live.encoder changes or the bridge restarts, a chain that cannot
be built is not tried again, and one that did not encode a picture is not
tried again for pictures of that size.
As the last sentence says, the bridge remembers two things, and they are not the same finding.
A chain that cannot be built is about the robot: a package, a driver or the chip is missing. Install what is missing and restart the bridge: GStreamer reads its plugins once, when the bridge first loads it. Where nothing had to be installed, changing the key to another value and back makes the bridge try again; publishing the value it already has changes nothing.
A chain that was built and did not encode a camera’s first picture is remembered for pictures of that size. That camera stays on software, and so does any other of the same size. A camera of another size is still tried on the chip.
An encoder whose driver stops answering costs one late frame, and the session goes on in software. A pipeline that does not shut down within two seconds is left behind, and the bridge says so in a warning of its own. That chain is not built again until the key changes or the bridge restarts.
The fallback catches an encoder that fails, not one that draws a wrong picture. Look at the stream once after choosing an encoder.
Which encoders are proven
None on a robot yet. Every chain is written from the documentation of its GStreamer element.
| Chain | Proven on hardware |
|---|---|
nvenc on a Jetson (nvv4l2h264enc) |
not yet: the first robot, a Jetson Orin NX, is next |
nvenc on a desktop card (nvh264enc) |
no |
vaapi (vah264enc, vah264lpenc, vaapih264enc) |
no |
v4l2 (v4l2h264enc) |
no |
What is proven is everything around the encoder element. The bridge’s test suite runs the whole pipeline, the choice of encoder and the fallback over a software stand-in, on all three ROS distributions.
Treat an encoder as untested until this table says otherwise, and prove it on your machine first:
python3 tools/encoder-proof/encoder_proof.py --backend nvenc
The tool is in the bridge repository,
under tools/encoder-proof/. It runs the encoder with a moving pattern and
checks what the bridge needs from it: one picture out for one picture in, a
keyframe when asked, a bitrate that follows, the picture that went in coming
out, and its colours right by the label the stream carries.
Its last line is the verdict. Send its output along if you report an encoder that works, or one that does not.
Bitrate
With a hardware encoder the bridge hands the encoder a camera’s
bitrate_kbps over its whole range.
The software encoders clamp it: VP8 to 250 to 1500 kbps, H.264 to 500 to
3000 kbps. A session that fell back from a hardware encoder keeps the whole
range.
Trying an encoder on the robot
As for the packet size, the key exists as a ROS parameter:
ros2 param set /fleetless_bridge live.encoder nvenc
A name that is not one of the five is refused by the set. Start a live stream, and read the bridge’s log: it says which chain encodes, or why none does.