[jacoblambert]

Summary

Adds load_image() in t4_devkit/common/io.py: reads one camera frame as an (H, W, 3) RGB array.

load_image(filepath, *, frame_index=None) -> np.ndarray

• With frame_index: the file is the channel's video (1.0), read with PyAV.
• Without it: the file is one image per frame (.jpg, .png, ...), read with Pillow. This covers 0.x datasets and 1.0 datasets whose images were not re-encoded (frame_index is null).

Nothing about the video is set in the reader

• PyAV picks the container and decoder from the file (AV1 → dav1d).
• The frame's own pixel format, bit depth, color range and color matrix drive the conversion to RGB. Range matters: reading our full-range video as limited range turns 250 into 255. Range and matrix are passed explicitly, so older PyAV versions behave the same.
• 8-bit video gives uint8 RGB; more than 8 bits (e.g. 10-bit) gives uint16.

Finding a frame

The frame's timestamp is computed from the stream's frame rate and time base (start_time + frame_index / (rate * time_base)). The reader seeks there and decodes up to it.

• Intra-only video (what the 1.0 encoder writes) lands on the frame directly. Other keyframe intervals still work, just slower.
• This needs a constant frame rate, as the spec requires. A frame that isn't found raises IndexError.

Speed

Measured on 100 real j6gen2 CAM_FRONT frames (2880x1860, AV1 preset 6, CRF 26, intra-only), read in random order: 20.5 ms/frame, vs 38.2 ms for the 0.x JPG with Pillow.

• FFmpeg's default decoder threading adds ~30 ms/frame latency after each seek, so the reader uses one decoding thread. That also suits data loaders with several workers.
• RGB conversion takes ~0.4 ms.
• One open container per file, process and thread, cached (keyed like the Parquet cache in feat(common): load LiDAR frames from Parquet, PCD or bin by their own layout #305, plus the thread, since a container keeps decoding state).

New dependency: PyAV (av>=13.1)

A regular dependency, not an optional extra: reading camera frames is core, and migrated 1.0 datasets store cameras as AV1. 13.1 is the oldest version checked. It installs at ~127 MB with its bundled FFmpeg, less than rerun-sdk or pyarrow, with wheels for Linux, macOS and Windows and no system FFmpeg needed.

Tests

tests/common/test_io.py:

• The 1.0 sample video: frames read out of order (1, then 0) have the expected size, dtype and pixels (red ramps 0–255 across the width, green is 128 × frame_index), which checks frame lookup and the color range. A frame past the end raises IndexError.
• A PNG file reads back with the same pixels.

[ktro2828]

LGTM