[jacoblambert]

Summary

Adds load_pointcloud() in t4_devkit/common/io.py: reads one LiDAR frame as one array per point field, in the dtype stored in the file.

load_pointcloud(filepath, frame_index=None, fields=None, columns=None) -> dict[str, np.ndarray]

• Parquet (1.0, one file per channel, one row group per frame): frame_index picks the frame. The layout comes from the file schema.
• PCD (one file per frame): the layout comes from the PCD header.
• .bin (one file per frame): no header, so the layout comes from fields (in 1.0, sensor.lidar.fields).
• columns selects fields. Compression and encodings are read from the file and never set in the loader, so any writer settings load.

Speed

Each Parquet file is opened once per process, memory-mapped, and cached (keyed by path, file mtime and process id, so a rewritten file or a forked worker gets its own handle). Measured on 60 real j6gen2 LIDAR_CONCAT frames (~320k points):

Method	ms / frame
Parquet	1.8
0.x .bin	~1.1

Reopening re-reads the footer (~1.5 ms), memory mapping saves ~1 ms, and returning columns avoids a copy.

Changes to existing code

• LidarPointCloud.from_file reads through load_pointcloud, so it also reads Parquet (new frame_index and fields arguments). For 0.x .bin without fields, the layout is V0_BIN_FIELDS (x, y, z, intensity, ring, return_type, time_stamp) cut to num_pts_feats (5 by default, 7 with return type and time stamp), all float32, as the 0.x converter writes them. More than 7 fields raises instead of guessing names.
• SegmentationPointCloud.from_file reuses that code instead of its own copy. It is marked deprecated in its docstring: in 1.0, point labels are columns of the LiDAR frame. No runtime warning yet, since 0.x rendering still uses it and there is no replacement until T4Devkit moves to 1.0.
• RadarPointCloud: the docstring says it is a nuScenes placeholder (it reads the 18-field nuScenes radar PCD, while T4 radar is stored as RadarTracks JSON), and from_file warns that T4 radar is not implemented. Radar is not part of 1.0 yet.
• Stamp moved to t4_devkit/common/timestamp.py; still importable from t4_devkit.dataclass.

Found while benchmarking (for the migration pilot)

pyarrow's default dictionary encoding hurts float compression. 60 frames: 317 MB with defaults, 181 MB with dictionary off and BYTE_STREAM_SPLIT on floats, 364 MB as 0.x .bin. Codec (zstd vs lz4) and encodings will be decided in the pilot, with the real writer.

Tests

• tests/common/test_io.py:
  • Parquet: every field comes back in the dtype stored in the file, and columns selects fields.
  • .bin: read with the layout given in fields.
  • PCD: read with the layout in its header.
• tests/dataclass/test_pointcloud.py:
  • LidarPointCloud reads a 1.0 frame from its Parquet row group.
  • A 0.x .bin with 7 fields loads, and one with 8 raises.
  • Existing PCD / .bin tests for LidarPointCloud and SegmentationPointCloud still pass.

[ktro2828]

LGTM