HDF5 → Parquet / MP4

01

The problem

HDF5 robotics datasets cannot be used directly by the target experimentation pipelines. JEPA experiments need independently usable video sequences and trajectories while preserving their correspondence within each episode.

02

My role

I designed and developed HDF5-to-Parquet-and-MP4 conversion to make robotics data usable in world-model experiments.

03

Constraints

• Preserve relationships between tabular data and video sequences.
• Produce formats usable across several experiments.
• Handle the specifics of robotics and multimodal datasets.

04

Architecture & pipeline

1. Read source HDF5 data.
2. Materialize tabular data and trajectories in Parquet.
3. Export video sequences as MP4.
4. Prepare these artifacts for training pipelines.

05

Decisions & iterations

Separating video and tabular data allows formats suited to each type of content.

Conversion is dedicated infrastructure, reusable beyond a single experimental protocol.

06

Results & limitations

The pipeline materializes tabular data in Parquet and video sequences in MP4 for JEPA / world-model experiments.

Video-to-trajectory correspondence remains a conversion constraint; the output schema and consistency checks still need documentation.

07

Next iterations

Document the output schema and consistency checks with conversion examples.