Changhui Jing, Haochen Tian, Hongchen Li, Tianyu Li, Caojun Wang, Zihao Zhang, Yanhao Liu, Chengen Xie, Bailin Li, Hao Zhao, Zhenjie Yang
- 📧 Primary Contact: Changhui Jing (changhui.jing@opendrivelab.com)
- 🖊️ Joint effort by Shanghai Innovation Institute, OpenDriveLab at HKU, Li Auto Inc., Tongji University, and Tsinghua University.
- 🚗 Simulation-grounded alignment: Render matched future videos for alternative candidate trajectories in reconstructed real-world scenes.
- 🔍 Selective video supervision: Prioritize higher-error futures among high-reward candidates and jointly train trajectory planning and video prediction.
- 📈 Improved planning: Achieve 91.3 PDMS on NAVSIM v1, 88.2 EPDMS on NAVSIM v2, and 38.8 average HD-Score on HUGSIM, with direct trajectory prediction at inference.
- Results release.
- More visualization results.
- Code release.
- Checkpoints release.
SimVTA is a simulation-grounded post-training framework for driving world action models. It extends supervision beyond logged trajectories using a 3D Gaussian Splatting simulator, combining trajectory GRPO with a flow-matching video loss that updates shared representations.
Reported results in the paper:
| Benchmark | Metric | SimVTA |
|---|---|---|
| NAVSIM v1 | PDMS | 91.3 (+5.1 over the 86.2 initialization) |
| NAVSIM v2 | EPDMS | 88.2 |
| HUGSIM | Average HD-Score | 38.8 |
Simulator-rendered futures for going straight (A), changing lanes left (B), and changing lanes right (C).
Video-trajectory alignment: World Engine renderings compared with SimVTA and Epona predictions.
We thank the authors and open-source contributors of the following projects for providing the models, simulation tools, and evaluation benchmarks that support this work:
- Epona | World Engine | NAVSIM | HUGSIM
We also thank the broader open-source community for its contributions to driving world models, autonomous driving, and 3D Gaussian Splatting.
All content in this repository is licensed under the Apache-2.0 license.
If our paper or code helps your research, please consider citing our work and giving this repository a star.