We present generative predictive control (GPC), a learning control framework that (i) clones a generative diffusion-based policy from expert demonstrations, (ii) trains a predictive action-conditioned world model from both expert demonstrations and random explorations, and (iii) synthesizes an online planner that ranks and optimizes the action proposals from (i) by looking ahead into the future using the world model from (ii). Across a variety of robotic manipulation tasks, we demonstrate that GPC consistently outperforms behavior cloning in both state-based and vision-based settings, in simulation and in the real world.
Push-T
Triangle Drawing
Block Stacking
Cube & Sphere Swapping
World Model Prediction in GPC-RANK.
World Model Prediction in GPC-OPT.
World Model Prediction in GPC-RANK.
World Model Prediction in GPC-OPT.
World Model Prediction in GPC-RANK.
World Model Prediction in GPC-OPT.
Baseline Test 0: Success.
GPC-RANK Test 0: Success.
GPC-OPT Test 0: Success.
Baseline Test 1: Failure.
GPC-RANK Test 1: Success.
GPC-OPT Test 1: Success.
Baseline Test 2: Success.
GPC-RANK Test 2: Success.
GPC-OPT Test 2: Success.
Baseline Test 3: Failure.
GPC-RANK Test 3: Failure.
GPC-OPT Test 3: Success.
Baseline Test 4: Failure.
GPC-RANK Test 4: Success.
GPC-OPT Test 4: Success.
Baseline Test 5: Failure.
GPC-RANK Test 5: Success.
GPC-OPT Test 5: Failure.
Baseline Test 6: Success.
GPC-RANK Test 6: Failure.
GPC-OPT Test 6: Failure.
Baseline Test 7: Failure.
GPC-RANK Test 7: Failure.
GPC-OPT Test 7: Success.
Baseline Test 8: Success.
GPC-RANK Test 8: Success.
GPC-OPT Test 8: Success.
Baseline Test 9: Success.
GPC-RANK Test 9: Success.
GPC-OPT Test 9: Failure.
Baseline Test 0: Failure.
GPC-RANK Test 0: Success.
GPC-OPT Test 0: Success.
Baseline Test 1: Success.
GPC-RANK Test 1: Success.
GPC-OPT Test 1: Success.
Baseline Test 2: Failure.
GPC-RANK Test 2: Success.
GPC-OPT Test 2: Failure.
Baseline Test 3: Failure.
GPC-RANK Test 3: Failure.
GPC-OPT Test 3: Failure.
Baseline Test 4: Success.
GPC-RANK Test 4: Success.
GPC-OPT Test 4: Success.
Baseline Test 0: Success.
GPC-RANK Test 0: Success.
GPC-OPT Test 0: Success.
Baseline Test 1: Success.
GPC-RANK Test 1: Success.
GPC-OPT Test 1: Success.
Baseline Test 2: Failure.
GPC-RANK Test 2: Success.
GPC-OPT Test 2: Success.
Baseline Test 3: Failure.
GPC-RANK Test 3: Failure.
GPC-OPT Test 3: Failure.
Baseline Test 4: Failure.
GPC-RANK Test 4: Failure.
GPC-OPT Test 4: Success.
Baseline Test 0: Success.
GPC-RANK Test 0: Success.
GPC-OPT Test 0: Success.
Baseline Test 1: Success.
GPC-RANK Test 1: Success.
GPC-OPT Test 1: Success.
Baseline Test 2: Failure.
GPC-RANK Test 2: Success.
GPC-OPT Test 2: Success.
Baseline Test 0: Failure.
GPC-RANK Test 0: Success.
Baseline Test 1: Success.
GPC-RANK Test 1: Success.
Baseline Test 2: Failure.
GPC-RANK Test 2: Success.
Baseline Test 3: Failure.
GPC-RANK Test 3: Success.
Baseline Test 4: Failure.
GPC-RANK Test 4: Failure.
Baseline Test 5: Failure.
GPC-RANK Test 5: Success.
Baseline Test 6: Success.
GPC-RANK Test 6: Success.
Baseline Test 7: Success.
GPC-RANK Test 7: Success.
Baseline Test 8: Failure.
GPC-RANK Test 8: Failure.
Baseline Test 9: Failure.
GPC-RANK Test 9: Failure.
@article{qi2025gpc,
title={Strengthening Generative Robot Policies through Predictive World Modeling},
author={Qi, Han and Yin, Haocheng and Du, Yilun and Yang, Heng},
journal={arXiv preprint arXiv:2502.00622},
year={2025}
}