GRASP

Gradient-Guided Sampling for
Contact-Rich Physics-Based Motion Retargeting

Yang Liu*Tong Wang*Yiming BaoHeng Yang

01 / 04

Real dex

2× speed
Scroll to reveal ↓

Overview

Real robots / Dexterous hands

GRASP
for dexterous hands.

Real-world demonstrations2× speed · muted playback

Real robots / Humanoid

GRASP
for humanoid motion.

Method

GRASP Gradient recovery from archived samples
01

Evaluate

Every simulated candidate leaves
a control–cost pair.

CANDIDATE CONTROLSCOST J1J2J3J4J5J6 Archive Measured by physics rollouts
Accepted Rejected Both retained
02

Recover

Fit the local cost structure.
Recover a direction from the archive.

LOCAL QUADRATIC FIT Local control coordinates · schematic
No differentiation through the simulator
03

Guide

Use the recovered gradient
to guide stochastic proposals.

GUIDANCE + EXPLORATION Current The fit guides where to propose
Propose new controls z′
04

Correct & reuse

The surrogate guides proposals. MH corrects for the proposal bias.

From the guided sampler
Proposed control z′
Physics rolloutEvaluate the true cost J(z′)
Metropolis–Hastings
Acceptance test

True target
+ forward / reverse proposal densities

Accept with probability α
Two possible outcomes

Either way, retain the evaluation.The pair (z′, J(z′)) enters the archive for the next fit.

Archive
Why the correction matters

The fitted model guides the proposal, but the acceptance test uses the original tempered target and both proposal densities. With the fitted model and proposal settings held fixed within a move block, MH preserves that target.

α = min(1,πλ(z′) q(z | z′)πλ(z) q(z′ | z))

πλ(z) ∝ p0(z) exp(−λβJ(z)), where J is evaluated by the simulator.

New evaluations return to the archive for the next fit
GRASP

The same evaluations supply candidate motions and local gradient guidance.

Conceptual illustration

GRASP fits local quadratic models to archived control–cost pairs, including rejected proposals. Their gradients guide new samples without differentiating through the simulator. Acceptance uses true simulator costs and a Metropolis–Hastings correction; new evaluations inform the next fit.

Simulation / Dexterous hands

GRASP
for dexterous simulation.

OakInk2 bimanual demonstrationsClips over 8 s sped up · speed marked
Browse trajectories →

Simulation / Humanoid

GRASP
for humanoid simulation.

Humanoid demonstrationsReal-time playback
Browse trajectories →

Results / Experiments

Quantitative results.

Bimanual object tracking

OakInk2 · Two Sharpa hands · 1,352 GRASP tasks

↑ Higher is better   ↓ Lower is better
Dexterous retargeting on OakInk2. Bold values are the best reported values in each column.
MethodSuccess (%) ↑Object position (m) ↓Object rotation (rad) ↓
Dex-retargeting28.60.080.62
SPIDER mjwp71.40.040.57
SPIDER mjwp_act77.10.040.42
Do-as-I-Do (Sharpa)81.00.030.15
↳ w/o transition reward79.00.030.14
↳ annealed sampling only72.00.080.32
GRASP94.20.030.12

GRASP evaluates the complete 1,352-task OakInk2 collection. Evaluation sets differ. Bold indicates the best reported value, including ties.

Evaluation details

A GRASP task succeeds when mean object position error is below 10 cm and mean rotation error is below 0.5 rad. Execution failures count as failures. Error means use jointly finite position–rotation error pairs. Baseline success rates follow their original reporting.

Cross-embodiment

The same optimizer configuration across 1,352 OakInk2 tasks per hand.

Success rate% · Higher is better ↑
Sharpa94.2%
Wuji92.4%
XHand95.0%
Object position errorcm · Lower is better ↓
Sharpa2.87
Wuji2.87
XHand2.87
Object rotation errorrad · Lower is better ↓
Sharpa0.117
Wuji0.117
XHand0.112

Bars show mean object errors. Errors use completed trajectories with finite metrics; execution failures count as failures in success rates.

Policy learning

Dexterous hand

Kinematic reference
GRASP
Four PPO learning curves comparing red Kinematic (TopoRetarget) and blue GRASP (ours): episode reward, episode length, object position error, and shaft-angle error.

PPO tracking on nine single-axis pen-spinning clips. Kinematic (TopoRetarget) denotes the original kinematic references; GRASP (ours) denotes GRASP-refined references. Curves show 100-iteration moving averages aggregated over the clips.

View full figure ↗

Humanoid

Kinematic reference
GRASP
Humanoid PPO object position and orientation tracking rewards, comparing red kinematic OmniRetarget references with blue GRASP-refined references.

Humanoid PPO tracking with three reference motions from OmniRetarget. Red denotes original kinematic references; blue denotes GRASP-refined references. Higher rewards indicate better tracking.

View full figure ↗

Abstract

We study physics-based retargeting of human demonstrations for robot learning: finding robot controls that reproduce reference motions under contact dynamics. We propose GRASP (Gradient Recovery from Archived Samples for Proposals), which combines sampling-based exploration with gradient-guided refinement without requiring a differentiable simulator.

GRASP-TO extends tempered sequential Monte Carlo by fitting local quadratic cost models to archived rollouts and using their gradients to guide Markov chain Monte Carlo rejuvenation. Metropolis–Hastings correction preserves the target distribution for each fixed proposal kernel, and a population analysis connects the recovered gradients to Gaussian smoothing. GRASP-MPC scales this optimizer to long demonstrations through warm-started receding-horizon planning.

Experiments on dexterous hands and humanoids show accurate retargeting with a shared tracking objective and no task-specific contact rewards, including 94.2% success on 1352 bimanual OakInk2 demonstrations. The refined references improve PPO tracking in pen spinning and humanoid–object interaction. We also demonstrate dexterous manipulation and whole-body humanoid motion on real hardware.

Cite

Citation.

@misc{liu2026grasp,
  title  = {{GRASP}: Gradient-Guided Sampling for Contact-Rich Physics-Based Motion Retargeting},
  author = {Liu, Yang and Wang, Tong and Bao, Yiming and Yang, Heng},
  year   = {2026},
  eprint = {2611.05754},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  doi    = {10.48550/arXiv.2611.05754},
  url    = {https://arxiv.org/abs/2611.05754}
}