PIVOT

Policy Improvement via Trust-Region Optimization from Deployment Experience

Shenghan Lin1,2,∗ · Qicheng Xu1,2,∗ · Zhecheng Yuan1,2,3,∗,† · Xiangchen Tian2 · Chenhao Lu1,2,3 · Zilin Kang2,3 · Muhan Wang2 · Xiuyu Xie2 · Huazhe Xu1,2,3

1 PokeBot    2 Tsinghua University IIIS    3 Shanghai Qi Zhi Institute

∗ Equal Contribution    † Project Leader

PIVOT post-trains an embodied foundation model from its own real-world deployment experience into consistent, monotonic gains across precise, contact-rich bimanual manipulation tasks.

01Approach

“For the things we have to learn before we can do them, we learn by doing them.”

— Aristotle, Nicomachean Ethics

We bring this idea to embodied foundation models: a robot that learns by doing. Deployed in the real world, our policy improves itself round after round from its own deployment experience — demonstrations, human corrections, and autonomous rollouts alike — turning each into steady, monotonic gains. We call this system PIVOT.

02Tasks

Task Demonstrations

After post-training, a single policy runs unattended over long horizons across contact-rich bimanual tasks, each requiring fine-grained contact control and recovery from small execution errors.

Laundry T-shirt folding from randomized crumpled poses, running continuously for one hour without intervention.
Paper Box Long-horizon multi-stage box assembly from a flat cutout, running autonomously over many consecutive runs (shown at 20× speed).
Sachet Threading sachets with high dexterity, succeeding across four different sachet pairs.
Zip-Tie Picking and threading a flexible zip tie at millimeter-level precision.

03Generalization

Generalization

We probe visual generalization on objects with appearances never seen during training: the Laundry policy folds unseen T-shirts, and the Zip-Tie policy threads colored cable ties. Both indicate that the learned contact-rich skills transfer beyond the training distribution.

Colorful Zip Ties Threading cable ties of unseen colors.
Unseen T-Shirts Folding T-shirts of unseen colors and patterns.

04Robustness

Robustness

We stress-test our post-trained policies against unexpected mishaps. The Paper Box policy makes robust corrections to small accidents mid-assembly, while the Laundry policy resists physical disturbances from a human, recognizes and re-does a poor fold, and recovers from small execution slips mid-fold.

Paper Box The policy makes robust corrections to small accidents that occur during box assembly, keeping the fold on track.
Human Disturbance Folding continues while a person physically perturbs the cloth and the arms mid-task.
Self-Correction When a fold comes out poorly, the policy notices and re-folds until the result is clean.
Error Recovery After a small execution slip mid-fold, the policy recovers and retries.

05Value Curve

Value Curve

As post-training proceeds, the policy’s own estimate of how well it is doing climbs round after round. We trace this learned value over successive iterations on three tasks — Laundry, Zip-Tie, and Sachet — where the curve rises steadily and monotonically as deployment experience accumulates.

Laundry Learned value rising over post-training iterations on T-shirt folding.
Zip-Tie Learned value rising over post-training iterations on zip-tie threading.
Sachet Learned value rising over post-training iterations on sachet threading.

06Results

Experiments

Fig. 1 Iterative improvement across tasks. Success rate (or throughput) at each post-training iteration. PIVOT yields stable, monotonic gains over successive iterations on all evaluated tasks.
Fig. 2 Policy-extraction comparison and ablations. Laundry throughput and zip-tie success rate across post-training iterations, comparing TRFO against alternative policy-extraction methods and ablation variants under the same data budget.
Fig. 3 Unified multi-task post-training. A single unified policy post-trained with TRFO performs on par with per-task specialists across tasks, suggesting the objective scales to diverse skill repertoires without per-task modification.