DeFI：解耦前向与逆向动力学预训练的机器人学习DeFI: Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining
Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining
Wenyao Zhang · Bozhou Zhang · Zekun Qi · Wenjun Zeng · Xin Jin · Li Zhang | Shanghai Jiao Tong University, Fudan University, Eastern Institute of Technology & Shanghai Innovation Institute
VLA（Vision-Language-Action）模型在构建通用机器人方面潜力巨大，但其将 2D 图像预测与 3D 动作预测耦合训练的方式存在本质矛盾，也限制了从海量无标注网络视频中学习。DeFI 通过将视觉前向动力学（future prediction）与逆向动力学（action inference）解耦预训练，分别利用各自最匹配的数据源，再融合为端到端微调架构——让两个模块先各自发光，再协同增益。VLA (Vision-Language-Action) models hold great promise for building generalist robots, yet coupling 2D image forecasting with 3D action prediction during training creates an intrinsic conflict, and it also limits learning from the vast amount of action-free web video. DeFI pretrains visual forward dynamics (future prediction) and inverse dynamics (action inference) separately, each on the data source that suits it best, and then merges them into an end-to-end finetuning architecture — letting the two modules first shine on their own, then reinforce each other.
现有 VLA 模型将视觉生成与动作预测捆绑训练，存在两大根本矛盾：其一，2D 图像空间的未来帧预测目标与 3D 空间的精细动作预测目标本质不对齐；其二，这种耦合训练方式使模型无法充分利用互联网上海量的、仅有视觉内容的无动作标注视频数据。Existing VLA models train visual generation and action prediction jointly, which raises two fundamental conflicts. First, the objective of forecasting future frames in 2D image space is intrinsically misaligned with the objective of predicting fine-grained actions in 3D space. Second, this entangled training scheme prevents the model from fully exploiting the vast amount of action-free, vision-only video data available on the internet.
"VLA models have shown great potential in building generalist robots, but still face a dilemma — misalignment of 2D image forecasting and 3D action prediction. Besides, such a vision-action entangled training manner limits model learning from large-scale, action-free web video data."
图1：DeFI 框架总览。左侧：将前向动力学模型（GFDM）在多样化人类与机器人视频上预训练用于未来预测，将逆向动力学模型（GIDM）通过自监督学习在无标注视频转换中推断潜在动作；两者整合后在下游任务上端到端微调。这一解耦预训练范式在 CALVIN、SimplerEnv 和真实世界部署中均取得当前最优结果。Figure 1: Overview of the DeFI framework. Left: the forward dynamics model (GFDM) is pretrained for future prediction on diverse human and robot videos, while the inverse dynamics model (GIDM) infers latent actions from action-free video transitions via self-supervised learning; the two are then integrated and finetuned end-to-end on downstream tasks. This disentangled pretraining paradigm achieves state-of-the-art results on CALVIN, SimplerEnv and real-world deployment.
4.51CALVIN ABC-D 平均任务链长度CALVIN ABC-D Average task length
SOTA三大评测均超越 所有先前方法Beats all prior methods on all three benchmarks
02方法 MethodMethod
DeFI 将 VLA 训练拆分为三个阶段：① 独立预训练 GFDM 用于视觉前向动力学；② 独立预训练 GIDM 用于逆向动力学；③ 将两者整合进统一架构后在机器人演示数据上端到端微调。两个模型先各自从最合适的数据中汲取知识，再协同合作共同提升下游任务性能。DeFI splits VLA training into three stages: ① pretrain GFDM alone for visual forward dynamics; ② pretrain GIDM alone for inverse dynamics; ③ integrate the two into a unified architecture and finetune end-to-end on robot demonstration data. The two models first absorb knowledge from the data that suits each of them best, then cooperate to jointly improve downstream task performance.
图2：DeFI 三阶段框架。(a) 阶段一：在海量人类与机器人视频上预训练 GFDM（前向动力学），目标为未来帧预测；(b) 阶段二：在无标注视频转换上以自监督学习预训练 GIDM（逆向动力学），推断隐式动作表征；(c) 阶段三：两模块耦合（Coupled Finetuning），在机器人演示数据上端到端微调，GFDM 输出作为视觉先验指导 GIDM 生成准确动作，并由 diffusion-based action adapter 解码为最终动作。Figure 2: The three-stage DeFI framework. (a) Stage 1: GFDM (forward dynamics) is pretrained on large-scale human and robot videos with a future-frame prediction objective; (b) Stage 2: GIDM (inverse dynamics) is pretrained on action-free video transitions with self-supervised learning to infer implicit action representations; (c) Stage 3: the two modules are coupled (Coupled Finetuning) and finetuned end-to-end on robot demonstration data, where the GFDM output serves as a visual prior that guides GIDM to produce accurate actions, which a diffusion-based action adapter decodes into the final actions.
General Forward Dynamics Model (GFDM)
GFDM 以当前观测 ot 和语言指令 l 为输入，预测未来视频帧序列。模型基于视频 VAE（2D 或 3D）在 mixed 数据集上预训练：包含多样化人类视频与机器人操作视频，并附加文本条件。由于不依赖动作标注，GFDM 可以利用互联网上大规模的无动作标注视频数据——这是此前 VLA 范式无法实现的。目标函数为预测 latent 的 noise prediction loss。GFDM takes the current observation ot and the language instruction l as input and predicts the future video frame sequence. The model builds on a video VAE (2D or 3D) and is pretrained on a mixed dataset of diverse human videos and robot manipulation videos with additional text conditioning. Because it does not rely on action labels, GFDM can exploit large-scale action-free video data from the internet — something previous VLA paradigms could not do. The objective is a noise prediction loss on the predicted latents.
General Inverse Dynamics Model (GIDM)
GIDM 以相邻视频帧对 (vt, vt+1) 为输入，通过自监督学习推断两帧之间发生的隐式动作（latent action）。具体而言，以 NonCausal-Transformer 编码未来帧到 latent action codebook（VQ-VAE 量化），再以 Causal-Transformer 从当前帧出发重建未来帧，从而学习有意义的动作表征而无需任何动作标注。预训练后，GIDM 的 latent 动作将作为 action adapter 的输入，并以下游任务的机器人本体动作（proprioceptive actions）作为监督信号进行微调，使用的是 diffusion-based adapter。GIDM takes a pair of adjacent video frames (vt, vt+1) as input and infers, through self-supervised learning, the implicit action (latent action) that occurred between the two frames. Concretely, a NonCausal-Transformer encodes the future frame into a latent action codebook (quantized by VQ-VAE), and a Causal-Transformer reconstructs the future frame starting from the current one, so that meaningful action representations are learned without any action labels. After pretraining, the latent actions of GIDM serve as input to the action adapter, which is finetuned with the robot proprioceptive actions of the downstream task as the supervision signal, using a diffusion-based adapter.
微调阶段，GFDM 先预测未来帧，将预测的视觉特征（intermediate embeddings）注入 GIDM，从而将丰富的视觉先验引导动作推断。GIDM 接收当前观测、语言指令及 GFDM 提供的未来视觉特征后，经由 diffusion-based action adapter 输出最终机器人动作序列。这一设计既保留了两个预训练模块各自的专长，又通过端到端的梯度传播实现了协同优化。During finetuning, GFDM first predicts future frames and injects the predicted visual features (intermediate embeddings) into GIDM, so that rich visual priors guide action inference. Given the current observation, the language instruction and the future visual features supplied by GFDM, GIDM outputs the final robot action sequence through a diffusion-based action adapter. This design preserves the individual expertise of the two pretrained modules while achieving joint optimization through end-to-end gradient propagation.
03实验 ExperimentsExperiments
在三大评测基准上与先前最优方法对比：CALVIN ABC-D（长序列多任务操作）、SimplerEnv（Fractal 与 Bridge 子集）以及真实世界 Franka Robot 部署。同时进行消融实验分析各组件贡献与预训练规模的影响。DeFI is compared with prior state-of-the-art methods on three benchmarks: CALVIN ABC-D (long-horizon multi-task manipulation), SimplerEnv (the Fractal and Bridge subsets) and real-world Franka Robot deployment. Ablations are also run to analyze the contribution of each component and the effect of pretraining scale.
CALVIN ABC-D benchmark
表1 & 表2：主要定量结果。CALVIN ABC-D 上，DeFI 的平均任务链长度（average task length，1000 次 rollout）达到 4.51，超越 VPP（4.08）、GR-2（3.97）、GROOT-N1（3.90）等先前 SOTA 方法。SimplerEnv-Fractal 上 DeFI 达到 51.2% 成功率，超越 OpenVLA-OFT（45.6%）和 π₀（48.4%）。表中加粗数字为 DeFI 结果，*表示作者复现基线值。Table 1 & Table 2: Main quantitative results. On CALVIN ABC-D, DeFI reaches an average task length of 4.51 (over 1000 rollouts), surpassing prior SOTA methods such as VPP (4.08), GR-2 (3.97) and GROOT-N1 (3.90). On SimplerEnv-Fractal DeFI attains a 51.2% success rate, beating OpenVLA-OFT (45.6%) and π₀ (48.4%). Bold numbers in the tables are the DeFI results; * marks baseline values reproduced by the authors.
Benchmark
次优方法Second best
DeFI（本文）DeFI (ours)
提升Gain
CALVIN ABC-D（Avg. Len.）CALVIN ABC-D (Avg. Len.)
VPP 4.08
4.51
+10.5%
SimplerEnv-Fractal（SR%）SimplerEnv-Fractal (SR%)
π₀ 48.4%
51.2%
+2.8pp
SimplerEnv-Bridge（SR%）SimplerEnv-Bridge (SR%)
OpenVLA-OFT 44.8%
49.8%
+5.0pp
Real-world（SR%）Real-world (SR%)
—
81.3%
—
真实世界实验Real-World Experiments
图3：真实世界实验。在 Franka Panda 机器人上部署，评测多种单步操作任务（put the mug, pour water 等）。DeFI 在单任务评估中取得 81.3% 的平均成功率，显著高于 baseline 方法。对复杂任务（如桌面摆放多步流程）的泛化能力也优于对比方法。Figure 3: Real-world experiments. Deployed on a Franka Panda robot and evaluated on a variety of single-step manipulation tasks (put the mug, pour water, etc.). DeFI reaches an average success rate of 81.3% in the single-task evaluation, clearly above the baseline methods. It also generalizes better than the compared methods on complex tasks such as multi-step tabletop arrangement.
消融分析 AblationsAblations
图4 & 消融表。左：在 CALVIN 上分别消融 GFDM 预训练与 GIDM 预训练，发现两者均贡献显著，完整 DeFI 最优（Avg. Len. 4.51 vs 仅用 GIDM-pretrain 的 4.19）。中：数据效率实验——使用 10%、50%、100% 的可用训练数据时，DeFI 始终优于 baseline，在 10% 数据条件下性能仅下降约 18%（vs 基线下降约 43%），体现出预训练带来的强泛化性。右：针对 GIDM 中逆向动力学不同离散化方法（VQ-VAE、DINO、IDM）的对比，VQ-VAE 配合 NonCausal 编码器效果最佳。Figure 4 & ablation tables. Left: ablating GFDM pretraining and GIDM pretraining separately on CALVIN shows that both contribute substantially, with the full DeFI performing best (Avg. Len. 4.51 vs 4.19 for GIDM-pretrain only). Middle: a data-efficiency study — with 10%, 50% and 100% of the available training data DeFI consistently outperforms the baseline, and with only 10% of the data its performance drops by about 18% (vs about 43% for the baseline), reflecting the strong generalization brought by pretraining. Right: comparing different discretization schemes for inverse dynamics in GIDM (VQ-VAE, DINO, IDM), VQ-VAE combined with a NonCausal encoder works best.
消融实验进一步发现：① 将 GFDM 预训练移除后性能下降约 0.32 Avg. Len.；② 移除 GIDM 预训练则下降约 0.36；③ 在 Coupled Finetuning 阶段，将 GFDM 的中间表征注入 GIDM 比仅使用预测帧提升更明显，验证了特征级融合的重要性。使用 VQ-VAE 量化隐式动作相比连续 latent 提升了动作表征的稳定性，减少了训练中的梯度干扰（gradient interference）。The ablations further show that ① removing GFDM pretraining costs about 0.32 Avg. Len.; ② removing GIDM pretraining costs about 0.36; ③ during Coupled Finetuning, injecting the intermediate representations of GFDM into GIDM helps markedly more than using the predicted frames alone, confirming the importance of feature-level fusion. Quantizing implicit actions with VQ-VAE improves the stability of action representations over continuous latents and reduces gradient interference during training.
04局限性 LimitationsLimitations
Note：论文在 Section 5 Conclusion 及附录中对局限性有部分说明；以下条目标注了哪些是作者明确陈述（stated）、哪些是从设计中推断（inferred）。Note: The paper discusses limitations in part in Section 5 Conclusion and in the appendix; the items below mark which ones are explicitly stated by the authors (stated) and which are inferred from the design (inferred).
依赖视频预测质量（stated）Dependence on video prediction quality (stated)
GFDM 的未来帧预测质量直接影响下游动作质量。作者在附录中指出，当视频预测出现较大误差（如快速运动或遮挡场景）时，注入 GIDM 的视觉先验可能引入噪声，进而影响动作精度。论文建议后续工作探索更鲁棒的视频生成基础模型。The quality of the future-frame prediction of GFDM directly affects downstream action quality. The authors note in the appendix that when video prediction has large errors (e.g. fast motion or occluded scenes), the visual prior injected into GIDM may introduce noise and thus degrade action precision. The paper suggests that future work explore more robust video generation foundation models.
GFDM 推断开销较大（inferred from design）GFDM inference is costly (inferred from design)
在推断阶段 GFDM 需要先生成未来帧（latent video），再传递给 GIDM 解码动作。相比直接动作回归的 VLA，这一两阶段推断流程增加了推断延迟，可能限制对实时控制频率要求极高的任务场景。论文中未报告推断速度数据。At inference time GFDM must first generate future frames (latent video) and then pass them to GIDM to decode actions. Compared with VLAs that regress actions directly, this two-stage inference pipeline adds latency and may limit task settings that demand very high real-time control frequencies. The paper reports no inference-speed numbers.
真实世界实验规模有限（inferred）Limited scale of the real-world experiments (inferred)
真实世界评测仅在 Franka Panda 平台上进行，任务种类和数量相对有限，且未涉及双臂、移动底座等更复杂的机器人系统。泛化到其他硬件平台的能力需进一步验证。Real-world evaluation is carried out only on the Franka Panda platform, with a relatively limited variety and number of tasks, and it does not cover more complex robot systems such as bimanual arms or mobile bases. Generalization to other hardware platforms remains to be verified.
SimplerEnv 上部分任务表现不及 baseline（stated）Trails the baseline on some SimplerEnv tasks (stated)
作者诚实地指出，在 SimplerEnv 的若干特定子任务上，DeFI 并未全面超越所有 baseline（如在某些 Bridge 子集任务上与 OpenVLA-OFT 接近），并将其归因于预训练视频数据域与 SimplerEnv 仿真环境的分布差距。The authors candidly point out that on several specific SimplerEnv subtasks DeFI does not outperform every baseline (for instance it is close to OpenVLA-OFT on some Bridge-subset tasks), and attribute this to the distribution gap between the pretraining video domain and the SimplerEnv simulation environment.
