Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

01 动机Motivation

机器人 visuomotor 策略学习面临三大核心挑战：多模态动作分布（同一状态存在多种合理行为）、 高维动作序列的时序相关性以及高精度控制要求。现有方法（显式策略、IBC 隐式策略、BET 序列模型）各有缺陷，均难以同时满足这三点。Robot visuomotor policy learning faces three core challenges: multimodal action distributions (multiple reasonable behaviors exist for the same state), temporal correlation within high-dimensional action sequences, and high-precision control requirements. Existing approaches (explicit policies, IBC implicit policies, BET sequence models) each have shortcomings, and none satisfies all three simultaneously.

"This work presents Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process on robot action space."

Policy Representations — **图 1：三种策略表示的对比。** (a) 显式策略直接预测动作，无法表达多峰分布； (b) 隐式策略（IBC）学习能量函数，通过优化寻找最优动作，但训练不稳定； (c) Diffusion Policy 通过学习梯度场将噪声迭代去噪为动作，天然支持多模态分布。**Figure 1: Comparison of three policy representations.** (a) Explicit policies predict actions directly and cannot express multimodal distributions; (b) implicit policies (IBC) learn an energy function and search for the optimal action through optimization, but training is unstable; (c) Diffusion Policy iteratively denoises noise into actions by learning a gradient field, and naturally supports multimodal distributions.

+46.9%平均超越现有 SOTA
（15 个任务）average gain over prior SOTA
(15 tasks)

95%真实 Push-T 任务
成功率（E2E Trans）success rate on the real
Push-T task (E2E Trans)

0.80真实 Push-T 的 IoU
（人类示范 0.84）IoU on the real Push-T
(human demonstration 0.84)

12测试任务总数
（含双臂操作）evaluated tasks in total
(including bimanual)

为什么现有方法不够用？Why are existing methods not enough?

显式策略（BC / LSTM-GMM / BET）Explicit policies (BC / LSTM-GMM / BET)

GMM 等混合模型建模能力有限，难以捕捉复杂多峰分布Mixture models such as GMM have limited modeling capacity and struggle to capture complex multimodal distributions
BET（Behavior Transformer）在精细操作任务中准确率显著下降BET (Behavior Transformer) shows a marked drop in accuracy on fine-grained manipulation tasks
高精度任务（如 Square、ToolHang）成功率偏低Success rates are low on high-precision tasks (such as Square and ToolHang)

隐式策略（IBC）Implicit policies (IBC)

训练极不稳定：训练 loss 下降，但策略成功率剧烈震荡，难以选择 checkpointExtremely unstable training: the training loss goes down, yet the policy success rate oscillates violently, making checkpoint selection difficult
推理时需要优化能量函数，计算开销大且容易陷入局部极小Inference requires optimizing an energy function, which is computationally expensive and easily trapped in local minima
在仿真多个任务上成功率接近 0%Success rates are close to 0% on several simulation tasks

02 方法Method

Diffusion Policy 将策略参数化为条件去噪扩散过程：给定最近 T_o 步观测 O_t，从高斯噪声出发，经过 K 步去噪（学习动作分布的分数函数梯度），输出 T_a 步动作序列，再通过 receding-horizon control 执行其中前 T_a 步并滑窗更新。论文提供两种网络骨干：CNN-based（DP-C）与Transformer-based（DP-T）。Diffusion Policy parameterizes the policy as a conditional denoising diffusion process: given the most recent T_o steps of observation O_t, it starts from Gaussian noise and, after K denoising steps (learning the gradient of the score function of the action distribution), outputs an action sequence of T_a steps; the leading T_a steps are then executed under receding-horizon control and the window slides forward. The paper provides two network backbones: CNN-based (DP-C) and Transformer-based (DP-T).

Diffusion Policy Overview — **图 2：Diffusion Policy 整体框架。** (a) 通用公式：T_o 步观测作为条件，输出 T_a 步动作； (b) CNN 变体：以 FiLM 将观测特征注入每层卷积； (c) Transformer 变体：观测 embedding 通过 multi-head cross-attention 与动作 token 交互，有效缓解过平滑（over-smoothing）问题。**Figure 2: Overall framework of Diffusion Policy.** (a) General formulation: T_o steps of observation act as the condition, and T_a steps of action are produced; (b) CNN variant: observation features are injected into every convolutional layer via FiLM; (c) Transformer variant: observation embeddings interact with action tokens through multi-head cross-attention, effectively mitigating the over-smoothing problem.

去噪扩散过程The denoising diffusion process

训练时对动作加噪，学习预测噪声（或分数）；推理时从 A^K ∼ 𝒩(0, I) 出发，迭代执行 K 步 DDPM 去噪（或 DDIM 加速推理）得到动作序列 A⁰。关键优势：能量函数无需显式定义，分布覆盖度由扩散过程天然保证，可表达任意复杂的多模态分布。During training, noise is added to the actions and the network learns to predict the noise (or the score); at inference the process starts from A^K ∼ 𝒩(0, I) and iteratively performs K DDPM denoising steps (or DDIM for accelerated inference) to obtain the action sequence A⁰. Key advantage: the energy function need not be defined explicitly, distribution coverage is guaranteed naturally by the diffusion process, and arbitrarily complex multimodal distributions can be expressed.

Receding-Horizon Control

每次推理预测未来 T_a 步动作，实际执行前 T_a 步，再以滑窗获取新观测、重新推理。这一机制在保持时序一致性（trajectory coherence）的同时保留闭环响应能力。论文消融实验表明 action horizon 的选择对性能影响显著。Each inference predicts T_a future action steps, of which the leading T_a steps are actually executed; new observations are then obtained with a sliding window and inference is run again. This mechanism preserves closed-loop responsiveness while maintaining trajectory coherence. The ablation study in the paper shows that the choice of action horizon has a significant effect on performance.

视觉条件输入Visual conditioning input

观测 O_t（RGB 图像或状态向量）仅作为条件输入去噪网络，而非被去噪的对象，从而保持动作空间扩散的简洁性。 CNN 变体用 FiLM conditioning；Transformer 变体用 cross-attention。The observation O_t (RGB image or state vector) is fed into the denoising network only as a condition, rather than being the object that is denoised, which keeps diffusion in the action space simple. The CNN variant uses FiLM conditioning; the Transformer variant uses cross-attention.

多模态行为建模Modeling multimodal behavior

扩散模型天然支持多峰（multimodal）分布：同一观测下策略可采样出截然不同但各自合理的动作轨迹，且每次 rollout 内部保持一致性，不会在执行中途切换模式。Diffusion models naturally support multimodal distributions: under the same observation the policy can sample markedly different yet individually reasonable action trajectories, and each rollout stays internally consistent, never switching mode midway through execution.

03 实验Experiments

实验覆盖 4 个基准（Robomimic 仿真、Push-T 仿真、真实 Push-T、多个真实操作任务），基线包括 BC、LSTM-GMM、IBC、BET。指标为任务成功率（sim）和 IoU / 成功率（real Push-T）。 DP-C = CNN-based Diffusion Policy，DP-T = Transformer-based Diffusion Policy。The experiments cover 4 benchmarks (Robomimic simulation, Push-T simulation, real-world Push-T, and several real-world manipulation tasks), with BC, LSTM-GMM, IBC and BET as baselines. The metrics are task success rate (sim) and IoU / success rate (real Push-T). DP-C = CNN-based Diffusion Policy, DP-T = Transformer-based Diffusion Policy.

Robomimic 仿真基准（State Policy，最大成功率 / 最后 10 checkpoint 均值）Robomimic simulation benchmark (State Policy, max success rate / mean of the last 10 checkpoints)

方法Method	Lift	Can	Square	Transport	ToolHang	Push-T
LSTM-GMM	1.00/0.96	1.00/0.93	1.00/0.91	1.00/0.81	0.95/0.73	0.86/0.59
IBC	0.79/0.41	0.15/0.02	0.00/0.00	0.01/0.01	0.00/0.00	0.00/0.00
BET	1.00/0.96	1.00/0.99	1.00/0.89	1.00/0.90	0.76/0.52	0.68/0.43
DP-C（ours）	1.00/0.98	1.00/0.97	1.00/0.96	1.00/0.96	1.00/0.93	0.97/0.82
DP-T（ours）	1.00/1.00	1.00/1.00	1.00/1.00	1.00/0.94	1.00/0.89	0.95/0.81

Robomimic 仿真基准（Visual Policy，最大/均值）Robomimic simulation benchmark (Visual Policy, max / mean)

方法Method	Lift	Can	Square	Transport	ToolHang	Push-T
LSTM-GMM	1.00/0.96	1.00/0.95	1.00/0.88	0.98/0.90	0.82/0.59	0.64/0.38
IBC	0.94/0.73	0.39/0.05	0.08/0.01	0.00/0.00	0.03/0.00	0.00/0.00
DP-C（ours）	1.00/1.00	1.00/1.00	1.00/0.97	1.00/0.96	0.98/0.92	0.98/0.84
DP-T（ours）	1.00/1.00	1.00/0.99	1.00/0.98	1.00/0.98	1.00/0.90	0.94/0.80

真实 Push-T 对比（20 次试验）Real-world Push-T comparison (20 trials)

方法Method	IoU	成功率Success rate	时长（秒）Duration (s)
人类示范Human demonstration	0.84	1.00	20.3
IBC (pos)	0.14	0.00	56.3
IBC (vel)	0.19	0.00	41.6
LSTM-GMM (pos)	0.24	0.20	47.3
LSTM-GMM (vel)	0.25	0.10	51.7
DP (R3M)	0.53	0.65	57.5
DP (ImgNet)	0.24	0.15	55.8
DP E2E Trans（ours）	0.80	0.95	22.9
DP E2E CNN（ours）	0.66	0.80	31.7

真实复杂操作任务（均采用 E2E Diffusion Policy）Real-world complex manipulation tasks (all with E2E Diffusion Policy)

任务Task	成功率Success rate	试验次数Trials	示范数Demos
Mug Flipping（6DoF 抓取+翻转杯子）Mug Flipping (6DoF grasping and flipping a mug)	90%	20	—
Sauce Pouring（酱料倒浇披萨）Sauce Pouring (pouring sauce onto a pizza)	79%	—	—
Sauce Spreading（酱料涂抹）Sauce Spreading (spreading the sauce)	100%	—	—
双臂 Egg Beater（打蛋器操作）Bimanual Egg Beater (operating an egg beater)	55%	20	210
双臂 Mat Unrolling（展开垫子）Bimanual Mat Unrolling (unrolling a mat)	75%	20	162
双臂 Shirt Folding（折叠衬衫）Bimanual Shirt Folding (folding a shirt)	75%	20	284

Real Push-T results — **图 4：真实 Push-T 实验对比。** 各列分别展示关键时刻的动作轨迹，最后一列为末态均值图像。 DP (E2E Trans) 达到 IoU 0.80、成功率 95%，接近人类示范水平（IoU 0.84），而 IBC 在所有变体上成功率均为 0%。**Figure 4: Comparison on the real-world Push-T experiment.** The columns show the action trajectories at key moments, and the last column is the mean image of the final state. DP (E2E Trans) reaches an IoU of 0.80 and a success rate of 95%, close to the human demonstration level (IoU 0.84), whereas IBC has a success rate of 0% across all of its variants.

消融实验Ablation studies

**图 5：消融实验。** 左图：action horizon（动作预测步长）存在最优区间——太短则时序一致性差，太长则响应性下降。右图：Position control 相比 velocity control 使 Diffusion Policy 对延迟更鲁棒，而传统方法（BCRNN、BET）切换到 position control 反而性能下降。**Figure 5: Ablation study.** Left: the action horizon (the number of predicted action steps) has an optimal range — too short and temporal consistency suffers, too long and responsiveness degrades. Right: position control makes Diffusion Policy more robust to latency than velocity control, whereas for conventional methods (BCRNN, BET) switching to position control instead degrades performance.

训练稳定性方面，论文专门与 IBC 对比（Figure 6）：IBC 的 evaluation success rate 剧烈震荡，即便训练 loss 单调下降也无法获得稳定收敛的策略；Diffusion Policy 则展现出平稳的训练曲线， checkpoint 选择不敏感。这一特性在实际应用中尤为重要。Regarding training stability, the paper makes a dedicated comparison with IBC (Figure 6): the evaluation success rate of IBC oscillates violently, and even when the training loss decreases monotonically no stably converged policy can be obtained; Diffusion Policy, by contrast, exhibits a smooth training curve and is insensitive to checkpoint selection. This property is particularly important in practice.

04 局限性Limitations

注：以下局限性混合了论文明确陈述（stated）与从方法设计推断（inferred）的内容，已分别标注。Note: The limitations below mix points explicitly stated in the paper (stated) with points inferred from the method design (inferred); each is labeled accordingly.

继承 Behavior Cloning 的数据依赖性 [stated]Inherits the data dependence of Behavior Cloning [stated]

Diffusion Policy 本质是一种 behavior cloning 方法，当示范数据不足或质量较差时，性能会显著下降。论文指出其改进并不能突破 BC 范式的根本限制—— 若示范覆盖的状态空间有限，策略在分布外状态的泛化性仍无法保证。Diffusion Policy is in essence a behavior cloning method, and its performance degrades markedly when demonstration data are insufficient or of low quality. The paper points out that its improvements do not break through the fundamental limitations of the BC paradigm — if the state space covered by the demonstrations is limited, generalization of the policy to out-of-distribution states still cannot be guaranteed.

推理延迟较高，不适合高频控制 [stated]Relatively high inference latency, not suited to high-frequency control [stated]

每次动作推理需要运行 K 步去噪（DDPM K=100，DDIM K=10~16），推理延迟明显高于直接回归策略。论文承认这使得 Diffusion Policy 在需要高频（如 > 10 Hz）控制的任务上可能遇到瓶颈。虽然 DDIM 加速推理可缓解此问题，但代价是生成质量可能略有下降。Every action inference requires running K denoising steps (DDPM K=100, DDIM K=10~16), so the inference latency is clearly higher than that of a direct regression policy. The paper acknowledges that this may create a bottleneck on tasks that require high-frequency control (e.g. > 10 Hz). DDIM accelerated inference can alleviate the problem, at the cost of a possible slight drop in generation quality.

Transformer 变体超参数敏感 [stated/inferred]The Transformer variant is sensitive to hyperparameters [stated/inferred]

DP-T（Transformer-based）在多数任务上性能优异，但论文指出其对超参数（如 action horizon、 observation horizon）的选择较为敏感，需要细致调优。此外，Transformer 变体的显存占用和训练时间也更高（inferred from architecture scale）。DP-T (Transformer-based) performs excellently on most tasks, but the paper points out that it is fairly sensitive to the choice of hyperparameters (such as action horizon and observation horizon) and requires careful tuning. In addition, the Transformer variant has a higher memory footprint and longer training time (inferred from architecture scale).

双臂复杂任务成功率仍有差距 [inferred]A gap remains in the success rate on complex bimanual tasks [inferred]

在双臂操作任务（Egg Beater 55%、Mat Unrolling 75%、Shirt Folding 75%）中，成功率相比单臂任务明显偏低。这部分源于任务本身的复杂性（多步骤、双臂协调、接触点丰富），也暗示当前方法在长时程、高接触任务中仍有较大提升空间（inferred）。On the bimanual manipulation tasks (Egg Beater 55%, Mat Unrolling 75%, Shirt Folding 75%), success rates are clearly lower than on single-arm tasks. This partly stems from the intrinsic complexity of the tasks (multiple stages, bimanual coordination, rich contact), and also suggests that current methods still have substantial room for improvement on long-horizon, high-contact tasks (inferred).