Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation

01 动机Motivation

人类天然具备视觉与触觉的协同感知能力，能在接触过程中即时做出毫米级精度调整。而现有机器人操作方法面临双重困境：一方面，遥操作系统普遍缺乏精细的触觉/力反馈，导致采集的演示数据质量受限；另一方面，以 action chunking 为代表的视觉模仿学习范式将动作分块执行，形成开环控制，根本上无法在接触过程中即时响应触觉信号。Humans are naturally endowed with coordinated visual and tactile perception, letting them make millimeter-level adjustments the instant contact happens. Existing robotic manipulation methods face a twofold predicament: on one hand, teleoperation systems generally lack fine-grained tactile/force feedback, which limits the quality of the demonstrations collected; on the other hand, the visual imitation learning paradigm typified by action chunking executes actions in blocks, forming open-loop control that fundamentally cannot respond to tactile signals during contact.

"Action chunking enables the policy to model complex behaviors but prevents immediate responses to tactile feedback during execution."

TactAR 与 RDP 概览 — 系统概览。左：TactAR 以 AR 方式将 3D 形变场可视化，叠加在机械臂末端，操作者通过 Meta Quest 3 实时感知接触力。右：RDP 的 slow-fast 推理流水线——慢策略（Latent Diffusion Policy）以低频生成 latent action chunk，快策略（Asymmetric Tokenizer）以高频读取触觉输入并在 latent 空间内自回归修正动作。System overview. Left: TactAR visualizes the 3D deformation field in AR, overlaid on the robot arm's end-effector, so the operator perceives contact forces in real time through a Meta Quest 3. Right: the slow-fast inference pipeline of RDP — the slow policy (Latent Diffusion Policy) generates latent action chunks at low frequency, while the fast policy (Asymmetric Tokenizer) reads tactile input at high frequency and autoregressively corrects the actions inside the latent space.

0.90RDP (GelSight) Peeling 综合得分（vs. 基线 0.39）RDP (GelSight) Peeling overall score (vs. baseline 0.39)

0.87RDP (Force) Wiping 综合得分（vs. 基线 0.50）RDP (Force) Wiping overall score (vs. baseline 0.50)

0.70RDP (Force) Bimanual Lifting 综合得分（vs. 基线 0.08）RDP (Force) Bimanual Lifting overall score (vs. baseline 0.08)

<1 msFast policy 推理延迟，理论支持 >300 Hz 触觉输入Fast policy inference latency; in theory supports >300 Hz tactile input

02 方法Method

本文方法由两个互补模块组成：TactAR 解决数据采集端的触觉反馈缺失问题，RDP 解决策略执行端的闭环响应问题。The method consists of two complementary modules: TactAR addresses the missing tactile feedback on the data-collection side, and RDP addresses closed-loop responsiveness on the policy-execution side.

TactAR：低成本通用触觉遥操作系统TactAR: A Low-Cost, General-Purpose Tactile Teleoperation System

TactAR 遥操作系统 — TactAR 系统概览。系统以 **3D 形变场**作为触觉/力反馈的统一表示，兼容 GelSight Mini、MCTac 等光学触觉传感器以及关节力矩传感器。3D 形变场通过 Meta Quest 3 渲染并"附着"于机器人末端，使操作者在 AR 空间中直接感知接触信息，系统整体成本约 $500。Overview of the TactAR system. The system adopts the **3D deformation field** as a unified representation of tactile/force feedback, and is compatible with optical tactile sensors such as GelSight Mini and MCTac as well as with joint torque sensors. The 3D deformation field is rendered by a Meta Quest 3 and "attached" to the robot end-effector, letting the operator perceive contact information directly in AR space; the overall system cost is about $500.

从凝胶标记的 2D 光流中提取形变场，再通过预定义标定矩阵投影为 3D 向量场。The deformation field is extracted from the 2D optical flow of the gel markers, then projected into a 3D vector field through a predefined calibration matrix.
支持多路 RGB 相机与触觉传感器的实时流传输，适配不同机器人本体。Real-time streaming of multiple RGB cameras and tactile sensors is supported, adapting to different robot embodiments.
用户研究（10 名用户，Peeling 任务）表明，有触觉反馈条件下的遥操作数据质量显著优于无反馈条件，接触力稳定性更高。A user study (10 users, Peeling task) shows that teleoperation data collected with tactile feedback is markedly better in quality than data collected without it, with more stable contact forces.

策略对比：action chunking 的开环困境Policy Comparison: The Open-Loop Dilemma of Action Chunking

控制流水线对比 — 四种控制模式对比。(a) 原始 action chunking：分块执行期间完全开环；(b) temporal ensemble：提供半闭环能力，但对平滑系数 τ 极度敏感（τ=0.2 抓取成功率 30%，τ=0.5 为 0%，τ=0.8 为 100%）；(c) RDP slow-fast 推理：具备闭环能力，可快速响应接触扰动；(d) 人类操作模式：同时兼顾高层规划与触觉实时反馈。Comparison of four control modes. (a) Vanilla action chunking: fully open-loop while a chunk is being executed; (b) temporal ensemble: offers semi-closed-loop capability but is extremely sensitive to the smoothing coefficient τ (grasp success rate 30% at τ=0.2, 0% at τ=0.5, 100% at τ=0.8); (c) RDP slow-fast inference: closed-loop, able to respond quickly to contact disturbances; (d) human operation mode: combining high-level planning with real-time tactile feedback.

Reactive Diffusion Policy (RDP)：Slow-Fast 双频架构Reactive Diffusion Policy (RDP): A Slow-Fast Dual-Frequency Architecture

RDP 框架。(a) 训练阶段：第一阶段训练 fast policy（Asymmetric Tokenizer，AT），包含 1D-CNN encoder 和 GRU decoder；第二阶段固定 AT，训练 slow policy（Latent Diffusion Policy，LDP）在 latent 空间上进行扩散去噪。(b) 推理阶段：LDP 以 1–2 Hz 低频基于视觉观测生成 latent action chunk；AT decoder 以 20–30 Hz 高频接收触觉输入，自回归地在 latent 空间修正动作块，输出最终关节角度序列。The RDP framework. (a) Training stage: the first stage trains the fast policy (Asymmetric Tokenizer, AT), comprising a 1D-CNN encoder and a GRU decoder; the second stage freezes AT and trains the slow policy (Latent Diffusion Policy, LDP) to run diffusion denoising in the latent space. (b) Inference stage: LDP generates latent action chunks from visual observations at a low frequency of 1–2 Hz; the AT decoder takes tactile input at 20–30 Hz and autoregressively corrects the action chunk in the latent space, outputting the final joint angle sequence.

Slow Policy — Latent Diffusion Policy (LDP)

以 低频（1–2 Hz）基于视觉观测预测 latent action chunk。Predicts latent action chunks from visual observations at low frequency (1–2 Hz).
在 latent 空间执行扩散去噪，保留高层策略意图，推理延迟约 100 ms。Runs diffusion denoising in the latent space, preserving the high-level policy intent, with an inference latency of about 100 ms.
采用 相对轨迹预测而非绝对坐标，实验证明相对预测性能显著优于绝对预测。Adopts relative trajectory prediction instead of absolute coordinates; experiments show that relative prediction performs markedly better than absolute prediction.

Fast Policy — Asymmetric Tokenizer (AT)

以 高频（20–30 Hz）读取触觉/力信号，修正 latent action chunk。Reads tactile/force signals at high frequency (20–30 Hz) and corrects the latent action chunk.
Encoder（1D-CNN）将动作序列压缩为 latent；Decoder（GRU）仅接收 latent 与触觉特征，保证 latent 仅编码高层策略，不含传感器噪声。The encoder (1D-CNN) compresses the action sequence into a latent; the decoder (GRU) receives only the latent and the tactile features, ensuring the latent encodes the high-level policy alone and carries no sensor noise.
推理延迟 <1 ms，理论支持 >300 Hz 触觉输入频率。Inference latency <1 ms, in theory supporting tactile input rates of >300 Hz.

触觉表示采用 PCA 对标记形变场进行降维，前四个主成分分别对应切向力（C1, C3）、扭转力矩（C2）和法向力（C4），提供紧凑且物理可解释的触觉编码。The tactile representation applies PCA to reduce the dimensionality of the marker deformation field; the first four principal components correspond respectively to tangential forces (C1, C3), torsional moment (C2) and normal force (C4), giving a compact and physically interpretable tactile encoding.

03 实验Experiments

在三项接触丰富的真实机器人任务上评估：Peeling（剥皮）、Wiping（擦拭）和 Bimanual Lifting（双臂抬升）。每项任务设置三种扰动条件：无扰动、接触前扰动、接触后扰动，以评估反应式闭环能力。基线包括原始 Diffusion Policy（DP）以及加入触觉图像/嵌入的 DP 变体。Evaluated on three contact-rich real-robot tasks: Peeling, Wiping and Bimanual Lifting. Each task is run under three disturbance conditions — no disturbance, pre-contact disturbance and post-contact disturbance — to assess reactive closed-loop capability. Baselines include the original Diffusion Policy (DP) and DP variants augmented with tactile images/embeddings.

三项实验任务。*Peeling*：用刮刀从物体表面剥离薄层，需精确控制法向力；*Wiping*：擦拭表面污渍，需稳定接触力；*Bimanual Lifting*：双臂协同夹持并提起软/硬杯，需实时感知夹持力以防止变形或脱落。The three experimental tasks. *Peeling*: peel a thin layer off an object's surface with a scraper, which demands precise control of the normal force; *Wiping*: wipe stains off a surface, which demands a stable contact force; *Bimanual Lifting*: two arms cooperatively clamp and lift a soft/rigid cup, which demands real-time sensing of the clamping force to prevent deformation or slipping.

Peeling 任务（TABLE II）Peeling Task (TABLE II)

方法Method	无扰动No Disturbance	接触前扰动Pre-Contact Disturbance	接触后扰动Post-Contact Disturbance	综合得分Overall Score
DP	0.56	0.58	0.19	0.44
DP w. tactile img.	0.60	0.49	0.16	0.41
DP w. tactile emb.	0.48	0.55	0.15	0.39
RDP (GelSight)	0.98	0.93	0.80	0.90
RDP (MCTac)	1.00	0.84	0.79	0.88
RDP (Force)	0.99	0.98	0.88	0.95

Wiping 任务（TABLE III）Wiping Task (TABLE III)

方法Method	无扰动No Disturbance	接触前扰动Pre-Contact Disturbance	接触后扰动Post-Contact Disturbance	综合得分Overall Score
DP	0.75	0.70	0.25	0.57
DP w. tactile emb.	0.60	0.75	0.15	0.50
RDP (GelSight)	0.85	0.95	0.50	0.77
RDP (Force)	0.95	0.85	0.80	0.87

Bimanual Lifting 任务（TABLE IV）Bimanual Lifting Task (TABLE IV)

方法Method	软杯夹持Soft Cup Grasp	软杯提升Soft Cup Lift	软杯得分Soft Cup Score	硬杯夹持Rigid Cup Grasp	硬杯提升Rigid Cup Lift	硬杯得分Rigid Cup Score	综合得分Overall Score
DP	0%	0%	0.00	0%	0%	0.00	0.00
DP w. tactile emb.	10%	10%	0.10	20%	10%	0.05	0.08
RDP (GelSight + MCTac)	100%	100%	0.55	90%	80%	0.40	0.48
RDP (Force)	100%	90%	0.80	90%	90%	0.60	0.70

RDP 推理过程可视化。红色（左）和蓝色（右）点表示慢策略预测的 action chunk；快策略根据触觉反馈在 latent 空间内自回归修正，实现亚毫米级精度的闭环调整，使机器人在受到扰动后能快速恢复正确接触姿态。Visualization of the RDP inference process. The red (left) and blue (right) points are the action chunk predicted by the slow policy; the fast policy corrects it autoregressively in the latent space from tactile feedback, achieving closed-loop adjustment at sub-millimeter precision so that the robot quickly recovers the correct contact pose after a disturbance.

消融实验（TABLE V）Ablation Study (TABLE V)

消融研究在 Peeling 任务上验证了 slow-fast 设计的必要性。Temporal ensemble 对平滑系数 τ 极度敏感：τ=0.2 时抓取成功率仅 30%，τ=0.5 时降至 0%，τ=0.8 时才达到 100%——而在接触后扰动条件下得分仍仅 0.15，远不及 RDP（GelSight）的 0.50。此外，将 action chunk 大小从 8 缩短至 2 会使抓取成功率从 100% 骤降至 20%，说明慢策略的长程规划能力至关重要。The ablation study on the Peeling task confirms the necessity of the slow-fast design. Temporal ensemble is extremely sensitive to the smoothing coefficient τ: the grasp success rate is only 30% at τ=0.2, falls to 0% at τ=0.5, and reaches 100% only at τ=0.8 — yet its score under post-contact disturbance is still merely 0.15, far below the 0.50 of RDP (GelSight). Moreover, shortening the action chunk size from 8 to 2 makes the grasp success rate plunge from 100% to 20%, showing that the long-horizon planning ability of the slow policy is essential.

配置Configuration	抓取成功率Grasp Success Rate	接触后扰动得分Post-Contact Disturbance Score
DP w. tactile emb. (chunk=8)	100%	0.15
DP w. tactile emb. (chunk=2)	20%	0.10
DP w. temporal ensemble (τ=0.2)	30%	0.05
DP w. temporal ensemble (τ=0.5)	0%	0.00
DP w. temporal ensemble (τ=0.8)	100%	0.15
RDP (GelSight)	100%	0.50

跨传感器兼容性Cross-Sensor Compatibility

RDP 在 Peeling 任务上使用三种不同传感器均取得强劲性能：GelSight Mini（综合得分 0.90）、MCTac（0.88）、关节力矩传感器（0.95），验证了 3D 形变场统一表示的跨传感器泛化能力。RDP attains strong performance on the Peeling task with three different sensors: GelSight Mini (overall score 0.90), MCTac (0.88) and a joint torque sensor (0.95), validating the cross-sensor generalization of the unified 3D deformation field representation.

04 局限性Limitations

说明：以下局限性均为论文作者明确陈述（stated），非推断。Note: All limitations listed below are explicitly stated by the paper's authors, not inferred.

AR 反馈直觉性不如直接手部操作AR feedback is less intuitive than direct hand operation

TactAR 的 AR 触觉可视化"not as intuitive or efficient as direct human-hand operations"，对新用户仍存在认知负担。TactAR's AR tactile visualization is "not as intuitive or efficient as direct human-hand operations" and still imposes a cognitive burden on new users.

仅支持两指夹爪，不适用于灵巧手Supports only two-finger grippers, not dexterous hands

当前系统"designed for two-finger grippers"，无法直接迁移至多指灵巧手或非标准末端执行器，限制了任务多样性。The current system is "designed for two-finger grippers" and cannot be transferred directly to multi-fingered dexterous hands or non-standard end-effectors, which limits task diversity.

快策略无法处理高频图像输入The fast policy cannot handle high-frequency image input

Fast policy 目前只能接收"high-frequency tactile / force input"，不支持高频视觉流，限制了纯视觉场景下的响应速度。The fast policy can currently take only "high-frequency tactile / force input" and does not support high-frequency visual streams, which limits responsiveness in vision-only settings.

当前限于单任务场景Currently restricted to single-task scenarios

算法目前"currently restricted to single-task scenarios"，尚未支持多任务或语言条件化泛化，是未来工作的重要方向。The algorithm is "currently restricted to single-task scenarios" and does not yet support multi-task or language-conditioned generalization, an important direction for future work.