我们应该从基于采样的规划器中学习接触丰富的操作策略吗？Should We Learn Contact-Rich Manipulation Policies from Sampling-Based Planners?

01 动机Motivation

Behavior Cloning（BC）在机器人操作领域取得了巨大成功，但绝大多数工作依赖人工遥操作收集演示数据。对于需要多接触点协调配合的接触丰富操作任务（如双臂搬运、灵巧手重定向），遥操作接口的局限性使得高质量演示的采集极为困难。Behavior Cloning (BC) has achieved great success in robotic manipulation, yet the vast majority of work relies on human teleoperation to collect demonstrations. For contact-rich manipulation tasks that require coordinating multiple contact points (such as bimanual carrying or dexterous in-hand reorientation), the limitations of teleoperation interfaces make high-quality demonstrations extremely hard to collect.

"我们的分析揭示，以 RRT（Rapidly Exploring Random Tree）为代表的流行基于采样的规划器虽然在运动规划中效率极高，却会产生具有不利的高熵特性的演示数据。""Our analysis reveals that popular sampling-based planners such as RRT (Rapidly Exploring Random Tree), although highly efficient for motion planning, produce demonstration data with unfavorably high entropy."
— 原文摘要— From the abstract

核心研究问题：能否用基于模型的规划与优化替代人工遥操作，为接触丰富的灵巧操作任务生成训练数据？RRT 等基于采样的规划器真的适合作为 BC 的数据来源吗？Core research question: can model-based planning and optimization replace human teleoperation to generate training data for contact-rich dexterous manipulation tasks? Are sampling-based planners such as RRT really suitable as a data source for BC?

为什么 RRT 演示会"难以学习"？Why are RRT demonstrations "hard to learn"?

作者从三个维度量化了演示熵：The authors quantify demonstration entropy along three dimensions:

• 速度方向熵（Velocity Direction Entropy）：RRT 规划路径迂回，在状态空间中各位置的物体运动方向高度发散，导致在同一状态下策略需要"预测"多个截然不同的动作；Velocity Direction Entropy: RRT plans circuitous paths, so object motion directions are highly divergent across locations in state space, forcing the policy to "predict" several completely different actions in one and the same state;
• 目标进展分布（Progress Toward Goal）：RRT 演示的目标进展分布分散，甚至偶尔出现负进展（远离目标），而 Greedy Search 演示则单调收敛；Progress Toward Goal: the goal-progress distribution of RRT demonstrations is dispersed and occasionally even exhibits negative progress (moving away from the goal), whereas Greedy Search demonstrations converge monotonically;
• 重新抓取熵（Regrasp Entropy）：RRT 的重抓取熵几乎始终接近 1，意味着在任意时刻执行重抓取的概率约为 50%，策略无从判断何时应执行重抓取动作。Regrasp Entropy: the regrasp entropy of RRT stays close to 1 almost throughout, meaning the probability of executing a regrasp at any given moment is about 50%, leaving the policy no way to tell when a regrasp should be performed.

02 方法Method

作者提出一套以一致性优先的数据生成流程，结合为接触操作定制的低熵规划器与基于扩散模型的目标条件 BC 框架，以实现从规划数据到可部署策略的完整通道。The authors propose a consistency-first data-generation pipeline that combines a low-entropy planner tailored to contact manipulation with a diffusion-based goal-conditioned BC framework, providing a complete path from planning data to a deployable policy.

核心问题：高熵演示何以产生？Core question: how does high-entropy demonstration data arise?

RRT 的核心机制是通过随机采样在状态空间中快速探索，这赋予了它出色的全局覆盖能力，但同时导致：对同一起始状态可能产生多条截然不同的解路径，聚合后形成高熵数据集。此外，RRT 的分叉探索方式使得接触切换（regrasp）的时机高度不确定。The core mechanism of RRT is fast exploration of the state space through random sampling, which grants it excellent global coverage but at the same time means that several completely different solution paths may be produced for the same start state, and aggregating them yields a high-entropy dataset. Moreover, RRT's branching exploration makes the timing of contact switches (regrasp) highly uncertain.

面向 IiwaBimanual 的 Greedy SearchGreedy Search for IiwaBimanual

对于双臂圆柱体旋转任务，作者设计了一种贪婪搜索规划器：在不采样子目标的情况下，迭代求解接触优化问题；仅在遭遇关节极限时才采样新的抓取姿态。该规划器以接触稳定性为首要目标，确保每一步都朝着目标方向稳步推进，从而产生低熵、单调收敛的演示。For the bimanual cylinder rotation task, the authors design a greedy search planner: it iteratively solves a contact optimization problem without sampling subgoals, and samples a new grasp pose only when a joint limit is encountered. The planner treats contact stability as its primary objective and ensures that every step advances steadily toward the goal, thereby producing low-entropy, monotonically converging demonstrations.

面向 AllegroHand 的 PRM 规划器A PRM planner for AllegroHand

对于 16-DoF 灵巧手方块重定向任务，作者使用概率路线图（PRM）规划器：以 24 个标准朝向作为图节点，通过预计算的 primitives（PitchPlus90、YawPlus45、YawMinus45）连接节点。这种结构"同时保证了完备性与一致性"。部署时采用混合策略（Hybrid Policy）：主策略（1000 条演示训练）执行任意目标重定向，调整策略（5000 条演示训练）负责从最近标准朝向到精确目标朝向的精细调节，两者配合使成功率提升约 10%。For the 16-DoF dexterous-hand cube reorientation task, the authors use a probabilistic roadmap (PRM) planner: 24 canonical orientations serve as graph nodes, connected through precomputed primitives (PitchPlus90, YawPlus45, YawMinus45). This structure "guarantees both completeness and consistency". At deployment a Hybrid Policy is adopted: the main policy (trained on 1000 demonstrations) performs reorientation to arbitrary goals, while the adjustment policy (trained on 5000 demonstrations) handles the fine adjustment from the nearest canonical orientation to the exact goal orientation; together they raise the success rate by about 10%.

基于扩散模型的目标条件行为克隆Diffusion-based goal-conditioned behavior cloning

策略学习采用 DDPM（Denoising Diffusion Probabilistic Model） 作为动作头，输入历史状态序列与目标状态，输出动作序列（而非单步动作），以提升时序一致性。通过 Feature-wise Linear Modulation（FiLM） 对观测与目标进行条件化，再将嵌入融合到去噪网络的各层中。训练使用 AdamW 优化器，学习率 1×10⁻⁴，批大小 256，训练 50 个 epoch。此外引入 Hindsight Goal Relabeling，大幅扩充有效训练样本。Policy learning adopts a DDPM (Denoising Diffusion Probabilistic Model) as the action head; it takes a history of states together with the goal state as input and outputs an action sequence rather than a single-step action, so as to improve temporal consistency. Observations and goals are conditioned via Feature-wise Linear Modulation (FiLM), and the embeddings are then fused into every layer of the denoising network. Training uses the AdamW optimizer with learning rate 1×10⁻⁴, batch size 256, for 50 epochs. In addition, Hindsight Goal Relabeling is introduced to greatly expand the pool of effective training samples.

03 实验Experiments

实验覆盖两个接触丰富操作任务：IiwaBimanual（两台 7-DoF 机械臂协作旋转直径 0.6m 圆柱体 180°）和 AllegroHand（16-DoF 灵巧手将 6cm 立方体重定向到目标朝向）。在仿真中全面评估，并在真实硬件上进行零样本迁移测试。The experiments cover two contact-rich manipulation tasks: IiwaBimanual (two 7-DoF arms cooperatively rotating a cylinder of 0.6m diameter by 180°) and AllegroHand (a 16-DoF dexterous hand reorienting a 6cm cube to a target orientation). Both are evaluated comprehensively in simulation and tested for zero-shot transfer on real hardware.

实验一：规划器类型 × 演示数量对 IiwaBimanual 的影响（TABLE I）Experiment 1: effect of planner type × number of demonstrations on IiwaBimanual (TABLE I)

评价指标：在 100 个随机初始位置上测试，位置误差 < 0.1m 且朝向误差 < 0.2 rad 视为成功。Evaluation metric: tested from 100 random initial positions; a trial counts as a success when the position error is < 0.1m and the orientation error is < 0.2 rad.

核心发现：Greedy Search 仅需 100 条演示即可达到 99% 成功率，而 Contact-RRT 即使增加到 5000 条演示也只能达到 84%，甚至低于 1000 条演示时的 88%，表明高熵演示不仅样本效率低下，且增大数据量并不能有效缓解问题。Key finding: Greedy Search reaches a 99% success rate with only 100 demonstrations, whereas Contact-RRT attains only 84% even when scaled up to 5000 demonstrations, which is even lower than the 88% it reaches with 1000 demonstrations, showing that high-entropy demonstrations are not only sample-inefficient but that enlarging the dataset does not effectively alleviate the problem.

实验二：仿真与真机综合性能（TABLE II）Experiment 2: overall performance in simulation and on hardware (TABLE II)

仿真评估取最优 checkpoint 结果；真机硬件测试为零样本迁移（无额外微调）。Simulation evaluation reports the best-checkpoint result; real-hardware testing is zero-shot transfer (without any extra fine-tuning).

关键消融：混合策略的贡献Key ablation: the contribution of the hybrid policy

对于 AllegroHand 任务，引入调整策略（Adjustment Policy）使最终成功率提升约 10%。主策略负责粗调，从任意初始朝向到达最近的标准朝向；调整策略负责精调，从标准朝向到精确目标朝向。两者的分工使策略在 Hard 变体（目标均匀采样自 SO(3)）上也能达到稳健表现。For the AllegroHand task, introducing an Adjustment Policy raises the final success rate by about 10%. The main policy handles coarse adjustment, moving from an arbitrary initial orientation to the nearest canonical orientation; the adjustment policy handles fine adjustment, going from the canonical orientation to the exact goal orientation. This division of labor lets the policy remain robust even on the Hard variant, whose goals are sampled uniformly from SO(3).

Sim-to-Real 迁移细节Sim-to-real transfer details

真机与仿真间存在系统误差：物体实际质量 1.25 kg（仿真为 1.0 kg），实际直径 0.59m（仿真为 0.6m）。尽管如此，IiwaBimanual 真机成功率达 90%，AllegroHand 达 62.5%，均实现了零样本硬件迁移，验证了低熵演示对策略泛化能力的积极作用。Systematic discrepancies exist between hardware and simulation: the object's actual mass is 1.25 kg (1.0 kg in simulation) and its actual diameter is 0.59m (0.6m in simulation). Even so, IiwaBimanual reaches a 90% success rate on hardware and AllegroHand 62.5%, both realizing zero-shot hardware transfer and confirming the positive effect of low-entropy demonstrations on policy generalization.

04 局限性Limitations