RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation

01 动机 · MotivationMotivation

双臂协作操作（bimanual manipulation）是机器人迈向通用操作能力的关键一步，却面临严重的数据瓶颈：现有数据集大多针对单一机器人平台、任务类型单一、缺乏高质量标注，且规模远不足以支撑大模型预训练。Bimanual manipulation is a key step toward general-purpose robotic manipulation, yet it faces a severe data bottleneck: existing datasets mostly target a single robot platform, cover a single task type, lack high-quality annotation, and are far too small to support large-model pre-training.

"Existing datasets are often limited in scale, diversity, or quality, failing to capture the breadth of bimanual manipulation tasks required for general robot learning."

RoboCOIN 数据集总览 — **图 1 · RoboCOIN 总览。**数据集支持 15 种不同机器人平台，涵盖 421 项双臂任务，分布于 16 种真实场景（居家、商业、工作），收录 18 万余条高质量遥操作示教轨迹，并提供一键安装接口（one-click installation）。**Figure 1 · RoboCOIN overview.** The dataset supports 15 different robot platforms, covers 421 bimanual tasks across 16 real-world scenes (home, commercial, work), collects more than 180000 high-quality teleoperated demonstration trajectories, and provides a one-click installation interface.

180K+高质量示教轨迹high-quality demonstrations

421双臂操作任务bimanual manipulation tasks

15机器人平台robot platforms

432操作对象类别object categories

**图 2 · 数据采集平台类型。**三大类平台：双臂机器人（dual-arm，如 AgileX Cobot Magic，配平行夹爪）、半人形机器人（half-humanoid，如 Realman RMC-AIDA-L，配外骨骼遥操作）以及人形机器人（humanoid，如 Unitree G1edu-u3，配动作捕捉设备）。**Figure 2 · Types of data collection platforms.** Three platform categories: dual-arm robots (e.g., AgileX Cobot Magic, with parallel grippers), half-humanoid robots (e.g., Realman RMC-AIDA-L, with exoskeleton teleoperation), and humanoid robots (e.g., Unitree G1edu-u3, with motion capture devices).

为什么需要 RoboCOIN？Why is RoboCOIN needed?

规模不足：现有双臂数据集最多仅有数千条轨迹，与单臂数据集的百万级规模差距巨大。Insufficient scale: existing bimanual datasets contain at most a few thousand trajectories, far short of the million-scale size of single-arm datasets.
平台单一：跨本体（cross-embodiment）泛化研究缺乏覆盖多种硬件形态的统一数据集。Single platform: research on cross-embodiment generalization lacks a unified dataset covering diverse hardware morphologies.
标注缺失：现有数据集大多仅有低层关节状态，缺乏语义级别的分层标注，难以支持需要精细推理的策略学习。Missing annotation: most existing datasets provide only low-level joint states and lack semantic-level hierarchical annotation, making it hard to support policy learning that requires fine-grained reasoning.
质量良莠不齐：遥操作本身会引入操作员偏差，缺乏系统性的数据质量过滤机制。Uneven quality: teleoperation itself introduces operator bias, and systematic data-quality filtering mechanisms are missing.

**图 3 · 数据集统计概览。**15 个平台分三类，16 种场景（居家 47.1%、商业 18.6%、工作 34.3%），39 种协作动作类型，以及 432 种操作对象（刚性 83.9%、铰链 4.2%、可变形 11.9%）。**Figure 3 · Dataset statistics.** 15 platforms in three categories, 16 scenes (home 47.1%, commercial 18.6%, work 34.3%), 39 types of collaborative action, and 432 object categories (rigid 83.9%, articulated 4.2%, deformable 11.9%).

02 方法 · MethodMethod

RoboCOIN 由三部分构成：大规模多平台双臂数据集本身、层级能力金字塔（Capability Pyramid）标注体系，以及 CoRobot 数据处理管线——其核心是机器人轨迹标记语言（Robot Trajectory Markup Language, RTML），用于自动化质量验证与过滤。RoboCOIN consists of three parts: the large-scale multi-platform bimanual dataset itself, the hierarchical Capability Pyramid annotation scheme, and the CoRobot data processing pipeline, whose core is the Robot Trajectory Markup Language (RTML) for automated quality verification and filtering.

分层能力金字塔（Hierarchical Capability Pyramid）Hierarchical Capability Pyramid

Trajectory-level：整条轨迹的全局语义描述（任务名、场景类型、涉及对象）。Trajectory-level: global semantic description of the whole trajectory (task name, scene type, objects involved).
Segment-level：将轨迹分解为若干子任务，给出时序对齐的自然语言标注。Segment-level: decomposes the trajectory into subtasks with temporally aligned natural-language annotations.
Frame-level：逐帧记录关节角度、末端执行器位姿、夹爪开合等运动学参数，配合物体检测器自动生成，人工精修。Frame-level: records per-frame kinematic parameters such as joint angles, end-effector poses and gripper opening, generated automatically with an object detector and refined manually.

CoRobot 数据处理管线The CoRobot Data Processing Pipeline

CoRobot 处理管线 — **图 5 · CoRobot 三组件示意。**（1）RTML 轨迹评估与过滤；（2）半自动分层标注管线（目标检测器 + 规则工具 + 人工精修）；（3）基于 LeRobot 框架的统一机器人接口，支持多平台坐标对齐与数据集原子化管理。**Figure 5 · The three components of CoRobot.** (1) RTML trajectory evaluation and filtering; (2) a semi-automatic hierarchical annotation pipeline (object detector + rule-based tools + manual refinement); (3) a unified robot interface built on the LeRobot framework, supporting multi-platform coordinate alignment and atomic dataset management.

RTML：机器人轨迹标记语言RTML: Robot Trajectory Markup Language

RTML 以 YAML 格式定义两类约束，对每条轨迹进行自动化验证：RTML defines two classes of constraints in YAML format and validates every trajectory automatically:

全局约束（Global Constraints）Global Constraints

工作空间边界（workspace boundaries）Workspace boundaries
速度 / 加速度上限（velocity / acceleration limits）Velocity / acceleration limits
轨迹时长范围（duration bounds）Trajectory duration bounds

局部相位约束（Local Phase Constraints）Local Phase Constraints

相位特定工作空间限制Phase-specific workspace limits
运动学细化（kinematic refinements）Kinematic refinements
空闲臂控制（idle-arm controls）Idle-arm controls
姿态约束 + 时序参数Pose constraints + timing parameters

双臂协作动作分类体系（39 种动作）Taxonomy of Bimanual Collaborative Actions (39 actions)

39 种双臂协作动作归为两大类：The 39 bimanual collaborative actions fall into two major categories:

低协作任务（Low-Collaboration, 58.9%）：双臂以较顺序方式运作，同步交互少。Low-Collaboration tasks (58.9%): the two arms act in a largely sequential manner with little synchronous interaction.
高协作任务（High-Collaboration, 41.1%）：双臂需紧密同步，部分或完全并行运动。High-Collaboration tasks (41.1%): the two arms must be tightly synchronized, moving partly or fully in parallel.

动作按功能分为四类：通用操作（General Manipulation）10 种、对象状态改变（Object State Change）9 种、对象关系改变（Object Relation Change）9 种、任务特定动作（Task-Specific）11 种。By function the actions fall into four groups: General Manipulation (10), Object State Change (9), Object Relation Change (9), and Task-Specific (11).

03 实验 · ExperimentsExperiments

实验在三类场景下验证 RoboCOIN 的价值：（1）RoboTwin 2.0 仿真基准上的跨本体策略迁移，（2）Realman RMC-AIDA-L 真实机器人上的分层标注收益，（3）Unitree G1edu-u3 上的 RTML 数据质量过滤效果。基准策略均为 π₀ 或 GR00T N1.5。Experiments validate the value of RoboCOIN in three settings: (1) cross-embodiment policy transfer on the RoboTwin 2.0 simulation benchmark, (2) the benefit of hierarchical annotation on the Realman RMC-AIDA-L real robot, and (3) the effect of RTML data quality filtering on Unitree G1edu-u3. The baseline policies are π₀ and GR00T N1.5.

仿真基准：跨本体迁移（RoboTwin 2.0）Simulation Benchmark: Cross-Embodiment Transfer (RoboTwin 2.0)

跨本体迁移实验结果 — **图 6 · π₀ 与 π₀-RoboCOIN 对比（RoboTwin 2.0）。**在 ARX-X5（14 任务）和 Franka Emika Panda（11 任务）两种平台上，使用 RoboCOIN 数据预训练后均获得显著提升。高协作任务提升更为突出。**Figure 6 · π₀ versus π₀-RoboCOIN (RoboTwin 2.0).** On both ARX-X5 (14 tasks) and Franka Emika Panda (11 tasks), pre-training on RoboCOIN data yields significant gains, which are most pronounced on high-collaboration tasks.

平台 / PlatformPlatform	基线（π₀）Baseline (π₀)	+RoboCOIN（π₀-RoboCOIN）+RoboCOIN (π₀-RoboCOIN)	相对提升Relative gain
ARX-X5（全部任务）ARX-X5 (all tasks)	17.3%	30.4%	+75.7%
ARX-X5（高协作任务）ARX-X5 (high-collaboration tasks)	—	—	+94.1%
Franka Emika Panda（全部任务）Franka Emika Panda (all tasks)	8.5%	26.6%	+212.9%

真实机器人：分层标注收益（Realman + π₀）Real Robot: Benefit of Hierarchical Annotation (Realman + π₀)

测试条件Test condition	基线（无 Pyramid）Baseline (no Pyramid)	+Capability Pyramid	相对提升Relative gain
分布内（In-distribution）In-distribution	28%	43%	+53.6%
分布外（Out-of-distribution）Out-of-distribution	22.5%	57.5%	+155.6%
OOD 性能下降幅度OOD performance drop	-43.8%	-17.9%	鲁棒性显著提升Robustness clearly improved

RTML 数据质量过滤（Unitree G1edu-u3 + GR00T N1.5）RTML Data Quality Filtering (Unitree G1edu-u3 + GR00T N1.5)

RTML 质量过滤分析 — **图 7 · RTML 过滤效果分析。**RTML 过滤掉 35.3% 的低质量轨迹；失败相位分布：抓取（grasping）52.7%、移动（moving）17.8%；失效原因：速度违规（velocity violations）46.2%、时长违规（duration violations）24.5%。GR00T-Fine（相位约束）相比原始数据提升 22.2%。**Figure 7 · Analysis of RTML filtering.** RTML removes 35.3% of low-quality trajectories; failing phases: grasping 52.7%, moving 17.8%; failure causes: velocity violations 46.2%, duration violations 24.5%. GR00T-Fine (phase constraints) improves over the raw data by 22.2%.

数据设置Data setting	任务成功率（相对原始数据）Task success rate (relative to raw data)
原始数据（Raw data）Raw data	基准baseline
RTML 过滤后After RTML filtering	提升（移除 35.3% 低质量轨迹）improved (35.3% of low-quality trajectories removed)
GR00T-Fine（相位级约束）GR00T-Fine (phase-level constraints)	+22.2%

消融实验总结Ablation Summary

消融结果表明：（1）Capability Pyramid 的三级标注均有独立贡献，去掉任一层级均会导致性能下降；（2）RTML 过滤对 OOD 泛化的帮助尤为明显，说明数据质量比数据数量更重要；（3）跨平台预训练（cross-embodiment pre-training）的收益在高协作任务上更为突出，表明复杂协调动作更依赖大规模先验。The ablations show that: (1) all three levels of the Capability Pyramid contribute independently, and removing any level degrades performance; (2) RTML filtering is especially helpful for OOD generalization, indicating that data quality matters more than data quantity; (3) the gains of cross-embodiment pre-training are more pronounced on high-collaboration tasks, suggesting that complex coordinated actions rely more heavily on large-scale priors.

04 局限性 · LimitationsLimitations

Note: 以下三点局限性均为作者在论文中明确陈述（stated by the authors）。All three limitations below are explicitly stated by the authors in the paper.

遥操作偏差（Teleoperation Bias）Teleoperation Bias

作者指出："Reliance on teleoperation may introduce inherent operator biases, leading to inter-operator variability in trajectory patterns." 不同操作员风格各异，导致同一任务的示教轨迹之间存在较大方差，可能影响策略学习的稳定性与泛化能力。The authors note: "Reliance on teleoperation may introduce inherent operator biases, leading to inter-operator variability in trajectory patterns." Operator styles differ, so demonstrations of the same task vary considerably, which may affect the stability and generalization of policy learning.

标注可扩展性瓶颈（Annotation Scalability Bottleneck）Annotation Scalability Bottleneck

"Manual annotation presents a scalability bottleneck due to its high labor costs and potential for subjectivity, which may impact data consistency." 尽管采用了半自动标注管线，人工精修环节仍然是扩展数据规模的主要瓶颈，且主观性难以完全消除。"Manual annotation presents a scalability bottleneck due to its high labor costs and potential for subjectivity, which may impact data consistency." Although a semi-automatic annotation pipeline is used, manual refinement remains the main bottleneck for scaling up the data, and subjectivity cannot be fully eliminated.

RTML 灵活性受限（RTML Flexibility Constraints）RTML Flexibility Constraints

"The current RTML framework depends heavily on expert-defined heuristics, potentially constraining its flexibility when applied to niche tasks or novel robotic platforms." RTML 的约束规则由专家手工设计，对新型机器人平台或非常规任务的适用性有限，需要额外的专家介入来定义新规则。"The current RTML framework depends heavily on expert-defined heuristics, potentially constraining its flexibility when applied to niche tasks or novel robotic platforms." The RTML constraint rules are hand-crafted by experts, so their applicability to novel robot platforms or unconventional tasks is limited and additional expert effort is needed to define new rules.