Alexander Khazatsky*, Karl Pertsch* et al.（共 101 位作者，13 家机构）· Stanford, UC Berkeley, CMU 等Alexander Khazatsky*, Karl Pertsch* et al. (101 authors, 13 institutions) · Stanford, UC Berkeley, CMU, etc.
DROID 是一个包含 76,000 条机器人演示轨迹（350 小时）的大规模公开数据集，由来自北美、亚洲和欧洲的 50 名采集者在 564 个真实场景中收集，覆盖 86 类任务。
在此数据集上联合训练的 diffusion policy，分布内成功率相较基线提升 22%，分布外提升 17%。DROID is a large-scale open dataset of 76,000 robot demonstration trajectories (350 hours), collected by 50 operators from North America, Asia and Europe across 564 real-world scenes and covering 86 task categories. A diffusion policy co-trained on this dataset improves the in-distribution success rate by 22% and the out-of-distribution success rate by 17% over the baselines.
通才型机器人操作策略的关键瓶颈是数据——现有数据集几乎全部在少数受控实验室环境中采集，场景单一、任务有限，导致策略在新场景下的泛化极差。
如何以可扩展的方式采集真实多样的操作数据，是迈向通用机器人的核心挑战。The key bottleneck for generalist robot manipulation policies is data: almost all existing datasets are collected in a small number of controlled laboratory environments, with monotonous scenes and limited tasks, which leaves policies generalizing very poorly to new scenes. How to collect realistic and diverse manipulation data in a scalable way is the central challenge on the road to general-purpose robots.
"Most existing datasets are mostly trained on data collected in a small number of environments with limited scene and task diversity… collecting robot data remains logistically challenging and expensive outside of controlled lab settings."
DROID 硬件平台：Franka Panda 7-DoF 机械臂 + Robotiq 2F-85 夹爪，配备两个 ZED 2 外部立体相机和一个腕部 ZED Mini 相机，以及 Oculus Quest 2 遥操作手柄，全套硬件安装于可移动升降桌上，便于在任意真实场景中部署采集。The DROID hardware platform: a Franka Panda 7-DoF arm with a Robotiq 2F-85 gripper, equipped with two ZED 2 external stereo cameras and one wrist-mounted ZED Mini camera, together with an Oculus Quest 2 teleoperation controller; the whole rig is mounted on a mobile height-adjustable desk so that it can be deployed for collection in arbitrary real-world scenes.
76,000成功演示轨迹successful demonstrations
350 h总交互时长total interaction time
564不同场景数量distinct scenes
86任务类别（动词）task categories (verbs)
02方法Method
DROID 的核心贡献是建立一套可在任意真实环境（办公室、厨房、户外等）快速部署的标准化采集系统，并在全球多个机构同步运行，以规模化方式覆盖真实世界的场景与任务多样性。The core contribution of DROID is a standardized collection system that can be deployed rapidly in arbitrary real environments (offices, kitchens, outdoors and so on) and run in parallel at many institutions worldwide, covering real-world scene and task diversity at scale.
硬件与数据采集流程Hardware and Data Collection Pipeline
每套 DROID 采集站包括：Franka Panda 机械臂（7-DoF）、两个可调角度的 ZED 2 外部立体相机（分辨率 1280×720，15 Hz）、一个腕部 ZED Mini 相机，以及 Oculus Quest 2 头显用于遥操作。
数据在 12 个月内由来自 18 个研究室、13 个机构的 50 名采集者在北美、亚洲和欧洲的 52 栋楼宇中完成，每个场景平均采集约 100 条轨迹（约 20 分钟）。
每条轨迹记录：3 路立体 RGB 视频流、7D 关节位姿/速度、6D 末端执行器位姿与速度、1D 夹爪状态，以及 1–3 条众包自然语言标注。Each DROID collection station comprises a Franka Panda arm (7-DoF), two angle-adjustable ZED 2 external stereo cameras (1280×720 resolution, 15 Hz), one wrist-mounted ZED Mini camera, and an Oculus Quest 2 headset for teleoperation. The data was gathered over 12 months by 50 operators from 18 labs at 13 institutions, in 52 buildings across North America, Asia and Europe, with about 100 trajectories (roughly 20 minutes) collected per scene on average. Every trajectory records 3 stereo RGB video streams, 7D joint positions/velocities, 6D end-effector pose and velocity, 1D gripper state, and 1–3 crowdsourced natural-language annotations.
任务分布热力图：横轴为操作动词（如 pick up, place, open…），纵轴为操作对象，颜色深浅代表轨迹数量。DROID 覆盖了"pick up"、"place"、"push"、"open/close"等多种动词，以及大量不同类别的日常物品，展现出远超既有数据集的任务多样性。Task distribution heatmap: the horizontal axis lists manipulation verbs (e.g. pick up, place, open…), the vertical axis the manipulated objects, and colour intensity encodes the number of trajectories. DROID covers verbs such as "pick up", "place", "push" and "open/close", along with a wide range of everyday object categories, exhibiting task diversity far beyond that of existing datasets.
场景多样性设计Designing for Scene Diversity
DROID 刻意要求每个采集者在不同建筑、不同房间布置下工作，并记录场景类型（实验室、厨房、办公室、餐厅等）及相机外参标定矩阵，以支持后续的跨场景泛化研究。数据集共覆盖 564 个唯一场景，远超此前同类数据集（最大约 24–33 个场景）。DROID deliberately requires every operator to work in different buildings and different room layouts, and records the scene type (lab, kitchen, office, dining room and so on) together with the camera extrinsic calibration matrices, so as to support later research on cross-scene generalization. In total the dataset covers 564 unique scenes, far more than earlier comparable datasets (at most about 24–33 scenes).
场景类型分布：数据集中场景涵盖实验室、厨房、办公室、休息室、图书馆、户外等多种环境，每种场景下均有大量轨迹覆盖，体现了真实世界"in-the-wild"采集的多样性。Distribution of scene types: the scenes in the dataset span labs, kitchens, offices, lounges, libraries, outdoor spaces and other environments, each covered by a large number of trajectories, reflecting the diversity of genuine "in-the-wild" collection.
策略训练框架（Co-training）Policy Training Framework (Co-training)
为验证数据集价值，作者训练了一个基于 diffusion policy 的通用操作策略：ResNet-50 视觉编码器（ImageNet 预训练）+ 冻结 DistilBERT 语言嵌入 + U-Net 扩散头，生成 16 步动作序列，输出绝对末端执行器平移、旋转与夹爪动作。
训练时采用 50/50 batch mixing：50% 目标任务数据 + 50% DROID 数据联合训练。To demonstrate the value of the dataset, the authors train a general manipulation policy based on diffusion policy: a ResNet-50 visual encoder (ImageNet pre-trained) + frozen DistilBERT language embeddings + a U-Net diffusion head, generating 16-step action sequences and outputting absolute end-effector translation, rotation and gripper actions. Training uses 50/50 batch mixing: 50% target-task data + 50% DROID data trained jointly.
03实验Experiments
作者在 6 个真实机器人操作任务上评估了 DROID co-training 的效果，任务从短时程（如关闭华夫饼机）到长时程（如煮扁豆）不等，并在分布内（in-distribution）和分布外（OOD）两种设定下测试泛化能力。The authors evaluate the effect of DROID co-training on 6 real-robot manipulation tasks, ranging from short-horizon (e.g. closing a waffle maker) to long-horizon (e.g. cooking lentils), and test generalization under both the in-distribution and the out-of-distribution (OOD) setting.
评估任务Evaluation Tasks
Close Waffle Maker（短时程，实验室，70 条演示）Close Waffle Maker (short-horizon, lab, 70 demonstrations)
Place Chips on Plate（短时程，实验室，50 条演示）Place Chips on Plate (short-horizon, lab, 50 demonstrations)
Put Apple in Pot（中时程，实验室，60 条演示）Put Apple in Pot (medium-horizon, lab, 60 demonstrations)
+22%（绝对值，vs. 最优基线）+22% (absolute, vs. best baseline)
+17%（绝对值，vs. 最优基线）+17% (absolute, vs. best baseline)
注：论文以平均成功率±标准误差展示，未逐任务列出精确数值，上表数字直接引自论文原文。Note: the paper reports mean success rates ± standard error and does not list exact per-task values; the numbers in the table above are quoted directly from the paper.
Co-training 效果对比（Figure 6）：在 6 个任务上，DROID co-training（绿色）在分布内和 OOD 两个评估设定下均全面超越"No Co-training"和"OXE Co-training"基线。OOD 测试变体包括：干扰物体、新颖物体、未见过的薯片袋包装、相机位置偏移等。Co-training comparison (Figure 6): across the 6 tasks, DROID co-training (green) outperforms the "No Co-training" and "OXE Co-training" baselines under both the in-distribution and the OOD evaluation setting. The OOD test variations include distractor objects, novel objects, unseen chip-bag packaging, and shifted camera positions.Cook Lentils 任务上的策略展开示例（Figure 7）："Policies co-trained with DROID perform smoother, more precise motions, allowing them to solve long-horizon tasks"——在最具挑战性的长时程任务上，DROID 训练的策略动作更流畅精准，而基线方法则失败。Example policy rollouts on the Cook Lentils task (Figure 7): "Policies co-trained with DROID perform smoother, more precise motions, allowing them to solve long-horizon tasks" — on the most challenging long-horizon task, the policy trained with DROID moves more smoothly and precisely, whereas the baseline methods fail.
场景多样性消融（Ablation）Scene Diversity Ablation
作者从完整 DROID 中抽取两个等大小子集（各 7,362 条轨迹）进行消融：The authors draw two equally sized subsets from the full DROID dataset (7,362 trajectories each) for an ablation:
DROID (7k, 20 Scenes)：仅取出现次数最多的 20 个场景的数据（高密度，低多样性）DROID (7k, 20 Scenes): only data from the 20 most frequently occurring scenes (high density, low diversity)
DROID (7k, Diverse Scenes)：在全数据集中均匀随机采样，保持高场景多样性DROID (7k, Diverse Scenes): uniform random sampling over the whole dataset, preserving high scene diversity
场景多样性消融（Figure 8）："Using the split with more diverse scenes yields better performance in the OOD evaluation setting"——在 OOD 设定下，多样场景子集显著优于同等规模的单一场景子集，表明场景多样性本身（而非仅靠数据量）是提升泛化的关键因素。Scene diversity ablation (Figure 8): "Using the split with more diverse scenes yields better performance in the OOD evaluation setting" — under the OOD setting the diverse-scene subset markedly outperforms the equally sized narrow-scene subset, showing that scene diversity itself, rather than data volume alone, is the key factor behind improved generalization.
04局限性Limitations
Note：论文未设专门的 Limitations 节；以下限制点部分为作者在附录与讨论中明确陈述（标注"stated"），部分为从系统设计推断（标注"inferred"）。Note: the paper has no dedicated Limitations section; some of the points below are explicitly stated by the authors in the appendix and the discussion (marked "stated"), while others are inferred from the system design (marked "inferred").
硬件单一性（stated）A single hardware platform (stated)
DROID 全部数据均来自 Franka Panda + Robotiq 夹爪平台，而 OXE 等数据集覆盖 22 种机器人形态。这意味着 DROID 训练的策略在跨机器人形态迁移方面的优势尚不明确，泛化范围主要体现在场景和任务层面，而非机器人本体层面。All DROID data comes from the Franka Panda + Robotiq gripper platform, whereas datasets such as OXE cover 22 robot embodiments. This means that the advantage of DROID-trained policies for cross-embodiment transfer remains unclear: the generalization they provide lies mainly at the scene and task level rather than at the robot embodiment level.
相机标定精度有限（stated）Limited camera calibration accuracy (stated)
论文附录 G 明确指出，外参标定参数"may not always be accurate due to checkerboard misalignment, inconsistent lighting, or errors inherent to OpenCV calibration"，这限制了利用精确三维几何信息进行更高层次空间推理的能力。Appendix G of the paper states explicitly that the extrinsic calibration parameters "may not always be accurate due to checkerboard misalignment, inconsistent lighting, or errors inherent to OpenCV calibration", which limits the ability to use precise 3D geometric information for higher-level spatial reasoning.
开放问题：如何最优利用多样数据（stated）Open question: how best to exploit diverse data (stated)
作者坦承，"how to best make use of such diverse data" 以及 "how can we train policies that perform tasks in new scenes without any in-domain data?" 仍是未解决的开放问题。Co-training 方案（50/50 混合）只是一种基础验证，最优利用方式有待进一步研究。The authors concede that "how to best make use of such diverse data" and "how can we train policies that perform tasks in new scenes without any in-domain data?" remain unresolved open questions. The co-training recipe (50/50 mixing) is only a basic validation, and the optimal way of exploiting the data awaits further study.
场景计数方法的保守性（stated）Conservativeness of the scene counting method (stated)
564 个"唯一场景"采用保守估计方法，若不同研究室将机器人放置于外观相同的实验台，可能被重复计数，导致实际场景多样性略低于标称值。The 564 "unique scenes" follow a conservative estimation method; if different labs place their robot on visually identical benches, those may be counted more than once, so the actual scene diversity may be slightly lower than the nominal figure.
硬件复制成本（inferred）Cost of replicating the hardware (inferred)
DROID 硬件套件（双 ZED 2 + ZED Mini + Oculus Quest 2 + Franka Panda）初始部署成本较高，且需要专业人员进行标定，限制了在资源受限机构中的大规模推广。（此点由作者在论文中隐含提及，属 inferred 推断。）The DROID hardware kit (two ZED 2 cameras + ZED Mini + Oculus Quest 2 + Franka Panda) has a high up-front deployment cost and requires trained personnel for calibration, which limits large-scale adoption at resource-constrained institutions. (This point is only implied by the authors in the paper, so it is an inferred limitation.)
