EgoScale 在超过 20,854 小时的第一视角（egocentric）人类操作视频上预训练一个基于 flow matching 的 VLA 模型，发现人类数据规模与验证损失之间存在严格的对数线性 scaling law（R² = 0.9983），并通过轻量级对齐 mid-training 将成功率提升 54%，同时实现跨机器人本体的泛化迁移。EgoScale pretrains a flow-matching-based VLA model on more than 20,854 hours of egocentric human manipulation video, finds a strict log-linear scaling law between human data scale and validation loss (R² = 0.9983), and raises the success rate by 54% through a lightweight aligned mid-training stage, while achieving generalized transfer across robot embodiments.
arXiv 2602.16710cs.RO · 2026-02-1820,854 小时人类视频20,854 hours of human video📄 arXiv:2602.16710PDF
机器人灵巧操作的核心瓶颈在于高质量训练数据的匮乏——遥操作采集成本极高，而人类每天都在无约束环境中自然产生数以万计小时的第一视角操作视频。如何让机器人从这些「免费」的人类行为数据中学习？The core bottleneck of dexterous robot manipulation is the scarcity of high-quality training data — teleoperated collection is extremely expensive, while humans naturally produce vast numbers of hours of egocentric manipulation video every day in unconstrained environments. How can a robot learn from this "free" human behavior data?
"We ask: can large-scale human data meaningfully support complex, dexterous manipulation at scale? … We find that effective transfer is fundamentally a scaling phenomenon."
现有人类-机器人迁移研究普遍存在两大局限：Existing human-to-robot transfer studies share two common limitations:
数据规模不足：先前工作通常依赖"相对较小的人类数据集，数量级约为数十至数百小时"（relatively small human datasets, typically on the order of tens to hundreds of hours），远不足以揭示规模效应。Insufficient data scale: prior work typically relies on "relatively small human datasets, typically on the order of tens to hundreds of hours", far too little to reveal scaling effects.
自由度局限：主流方法聚焦低 DoF 系统，对 22 自由度灵巧手等高 DoF 本体的迁移未有充分研究。Limited degrees of freedom: mainstream methods focus on low-DoF systems; transfer to high-DoF embodiments such as 22-DoF dexterous hands has not been adequately studied.
图 1 — EgoScale 两阶段人类-机器人学习框架。Stage I 在 20,854 小时第一视角视频上预训练 VLA（wrist motion + retargeted 22-DoF 灵巧手动作）；Stage II 利用少量对齐的人类-机器人 mid-training 数据适配到目标机器人本体，最终在真实机器人上评估。Figure 1 — EgoScale's two-stage human-to-robot learning framework. Stage I pretrains a VLA on 20,854 hours of egocentric video (wrist motion + retargeted 22-DoF dexterous hand actions); Stage II uses a small amount of aligned human-robot mid-training data to adapt to the target robot embodiment, with final evaluation on a real robot.
20,854 h第一视角人类操作视频（比先前工作多 20×）hours of egocentric human manipulation video (20× more than prior work)
+54%相对 no-pretraining 基线的成功率提升（22-DoF 灵巧手）success-rate gain over the no-pretraining baseline (22-DoF dexterous hand)
R²=0.9983scaling law 拟合优度goodness of fit of the scaling law
EgoScale 将人类-机器人迁移分解为三个递进阶段：大规模人类预训练（Stage I）→ 对齐 mid-training（Stage II）→ 少量机器人后训练（Stage III），核心是设计跨本体通用的动作表示与轻量级适配接口。EgoScale decomposes human-to-robot transfer into three progressive stages: large-scale human pretraining (Stage I) → aligned mid-training (Stage II) → small-scale robot post-training (Stage III), centered on designing an action representation that is shared across embodiments together with a lightweight adaptation interface.
数据采集流程。大规模 in-the-wild 第一视角 RGB 视频通过现成 SLAM 估计相机位姿，通过手部姿态估计获取 21 个关键点；EgoDex 子集（Apple Vision Pro）提供高精度追踪锚点。Data collection pipeline. Large-scale in-the-wild egocentric RGB video is processed with off-the-shelf SLAM to estimate camera poses and with hand-pose estimation to obtain 21 keypoints; the EgoDex subset (Apple Vision Pro) provides high-precision tracking anchors.EgoScale 模型架构。视觉-语言骨干（frozen in Stage II）+ DiT action expert（基于 flow matching）+ 本体特定的轻量 MLP 适配器（仅用于输入/输出接口）。EgoScale model architecture. A vision-language backbone (frozen in Stage II) + a DiT action expert (based on flow matching) + embodiment-specific lightweight MLP adapters (used only for the input/output interface).
Stage I — 大规模人类预训练Stage I — Large-Scale Human Pretraining
使用 20,854 小时第一视角视频，以 相对腕部运动（relative SE(3) end-effector pose，公式：ΔWᵗ = (W₀ʷ)⁻¹ Wᵗʷ）和 retargeted 22-DoF 灵巧手关节动作作为监督信号。手部动作通过基于 CasADi/IPOPT 的非线性优化（每帧求解，带关节限位约束，指数滤波去抖）从 21 关键点 SE(3) 变换中重定向到 Sharpa 手的 URDF 关节空间。训练配置：256 块 GB200 GPU，100K steps，batch size 8,192，学习率 5×10⁻⁵。Using 20,854 hours of egocentric video, the supervision signals are relative wrist motion (relative SE(3) end-effector pose, formula: ΔWᵗ = (W₀ʷ)⁻¹ Wᵗʷ) and retargeted 22-DoF dexterous hand joint actions. Hand actions are retargeted from 21-keypoint SE(3) transforms into the URDF joint space of the Sharpa hand through nonlinear optimization based on CasADi/IPOPT (solved per frame, with joint-limit constraints and exponential filtering for de-jittering). Training configuration: 256 GB200 GPUs, 100K steps, batch size 8,192, learning rate 5×10⁻⁵.
Stage II — 对齐 mid-training（关键设计）Stage II — Aligned Mid-Training (Key Design)
利用 50 小时人类 + 4 小时机器人的配对视角数据（344 个桌面任务，Vive 追踪器记录腕部，Manus 手套记录手部），冻结视觉-语言骨干，解冻 vision encoder 和 DiT action expert 进行适配。这一阶段的核心作用是：将预训练中学到的通用动作先验"对齐"到机器人控制空间，同时通过共享 SE(3) 腕部表示保留跨本体泛化能力。Using paired-view data of 50 hours of human plus 4 hours of robot demonstrations (344 tabletop tasks, with a Vive tracker recording the wrist and Manus gloves recording the hand), the vision-language backbone is frozen while the vision encoder and the DiT action expert are unfrozen for adaptation. The core role of this stage is to "align" the generic action priors learned in pretraining to the robot control space, while preserving cross-embodiment generalization through the shared SE(3) wrist representation.
动作表示设计Action Representation Design
对比三种手部动作表示：joint-space（22-DoF 关节角）、wrist-only（仅 SE(3) 腕部）、fingertip-based（指尖位置）。实验表明 wrist-only 在接触敏感任务中表现最差（关节角精度缺失），fingertip-based 误差累积不稳定，joint-space 在所有任务中最为一致，被选为默认表示。Three hand-action representations are compared: joint-space (22-DoF joint angles), wrist-only (SE(3) wrist only), and fingertip-based (fingertip positions). Experiments show that wrist-only performs worst on contact-sensitive tasks (lacking joint-angle precision) and that fingertip-based accumulates error unstably, while joint-space is the most consistent across all tasks and is therefore chosen as the default representation.
图 3 — 五个灵巧操作评估任务（Galaxea R1 Pro）。从左到右：Shirt Rolling（衬衫卷叠）、Card Sorting（卡片分拣）、Dexterous Tong Use（夹钳取物）、Bottle Cap Unscrewing（开瓶盖）、Syringe Liquid Transfer（注射器取液），均为长时域、多步骤任务，使用连续评分 [0, 1] 衡量进度。Figure 3 — The five dexterous manipulation evaluation tasks (Galaxea R1 Pro). From left to right: Shirt Rolling, Card Sorting, Dexterous Tong Use, Bottle Cap Unscrewing, and Syringe Liquid Transfer, all long-horizon multi-step tasks whose progress is measured by a continuous score in [0, 1].
03实验Experiments
实验在 Galaxea R1 Pro（22-DoF 灵巧手）和 Unitree G1（7-DoF 三指手）两个机器人本体上进行，每任务 10 次独立评估（多实例任务 16 次），评分采用图像叠加初始化保证一致性。Experiments are run on two robot embodiments, the Galaxea R1 Pro (22-DoF dexterous hand) and the Unitree G1 (7-DoF three-fingered hand), with 10 independent evaluations per task (16 for multi-instance tasks); scoring uses image-overlay initialization to ensure consistency.
Scaling Law（核心发现）Scaling Law (Core Finding)
"A clear log-linear scaling law: validation loss follows L = 0.024 − 0.003 · ln(D)" with "R² of 0.9983"，其中 D 为人类数据小时数。, where D is the number of hours of human data.
task completion score 从 1k 小时的 0.30 提升至 20k 小时的 0.71，且"在探索范围内未见饱和迹象"（no signs of saturation in the explored regime）。The task completion score rises from 0.30 at 1k hours to 0.71 at 20k hours, with "no signs of saturation in the explored regime".
图 5 — Scaling behavior。左图：不同规模人类数据预训练的 validation loss 随训练步数的变化；右图：验证损失与真实机器人任务完成率的强正相关（R² = 0.9983），证明 loss 是机器人性能的可靠预测指标。Figure 5 — Scaling behavior. Left: validation loss versus training steps for pretraining on human data of different scales. Right: the strong positive correlation between validation loss and real-robot task completion rate (R² = 0.9983), showing that the loss is a reliable predictor of robot performance.
相对 baseline +55% 平均成功率+55% average success rate over the baseline
Human Pretrain + Mid-Training
相对 baseline +54% 平均成功率（最终系统）+54% average success rate over the baseline (final system)
注：论文以条形图呈现各任务的 task completion score，未给出逐任务数字表格；上表数字均直接引用原文摘要与正文。Note: the paper reports the task completion score of each task as a bar chart and gives no per-task numeric table; the numbers in the table above are quoted directly from the abstract and body of the paper.
图 4 — 主要实验结果。对比 Human Pretrain + Mid-Training、Human Pretraining only 和 No Pretraining 三种方法在五个灵巧操作任务上的 task completion score。预训练对所有任务均有显著提升。Figure 4 — Main experimental results. Task completion score of Human Pretrain + Mid-Training, Human Pretraining only and No Pretraining on the five dexterous manipulation tasks. Pretraining brings a marked improvement on every task.
One-Shot 迁移One-Shot Transfer
mid-training 使模型获得共享运动原语（shared motion primitives），仅用单个机器人演示即可泛化到新任务：Mid-training gives the model shared motion primitives, so that a single robot demonstration suffices to generalize to a new task:
在 Stage II mid-training 中加入少量 G1 play data，通过 embodiment-specific MLP 适配器适配 7-DoF 手部接口，相比 G1 直接训练基线实现 30% 绝对提升，且行为更流畅。Adding a small amount of G1 play data to the Stage II mid-training and adapting the 7-DoF hand interface through an embodiment-specific MLP adapter achieves a 30% absolute improvement over the baseline trained directly on G1, with smoother behavior.
动作表示消融Action Representation Ablation
joint-space 手部动作在所有 5 个任务中均优于 wrist-only 和 fingertip-based 表示，尤其在需要精细接触调控的任务（Bottle Cap、Syringe）中差距最为显著。Joint-space hand actions outperform the wrist-only and fingertip-based representations on all 5 tasks, and the gap is most pronounced on tasks that require fine contact modulation (Bottle Cap, Syringe).
04局限性Limitations
注：本文无独立 Limitations 章节。以下各点标注为 论文明确陈述 或 设计推断，均来自正文讨论或方法设计本身。Note: this paper has no standalone Limitations section. Each point below is labeled explicitly stated in the paper or inferred from the design, and all come from the discussion in the body or from the method design itself.
Scaling law 的外推范围未知（论文明确陈述）The extrapolation range of the scaling law is unknown (explicitly stated in the paper)
scaling law 仅在 1k—20k 小时数据范围内验证，"no signs of saturation in the explored regime"——即 20k 小时以外的规模效应尚无实证，计算成本（256 × GB200 GPU）也制约进一步探索。The scaling law is validated only over the 1k—20k hour data range, "no signs of saturation in the explored regime" — that is, scaling behavior beyond 20k hours has no empirical evidence, and the compute cost (256 × GB200 GPUs) also constrains further exploration.
SLAM 与手部姿态估计引入噪声（设计推断）SLAM and hand-pose estimation introduce noise (inferred from the design)
大规模数据依赖"off-the-shelf SLAM and hand-pose estimation pipelines"，其估计质量因场景而异，噪声容限（robustness threshold）未量化。尽管论文认为规模与多样性可以补偿噪声，但极端噪声情况下的失效模式未有讨论。The large-scale data relies on "off-the-shelf SLAM and hand-pose estimation pipelines", whose estimation quality varies across scenes, and the noise tolerance (robustness threshold) is not quantified. Although the paper argues that scale and diversity can compensate for noise, failure modes under extreme noise are not discussed.
评估任务以桌面操作为主（设计推断）Evaluation tasks are mainly tabletop manipulation (inferred from the design)
五个评估任务均为桌面场景（tabletop manipulation）；移动操作、开放世界场景以及与运动（locomotion）结合的场景未涉及，限制了结论的泛化范围。All five evaluation tasks are tabletop scenes (tabletop manipulation); mobile manipulation, open-world scenes and settings combined with locomotion are not covered, which limits how far the conclusions generalize.
计算成本高，面向大型实验室（设计推断）High compute cost, aimed at large laboratories (inferred from the design)
Stage I 预训练需要 256 块 GB200 GPU 运行 100K steps，这一规模对学术小实验室和工业小团队存在较高门槛。论文未提供缩减计算预算的替代方案。Stage I pretraining requires 256 GB200 GPUs running 100K steps, a scale that poses a high barrier for small academic labs and small industrial teams. The paper offers no alternative for reducing the compute budget.
