SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control

01 动机Motivation

大语言模型已证明"规模即能力"，但 humanoid 控制领域至今未能复现这一增益：现有神经控制器参数规模有限、行为种类单一、训练资源匮乏。 SONIC 的出发点正是弥合这一差距——证明在 humanoid 控制上同样存在清晰的 scaling law。Large language models have already proven that "scale is capability", yet the field of humanoid control has so far failed to reproduce such gains: existing neural controllers are modest in parameter size, cover a narrow set of behaviors, and are trained with scarce resources. SONIC sets out precisely to close this gap, showing that a clear scaling law holds for humanoid control as well.

"Despite the rise of billion-parameter foundation models trained across thousands of GPUs, similar scaling gains have not been shown for humanoid control. Current neural controllers for humanoids remain modest in size, target a limited set of behaviors, and are trained on a handful of GPUs."

Motion tracking 天然适合 scaling：运动捕捉数据提供密集的监督信号，无需手工设计奖励函数；多样化数据集隐式赋予策略人体运动先验（human motion prior）。 SONIC 同时在三个维度上扩展：网络容量（1.2M → 42M 参数）、数据量（100M+ 帧，700 小时高质量 mocap）、计算量（21,000 GPU 小时）。Motion tracking is naturally suited to scaling: motion capture data provides a dense supervision signal without hand-designed reward functions, and diverse datasets implicitly endow the policy with a human motion prior. SONIC scales along three axes at once: network capacity (1.2M → 42M parameters), data volume (100M+ frames, 700 hours of high-quality mocap) and compute (21,000 GPU-hours).

02 方法Method

SONIC 的核心是一个编解码框架，配备三种专用编码器，将异构控制信号（机器人关节、人体 SMPL 关节、稀疏关键点）统一映射到共享 latent 空间， 再经 Finite Scalar Quantization（FSQ）量化为 universal token，最终由统一解码器输出 29 维关节位置目标。At the core of SONIC is an encoder-decoder framework equipped with three dedicated encoders that map heterogeneous control signals (robot joints, human SMPL joints, sparse keypoints) into a shared latent space; the latent is then quantized by Finite Scalar Quantization (FSQ) into a universal token, and a unified decoder finally outputs 29-dimensional joint position targets.

Unified Token Space（统一 token 空间）Unified Token Space

三条编码器各有分工：The three encoders divide the labor as follows:

• Robot Motion Encoder（ℰ_r）：接收未来 10 帧（0.1s 间隔）机器人关节位置/速度，适用于直接机器人运动跟踪。Robot Motion Encoder (ℰ_r): takes the next 10 frames (0.1s interval) of robot joint positions/velocities, and is suited to direct robot motion tracking.
• Human Motion Encoder（ℰ_h）：编码未来 10 帧（0.02s 间隔）人体 3D SMPL 关节位置，无需显式 retargeting。Human Motion Encoder (ℰ_h): encodes the next 10 frames (0.02s interval) of human 3D SMPL joint positions, without explicit retargeting.
• Hybrid Motion Encoder（ℰ_m）：融合稀疏上肢关键点（头部、双手、手指角度、腰部高度）与下肢机器人运动，支持 3-point VR teleoperation。Hybrid Motion Encoder (ℰ_m): fuses sparse upper-body keypoints (head, both hands, finger angles, waist height) with lower-body robot motion, supporting 3-point VR teleoperation.

所有编码器通过多层感知机（隐藏层：[2048, 1024, 512, 512]）映射至共享 latent，经 FSQ 量化为 universal token。 辅助重建解码器 𝒟_r 重建机器人运动，隐式实现 human-to-robot retargeting 与特征对齐。All encoders map into the shared latent through a multilayer perceptron (hidden layers: [2048, 1024, 512, 512]) and are quantized by FSQ into a universal token. The auxiliary reconstruction decoder 𝒟_r reconstructs robot motion, implicitly realizing human-to-robot retargeting and feature alignment.

训练目标：四项联合损失Training Objective: Four Joint Losses

训练损失由四项组成：The training loss consists of four terms:

• ℒ_ppo：标准 Proximal Policy Optimization 强化学习损失。ℒ_ppo: the standard Proximal Policy Optimization reinforcement learning loss.
• ℒ_recon：重建损失，使模型学习 human→robot 的隐式 retargeting。ℒ_recon: a reconstruction loss that lets the model learn implicit human→robot retargeting.
• ℒ_token：对齐损失，"measures the discrepancy between the robot token and the human motion token"，驱动跨 embodiment 的表征对齐。ℒ_token: an alignment loss that "measures the discrepancy between the robot token and the human motion token", driving representation alignment across embodiments.
• ℒ_cycle：循环一致性损失，确保跨模态转换的保真度。ℒ_cycle: a cycle-consistency loss ensuring the fidelity of cross-modal conversion.

实时运动规划器（Kinematic Planner）Real-Time Kinematic Planner

为将 motion tracking 能力桥接到实际任务（如导航），SONIC 额外引入实时运动规划器：在标准笔记本上延迟 <5 ms，在 Jetson Orin GPU 上 12 ms； 每 100 ms 或接到新指令时重新规划；每段运动时长 0.8–2.4s 自动确定。速度指令范围 0.0–6.0 m/s，支持 0–360° 任意方向。To bridge motion tracking capability to practical tasks such as navigation, SONIC additionally introduces a real-time kinematic planner: latency is <5 ms on a standard laptop and 12 ms on a Jetson Orin GPU; replanning happens every 100 ms or whenever a new command arrives, and the duration of each motion segment, 0.8–2.4s, is determined automatically. Velocity commands range over 0.0–6.0 m/s and support arbitrary directions in 0–360°.

Scaling 配置Scaling Setup

训练使用 4,096 个并行环境/GPU，每环境 24 步，5 epochs，actor 学习率 2×10^-5。 领域随机化涵盖摩擦系数（μ_s: 0.3–1.6，μ_a: 0.3–1.2）、质心偏移、外力扰动和运动扰动。 数据采用自适应运动采样（Adaptive Motion Sampling），以失败率为权重（β=200，混合参数 α=0.1）动态分配训练难度。Training uses 4,096 parallel environments per GPU, 24 steps per environment, 5 epochs, and an actor learning rate of 2×10^-5. Domain randomization covers friction coefficients (μ_s: 0.3–1.6, μ_a: 0.3–1.2), center-of-mass offsets, external force perturbations and motion perturbations. The data uses Adaptive Motion Sampling, weighting by failure rate (β=200, mixing parameter α=0.1) to dynamically allocate training difficulty.

03 实验Experiments

评测基准：9 小时重新定向的 AMASS 数据（1,602 条轨迹），规模显著大于此前工作。 核心指标：轨迹成功率 + MPJPE（Mean Per-Joint Position Error，mm）。 基线方法：Any2Track、BeyondMimic、GMT。Evaluation benchmark: 9 hours of retargeted AMASS data (1,602 trajectories), substantially larger than prior work. Core metrics: trajectory success rate and MPJPE (Mean Per-Joint Position Error, mm). Baselines: Any2Track, BeyondMimic, GMT.

Scaling 效果Scaling Results

SONIC 在三个维度上均呈现单调性能提升：网络规模（1.2M → 42M 参数）、数据量（到 100M+ 帧）、计算量（到 21k GPU 小时）， 其中数据多样性带来的增益最为显著。性能随计算量稳定提升，表明 motion tracking 具备良好的 scaling law 特性。SONIC shows monotone performance improvements along all three axes: network size (1.2M → 42M parameters), data volume (up to 100M+ frames) and compute (up to 21k GPU-hours), with the gain from data diversity being the most significant. Performance improves steadily with compute, indicating that motion tracking possesses favorable scaling-law behavior.

运动跟踪性能Motion Tracking Performance

指标Metric	结果Result	说明Notes
真实世界 50 条轨迹成功率Success rate on 50 real-world trajectories	100%	涵盖舞蹈、跳跃、移动操作Covers dancing, jumping and mobile manipulation
超越基线Beats the baselines	全部指标All metrics	vs. Any2Track, BeyondMimic, GMT：成功率 + MPJPE 均优vs. Any2Track, BeyondMimic, GMT: better on both success rate and MPJPE
策略泛化Policy generalization	通过Pass	可泛化到训练集外的未见运动Generalizes to unseen motions outside the training set

VR Teleoperation（3-point 接口）VR Teleoperation (3-point interface)

指标Metric	均值Mean	95th 百分位95th percentile
端到端延迟End-to-end latency	121.9 ms	—
右腕位置误差Right wrist position error	6 cm	13.3 cm
右腕朝向误差Right wrist orientation error	0.145 rad (8.32°)	0.267 rad (15.31°)
采集 demonstrationDemonstrations collected	300 条300 trajectories	用于下游 VLA 微调Used for downstream VLA fine-tuning

VLA 自主操作（Apple-to-Plate 任务）VLA Autonomous Manipulation (Apple-to-Plate Task)

在 VR teleoperation 采集的 300 条轨迹上微调 GR00T N1.5 视觉-语言-动作模型（vision-language-action model）， 然后通过 unified token space 直接将 VLA 输出的运动指令送入 SONIC 控制器，无需任何额外适配器。A GR00T N1.5 vision-language-action model is fine-tuned on the 300 trajectories collected through VR teleoperation; the motion commands output by the VLA are then fed directly into the SONIC controller through the unified token space, without any additional adapter.

消融实验Ablations

消融验证了 unified token space 各组件的必要性：去掉 ℒ_token 对齐损失后，跨 embodiment 跟踪精度显著下降； 去掉 ℒ_cycle 循环一致性损失后，模态转换保真度下降。数据规模是 scaling 的最大贡献因素（文中明确指出"dataset size providing the most substantial gains"）。The ablations verify the necessity of each component of the unified token space: removing the ℒ_token alignment loss markedly degrades cross-embodiment tracking accuracy, and removing the ℒ_cycle cycle-consistency loss degrades the fidelity of modality conversion. Dataset size is the largest contributor to scaling (the paper explicitly notes "dataset size providing the most substantial gains").

04 局限性Limitations

安全性、合规性与能效问题尚未正式处理（stated）Safety, compliance and energy efficiency not yet formally addressed (stated)

论文原文指出："formal treatment of safety, compliance, and energy efficiency for extended deployments" 是尚待解决的问题。 当前系统在长期部署场景中的安全边界和能耗表现尚不明确。The paper states that a "formal treatment of safety, compliance, and energy efficiency for extended deployments" remains an open problem. The safety boundaries and energy behavior of the current system in long-term deployment scenarios are still unclear.

部署中噪声输入的鲁棒性（stated）Robustness to noisy input during deployment (stated)

作者明确提及"combating noisy input during deployments"是待解决的挑战。 视频遥控路径下，实时姿态估计（≥60 fps）在光照、遮挡等不利条件下的鲁棒性仍有改善空间。The authors explicitly mention "combating noisy input during deployments" as an open challenge. Along the video teleoperation path, the robustness of real-time pose estimation (≥60 fps) under adverse conditions such as poor lighting and occlusion still leaves room for improvement.

模态间隙与联合训练（stated）Modality gaps and joint training (stated)

论文将"exploring joint training of planner, tokenizers, and policy to reduce modality gaps"列为未来工作， 说明当前分阶段训练管线存在模态对齐误差的累积问题。The paper lists "exploring joint training of planner, tokenizers, and policy to reduce modality gaps" as future work, indicating that the current staged training pipeline suffers from accumulated modality alignment error.

Scaling law 覆盖范围有限（inferred）Limited coverage of the scaling law (inferred)

当前 scaling 实验在单一机器人平台（Unitree H1）上进行。跨机器人体型（embodiment morphology）的 scaling 规律是否成立，论文尚未验证。 作者将"scaling laws across more diverse datasets"列为未来方向。The current scaling experiments are conducted on a single robot platform (Unitree H1). Whether the scaling law holds across robot morphologies (embodiment morphology) has not been verified in the paper. The authors list "scaling laws across more diverse datasets" as a future direction.