GigaWorld-Policy: An Efficient Action-Centered World-Action Model

01 动机Motivation

现有世界-动作模型（World-Action Models）在推理时面临两大核心瓶颈：一是推理开销过大——联合推断未来视觉动态与动作序列导致延迟极高；二是表征耦合问题——动作预测质量与视频预测质量强绑定，一旦视频生成出错，动作也随之降级。此外，基于 VLM 的 VLA 方法存在"监督稀疏"（supervision sparsity）问题：动作标签相对于高维观测过于稀疏，容易使模型将不同场景压缩为重复行为，而非学习有物理意义的动作。Existing World-Action Models face two core bottlenecks at inference time: first, excessive inference overhead—jointly inferring future visual dynamics and action sequences results in very high latency; second, representation coupling—action prediction quality is tightly bound to video prediction quality, so once video generation goes wrong, the actions degrade with it. In addition, VLM-based VLA methods suffer from "supervision sparsity": action labels are far too sparse relative to high-dimensional observations, which tends to make the model compress different scenes into repetitive behaviors rather than learn physically meaningful actions.

"jointly reasoning over future visual dynamics and corresponding actions incurs substantial inference overhead" ——论文对当前世界-动作模型推理瓶颈的直接概括— The paper's direct characterization of the inference bottleneck of current world-action models

推理频率与任务成功率对比 — 图1：在 A100 GPU 真实部署环境下，GigaWorld-Policy 与各基线方法的推理频率（Inference Frequency）与任务成功率（Task Success Rate）对比。GigaWorld-Policy 在保持竞争力成功率的同时，实现了显著更低的推理延迟。Figure 1: Inference frequency and task success rate of GigaWorld-Policy and the baseline methods under real deployment on an A100 GPU. GigaWorld-Policy attains markedly lower inference latency while keeping a competitive success rate.

9×比 Motus 推理速度提升（360ms vs 3231ms）Inference speedup over Motus (360ms vs 3231ms)

0.86RoboTwin 2.0 仿真平均成功率（clean）RoboTwin 2.0 simulation average success rate (clean)

0.83真实机器人平均成功率（20次试验/任务）Real-robot average success rate (20 trials per task)

10%达到基线最优所需的训练数据量Training data required to match the best baseline

02 方法Method

GigaWorld-Policy 以 5B 参数量的 diffusion Transformer 为骨干，将策略学习形式化为两个耦合组件：从当前观测预测动作序列（主任务），同时以预测动作为条件生成未来视频（辅助任务）。核心创新在于因果注意力掩码（causal attention mask）——它确保未来视频 token 的信息绝对不会回流至动作 token，从而消除视觉-运动表征的纠缠，同时为动作学习提供更稠密的监督信号。GigaWorld-Policy takes a 5B-parameter diffusion Transformer as its backbone and formalizes policy learning as two coupled components: predicting the action sequence from the current observation (main task), while generating future video conditioned on the predicted actions (auxiliary task). The core innovation is the causal attention mask—it guarantees that information from future video tokens never flows back to action tokens, thereby removing the entanglement of visual and motor representations while supplying denser supervision for action learning.

四种世界-动作模型架构对比 — 图2：四种世界-动作模型架构示意。(a) 基于 VLM 的 VLA + 辅助监督；(b) 联合双向动作-视频预测； (c) 两阶段先视频后动作流水线；(d) 本文提出的以动作为中心的设计，视频生成为可选路径。 GigaWorld-Policy 属于 (d)：推理时可直接跳过视频分支，大幅降低延迟。Figure 2: Four world-action model architectures. (a) VLM-based VLA plus auxiliary supervision; (b) joint bidirectional action-video prediction; (c) two-stage video-then-action pipeline; (d) the action-centered design proposed here, in which video generation is an optional path. GigaWorld-Policy belongs to (d): the video branch can be skipped outright at inference, greatly reducing latency.

因果注意力掩码（Causal Self-Attention Mask）Causal Self-Attention Mask

模型采用分块因果依赖设计（blockwise causal dependency）：动作 token 仅能 attend 到状态和观测 token，而未来视频 token 可以 attend 到动作 token，但反向路径被彻底切断。这一设计使得推理时可以只解码动作分支，完全不实例化视频 token，从而实现低延迟控制；训练时视频分支则作为辅助损失，为动作头提供更丰富的梯度信号。动作与视频均采用 flow-matching 目标函数（Equations 8–10）进行联合训练。The model adopts a blockwise causal dependency design: action tokens can attend only to state and observation tokens, while future video tokens may attend to action tokens, but the reverse path is cut off entirely. This design lets inference decode the action branch alone, never instantiating video tokens, which yields low-latency control; during training the video branch serves as an auxiliary loss that provides a richer gradient signal to the action head. Actions and video are trained jointly under flow-matching objectives (Equations 8–10).

注意力掩码示意图 — 图3：blockwise 因果注意力掩码结构。动作 token（Actions）只能看到状态与观测，未来视频 token 可向前 attend 到动作，但不能反向影响动作预测。Figure 3: Structure of the blockwise causal attention mask. Action tokens see only state and observations; future video tokens may attend forward to the actions, but cannot influence action prediction in return.

三阶段预训练流程（Three-Stage Pre-training）Three-Stage Pre-training

模型采用渐进式预训练策略：The model follows a progressive pre-training strategy:

Stage 1 — 大规模网络视频预训练：在互联网视频数据上初始化 diffusion Transformer 基础模型，学习通用视觉表征与物理先验。Stage 1 — Large-scale web video pre-training: Initialize the diffusion Transformer foundation model on internet video data, learning general visual representations and physical priors.
Stage 2 — 具身数据预训练（Embodied Pre-training）：在约 10,000 小时的机器人与第一人称人类演示数据上继续训练，涵盖 Agibot、RDT、RoboMind、ATARA（机器人视频）以及 EgoDex、Ego4D（以人为中心的演示）。Stage 2 — Embodied Pre-training: Continue training on roughly 10,000 hours of robot and egocentric human demonstration data, covering Agibot, RDT, RoboMind and ATARA (robot video) as well as EgoDex and Ego4D (human-centered demonstrations).
Stage 3 — 策略后训练（Policy Post-training）：在目标机器人轨迹数据上进行动作-语言-图像对齐微调，适配具体任务。Stage 3 — Policy Post-training: Perform action-language-image alignment fine-tuning on target-robot trajectory data to adapt to the concrete task.

图4：GigaWorld-Policy 三阶段训练流水线示意。从通用视频基础模型出发，经具身数据预训练，最终通过策略后训练对接目标机器人，推理时可选择性地生成未来视频。Figure 4: The three-stage training pipeline of GigaWorld-Policy. Starting from a general video foundation model, it passes through embodied pre-training and finally policy post-training to interface with the target robot; future video can be generated selectively at inference.

多视角观测融合Multi-view Observation Fusion

系统接收多视角 RGB 输入（Multi-view RGB），按 Equation 4 将其拼合为单张图像后送入共享 Transformer 块，与状态 token、动作 token、视频 token 一并处理。这种统一 token 化设计简化了模型结构，避免了针对不同模态引入独立编码器。The system receives Multi-view RGB input and, following Equation 4, stitches it into a single image before feeding it into shared Transformer blocks, where it is processed together with state tokens, action tokens and video tokens. This unified tokenization simplifies the model structure and avoids introducing separate encoders for the different modalities.

03 实验Experiments

实验在两个场景下评估 GigaWorld-Policy： (1) 仿真——RoboTwin 2.0 基准，50 个任务，分 clean（标准环境）和 randomized（随机初始状态）两种设置； (2) 真实世界——AgileX PiPER 6 自由度机械臂，每个任务执行 20 次试验，涉及 QR 码扫描、垃圾清扫、碗叠放、桌面清洁等任务。基线方法包括：π₀.₅、X-VLA、Motus、GigaBrain-0、Cosmos-Policy。The experiments evaluate GigaWorld-Policy in two settings: (1) simulation—the RoboTwin 2.0 benchmark with 50 tasks, under a clean (standard environment) and a randomized (random initial state) setting; (2) real world—an AgileX PiPER 6-DoF arm, with 20 trials executed per task, covering QR-code scanning, trash sweeping, bowl stacking and table cleaning. Baselines include π₀.₅, X-VLA, Motus, GigaBrain-0 and Cosmos-Policy.

仿真基准（RoboTwin 2.0）Simulation Benchmark (RoboTwin 2.0)

方法Method	Clean SR	Randomized SR	推理延迟 (A100)Inference Latency (A100)
π₀.₅	0.43	0.44	225ms
X-VLA	0.73	0.73	—
Motus	0.89	0.87	3231ms
GigaBrain-0	—	—	452ms
GigaWorld-Policy（本文）GigaWorld-Policy (ours)	0.86	0.85	360ms

注：Motus 在仿真成功率上略高（0.89 vs 0.86），但推理延迟达 3231ms，为 GigaWorld-Policy（360ms）的 9×；π₀.₅ 延迟最低（225ms），但成功率仅 0.43/0.44。Note: Motus is slightly higher in simulated success rate (0.89 vs 0.86), but its inference latency reaches 3231ms, 9× that of GigaWorld-Policy (360ms); π₀.₅ has the lowest latency (225ms) but a success rate of only 0.43/0.44.

真实世界部署（AgileX PiPER，20次/任务）Real-World Deployment (AgileX PiPER, 20 trials per task)

方法Method	平均成功率Average Success Rate
Cosmos-Policy	0.58
GigaBrain-0	0.68
π₀.₅	0.69
Motus	0.76
GigaWorld-Policy（本文）GigaWorld-Policy (ours)	0.83

数据效率（Data Efficiency）Data Efficiency

在训练数据量实验中，GigaWorld-Policy 仅使用 10% 的训练数据即可达到 VLA 基线方法的最优成功率，展现出显著更强的样本效率。In the training-data-volume experiment, GigaWorld-Policy needs only 10% of the training data to reach the best success rate of the VLA baseline methods, showing markedly stronger sample efficiency.

数据效率对比曲线 — 图5：不同训练数据比例下各方法的成功率曲线。GigaWorld-Policy 在 10% 数据量处已达到 VLA 基线的满数据性能上限，随数据增加持续提升。Figure 5: Success-rate curves of each method across training-data fractions. GigaWorld-Policy already reaches the full-data performance ceiling of the VLA baselines at 10% of the data, and keeps improving as data grows.

消融实验（Ablations）Ablations

预训练策略对最终真实世界成功率的影响（消融实验，真实机器人测试）：Effect of the pre-training strategy on final real-world success rate (ablation study, tested on the real robot):

配置Setup	成功率（SR）Success Rate (SR)
无预训练（from scratch）No pre-training (from scratch)	0.45
仅视频基础模型初始化Video foundation model initialization only	0.57
仅具身数据预训练Embodied data pre-training only	0.73
完整三阶段预训练（full）Full three-stage pre-training (full)	0.83

未来帧间距（frame spacing）的消融实验表明，Δ=12 时性能最优（SR = 0.83）。具身预训练数据量实验显示，随着具身数据比例从 0% 增至 100%，模型性能单调提升（图8），验证了大规模具身预训练对下游策略学习的持续正收益。An ablation on future frame spacing shows that performance is best at Δ=12 (SR = 0.83). The embodied pre-training data-volume experiment shows that as the embodied data fraction rises from 0% to 100%, model performance improves monotonically (Figure 8), confirming the sustained positive return of large-scale embodied pre-training for downstream policy learning.

04 局限性Limitations

注意：原论文无独立的"Limitations"章节。以下各点综合了论文设计决策中可推断出的隐含约束，均标注为 inferred（推断）。Note: The original paper has no standalone "Limitations" section. The points below synthesize implicit constraints inferable from the paper's design decisions, and are all marked inferred.

依赖大规模具身预训练数据（inferred）Reliance on large-scale embodied pre-training data (inferred)

模型 Stage 2 需要约 10,000 小时机器人与第一人称演示数据（Agibot、RDT、RoboMind、ATARA、EgoDex、Ego4D）。对于数据采集成本较高的新场景或新型机器人，完整三阶段预训练流程的复现难度较大。Stage 2 of the model requires roughly 10,000 hours of robot and egocentric demonstration data (Agibot, RDT, RoboMind, ATARA, EgoDex, Ego4D). For new scenes or new robot platforms where data collection is costly, the full three-stage pre-training pipeline is hard to reproduce.

评估机器人形态单一（inferred）Single robot embodiment in the evaluation (inferred)

真实世界实验仅在 AgileX PiPER 6-DoF 机械臂上进行，且限于桌面操作类任务（QR 码扫描、垃圾清扫、碗叠放、桌面清洁）。跨机器人形态（如双臂、移动操作）的泛化能力尚未验证。Real-world experiments are run only on an AgileX PiPER 6-DoF arm and are confined to tabletop manipulation tasks (QR-code scanning, trash sweeping, bowl stacking, table cleaning). Generalization across robot embodiments, such as bimanual or mobile manipulation, remains unverified.

无零样本跨任务迁移（inferred）No zero-shot cross-task transfer (inferred)

各具体任务均需要针对性的 Stage 3 后训练（task-specific post-training），而非真正的零样本迁移或少样本适应。在新任务上直接部署时，需要收集并标注新的轨迹数据。Every concrete task requires dedicated Stage 3 task-specific post-training rather than genuine zero-shot transfer or few-shot adaptation. Deploying directly on a new task requires collecting and annotating new trajectory data.

视频辅助分支的实际贡献量化不足（inferred）Insufficient quantification of the video auxiliary branch (inferred)

消融实验主要针对预训练阶段的配置，对"是否使用视频辅助损失"这一核心设计选择的独立消融结果在论文中未被单独列出，难以精确量化视频生成分支对最终策略性能的边际贡献。The ablations mainly target configurations of the pre-training stage; an isolated ablation of the core design choice of "whether to use the video auxiliary loss" is not reported separately in the paper, making it hard to precisely quantify the marginal contribution of the video generation branch to final policy performance.