π₀.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities
首个在灵巧操作、指令跟随与跨形态泛化上同时媲美专家模型的通用机器人基础模型The first generalist robotic foundation model to match specialist models simultaneously on dexterous manipulation, instruction following and cross-embodiment generalization
Physical Intelligence（含 Chelsea Finn, Sergey Levine, Karl Pertsch, Allen Z. Ren 等 87 位作者）· arXiv 2026Physical Intelligence (including Chelsea Finn, Sergey Levine, Karl Pertsch, Allen Z. Ren and 87 authors in total) · arXiv 2026
π₀.7 是一个约 50 亿参数的 vision-language-action 模型，通过在训练时引入多样化上下文条件——包括子任务自然语言指令、子目标图像和 episode 元数据（速度、质量评分、是否失误）——使模型能够消歧异构数据集，并涌现出组合泛化能力。其在无需任务特定微调的情况下，在折叠衣物、制作咖啡、跨形态灵巧操作等高难度任务上与 RL 专家模型持平或更优。π₀.7 is a vision-language-action model with roughly five billion parameters. By introducing diverse context conditioning during training—subtask natural-language instructions, subgoal images and episode metadata (speed, quality score, whether a mistake occurred)—the model can disambiguate heterogeneous datasets and exhibits emergent compositional generalization. Without any task-specific fine-tuning, it matches or surpasses RL specialist models on demanding tasks such as folding laundry, making coffee and cross-embodiment dexterous manipulation.
robotic foundation modelVLAflow matchingcross-embodiment组合泛化compositional generalizationdiverse context conditioning机器人操作robot manipulationvision-language-action
01动机 MotivationMotivation
尽管大型语言模型已展现出强大的组合泛化能力——能够将已学知识以前所未有的方式组合应用——现有机器人基础模型却在这一核心能力上严重缺失：它们往往需要对每项新任务进行专项微调，且无法在陌生环境或新形态机器人上零样本泛化。Although large language models have demonstrated strong compositional generalization—recombining learned knowledge in unprecedented ways—existing robotic foundation models are severely lacking in this core capability: they usually require dedicated fine-tuning for every new task, and they fail to generalize zero-shot to unfamiliar environments or new robot embodiments.
"Prior VLA models lack robust compositional generalization and often require task-specific fine-tuning despite being trained on large datasets."
图 1：实验涉及的多种机器人平台。从左至右：双臂移动操作平台（Mobile Bimanual）、静态双臂 BiPi 机器人、以及配备 Robotiq 夹爪的 UR5e 双臂系统。跨形态实验中，模型需要在这些形态差异显著的机器人之间实现零样本迁移。Figure 1: The robot platforms involved in the experiments. From left to right: a mobile bimanual manipulation platform (Mobile Bimanual), a stationary bimanual BiPi robot, and a UR5e bimanual system equipped with Robotiq grippers. In the cross-embodiment experiments the model must transfer zero-shot between robots whose embodiments differ substantially.
作者的核心假设是：在训练时提供关于"做什么"和"怎么做"的详细上下文信息，可以消除异构数据集中的歧义，进而解锁涌现式泛化能力（emergent generalization）。这种能力使得模型能够：The authors' core hypothesis is that supplying detailed context about "what to do" and "how to do it" at training time removes the ambiguity in heterogeneous datasets and thereby unlocks emergent generalization. This capability enables the model to:
在从未见过的厨房和卧室环境中，跟随开放式自然语言指令完成任务follow open-ended natural-language instructions in never-before-seen kitchen and bedroom environments
无需专项微调，直接完成制作咖啡、折叠衣物等高度灵巧的操作任务perform highly dexterous manipulation tasks such as making coffee and folding laundry without any dedicated fine-tuning
将在某一机器人形态上学到的技能零样本迁移到形态差异显著的目标机器人（跨形态泛化）transfer skills learned on one robot embodiment zero-shot to a target robot with a markedly different embodiment (cross-embodiment generalization)
通过语言 coaching，在没有低层动作数据的情况下习得全新的长时序任务acquire entirely new long-horizon tasks through language coaching, without low-level action data
~5B模型总参数量Total model parameters
80%跨形态灵巧折叠 T-shirt 成功率（匹配人类遥操作基线 80.6%）Cross-embodiment dexterous T-shirt folding success rate (matching the 80.6% human tele-operation baseline)
π₀.7 的核心创新在于多样化上下文条件训练（diverse context conditioning）：通过在训练时同时提供子任务自然语言指令、子目标图像和 episode 元数据，使模型能够理解并利用各种质量和来源的训练数据，并在推理时通过灵活配置这些条件来引导模型行为。The core innovation of π₀.7 is training with diverse context conditioning: by simultaneously providing subtask natural-language instructions, subgoal images and episode metadata during training, the model can understand and exploit training data of varying quality and provenance, and at inference time these conditions can be configured flexibly to steer its behavior.
图 2：π₀.7 模型架构。模型由三部分组成：4B 参数的 Gemma3 VLM 骨干网络（处理视觉与语言输入）、memory-style 视频历史编码器（MEM，用于记忆依赖型任务），以及 860M 参数的 flow-matching action expert（负责生成动作序列）。训练时，上下文信息包括：语言指令、子目标图像、episode 元数据（速度、质量、是否失误）和控制模式标识。Figure 2: the π₀.7 model architecture. The model comprises three parts: a 4B-parameter Gemma3 VLM backbone (processing visual and language inputs), a memory-style video history encoder (MEM, for memory-dependent tasks), and an 860M-parameter flow-matching action expert (generating action sequences). During training the context includes language instructions, subgoal images, episode metadata (speed, quality, whether a mistake occurred) and a control-mode identifier.图 3：多样化提示（Diverse Prompting）策略示意图。训练时的上下文由四类信息组成：子任务指令（subtask instructions）、子目标图像（subgoal images）、episode 元数据（episode metadata）和控制模式（control mode）。每类信息在训练时均以一定概率随机 dropout，确保模型在推理时能够灵活应对不同的提示组合。图中展示了折叠衬衫任务中子目标图像与元数据的组合提示示例。Figure 3: the diverse prompting strategy. The training context consists of four kinds of information: subtask instructions, subgoal images, episode metadata and control mode. Each kind is randomly dropped out with some probability during training, ensuring that the model can flexibly handle different prompt combinations at inference time. The figure shows an example of combining subgoal images and metadata as a prompt for the shirt-folding task.
训练时的上下文由四类信息构成，每类均以随机概率 dropout：The training context is made up of four kinds of information, each dropped out with a random probability:
子任务指令（Subtask Instructions）：中间层语义任务描述（如 "open the fridge door"），支持逐步引导和测试时的语言 coaching。Subtask Instructions: mid-level semantic task descriptions (such as "open the fridge door"), supporting step-by-step guidance and language coaching at test time.
子目标图像（Subgoal Images）：描述期望近未来场景状态的多视角视觉目标，由基于 BAGEL 初始化的轻量级 world model 生成，用于表达语言难以描述的信息。子目标图像被加入 25% 的训练样本，本身以 30% 的概率随机丢弃；训练样本中以 25% 概率采样片段端点的真实未来帧，75% 概率在 0–4 秒范围内均匀采样，并混合真实帧和生成帧以减小 train-test mismatch。Subgoal Images: multi-view visual goals describing the desired near-future scene state, generated by a lightweight world model initialized from BAGEL, used to express information that language describes poorly. Subgoal images are added to 25% of the training samples and are themselves randomly dropped with probability 30%; among the training samples, the true future frame at the segment endpoint is sampled with probability 25% and a frame is sampled uniformly within a 0–4 second window with probability 75%, with real and generated frames mixed to reduce the train-test mismatch.
Episode 元数据（Episode Metadata）：包括整体速度（以 500 步为单位离散化）、质量评分（1–5 分）和失误标记。这些标签使模型能够理解不同质量和策略的数据来源。整体以 15% 概率完全丢弃，各子项以 5% 概率单独丢弃。Episode Metadata: the overall speed (discretized in units of 500 steps), a quality score (1–5) and a mistake flag. These labels let the model make sense of data sources of differing quality and strategy. The metadata as a whole is dropped entirely with probability 15%, and each sub-item is dropped individually with probability 5%.
控制模式（Control Mode）：文本标识符，指定关节级或末端执行器动作模式。Control Mode: a text identifier specifying joint-level or end-effector action mode.
核心组件 2：训练数据多样性Core Component 2: Training Data Diversity
π₀.7 的训练数据涵盖前所未有的多样性：多种机器人平台的遥操作演示数据、包含失败和次优行为的自主推演数据、以人为中心的视频数据、以及网络非机器人数据（物体定位、VQA、文字预测、视频-语言任务）。值得注意的是，模型通过纳入 RL 专家模型的评估数据来蒸馏其行为，而无需重新采集低层动作数据。The training data of π₀.7 spans unprecedented diversity: tele-operated demonstrations from many robot platforms, autonomous rollout data containing failures and sub-optimal behavior, human-centric video data, and non-robot web data (object localization, VQA, text prediction, video-language tasks). Notably, the model distills the behavior of RL specialist models by incorporating their evaluation data, without re-collecting low-level action data.
推理配置Inference Configuration
推理时固定配置（Algorithm 1）：控制模式始终提供；速度设为任务 episode 长度的第 15 百分位（偏快）；质量始终设为最高（5 分）；失误标记设为 false。子任务指令由高层策略或人类监督提供；子目标图像每 4 秒刷新一次或在意图改变时更新。每次推理通过 5 步去噪生成 50 步动作 chunk，执行其中 15–25 步；支持 classifier-free guidance（CFG），权重 β ∈ {1.3, 1.7, 2.2}。Fixed configuration at inference time (Algorithm 1): the control mode is always provided; speed is set to the 15th percentile of the task episode length (i.e. fast); quality is always set to the maximum (5); the mistake flag is set to false. Subtask instructions are provided by a high-level policy or by human supervision; subgoal images are refreshed every 4 seconds or whenever the intent changes. Each inference pass generates a 50-step action chunk with 5 denoising steps, of which 15–25 steps are executed; classifier-free guidance (CFG) is supported, with weight β ∈ {1.3, 1.7, 2.2}.
03实验 ExperimentsExperiments
实验评估覆盖四个核心维度：开箱即用灵巧操作性能、指令跟随能力、跨形态迁移、以及组合任务泛化，所有任务均与 π₀.5 和 π₀.6 等基线模型对比。The evaluation covers four core dimensions: out-of-the-box dexterous manipulation performance, instruction-following ability, cross-embodiment transfer, and compositional task generalization, with all tasks compared against baseline models such as π₀.5 and π₀.6.
π₀.7 无需任何任务特定后训练，即可在多个高难度灵巧操作任务上与 RL 微调专家模型（π₀.6*）媲美：Without any task-specific post-training, π₀.7 already rivals RL fine-tuned specialist models (π₀.6*) on several demanding dexterous manipulation tasks:
图 4：开箱即用灵巧操作性能。上行：咖啡机制作浓缩咖啡（Espresso Making）、搭建纸箱（Box Building）、折叠衣物（Laundry Folding），比较成功率和归一化吞吐量（relative to specialist π₀.6* baseline）。下行：Robot Olympics 任务（花生酱三明治、衬衫内外翻转、导航穿门）及更多灵巧任务，使用 task progress 指标。π₀.7 在大多数任务上与或超越专家基线。Figure 4: out-of-the-box dexterous manipulation performance. Top row: espresso making with a coffee machine (Espresso Making), box building (Box Building) and laundry folding (Laundry Folding), comparing success rate and normalized throughput (relative to specialist π₀.6* baseline). Bottom row: Robot Olympics tasks (peanut-butter sandwich, turning a shirt right-side out, navigating through a door) and further dexterous tasks, measured by task progress. π₀.7 matches or exceeds the specialist baseline on most tasks.
在 6 个全新环境（4 个未见过的厨房 + 2 个未见过的卧室）、14 种场景下，每场景 3–6 条开放式指令评估，π₀.7 整体成功率显著优于 π₀.5 和 π₀.6。对于包含复杂指代关系的指令（如 "pick up the fruit on the largest plate"），π₀.7 相对基线的优势更为突出；配合子目标图像（π₀.7 GC）可进一步提升。数据集偏见破解测试（"Reverse Fridge to Microwave"）中，子目标图像对成功率至关重要。Evaluated in 6 entirely new environments (4 unseen kitchens + 2 unseen bedrooms) across 14 scenes, with 3–6 open-ended instructions per scene, π₀.7 achieves a markedly higher overall success rate than π₀.5 and π₀.6. For instructions containing complex referring expressions (such as "pick up the fruit on the largest plate"), the advantage of π₀.7 over the baselines is even more pronounced; combining it with subgoal images (π₀.7 GC) improves it further. In the dataset-bias-breaking test ("Reverse Fridge to Microwave"), subgoal images are critical to the success rate.
跨形态迁移（Cross-Embodiment Transfer）Cross-Embodiment Transfer
图 5：跨形态迁移结果。左图：从简单重新排列任务到复杂跨形态灵巧操作的逐步提升（Table Setting, Bag in Backpack, Shirt Bagging 等）。右图：将折叠 T-shirt / 毛巾任务从轻型双臂机器人迁移到重型 UR5e 双臂系统。π₀.7 task progress 达 85.6%，success rate 达 80%，与 10 名受试者（平均 375 小时经验）零样本遥操作的 task progress 90.9%、success rate 80.6% 相当。模型还涌现出适应目标形态运动学的全新策略（如将两臂持袋改为单臂拾取，将倾斜抓取改为垂直抓取）。Figure 5: cross-embodiment transfer results. Left: the progression from simple rearrangement tasks to complex cross-embodiment dexterous manipulation (Table Setting, Bag in Backpack, Shirt Bagging, etc.). Right: transferring T-shirt / towel folding from a lightweight bimanual robot to a heavy-duty UR5e bimanual system. π₀.7 reaches 85.6% task progress and 80% success rate, comparable to 10 human subjects (with 375 hours of experience on average) tele-operating zero-shot at 90.9% task progress and 80.6% success rate. The model also exhibits entirely new strategies adapted to the kinematics of the target embodiment (for example switching from holding the bag with both arms to picking it up with a single arm, and from a tilted grasp to a vertical grasp).
在完全无训练数据的短时序任务上（舀米饭、转动风扇/齿轮、擦拭物体、按 French press 活塞等），π₀.7 零样本成功率达 55–75%，且语言条件与子目标图像条件性能相当。通过语言 coaching 在未见长时序任务（装载/卸载空气炸锅、制作吐司贝果）上，π₀.7 达到约 70–85% 成功率（π₀.6 约 10–20%）。进一步地，在 coaching 数据上训练高层策略后，可实现自主执行，性能仅比有人 coaching 低约 5–10%。On short-horizon tasks with no training data at all (scooping rice, turning a fan/gear, wiping objects, pressing a French press plunger, etc.), π₀.7 attains a zero-shot success rate of 55–75%, with language conditioning and subgoal-image conditioning performing comparably. With language coaching on unseen long-horizon tasks (loading/unloading an air fryer, making a toasted bagel), π₀.7 reaches roughly 70–85% success (π₀.6 roughly 10–20%). Furthermore, after training a high-level policy on the coaching data, autonomous execution becomes possible, only about 5–10% below human-coached performance.
消融实验（Ablations）Ablations
图 6：数据质量与多样性消融实验。左图：随着数据集从 top-30% 扩大到全量（平均质量下降），带元数据的 π₀.7 持续提升，无元数据版本则性能下降——表明元数据对混合质量数据的学习至关重要。右图：移除最多样化的 20% 数据导致 15–30% 性能下降（移除随机 20% 几乎无影响），证明任务多样性是组合泛化的关键驱动因素。Figure 6: ablations on data quality and diversity. Left: as the dataset grows from the top-30% subset to the full set (lowering average quality), π₀.7 with metadata keeps improving while the version without metadata degrades—showing that metadata is essential for learning from mixed-quality data. Right: removing the most diverse 20% of the data causes a 15–30% performance drop (removing a random 20% has almost no effect), proving that task diversity is a key driver of compositional generalization.
关键消融结论：Key ablation findings:
无元数据（π₀.7 no metadata）：吞吐量下降 15–30%No metadata (π₀.7 no metadata): throughput drops by 15–30%
无评估数据（π₀.7 no eval data）：吞吐量下降 20–40%No evaluation data (π₀.7 no eval data): throughput drops by 20–40%
移除最多样 20% 数据：性能下降 15–30%（随机移除无明显影响）Removing the most diverse 20% of the data: performance drops by 15–30% (random removal has no noticeable effect)
04局限性 LimitationsLimitations
Note:以下局限性均由原文作者明确陈述（stated by the authors）。All limitations below are explicitly stated by the authors.
未见任务/形态的成功率仍有差距（Reduced Zero-Shot Success）Reduced Zero-Shot Success on Unseen Tasks and Embodiments
未见任务和形态的成功率为 60–80%，相比分布内任务的 >90% 仍有明显差距。这表明即使是 π₀.7，在真正陌生的场景下仍无法做到完全可靠。Success rates on unseen tasks and embodiments are 60–80%, still clearly short of the >90% achieved on in-distribution tasks. This shows that even π₀.7 is not yet fully reliable in genuinely unfamiliar scenarios.
随着训练数据规模和多样性极大扩展，准确区分"见过"和"未见过"的任务边界变得非常困难。某些技能可能以不同标签或作为其他任务的附带行为出现在数据中，这使得泛化声明的可靠性难以精确评估。As the scale and diversity of the training data expand enormously, accurately distinguishing the boundary between "seen" and "unseen" tasks becomes very difficult. Some skills may appear in the data under different labels or as incidental behavior within other tasks, which makes the reliability of generalization claims hard to assess precisely.
泛化主要依赖行为重组（Generalization via Remixing）Generalization via Remixing
模型实现组合泛化的主要途径可能是重新组合已有行为，而非发现真正全新的能力。作者认为这在功能上是可接受的，但这一局限性值得在更严格的测试条件下进一步审视。The main route by which the model achieves compositional generalization may be recombining existing behaviors rather than discovering genuinely new capabilities. The authors consider this functionally acceptable, but the limitation deserves further scrutiny under stricter test conditions.
复杂长时序任务仍需详细 coaching（Task-Specific Coaching Still Required）Task-Specific Coaching Still Required
对于复杂的长时序任务，模型仍依赖详细的步骤级语言 coaching 或已训练好的高层策略，尚不能完全依赖粗粒度指令自主完成任务。For complex long-horizon tasks the model still relies on detailed step-level language coaching or on a trained high-level policy, and cannot yet complete tasks autonomously from coarse-grained instructions alone.
