Learning to Model the World with Language

01 动机Motivation

现有具身 AI 智能体在处理语言时存在严重局限——它们往往只能接受简单的任务指令（如"拿起苹果"），却无法利用更丰富的语言形式：状态描述（"碗在厨房里"）、动态解释（"这个按钮会关掉电视"）、以及纠错反馈。当这些多样化语言信号涌入时，model-free 的基线方法（如 R2D2）甚至会出现性能退化的现象。Existing embodied AI agents are severely limited in how they handle language — they can usually accept only simple task instructions (such as "pick up the apple"), and cannot exploit richer forms of language: state descriptions ("the bowl is in the kitchen"), dynamics explanations ("this button turns off the TV"), or corrective feedback. When such diverse language signals flood in, model-free baselines (such as R2D2) even show performance degradation.

"We argue that agents should interpret such diverse language as a signal that helps them predict the future — what they will observe and what reward they will receive — rather than as a direct command to execute."

02 方法Method

Dynalang 以 DreamerV3 为基础，将其扩展为多模态世界模型：每个时间步同时编码一帧图像和一个语言 token，通过 GRU 序列模型在潜空间中进行未来预测，再由 Actor-Critic 在世界模型的"想象"轨迹中学习策略——语言理解完全由未来预测的自监督信号驱动。Dynalang builds on DreamerV3 and extends it into a multimodal world model: at every time step it encodes one image frame and one language token together, predicts the future in latent space with a GRU sequence model, and then learns a policy with an Actor-Critic on "imagined" trajectories of the world model — language understanding is driven entirely by the self-supervised signal of future prediction.

设计亮点：语言作为预测信号Design Highlight: Language as a Prediction Signal

核心洞察在于将语言的监督角色与行动角色解耦：语言提示经由未来预测损失自监督地嵌入世界模型，无需人工标注语言含义。这使得动态解释（"按下蓝色按钮会打开门"）和纠错（"你走错方向了"）这类在传统 instruction-following 范式中难以处理的语言类型，自然地融入统一框架。The core insight is to decouple the supervisory role of language from its action role: language hints are embedded into the world model in a self-supervised way through the future-prediction loss, with no need to annotate their meaning by hand. This lets dynamics explanations ("pressing the blue button opens the door") and corrections ("you are going the wrong way") — language types that are hard to handle in the traditional instruction-following paradigm — fit naturally into one unified framework.

03 实验Experiments

在四个具有代表性的语言-具身任务上评测 Dynalang，基线包括 model-free 方法（IMPALA、R2D2）与 model-based 方法（多种语言条件化 DreamerV3 变体），以及任务专用架构 EMMA。Dynalang is evaluated on four representative language-embodied tasks; the baselines include model-free methods (IMPALA, R2D2) and model-based methods (several language-conditioned DreamerV3 variants), as well as the task-specific architecture EMMA.

HomeGrid — 多样化语言提示HomeGrid — Diverse Language Hints

格子世界中，Agent 需完成物体操纵任务，并接收四类语言：仅任务（task only）、加状态描述（+ state）、加动态解释（+ dynamics）、加纠错（+ corrections）。关键发现：In a grid world the agent must complete object-manipulation tasks while receiving four kinds of language: task only, plus state descriptions (+ state), plus dynamics explanations (+ dynamics), and plus corrections (+ corrections). Key findings:

• Dynalang 随语言类型增多单调提升；Dynalang improves monotonically as more language types are added;
• R2D2 在语言变多时性能退化（无法利用多样化语言）；R2D2 degrades as language becomes more varied (it cannot exploit diverse language);
• IMPALA 无法完成任务。IMPALA fails to complete the tasks.

Messenger — 游戏手册阅读Messenger — Reading the Game Manual

Agent 需阅读描述随机化实体角色与规则的游戏手册，完成消息传递并规避敌人。难度分三阶段（S1/S2/S3）。The agent must read a game manual describing randomized entity roles and rules, deliver the message and avoid enemies. Difficulty is split into three stages (S1/S2/S3).

• S1：Dynalang 大幅领先所有 model-based 基线，尽管从零训练 token 嵌入；S1: Dynalang leads by a wide margin over all model-based baselines, even though token embeddings are trained from scratch;
• S3：其他方法完全失败（零成绩），Dynalang 唯一取得非零性能；S3: all other methods fail completely (zero score), and Dynalang alone attains non-zero performance;
• 文本预训练（in-domain 手册 或 TinyStories ~500M tokens）进一步提升 S2 成绩，且 TinyStories 预训练超越 T5 嵌入基线的最终性能。text pretraining (in-domain manuals or TinyStories ~500M tokens) further improves S2 scores, and TinyStories pretraining surpasses the final performance of the T5 embedding baseline.

VLN-CE — 真实感室内导航VLN-CE — Photorealistic Indoor Navigation

在 Matterport3D 真实感房屋中，Agent 须跟随自然语言指令导航至目标位置（success rate 衡量）。Dynalang 在 success rate 上显著超越 R2D2 model-free 基线，但与使用演示数据或专用架构的 SOTA 方法仍有差距（paper 明确承认）。In photorealistic Matterport3D houses the agent must follow natural-language instructions to navigate to a target location (measured by success rate). Dynalang clearly surpasses the R2D2 model-free baseline in success rate, but still trails SOTA methods that use demonstration data or specialized architectures (as the paper explicitly acknowledges).

LangRoom — 具身问答与文本生成LangRoom — Embodied QA and Text Generation

Agent 需导航找到颜色目标，并通过生成语言 action 回答问题。在大词表（10,000 token）下：The agent must navigate to find a color target and answer the question by generating a language action. With a large vocabulary (10,000 tokens):

• One-hot token 编码从零训练失败；training one-hot token encodings from scratch fails;
• 用世界模型预测正则化语言 action 后，性能恢复；after regularizing the language action with world-model prediction, performance recovers;
• 预训练（in-domain 或 TinyStories）进一步超越 T5 嵌入基线，体现多模态生成模型的迁移学习潜力。pretraining (in-domain or TinyStories) further surpasses the T5 embedding baseline, showing the transfer-learning potential of multimodal generative models.

消融分析Ablation Analysis

在 Messenger S1 上，作者系统对比了多种语言条件化策略（GRU 嵌入、SentenceBERT、T5 + image adapter、cross-attention、two-way cross-attention 等变体），Dynalang 在无需专用对齐模块的情况下全面超越所有变体，验证了"语言作为预测信号"这一核心设计的有效性。On Messenger S1 the authors systematically compare a range of language-conditioning strategies (GRU embeddings, SentenceBERT, T5 + image adapter, cross-attention, two-way cross-attention and other variants); without any dedicated alignment module Dynalang outperforms every variant across the board, confirming the effectiveness of the core design of "language as a prediction signal".

04 局限性Limitations