FlowVLA 提出"视觉思维链（Visual Chain of Thought）"范式：在预测未来帧外观之前，先显式推断光流（运动动态）。与直接预测下一帧的传统 VLA 世界模型相比，FlowVLA 生成的视觉预测更符合物理规律，同时在多个机器人操作基准上取得最优策略性能，并在数据效率上显著优于基线。FlowVLA proposes a "Visual Chain of Thought" paradigm: before predicting the appearance of future frames, it explicitly infers optical flow (motion dynamics) first. Compared with conventional VLA world models that directly predict the next frame, FlowVLA produces visual forecasts that are more physically plausible, achieves state-of-the-art policy performance on multiple robot manipulation benchmarks, and markedly surpasses the baselines in data efficiency.
VLAoptical flowvisual chain-of-thoughtworld model机器人操作robot manipulationautoregressive transformermotion reasoningsample efficiency策略学习policy learning视觉语言动作模型vision-language-action model
01动机 MotivationMotivation
当前主流 VLA 模型依赖基于"next-frame prediction"训练的世界模型，试图直接预测未来帧的外观，而不显式建模底层的运动动态。这导致两个核心问题：预测在物理上不合理（物体凭空消失、机械臂轨迹混乱），且预训练目标与下游动作生成之间存在明显的领域鸿沟。Mainstream VLA models today rely on world models trained with "next-frame prediction", attempting to predict the appearance of future frames directly without explicitly modelling the underlying motion dynamics. This causes two core problems: the forecasts are physically implausible (objects vanish into thin air, manipulator trajectories become erratic), and a clear domain gap exists between the pre-training objective and downstream action generation.
"This approach attempts to predict future frame appearance without explicitly reasoning about underlying dynamics, leading to physically implausible visual forecasts and inefficient policy learning."
图 1：FlowVLA 两阶段训练范式。Stage 1（世界模型预训练）：模型学习生成交错的运动 token（光流）和未来外观 token（下一帧）序列；Stage 2（策略微调）：在冻结或更新的世界模型基础上，继续训练动作预测头，损失仅计算在 action token 上。Figure 1: The two-stage training paradigm of FlowVLA.Stage 1 (world model pre-training): the model learns to generate an interleaved sequence of motion tokens (optical flow) and future appearance tokens (the next frame); Stage 2 (policy fine-tuning): on top of the frozen or updated world model, an action prediction head is further trained, with the loss computed only over the action tokens.
88.1%LIBERO 平均成功率（vs. UniVLA 84.0%）LIBERO average success rate (vs. UniVLA 84.0%)
74.0%SimplerEnv 平均成功率（vs. UniVLA 65.6%）SimplerEnv average success rate (vs. UniVLA 65.6%)
44.0%真实机器人平均成功率（vs. UniVLA 31.0%）Real-robot average success rate (vs. UniVLA 31.0%)
+55%低数据场景下相对基线的成功率提升Success-rate gain over the baseline in the low-data regime
传统世界模型陷入"pixel-copying trap"：模型只是复制静态背景而非理解时空动态，导致"blurry, inconsistent, and physically implausible long-horizon forecasts"。FlowVLA 通过在帧预测前插入光流预测步骤，强迫模型先理解"场景如何运动"，再推断"场景将呈现什么样子"。Conventional world models fall into a "pixel-copying trap": the model merely copies the static background instead of understanding spatio-temporal dynamics, which leads to "blurry, inconsistent, and physically implausible long-horizon forecasts". By inserting an optical-flow prediction step before frame prediction, FlowVLA forces the model to first understand "how the scene moves" and only then infer "what the scene will look like".
02方法 MethodMethod
FlowVLA 将世界模型的预测目标从 P(vt+1|vt, L) 重新表述为 P(vt+1, ft|vt, L)，分解为两步：先由当前帧和语言指令预测光流（Motion Reasoning），再由光流和当前帧预测未来帧（Appearance Generation）。整个系统采用统一的 autoregressive Transformer，共享同一个 VQ-GAN tokenizer 处理 RGB 帧和光流场。FlowVLA reformulates the prediction objective of the world model from P(vt+1|vt, L) into P(vt+1, ft|vt, L), decomposing it into two steps: first predict the optical flow from the current frame and the language instruction (Motion Reasoning), then predict the future frame from the flow and the current frame (Appearance Generation). The whole system adopts a unified autoregressive Transformer and shares one and the same VQ-GAN tokenizer for RGB frames and flow fields.
图 2：FlowVLA 模型架构。RGB 帧和光流场使用同一个 VQ-GAN tokenizer 离散化为 token 序列，送入 autoregressive Transformer 进行联合建模。图中分开画出的 Image Tokenizer 和 Flow Tokenizer"in practice, they are the exact same module applied to both appearance and motion inputs"。Figure 2: The FlowVLA model architecture.RGB frames and optical-flow fields are discretised into token sequences by the same VQ-GAN tokenizer and fed into an autoregressive Transformer for joint modelling. Although the Image Tokenizer and the Flow Tokenizer are drawn separately in the figure, "in practice, they are the exact same module applied to both appearance and motion inputs".
Visual Chain of Thought（视觉思维链）Visual Chain of Thought
FlowVLA 的核心创新是将预测序列从 vt → vt+1 扩展为 vt → ft → vt+1，其中 ft 是 t 时刻的光流场。世界模型预训练的完整序列为：The core innovation of FlowVLA is to extend the prediction sequence from vt → vt+1 into vt → ft → vt+1, where ft is the optical-flow field at time t. The full sequence used for world-model pre-training is:
Swm = {Linstr, v0, f0, v1, f1, …, vT, fT}
训练目标为：The training objective is: LWM = Σ [LCE(ft | S<vt+1) + λ · LCE(vt+1 | S<vt+1, ft)] 第一项约束模型正确预测运动动态，第二项约束在运动条件下生成合理的未来外观。The first term constrains the model to predict motion dynamics correctly; the second constrains it to generate plausible future appearance conditioned on that motion.
统一光流 TokenizationUnified Flow Tokenization
光流场通过 VideoJAM 技术转换为 3 通道 RGB 表示，再经过非线性归一化处理：Flow fields are converted into a 3-channel RGB representation by the VideoJAM technique and then passed through a non-linear normalisation:
mnorm = min(1.0, m / (σ · √(H² + W²)))，其中 σ = 0.15mnorm = min(1.0, m / (σ · √(H² + W²))), where σ = 0.15
归一化后的光流 RGB 与原始 RGB 帧使用完全相同的 VQ-GAN tokenizer 离散化。这种设计避免了为光流单独训练编码器的额外开销，同时允许模型在统一的 token 空间内对运动和外观进行联合建模。The normalised flow RGB and the original RGB frames are discretised by exactly the same VQ-GAN tokenizer. This design avoids the extra cost of training a separate encoder for optical flow, and at the same time lets the model jointly model motion and appearance within a unified token space.
两阶段训练流程Two-Stage Training Pipeline
Stage 1：世界模型预训练Stage 1: World Model Pre-training
在大规模视频数据上，以交错序列格式预训练 autoregressive Transformer，同时学习光流预测和下一帧预测。预训练使模型具备物理合理的视觉预测能力。The autoregressive Transformer is pre-trained on large-scale video data in the interleaved sequence format, learning flow prediction and next-frame prediction at once. Pre-training endows the model with physically plausible visual forecasting ability.
Stage 2：策略微调Stage 2: Policy Fine-tuning
在机器人操作数据集上微调，动作 token 被附加到序列末尾，损失"only over the action tokens"。世界模型预训练的视觉表征为策略学习提供了良好初始化。Fine-tuning is performed on robot manipulation datasets: action tokens are appended to the end of the sequence and the loss is taken "only over the action tokens". The visual representations from world-model pre-training provide a good initialisation for policy learning.
03实验 ExperimentsExperiments
在三个层次的基准上评估：LIBERO（仿真，4 个 task suite）、SimplerEnv（仿真，视觉域迁移鲁棒性）、AgileX Cobot 双臂真实机器人（4 个操作任务）。主要对比基线为 UniVLA 和 WorldVLA。Evaluation is carried out on benchmarks at three levels: LIBERO (simulation, 4 task suites), SimplerEnv (simulation, robustness to visual domain shift), and the AgileX Cobot dual-arm real robot (4 manipulation tasks). The main baselines for comparison are UniVLA and WorldVLA.
FlowVLA 在平均成功率上超过 UniVLA 4.1 个百分点，在长时序任务（LIBERO-Long）上的提升最为显著（72.6% vs. 63.0%），体现了运动推理对长视野规划的重要性。FlowVLA exceeds UniVLA by 4.1 percentage points in average success rate, and the gain is most pronounced on the long-horizon suite (LIBERO-Long), at 72.6% vs. 63.0%, which shows how important motion reasoning is for long-horizon planning.
SimplerEnv 考察视觉域迁移鲁棒性。FlowVLA 在需要精细空间推理的"Stack Block"任务上相对 UniVLA 提升 20.9 个百分点（62.5% vs. 41.6%）。SimplerEnv probes robustness to visual domain shift. On the "Stack Block" task, which demands fine-grained spatial reasoning, FlowVLA improves over UniVLA by 20.9 percentage points (62.5% vs. 41.6%).
真实机器人实验（Table 3）Real-Robot Experiments (Table 3)
方法Method
Pick & Place
Grasp & Toss
Handover
Place Vegetable
平均Average
UniVLA
25.0
40.0
19.0
40.0
31.0
FlowVLA
30.0
50.0
36.0
60.0
44.0
图 3：AgileX Cobot 双臂实验平台及四项操作任务。包含单臂和双臂协作任务，覆盖抓取、放置、传递等典型操作场景。Figure 3: The AgileX Cobot dual-arm experimental platform and its four manipulation tasks.They include both single-arm and dual-arm cooperative tasks, covering typical manipulation scenarios such as grasping, placing and handover.
视觉预测质量分析Visual Prediction Quality Analysis
图 4：物理合理性对比。基线（无光流中间步骤）出现"disappearing manipulator or erratic object behavior"等失败模式；FlowVLA 生成"stable and physically coherent predictions"。Figure 4: Comparison of physical plausibility.The baseline (without the intermediate optical-flow step) exhibits failure modes such as "disappearing manipulator or erratic object behavior"; FlowVLA produces "stable and physically coherent predictions".图 5：语义对齐分析。基线模型在给定语言指令时无法将预测与指令对齐；FlowVLA"correctly interprets the command"，预测结果与语言指令保持一致。Figure 5: Semantic alignment analysis.Given a language instruction, the baseline model fails to align its prediction with the instruction; FlowVLA "correctly interprets the command", and its prediction stays consistent with the language instruction.
样本效率与收敛速度（Figure 6）Sample Efficiency and Convergence Speed (Figure 6)
图 6：收敛速度对比。使用全量数据时，FlowVLA 在约 2k 步即可达到基线 6k 步的性能峰值；在 50% 数据的低资源场景下，FlowVLA 峰值成功率为 0.48，比基线的 0.31 高出 55%。Figure 6: Comparison of convergence speed.With the full dataset, FlowVLA reaches at about 2k steps the peak performance the baseline needs 6k steps to attain; in the low-resource setting with 50% of the data, FlowVLA peaks at a success rate of 0.48, 55% higher than the baseline’s 0.31.
消融实验（Table 4）Ablation Study (Table 4)
配置Configuration
成功率 (%)Success rate (%)
完整 FlowVLAFull FlowVLA
73.0
移除 CoT（无光流预测步骤）Remove CoT (no flow prediction step)
64.0
移除 flow loss（光流无监督）Remove flow loss (flow left unsupervised)
三项组件缺一不可：Visual CoT 结构（+9.0 pp）、直接光流损失监督（+3.5 pp）、以及交错序列格式（vs. 分组格式 +23.6 pp）均对最终性能有显著贡献。交错格式的大幅优势表明，将运动 token 与外观 token 紧密交织对于捕获时序依赖至关重要。None of the three components can be dropped: the Visual CoT structure (+9.0 pp), direct supervision by the flow loss (+3.5 pp) and the interleaved sequence format (vs. the grouped format, +23.6 pp) all contribute significantly to the final performance. The large advantage of the interleaved format indicates that tightly interweaving motion tokens with appearance tokens is essential for capturing temporal dependencies.
04局限性 LimitationsLimitations
Note:论文未单独设置 Limitations 章节。以下各点：前两项为从论文设计与实验中推断（inferred）的局限，第三项为作者在正文中明确提及（stated）的局限。The paper has no separate Limitations section. Of the points below, the first two are limitations inferred from the design and experiments of the paper, while the third is a limitation the authors state explicitly in the main text.
光流质量的上限依赖（inferred）Ceiling imposed by optical-flow quality (inferred)
FlowVLA 的视觉思维链依赖光流预测的准确性作为中间步骤。若场景中存在快速运动、遮挡或纹理稀疏区域，光流估计本身可能失准，进而传递误差到未来帧预测和动作生成。论文未讨论光流预测失败时的 fallback 策略。The visual chain of thought of FlowVLA relies on the accuracy of flow prediction as its intermediate step. If the scene contains fast motion, occlusion or texture-sparse regions, the flow estimate itself may be inaccurate and propagate errors into future-frame prediction and action generation. The paper does not discuss any fallback strategy for when flow prediction fails.
序列长度与计算开销（inferred）Sequence length and computational overhead (inferred)
插入光流 token 将预训练序列长度约翻倍（Swm 同时包含 ft 和 vt+1），增加了预训练阶段的计算开销。论文未提供与基线的 FLOPs 或推理延迟对比数据，在实时机器人控制场景中的延迟影响尚不明确。Inserting flow tokens roughly doubles the pre-training sequence length (Swm contains both ft and vt+1), which increases the computational cost of the pre-training stage. The paper provides no FLOPs or inference-latency comparison against the baselines, so the latency impact in real-time robot control scenarios remains unclear.
真实机器人评测规模有限（stated）Limited scale of the real-robot evaluation (stated)
真实机器人实验（Table 3）仅在 AgileX Cobot 双臂平台上进行 4 项任务评测，每项任务仅报告单一成功率数值，缺乏跨平台、跨机器人形态的泛化性验证。作者承认这是当前工作的局限，并将更大规模的真实世界评测作为未来工作方向。The real-robot experiments (Table 3) evaluate only 4 tasks on the AgileX Cobot dual-arm platform, and report a single success-rate number per task, lacking any cross-platform or cross-embodiment generalisation check. The authors acknowledge this as a limitation of the current work and leave larger-scale real-world evaluation to future work.
