ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models
在动作空间中进行链式推理，弥合语义与运动控制的鸿沟Chain-of-thought reasoning in action space, bridging the gap between semantics and motion control
Linqing Zhong, Yi Liu, Yifei Wei, Ziyu Xiong, Maoqing Yao, Si Liu, Guanghui Ren · Beihang University & AgiBot
ACoT-VLA 提出了一种全新的 Action Chain-of-Thought（ACoT）范式：不同于已有方法在语言空间或视觉空间中进行中间推理，ACoT 直接在动作空间中构建"粗粒度动作意图序列"，为最终策略提供同质化的运动先验。架构由 Explicit Action Reasoner（显式参考轨迹合成）和 Implicit Action Reasoner（隐式动作先验提取）两部分组成，协同引导最终的 Action-Guided Prediction 解码，在仿真与真实机器人实验中均取得最先进性能。ACoT-VLA introduces a new Action Chain-of-Thought (ACoT) paradigm: unlike existing methods that perform intermediate reasoning in language space or visual space, ACoT builds a "coarse-grained action intention sequence" directly in action space, providing the final policy with a homogeneous motion prior. The architecture consists of an Explicit Action Reasoner (synthesis of explicit reference trajectories) and an Implicit Action Reasoner (extraction of implicit action priors), which jointly guide the final Action-Guided Prediction decoding, achieving state-of-the-art performance in both simulation and real-robot experiments.
现有 VLA 模型从互联网规模的语义数据中获取了丰富知识，但缺乏对物理动力学的理解。语言 CoT 和视觉 CoT 两种主流中间推理范式，均因"语义-运动异质性（semantic-kinematic gap）"而难以为精确执行提供有效引导。Existing VLA models acquire rich knowledge from internet-scale semantic data, yet lack an understanding of physical dynamics. The two mainstream intermediate-reasoning paradigms, language CoT and visual CoT, both struggle to provide effective guidance for precise execution because of the "semantic-kinematic gap".
"Language CoT predicts sub-tasks as intermediate reasoning. Visual CoT synthesizes a goal image to provide guidance for action policy. Our proposed Action CoT directly operates in action space and provides homogeneous action guidance."
图 1： 三种 Chain-of-Thought 范式对比。(a) Language CoT 在语言空间预测子任务；(b) Visual CoT 合成目标图像；(c) ACoT 直接在动作空间构建粗粒度意图序列，提供与执行同质化的引导。Figure 1: Comparison of three Chain-of-Thought paradigms. (a) Language CoT predicts sub-tasks in language space; (b) Visual CoT synthesizes a goal image; (c) ACoT builds a coarse-grained intention sequence directly in action space, providing guidance homogeneous with execution.
98.5%LIBERO 平均成功率（4个任务集）LIBERO average success rate (4 task suites)
+1.6%超越前 SOTA π0.5 的绝对提升Absolute gain over the previous SOTA π0.5
86.6%LIBERO-Plus 零样本迁移成功率LIBERO-Plus zero-shot transfer success rate
ACoT-VLA 以预训练 VLM（Gemma 2B + SigLIP）为骨干，在其特征之上并联两个推理器，最终由 Action-Guided Prediction（AGP）head 融合两路引导，通过扩散去噪输出可执行动作序列。ACoT-VLA takes a pre-trained VLM (Gemma 2B + SigLIP) as its backbone and runs two reasoners in parallel on top of its features; an Action-Guided Prediction (AGP) head finally fuses the two guidance streams and outputs an executable action sequence through diffusion denoising.
图 2： ACoT-VLA 整体架构。(a) Explicit Action Reasoner (EAR)：基于 Transformer，对含噪动作序列施加 self-attention 及与 VLM 特征的 cross-attention，合成粗粒度参考轨迹；(b) Implicit Action Reasoner (IAR)：用可学习 query 对 VLM 多层内部表征进行 cross-attention，提取隐式动作先验；(c) Action-Guided Prediction (AGP) head：通过双路 cross-attention 融合显式与隐式引导，再经 self-attention 后解码最终动作。Figure 2: Overall architecture of ACoT-VLA. (a) Explicit Action Reasoner (EAR): a Transformer that applies self-attention over the noisy action sequence together with cross-attention to VLM features, synthesizing a coarse-grained reference trajectory; (b) Implicit Action Reasoner (IAR): learnable queries cross-attend to the multi-layer internal representations of the VLM to extract implicit action priors; (c) Action-Guided Prediction (AGP) head: fuses explicit and implicit guidance through dual cross-attention, then decodes the final action after self-attention.
Explicit Action Reasoner (EAR)
EAR 是一个轻量 Transformer 模块，输入含噪动作序列，通过 self-attention 建模序列内部关系，再通过与 VLM 特征的 cross-attention 注入视觉语言上下文，输出粗粒度参考轨迹作为显式运动引导。EAR 的监督信号来自于 ground-truth 动作的加噪版本，损失权重 λ₁ = 0.5。EAR is a lightweight Transformer module that takes a noisy action sequence as input, models intra-sequence relations through self-attention, injects vision-language context through cross-attention with VLM features, and outputs a coarse-grained reference trajectory as explicit motion guidance. Its supervision signal comes from a noised version of the ground-truth actions, with loss weight λ₁ = 0.5.
Implicit Action Reasoner (IAR)
IAR 使用一组可学习 query，通过 cross-attention 对 VLM 各层内部表征进行聚合，提炼与动作相关的隐式先验。为抑制噪声，IAR 对 key-value 对进行下采样。提取到的隐式先验与 EAR 的显式轨迹共同输入 AGP head，起到互补的增益效果，损失权重 λ₂ = 0.5。IAR uses a set of learnable queries that aggregate the internal representations of every VLM layer through cross-attention, distilling action-related implicit priors. To suppress noise, IAR downsamples the key-value pairs. The extracted implicit priors are fed into the AGP head together with the explicit trajectory from EAR, producing complementary gains, with loss weight λ₂ = 0.5.
在 LIBERO、LIBERO-Plus（零样本迁移）、VLABench 三个仿真 benchmark 以及 AgiBot G1 真实机器人平台上进行全面评测，与 40+ 条 baseline 进行对比，包括 Diffusion Policy、OpenVLA、π0、π0.5、WorldVLA、DreamVLA 等。Comprehensive evaluation on three simulation benchmarks — LIBERO, LIBERO-Plus (zero-shot transfer) and VLABench — as well as on the AgiBot G1 real-robot platform, compared against 40+ baselines including Diffusion Policy, OpenVLA, π0, π0.5, WorldVLA and DreamVLA.
图 3： 真实世界三个操作任务可视化：Wipe Stain（擦拭污渍）、Pour Water（倒水）、Open-set Pick（开集拾取）。Figure 3: Visualization of the three real-world manipulation tasks: Wipe Stain, Pour Water and Open-set Pick.图 4： 真实世界实验定量结果。ACoT-VLA 在 AgiBot G1 平台上平均成功率为 66.7%，超过 π0.5 的 61.0%，同时展示了在 AgileX 平台上的跨平台适应能力。Figure 4: Quantitative results of the real-world experiments. ACoT-VLA attains an average success rate of 66.7% on the AgiBot G1 platform, surpassing the 61.0% of π0.5, while also demonstrating cross-platform adaptability on the AgileX platform.
消融实验Ablation Studies
在 LIBERO benchmark 上逐步加入各组件的消融分析：Ablation analysis on the LIBERO benchmark, adding each component step by step:
配置Configuration
LIBERO 平均 SRLIBERO Average SR
Δ vs baseline
π0.5（baseline）π0.5 (baseline)
96.9%
—
+ EAR only
98.3%
+1.4%
+ IAR only
98.1%
+1.2%
+ EAR + IAR（完整 ACoT-VLA）+ EAR + IAR (full ACoT-VLA)
98.5%
+1.6%
EAR 和 IAR 各自均带来显著提升，两者组合呈现协同增益（synergistic benefits），说明显式轨迹引导与隐式行为先验具有互补性。EAR and IAR each bring notable improvements, and their combination exhibits synergistic benefits, showing that explicit trajectory guidance and implicit behavioral priors are complementary.
04局限性Limitations
注：论文将局限性讨论置于 Appendix D（Future Works），正文未单独列出；以下各条结合作者陈述（stated）及设计特点推断（inferred）标注。Note: The paper places its limitations discussion in Appendix D (Future Works) and does not list them separately in the main text; each item below is labeled according to whether it is stated by the authors (stated) or inferred from design characteristics (inferred).
粗粒度动作推理的精度瓶颈（stated）Precision bottleneck of coarse-grained action reasoning (stated)
EAR 生成的参考轨迹为"粗粒度（coarse-grained）"意图，在高精度、高速度操作任务中可能不足以覆盖所有运动细节。作者明确表示未来工作将探索更精细的动作空间表征。The reference trajectory produced by EAR is a "coarse-grained" intention and may be insufficient to cover every motion detail in high-precision, high-speed manipulation tasks. The authors explicitly state that future work will explore finer-grained action-space representations.
计算资源依赖较重（inferred）Heavy dependence on computational resources (inferred)
训练使用单节点 8× NVIDIA H100 GPU，双路推理器（EAR + IAR）与 VLM 骨干同时前向传播，相比单一 baseline（π0.5）推理开销更高，对资源受限平台的部署存在挑战。Training uses a single node with 8× NVIDIA H100 GPUs, and the dual reasoners (EAR + IAR) run forward together with the VLM backbone, so inference cost is higher than that of the single baseline (π0.5), which poses challenges for deployment on resource-constrained platforms.
真实世界实验仅在 AgiBot G1 和 AgileX 两款平台上进行验证，跨具身（cross-embodiment）泛化能力尚未在更多机器人类型上得到充分证明；作者将扩展至更大规模跨实体场景列为未来方向之一。Real-world experiments are validated on only two platforms, AgiBot G1 and AgileX, so cross-embodiment generalization has not been fully demonstrated across a wider range of robot types; the authors list extending to larger-scale cross-embodiment scenarios as one of their future directions.
模型容量扩展研究不足（inferred）Insufficient study of model-capacity scaling (inferred)
当前骨干为 Gemma 2B，作者指出"探索更大模型容量（scaling to larger model capacities）"是重要的后续方向，现有结论对更大规模 VLM 的有效性尚未验证。The current backbone is Gemma 2B, and the authors note that "scaling to larger model capacities" is an important follow-up direction; the effectiveness of the present conclusions on larger-scale VLMs has not yet been verified.
