A Survey on Vision-Language-Action Models for Embodied AI

01 动机Motivation

传统基于强化学习的机器人策略"largely focused on addressing a limited set of tasks within controlled environments"，难以泛化到真实复杂场景。随着大型语言模型（LLM）和视觉-语言模型（VLM）的崛起，将语言理解、视觉感知与机器人动作生成统一到同一模型的需求日益迫切——这正是 VLA 模型的核心出发点。Traditional reinforcement-learning-based robot policies have "largely focused on addressing a limited set of tasks within controlled environments", and struggle to generalize to real, complex scenes. With the rise of large language models (LLM) and vision-language models (VLM), the need to unify language understanding, visual perception and robot action generation within a single model has become increasingly urgent — and this is precisely the starting point of VLA models.

"Embodied AI is widely recognized as a cornerstone of artificial general intelligence (AGI)."

VLA general architecture — **图 1：VLA 模型的通用架构。**三种代表性动作预测方式（回归、离散化、扩散）被展示为并列路径。VLA 模型以多模态观测（视觉 + 语言指令）为输入，通过不同 action head 输出机器人动作序列。图中虚线框标注了相关组件（预训练视觉表征、世界模型等）。**Figure 1: The general architecture of VLA models.**Three representative action-prediction schemes (regression, discretization, diffusion) are shown as parallel paths. A VLA model takes multimodal observations (vision + language instruction) as input and outputs a robot action sequence through different action heads. The dashed boxes mark related components (pre-trained visual representations, world models, etc.).

3研究方向
（组件 / 低层策略 / 任务规划）research directions
(components / low-level policies / task planning)

50+低层控制策略
系统梳理（Table III）low-level control policies
systematically reviewed (Table III)

400MCLIP 训练图文对
（WIT 数据集）CLIP training image-text pairs
(WIT dataset)

8未来挑战方向
（Section VI）future challenge directions
(Section VI)

VLA evolution timeline — **图 2(b)：从单模态模型到 VLA 模型的演化时间线。**2017 年 Transformer 奠基，2021 年 Gato / Decision Transformer 出现，2023 年 RT-2 正式提出"VLA"术语，2024–2026 年 OpenVLA、π₀、RDT 等大参数量 VLA 快速涌现。**Figure 2(b): Evolution timeline from unimodal models to VLA models.**Transformer laid the foundation in 2017, Gato / Decision Transformer appeared in 2021, RT-2 formally introduced the term "VLA" in 2023, and from 2024 to 2026 large-parameter VLAs such as OpenVLA, π₀ and RDT emerged rapidly.

02 VLA 分类体系VLA Taxonomy

本综述提出一个层级化分类框架（hierarchical framework），将 VLA 系统分为三大研究方向：① VLA 组件（Components）、② 低层控制策略（Low-Level Control Policies）、③ 高层任务规划器（High-Level Task Planners）。低层策略负责执行具体的动作序列，高层规划器负责把复杂指令分解为可执行子任务。This survey proposes a hierarchical framework that divides VLA systems into three major research directions: ① VLA Components, ② Low-Level Control Policies, and ③ High-Level Task Planners. Low-level policies are responsible for executing concrete action sequences, while high-level planners decompose complex instructions into executable subtasks.

Hierarchical robot policy — **图 4：层级机器人策略示意。Figure 4: Illustration of a hierarchical robot policy.**"The high-level task planner decomposes the user instruction into subtasks, which are then executed step by step by the low-level control policy." 两层架构使系统具备长时程任务执行能力。The two-layer architecture gives the system the ability to execute long-horizon tasks.

① VLA 组件 (Components)① VLA Components

预训练视觉表征 (PVR)：CLIP、R3M、VC-1、TheiaPre-trained Visual Representations (PVR): CLIP, R3M, VC-1, Theia
视频表征学习Video representation learning
Dynamics Learning：Vi-PRoM、MaskDPDynamics Learning: Vi-PRoM, MaskDP
World Models：用于策略预测World Models: used for policy prediction
Reasoning 模块Reasoning modules
Policy Steering（如 RLHF）Policy Steering (e.g., RLHF)

② 低层控制策略 (Low-Level Policies)② Low-Level Control Policies

Transformer-based：RT-1、RT-2、OctoTransformer-based: RT-1, RT-2, Octo
Diffusion-based：Diffusion Policy、MDT、RDT-1BDiffusion-based: Diffusion Policy, MDT, RDT-1B
3D Vision：基于点云 / 3D 语义特征3D Vision: based on point clouds / 3D semantic features
Point-based Actions
Large VLAs：OpenVLA（7B）、π₀（22B）Large VLAs: OpenVLA (7B), π₀ (22B)
Motion Planning 集成Motion Planning integration

③ 高层任务规划器 (Task Planners)③ High-Level Task Planners

Monolithic：PaLM-E、EmbodiedGPT、LEOMonolithic: PaLM-E, EmbodiedGPT, LEO
Modular：工具调用链、代码生成规划Modular: tool-invocation chains, code-generation planning
3D-LLM：融合 3D 场景理解3D-LLM: integrating 3D scene understanding
长时程任务分解Long-horizon task decomposition
多步推理与反馈Multi-step reasoning and feedback

预训练视觉表征（PVR）对比Comparison of Pre-trained Visual Representations (PVR)

Table I 系统对比了常用 PVR 方法，包括网络类型、预训练目标及适用机器人任务：Table I systematically compares common PVR methods, covering network type, pre-training objective and applicable robot tasks:

方法Method	网络类型Network Type	预训练目标Pre-training Objective	特点Characteristics
CLIP	ViT / ResNet	对比学习（图文对齐）Contrastive learning (image-text alignment)	400M 图文对，泛化强400M image-text pairs, strong generalization
R3M	ResNet-50	时序对比 + 语言对齐Temporal contrast + language alignment	专为机器人操作设计Designed specifically for robot manipulation
VC-1	ViT-B	MAE + 像素重建MAE + pixel reconstruction	像素级细节更优Better pixel-level detail
DINOv2	ViT	自监督蒸馏Self-supervised distillation	强空间特征，无标签Strong spatial features, label-free
Theia	ViT	多任务综合Multi-task synthesis	综合评测最优Best in overall evaluation

动作预测三大范式Three Paradigms of Action Prediction

Transformer-based 策略Transformer-based policies

将动作序列建模为 token 序列（如 RT-1 离散化为 256 bins/维度），利用 causal Transformer 做 next-token prediction。RT-2 进一步将动作 token 与语言 token 混合，直接从 VLM 输出动作。Action sequences are modeled as token sequences (e.g., RT-1 discretizes each dimension into 256 bins), and a causal Transformer performs next-token prediction. RT-2 goes further by mixing action tokens with language tokens, emitting actions directly from the VLM.

Diffusion-based 策略Diffusion-based policies

以 Diffusion Policy 为代表，将动作生成建模为去噪过程（DDPM），能够建模多模态动作分布，避免均值回归问题。RDT-1B 扩展至 1.2B 参数并展示出 "zero-shot generalization"。Represented by Diffusion Policy, action generation is modeled as a denoising process (DDPM), which can capture multimodal action distributions and avoid the mean-regression problem. RDT-1B scales to 1.2B parameters and demonstrates "zero-shot generalization".

Embodied AI concepts Venn diagram — **图 2(a)：具身 AI 核心概念 Venn 图。**VLA 模型处于 Vision、Language、Action 三个模态的交叉区域，涵盖感知、理解和行动三个层面，是实现通用机器人智能的关键桥梁。**Figure 2(a): Venn diagram of the core embodied-AI concepts.**VLA models sit at the intersection of the Vision, Language and Action modalities, spanning the levels of perception, understanding and action, and are a key bridge toward general robotic intelligence.

03 代表性 VLA 系统与资源Representative VLA Systems and Resources

综述梳理了 50+ 低层控制策略（Table III）和数十种高层规划器（Table IV），并整合了主流数据集与 Benchmark。以下展示 RT 系列和大型 VLA 的关键里程碑，以及核心挑战。The survey reviews 50+ low-level control policies (Table III) and dozens of high-level planners (Table IV), and consolidates mainstream datasets and benchmarks. Below are the key milestones of the RT series and of large VLAs, together with the core challenges.

RT 系列里程碑Milestones of the RT Series

方法Method	年份Year	核心创新Key Innovation	规模Scale
RT-1	2022	大规模多任务机器人 TransformerLarge-scale multi-task robot Transformer	~35M 参数~35M parameters
RT-2	2023	首次提出"VLA"，将 VLM 输出动作 tokenFirst to introduce "VLA", letting the VLM output action tokens	55B（PaLI-X）55B (PaLI-X)
RT-H	2024	层级化动作 token（语言 → 子任务 → 动作）Hierarchical action tokens (language → subtask → action)	–
RT-X / RT-2-X	2023	OXE 跨机器人数据集，"orders of magnitude larger"OXE cross-robot dataset, "orders of magnitude larger"	55B

大参数量 VLA（Large VLAs）Large-Parameter VLAs (Large VLAs)

方法Method	参数量Parameters	特点Characteristics
OpenVLA	7B	开源 VLA，基于 LLaMA，支持微调Open-source VLA based on LLaMA, supports fine-tuning
π₀（pi zero）π₀ (pi zero)	~22B	流匹配（flow matching）动作生成Action generation via flow matching
RDT-1B	1.2B	扩散 Transformer，"zero-shot generalization"Diffusion Transformer, "zero-shot generalization"
RoboMamba	–	Mamba 架构，推理效率更高Mamba architecture, higher inference efficiency
SpatialVLA	–	空间感知增强 VLASpatially-aware enhanced VLA

主要 Benchmark 与数据集Main Benchmarks and Datasets

综述在 Table V 整理了主流数据集，涵盖真实环境与仿真环境：In Table V the survey organizes mainstream datasets, covering both real-world and simulated environments:

OXE（Open X-Embodiment）：跨机器人多任务聚合数据集，规模"orders of magnitude larger than previous datasets"OXE (Open X-Embodiment): a cross-robot multi-task aggregated dataset, "orders of magnitude larger than previous datasets"
RLBench：100+ 操作任务的仿真 BenchmarkRLBench: a simulation benchmark with 100+ manipulation tasks
LIBERO：终身学习 / 持续学习评测LIBERO: lifelong / continual learning evaluation
MetaWorld：50 个操作任务，标准化 BenchmarkMetaWorld: 50 manipulation tasks, a standardized benchmark
CALVIN：语言条件长时程操作评测CALVIN: language-conditioned long-horizon manipulation evaluation

未来挑战（Section VI 八大方向）Future Challenges (eight directions in Section VI)

Safety First

真实部署中的安全约束与 fail-safe 机制Safety constraints and fail-safe mechanisms in real-world deployment

Datasets & Benchmarks

数据稀缺、跨机器人一致性评测Data scarcity, and consistent evaluation across robots

Foundation Models & Generalization

跨域迁移、开放世界泛化Cross-domain transfer and open-world generalization

Multimodality

触觉、力觉、声音等多模态融合Fusion of tactile, force, audio and other modalities

Long-Horizon Tasks

复杂序列任务的分解与执行Decomposition and execution of complex sequential tasks

Real-Time Responsiveness

LVLA 推理延迟在动态环境中的影响The impact of LVLA inference latency in dynamic environments

Multiagent Systems

多具身智能体协作与协调Cooperation and coordination among multiple embodied agents

Ethical & Societal Implications

伦理规范与社会影响评估Ethical norms and assessment of societal impact

04 局限性Limitations

说明：以下局限性部分为作者在论文中明确陈述（标注"stated"），部分为从综述内容中推断（标注"inferred"）。Note:Some of the limitations below are explicitly stated by the authors in the paper (marked "stated"), while others are inferred from the survey's content (marked "inferred").

推理速度瓶颈（stated）Inference speed bottleneck (stated)

作者明确指出大型 VLA 的"slow inference speed can significantly impact performance in dynamic environments, as changes may occur during inference"，需要量化压缩、early-exit 等技术缓解。The authors explicitly point out that for large VLAs "slow inference speed can significantly impact performance in dynamic environments, as changes may occur during inference", which calls for mitigation through quantization, compression, early-exit and similar techniques.

离散动作表示的局限（stated）Limitations of discrete action representations (stated)

离散化动作空间会导致"early grasping issues"，且对于"pouring water into a cup"等需要额外自由度的任务，SE(2) 动作离散化不足以描述真实操作需求。A discretized action space leads to "early grasping issues", and for tasks that require additional degrees of freedom, such as "pouring water into a cup", SE(2) action discretization is not expressive enough for real manipulation needs.

数据稀缺与跨具身泛化（stated）Data scarcity and cross-embodiment generalization (stated)

机器人学习领域数据获取成本高，跨不同机器人形态的泛化仍是重大挑战。OXE 等跨机器人数据集是初步尝试，但与语言领域数据规模仍有量级差距。Data acquisition in robot learning is costly, and generalizing across different robot morphologies remains a major challenge. Cross-robot datasets such as OXE are a preliminary attempt, but their scale still lags the language domain by orders of magnitude.

综述覆盖的时效性（inferred）Timeliness of the survey's coverage (inferred)

VLA 领域发展极快（截至 v8，2026 年 5 月），综述所梳理的方法可能在出版后迅速过时。作者维护的 GitHub 仓库是持续更新的补充资源，但仍难以做到实时追踪。The VLA field is advancing extremely fast (as of v8, May 2026), so the methods reviewed may become outdated soon after publication. The GitHub repository maintained by the authors is a continuously updated supplement, yet real-time tracking remains hard to achieve.

定量性能对比缺失（inferred）Absence of quantitative performance comparison (inferred)

综述以定性分类为主，缺少统一 Benchmark 上的定量对比表格（成功率、延迟等），使读者难以直接判断各方法的相对优劣。这在一定程度上降低了综述作为选型参考的实用性。The survey is mainly a qualitative taxonomy and lacks quantitative comparison tables on a unified benchmark (success rate, latency, etc.), which makes it hard for readers to directly judge the relative merits of each method. To some extent this reduces its practical value as a reference for method selection.