A Vision-Language-Action Model for Affordable and Efficient Robotics
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, Remi Cadene · Hugging Face / Sorbonne University / Valeo.ai / ENS Paris-Saclay
SmolVLA 是一个仅 0.45B 参数的紧凑型视觉-语言-动作模型，可在单张 GPU 上训练、在 CPU 上部署。通过 layer skipping、flow matching action expert 与异步推理三项核心设计，SmolVLA 在 LIBERO 仿真基准上以 87.3% 的成功率超越 10 倍体量的 π₀，并在真实机器人任务中展现出更快的任务完成速度。SmolVLA is a compact vision-language-action model with only 0.45B parameters, trainable on a single GPU and deployable on CPU. Through three core designs -- layer skipping, a flow matching action expert, and asynchronous inference -- SmolVLA surpasses the ten-times-larger π₀ with an 87.3% success rate on the LIBERO simulation benchmark, and completes real-robot tasks faster.
当前的视觉-语言-动作（VLA）模型通常拥有数十亿参数，导致训练成本极高、难以在真实场景中部署。这一"规模崇拜"使得机器人学习研究对大多数团队而言门槛极高。Current vision-language-action (VLA) models typically carry billions of parameters, which makes training extremely costly and real-world deployment difficult. This "cult of scale" puts robot learning research out of reach for most teams.
"existing VLA systems are typically massive–often with billions of parameters–leading to high training costs and limited real-world deployability."
SmolVLA 的核心主张是：性能优秀的 VLA 不必庞大。通过精心的架构设计和利用社区贡献的开放数据集，0.45B 参数的 SmolVLA 在多个基准上超越了 3B 参数的 π₀，同时大幅降低了训练与部署的资源门槛。The core claim of SmolVLA is that a strong VLA need not be large. Through careful architectural design and the use of openly contributed community datasets, the 0.45B-parameter SmolVLA surpasses the 3B-parameter π₀ on multiple benchmarks while sharply lowering the resource barrier to training and deployment.
SmolVLA 系统概览。左侧为紧凑型 VLM 骨干（SmolVLM-2，使用前 16 层，剪掉后半部分层），右侧为 Action Expert，采用交错 cross-attention 与 self-attention 的 flow matching 解码器，接受来自 VLM 的视觉/语言 token 并输出动作序列。Overview of the SmolVLA system. On the left is the compact VLM backbone (SmolVLM-2, using the first 16 layers with the latter half pruned away); on the right is the Action Expert, a flow matching decoder that interleaves cross-attention and self-attention, taking visual/language tokens from the VLM and emitting action sequences.
0.45B总参数量（含 action expert）Total parameters (including the action expert)
87.3%LIBERO 平均成功率（超越 3.3B π₀ 的 86.0%）LIBERO average success rate (beating the 86.0% of the 3.3B π₀)
57.3%Meta-World 平均成功率（超越 π₀ 的 47.9%）Meta-World average success rate (beating the 47.9% of π₀)
78.3%SO100 真实机器人平均成功率（超越 π₀ 的 61.7%）SO100 real-robot average success rate (beating the 61.7% of π₀)
02方法Method
SmolVLA 由三个核心组件构成：（1）通过 layer skipping 压缩的 SmolVLM-2 视觉语言骨干；（2）基于 flow matching 的 Action Expert；（3）将感知与执行解耦的异步推理系统。SmolVLA is built from three core components: (1) a SmolVLM-2 vision-language backbone compressed by layer skipping; (2) an Action Expert based on flow matching; (3) an asynchronous inference system that decouples perception from execution.
SmolVLA 直接复用预训练的 SmolVLM-2（含 SigLIP 视觉编码器），但仅保留 LLM 的前 16 层（共 32 层），舍弃后半部分。实验表明，"skipping layers from a large VLM yields better results than downsizing"：layer skipping 在 LIBERO 上达到 78.5%，而重新训练的缩小版 VLM 仅 75.8%。视觉 token 通过 pixel shuffling 压缩至每帧仅 64 个，进一步降低序列长度。SmolVLA directly reuses the pretrained SmolVLM-2 (including its SigLIP visual encoder) but keeps only the first 16 of the LLM's 32 layers, discarding the latter half. Experiments show that "skipping layers from a large VLM yields better results than downsizing": layer skipping reaches 78.5% on LIBERO, whereas a retrained, downsized VLM reaches only 75.8%. Visual tokens are compressed by pixel shuffling to just 64 per frame, further reducing sequence length.
Flow Matching Action Expert
Action Expert 是一个约 100M 参数的独立模块，采用交错排列的 cross-attention（CA）和 self-attention（SA）层。CA 从 VLM 输出的视觉/语言 token 中提取语义条件，SA 在动作序列内部建模时序依赖。训练使用 flow matching 目标，以 chunk size n 预测一段连续动作。消融实验证明：The Action Expert is a standalone module of about 100M parameters, built from interleaved cross-attention (CA) and self-attention (SA) layers. CA extracts semantic conditioning from the visual/language tokens emitted by the VLM, while SA models temporal dependencies inside the action sequence. Training uses a flow matching objective, predicting a stretch of continuous actions with chunk size n. Ablations show:
交错 CA+SA 最优（85.5%），优于纯 CA（79.0%）和纯 SA（74.5%）Interleaved CA+SA is best (85.5%), better than CA alone (79.0%) and SA alone (74.5%)
Flow matching 显著优于 regression（80.25% vs 75.25%）Flow matching clearly outperforms regression (80.25% vs 75.25%)
将机器人状态输入 VLM（而非 expert）效果更好（80.3% vs 73.3%）Feeding the robot state into the VLM (rather than the expert) works better (80.3% vs 73.3%)
异步推理架构。PolicyServer 运行感知与动作预测（可在远程 GPU 服务器），RobotClient 执行动作队列（本地 CPU）。两者并行运行，感知不再阻塞执行，实现更高频率的控制。Asynchronous inference architecture. The PolicyServer runs perception and action prediction (possibly on a remote GPU server), while the RobotClient executes the action queue (on the local CPU). The two run in parallel, so perception no longer blocks execution, enabling higher-frequency control.
异步推理：解耦感知与执行Asynchronous Inference: Decoupling Perception from Execution
传统同步推理中，机器人在等待模型推理时处于静止状态，严重限制控制频率。SmolVLA 引入 asynchronous inference：PolicyServer 持续运行 VLA 推理并将动作块写入共享队列，RobotClient 持续从队列取出动作执行，两者完全并行。当队列长度超过阈值 g 时，重新触发感知更新。该设计不仅提升了速度，还允许将模型部署在远程 GPU 服务器，本体侧只需 CPU。Under conventional synchronous inference the robot stands still while waiting for the model, which severely limits control frequency. SmolVLA introduces asynchronous inference: the PolicyServer keeps running VLA inference and writes action chunks into a shared queue, while the RobotClient keeps popping actions from the queue and executing them, the two fully in parallel. When the queue length exceeds a threshold g, a perception update is re-triggered. This design not only raises speed but also allows the model to be deployed on a remote GPU server, leaving only a CPU on the robot itself.
预训练数据集Pretraining Dataset
SmolVLA 使用来自 Hugging Face Hub 的 481 个社区数据集进行预训练，经过 embodiment 类型、episode 数量、数据质量、帧覆盖度等过滤后，最终保留约 22,900 条轨迹、10.6M 帧。该规模"比其他 SOTA VLA 方法小至少一个数量级"，所有数据均公开可得，无专有数据。预训练使用 4 张 GPU，总计约 30,000 GPU 小时。SmolVLA is pretrained on 481 community datasets from the Hugging Face Hub; after filtering by embodiment type, episode count, data quality and frame coverage, about 22,900 trajectories and 10.6M frames remain. This scale is "at least one order of magnitude smaller than other SOTA VLA methods", and all data is publicly available, with none of it proprietary. Pretraining used 4 GPUs for roughly 30,000 GPU hours in total.
03实验Experiments
在 LIBERO 仿真基准、Meta-World 多任务基准和真实机器人（SO100 / SO101）上与 Diffusion Policy、Octo、OpenVLA、π₀ 等基线对比。Compared against baselines such as Diffusion Policy, Octo, OpenVLA and π₀ on the LIBERO simulation benchmark, the Meta-World multi-task benchmark, and real robots (SO100 / SO101).
LIBERO 仿真基准LIBERO Simulation Benchmark
方法Method
参数量Params
Spatial
Object
Goal
Long
平均Average
Diffusion Policy
—
78.3%
92.5%
68.3%
50.5%
72.4%
Octo
0.09B
78.9%
85.7%
84.6%
51.1%
75.1%
OpenVLA
7B
84.7%
88.4%
79.2%
53.7%
76.5%
π₀ (pretrained)
3.3B
90%
86%
95%
73%
86.0%
SmolVLA
0.45B
90%
96%
92%
71%
87.3%
SmolVLA 以 0.45B 参数超越了 3.3B 参数的 π₀（pretrained），平均成功率 87.3% vs 86.0%。With 0.45B parameters SmolVLA surpasses the 3.3B-parameter π₀ (pretrained), at an average success rate of 87.3% vs 86.0%.
Meta-World 多任务基准Meta-World Multi-Task Benchmark
方法Method
参数量Params
Easy
Medium
Hard
Very Hard
平均Average
Diffusion Policy
—
23.1%
10.7%
1.9%
6.1%
10.5%
TinyVLA
—
77.6%
21.5%
11.4%
15.8%
31.6%
π₀ (pretrained)
3.5B
71.8%
48.2%
41.7%
30.0%
47.9%
SmolVLA
0.45B
82.5%
41.8%
45.0%
60.0%
57.3%
真实机器人：SO100Real Robot: SO100
方法Method
Pick-Place
Stacking
Sorting
平均Average
ACT (single-task)
70%
50%
25%
48.3%
π₀ (multi-task)
100%
40%
45%
61.7%
SmolVLA (multi-task)
75%
90%
70%
78.3%
真实机器人：SO101Real Robot: SO101
方法Method
In-Distribution
Out-of-Distribution
ACT
70%
40%
SmolVLA
90%
50%
四类真实机器人任务的起始帧与终止帧，涵盖 SO100 和 SO101 机器人平台。任务包括抓取放置、堆叠和分类等不同难度的操作。Start and end frames of four kinds of real-robot task, spanning the SO100 and SO101 robot platforms. The tasks include pick-and-place, stacking and sorting, at differing levels of difficulty.
异步推理加速效果Speedup from Asynchronous Inference
不同阈值 g 下的动作队列随时间演化。异步推理在 pick-place 任务上将完成时间从 13.75s 缩短至 9.7s（约快 30%），在固定时间内成功次数提升 2.1 倍（19 次 vs 9 次）。Evolution of the action queue over time under different thresholds g. On the pick-place task, asynchronous inference shortens completion time from 13.75s to 9.7s (about 30% faster), and raises the number of successes within a fixed time budget by a factor of 2.1 (19 vs 9).
消融实验要点Key Ablations
关键设计选择的消融（均在 LIBERO 上测试）：Ablations of the key design choices (all tested on LIBERO):
Attention 结构：交错 CA+SA 最优（85.5%），纯 CA（79.0%）和纯 SA（74.5%）均更差Attention structure: interleaved CA+SA is best (85.5%); CA alone (79.0%) and SA alone (74.5%) are both worse
Layer skipping：保留前 16 层（78.5%）优于直接缩小 VLM（75.8%）Layer skipping: keeping the first 16 layers (78.5%) beats directly downsizing the VLM (75.8%)
Chunk size：n=10 最优（84.0%），过大（n=100: 74.5%）或过小（n=1: 50.0%）均下降Chunk size: n=10 is optimal (84.0%); too large (n=100: 74.5%) or too small (n=1: 50.0%) both degrade it
状态输入位置：将机器人状态输入 VLM（80.3%）优于输入 action expert（73.3%）State input location: feeding the robot state into the VLM (80.3%) beats feeding it into the action expert (73.3%)
04局限性Limitations
Note:以下局限性由作者在论文中明确陈述（stated），无需推断标注。The limitations below are explicitly stated by the authors in the paper; no inference labelling is needed.
预训练数据规模有限且来源单一Pretraining data is limited in scale and drawn from a single source
预训练数据集约 22,900 条轨迹（10.6M 帧），"at least one order of magnitude smaller than other state-of-the-art" VLA 方案。全部数据来自 SO100 单一 embodiment，泛化到不同机器人形态时需要额外 fine-tuning，且跨 embodiment 迁移能力尚未充分验证。The pretraining dataset holds about 22,900 trajectories (10.6M frames), "at least one order of magnitude smaller than other state-of-the-art" VLA solutions. All of the data comes from the single SO100 embodiment, so generalizing to different robot morphologies requires additional fine-tuning, and cross-embodiment transfer has not been fully validated.
长时序任务表现受限Limited performance on long-horizon tasks
在 LIBERO-Long（需要更长规划视野）上，SmolVLA 达到 71%，低于其在 Object（96%）和 Spatial（90%）子集的表现，表明对长时序、多步骤任务的建模能力仍有提升空间。On LIBERO-Long (which demands a longer planning horizon) SmolVLA reaches 71%, below its performance on the Object (96%) and Spatial (90%) subsets, indicating that its modelling of long-horizon, multi-step tasks still has room to improve.
VLM 骨干的预训练任务与机器人控制存在偏差The VLM backbone's pretraining tasks are mismatched with robot control
作者指出，VLM 骨干 SmolVLM-2 "pretrained mainly on document reading and OCR tasks"，其视觉语义理解与机器人操控场景存在领域差距，这可能限制其对操作场景的细粒度理解（如物体姿态、接触关系）。The authors point out that the VLM backbone SmolVLM-2 was "pretrained mainly on document reading and OCR tasks", so its visual semantic understanding has a domain gap with robot manipulation scenes, which may limit its fine-grained understanding of manipulation settings (such as object pose and contact relations).
异步推理引入系统复杂性Asynchronous inference introduces system complexity
异步推理系统需要在 PolicyServer 和 RobotClient 之间维护共享动作队列和状态同步，引入了额外的工程复杂性。队列阈值超参数 g 的选择对实际性能有影响，需针对不同任务调优。The asynchronous inference system has to maintain a shared action queue and state synchronization between the PolicyServer and the RobotClient, which introduces extra engineering complexity. The choice of the queue threshold hyperparameter g affects real-world performance and must be tuned per task.
