Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifei Feng, Jian Tang
TinyVLA 采用轻量级视觉语言模型（70M–1.4B 参数）结合 Diffusion Policy 解码器，彻底摆脱了大规模机器人预训练数据集的依赖，同时将推理延迟压缩至 OpenVLA 的 1/20，在真实机器人操作任务上平均成功率超越 OpenVLA 达 25.7 个百分点。TinyVLA couples a lightweight vision-language model (70M–1.4B parameters) with a Diffusion Policy decoder, entirely removing the dependence on large-scale robot pre-training datasets, while compressing inference latency to 1/20 of OpenVLA's and surpassing OpenVLA by 25.7 percentage points in average success rate on real-robot manipulation tasks.
现有 VLA 模型（如 OpenVLA）存在两大核心瓶颈：其一，需要在包含 970K 条样本的大规模机器人数据集上进行耗时预训练；其二，基于 7B+ 参数语言模型的自回归 token 生成导致推理延迟高达 292ms，无法满足实时控制需求。Existing VLA models such as OpenVLA face two core bottlenecks. First, they require time-consuming pre-training on a large-scale robot dataset of 970K samples. Second, autoregressive token generation on top of a 7B+ parameter language model drives inference latency up to 292ms, which cannot meet real-time control requirements.
"Our framework achieves faster inference speeds, and improved data efficiency, eliminating the need for pre-training stage."
图 1：推理延迟与平均成功率对比（真实世界 Franka 机械臂，5 个任务）。气泡大小表示模型参数量。TinyVLA-H 在 A6000 GPU 上仅需 14ms 推理延迟，比 OpenVLA（292ms）快 20 倍，同时平均成功率高出约 25.7%。Figure 1: Inference latency versus average success rate (real-world Franka arm, 5 tasks). Bubble size denotes model parameter count. On an A6000 GPU, TinyVLA-H needs only 14ms of inference latency, 20 times faster than OpenVLA (292ms), while its average success rate is about 25.7% higher.
TinyVLA 由三部分组成：（1）预训练的紧凑型 VLM 骨干（70M–1.4B 参数，基于 LLaVA 管线 + Pythia 语言后端）；（2）LoRA 参数高效微调；（3）Diffusion Policy 动作解码器。整个框架无需机器人数据预训练，直接在少量示范数据上端到端微调。TinyVLA has three components: (1) a pre-trained compact VLM backbone (70M–1.4B parameters, built on the LLaVA pipeline with a Pythia language backend); (2) LoRA parameter-efficient fine-tuning; (3) a Diffusion Policy action decoder. The whole framework requires no robot-data pre-training and is fine-tuned end-to-end directly on a small amount of demonstration data.
图 2：TinyVLA 模型架构。左侧为 VLM 预训练管线（图文对齐），右侧为机器人数据微调阶段。VLM 特征直接送入 Diffusion Policy 解码器预测动作噪声，取代自回归 token 生成。Figure 2: TinyVLA model architecture. The left side is the VLM pre-training pipeline (image-text alignment); the right side is the robot-data fine-tuning stage. VLM features are fed directly into the Diffusion Policy decoder to predict action noise, replacing autoregressive token generation.
紧凑型 VLM 骨干Compact VLM Backbone
作者选用参数量在 70M–1.4B 之间的小型视觉语言模型，通过 LLaVA 训练管线将视觉编码器与 Pythia 语言模型对齐。这使得 TinyVLA 在保留语言理解和视觉感知能力的同时，将参数规模压缩为 OpenVLA（7B）的约 1/5。The authors adopt small vision-language models of 70M–1.4B parameters and align the visual encoder with the Pythia language model through the LLaVA training pipeline. This lets TinyVLA retain language understanding and visual perception while shrinking the parameter scale to roughly 1/5 of OpenVLA (7B).
LoRA 参数高效微调LoRA Parameter-Efficient Fine-Tuning
在机器人数据微调阶段，作者采用 LoRA（Low-Rank Adaptation），将低秩矩阵注入 Transformer 的 Q、K、V attention 层。文中指出 "trainable parameters constitute only 5.0% of the entire transformer's parameters"，大幅降低计算开销。In the robot-data fine-tuning stage the authors adopt LoRA (Low-Rank Adaptation), injecting low-rank matrices into the Q, K and V attention layers of the Transformer. The paper notes that "trainable parameters constitute only 5.0% of the entire transformer's parameters", greatly reducing the computational cost.
Diffusion Policy 解码器Diffusion Policy Decoder
动作生成不再依赖自回归离散 token，而是通过 Diffusion Policy 头部预测噪声、迭代去噪得到连续动作序列。这一设计同时带来两项优势：避免了大词表 softmax 的计算瓶颈，且连续动作表示天然适合精细的机械臂控制任务。Action generation no longer relies on autoregressive discrete tokens; instead a Diffusion Policy head predicts noise and iteratively denoises it into a continuous action sequence. This design brings two benefits at once: it avoids the computational bottleneck of a large-vocabulary softmax, and the continuous action representation is naturally suited to fine-grained arm control tasks.
数据效率Data Efficiency
每个任务仅需 100 条演示轨迹（5 个任务共 500 条），无需 OpenVLA 所需的 970K 条大规模预训练数据，即可达到甚至超越其性能。Each task needs only 100 demonstration trajectories (500 in total for 5 tasks), and without the 970K large-scale pre-training samples OpenVLA requires it still matches or surpasses OpenVLA's performance.
无预训练阶段No Pre-Training Stage
直接使用公开发布的多模态预训练 VLM 权重作为初始化，省去机器人专属预训练阶段，显著降低部署门槛。Publicly released multimodal pre-trained VLM weights are used directly as initialization, dispensing with a robot-specific pre-training stage and markedly lowering the deployment barrier.
03实验Experiments
实验涵盖真实世界单臂 Franka 机械臂（5 个操作任务）、双臂 UR5（3 个任务）及 MetaWorld 仿真环境（50 个任务）。基线模型包括 OpenVLA-7B 和 Diffusion Policy。Experiments cover a real-world single-arm Franka robot (5 manipulation tasks), a bimanual UR5 (3 tasks) and the MetaWorld simulation environment (50 tasks). The baselines are OpenVLA-7B and Diffusion Policy.
真实机械臂对比（Table II）Real-Robot Comparison (Table II)
模型Model
PlaceTennis
FlipMug
StackCubes
CloseDrawer
OpenBox
平均Average
TinyVLA-H
90.0%
98.3%
98.3%
96.7%
86.7%
94.0%
OpenVLA
83.3%
51.7%
40.0%
85.0%
81.7%
68.3%
Diffusion Policy
16.7%
30.0%
3.3%
73.3%
53.3%
35.3%
TinyVLA-H 平均成功率 94.0%，比 OpenVLA（68.3%）高出 25.7%，同时使用参数量少 5.5 倍。TinyVLA-H reaches an average success rate of 94.0%, 25.7% higher than OpenVLA (68.3%), while using 5.5 times fewer parameters.
图 5：视角泛化实验（View Generalization）。在新环境中测试 3 个任务，评估 8 种相机视角变化（每台摄像机各 4 种），每种配置测试 2 次。TinyVLA-H 在最大 30° 视角偏移下仍保持高成功率，而 Diffusion Policy 在轻微视角变化下即失效。Figure 5: View generalization experiments. 3 tasks are tested in a new environment under 8 camera viewpoint variations (4 per camera), with every configuration run 2 times. TinyVLA-H keeps a high success rate under viewpoint shifts of up to 30°, whereas Diffusion Policy already fails under slight viewpoint changes.
双臂机器人（Table III）Bimanual Robot (Table III)
在双臂 UR5 平台上，TinyVLA-H 平均成功率为 44.5%，OpenVLA 为 0%（完全失败），原因是 OpenVLA 的预训练数据仅覆盖单臂场景，无法泛化至双臂操作。On the bimanual UR5 platform TinyVLA-H attains an average success rate of 44.5% while OpenVLA reaches 0% (complete failure), because OpenVLA's pre-training data covers only single-arm scenarios and does not generalize to bimanual manipulation.
泛化能力Generalization
图 4：指令泛化（Instruction Generalization）。三种难度递增的指令泛化实验：语义重表述、新颖指令、未见物体颜色描述。TinyVLA-H 均能正确理解并执行。Figure 4: Instruction generalization. Three instruction-generalization experiments of increasing difficulty: semantic rephrasing, novel instructions, and descriptions of unseen object colors. TinyVLA-H understands and executes all of them correctly.
指令泛化：能够理解语义重表述、新颖描述及未见颜色等指令变体。Instruction generalization: it understands instruction variants such as semantic rephrasings, novel descriptions and unseen colors.
视角泛化：摄像机视角偏移最大 30° 仍保持高成功率；Diffusion Policy 在轻微偏移下即失效。View generalization: a high success rate is retained under camera viewpoint shifts of up to 30°; Diffusion Policy already fails under slight shifts.
背景泛化：在 6 种不同背景下无需数据增强仍表现稳定。Background generalization: performance stays stable across 6 different backgrounds without data augmentation.
干扰物与光照泛化：在多种干扰物摆放和低光照条件下（OpenVLA 在低光照下失败）保持鲁棒性。Distractor and lighting generalization: robustness is maintained under various distractor placements and low-light conditions (where OpenVLA fails).
空间泛化：在训练区域之外的多个位置完成任务；OpenVLA 在此部分略优，作者归因于其大规模预训练数据覆盖了更多动作多样性。Spatial generalization: tasks are completed at multiple positions outside the training region; OpenVLA is slightly better here, which the authors attribute to its large-scale pre-training data covering more diverse actions.
消融实验Ablation Studies
图 10：不同规模预训练 VLM 的失败类型分析。TinyVLA-0.4B 常见指令误解错误；TinyVLA-1.3B 解决了语言理解问题；TinyVLA-3B（PaliGemma）进一步提升定位精度。Figure 10: Failure-type analysis across pre-trained VLMs of different scales. TinyVLA-0.4B commonly makes instruction-misunderstanding errors; TinyVLA-1.3B resolves the language-understanding problem; TinyVLA-3B (PaliGemma) further improves localization accuracy.
动作解码器消融（Table V）显示：MLP 解码器在全部 5 个任务上均为 0% 成功率；Action Chunking Transformer 平均约 11.6%；Diffusion Policy 解码器达到 94.0%，表明连续扩散动作建模对于精细操作至关重要。The action-decoder ablation (Table V) shows that the MLP decoder yields 0% success on all 5 tasks; the Action Chunking Transformer averages about 11.6%; the Diffusion Policy decoder reaches 94.0%, indicating that continuous diffusion-based action modeling is essential for fine-grained manipulation.
TinyVLA-H 平均成功率 31.6%，Diffusion Policy 基线为 10.5%；在困难任务上，TinyVLA-H 的表现约为基线的 6 倍（"sixfold better"）。TinyVLA-H attains an average success rate of 31.6% against 10.5% for the Diffusion Policy baseline; on hard tasks TinyVLA-H performs about 6 times better than the baseline ("sixfold better").
04局限性Limitations
注：本文没有设置独立的 Limitations 章节。以下第 1 条为作者在正文中明确陈述的限制，第 2–3 条为基于设计的推断（inferred）。Note: The paper has no standalone Limitations section. Item 1 below is a limitation the authors state explicitly in the text; items 2–3 are inferred from the design (inferred).
大规模预训练在分布外动作多样性上的优势（stated）Advantage of large-scale pre-training on out-of-distribution action diversity (stated)
作者在空间泛化实验中明确指出："OpenVLA performs slightly better than our approach, likely because it is trained on large-scale robotic data, allowing the model to 'see' more diverse robot actions during pre-training。" 当测试位置严重偏离训练分布时，TinyVLA 相比经过大规模预训练的 OpenVLA 存在一定劣势。In the spatial generalization experiment the authors state explicitly: "OpenVLA performs slightly better than our approach, likely because it is trained on large-scale robotic data, allowing the model to 'see' more diverse robot actions during pre-training." When test positions deviate sharply from the training distribution, TinyVLA is at some disadvantage relative to the heavily pre-trained OpenVLA.
虽然 TinyVLA-H 整体推理延迟仅 14ms，但文中 Table IV 暗示：若将 OpenVLA 的 7B 骨干替换为同等参数量的 1B 骨干，延迟从 292ms 降至 140ms（约 2×），而 TinyVLA-H 达到 14ms 的原因不仅在于骨干更小，还来自于 Diffusion Policy 自身的迭代去噪步数设置。在实时控制频率更高的场景下，去噪步数与延迟的权衡尚未充分讨论。Although TinyVLA-H's overall inference latency is only 14ms, Table IV hints that replacing OpenVLA's 7B backbone with a 1B backbone of comparable size lowers latency from 292ms to 140ms (about 2×), so TinyVLA-H reaching 14ms stems not only from a smaller backbone but also from the number of iterative denoising steps configured in the Diffusion Policy itself. For scenarios with higher real-time control frequencies, the trade-off between denoising steps and latency is not fully discussed.
任务特定数据收集仍为必要前提（inferred）Task-specific data collection remains a prerequisite (inferred)
尽管 TinyVLA 消除了大规模机器人预训练的需求，但每个新任务仍需收集约 100 条高质量演示轨迹。对于无法轻松采集人工示范的场景（如危险环境、远程操作），数据采集本身仍是瓶颈。Although TinyVLA removes the need for large-scale robot pre-training, every new task still requires collecting about 100 high-quality demonstration trajectories. In settings where human demonstrations cannot easily be gathered (hazardous environments, teleoperation), data collection itself remains the bottleneck.
