机器人 · Vision-Language-Action · 实时控制Robotics · Vision-Language-Action · Real-Time Control
VLASH：基于未来状态感知异步推理的实时 VLAVLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference
Real-Time VLAs via Future-State-Aware Asynchronous Inference
Jiaming Tang, Yufei Sun, Yilong Zhao, Shang Yang, Yujun Lin, Zhuoyang Zhang, James Hou, Yao Lu, Zhijian Liu, Song Han · 2025
Vision-Language-Action (VLA) 模型推理速度慢，导致机器人控制出现"动作停顿"现象。VLASH 提出将推理与执行并行化，并通过滚动预测未来执行时刻的机器人状态来消除时序错位，从而在不损失精度的前提下实现实时控制。Vision-Language-Action (VLA) models are slow at inference, which makes robot control exhibit "motion stalls". VLASH parallelizes inference and execution and removes the temporal misalignment by rolling forward a prediction of the robot state at the future execution moment, thereby achieving real-time control without sacrificing accuracy.
当前 VLA 模型推理延迟高（数百毫秒），若采用同步推理，机器人在等待结果期间必须停止运动，造成"动作停顿"，严重降低任务效率；而直接切换为异步推理（执行与推理并行），则会引入时序错位问题——模型推理时的机器人状态与实际执行时的状态不一致，导致控制不稳定甚至失败。Current VLA models have high inference latency (hundreds of milliseconds). With synchronous inference, the robot must stop moving while it waits for the result, producing "motion stalls" that severely reduce task efficiency; switching directly to asynchronous inference (execution running in parallel with inference) introduces the temporal misalignment problem — the robot state at inference time differs from the state at actual execution time, which makes control unstable or even causes failure.
"Asynchronous inference… introduces a fundamental challenge: the robot's execution-time state diverges from the prediction-time state due to inference latency Δ, causing severe instability and degraded control accuracy."
图1：时序错位问题。同步推理中机器人在推理期间停止（上），导致"动作停顿"；朴素异步推理（中）使机器人持续运动，但预测所用状态 s₁ 与实际执行时状态 s₃ 存在偏差（Δ=2步）；VLASH（下）将模型条件化于滚动预测的未来状态 s_{t+Δ}，消除时序错位。Figure 1: The temporal misalignment problem. Under synchronous inference the robot stops while inference runs (top), causing "motion stalls"; under naive asynchronous inference (middle) the robot keeps moving, but the state s₁ used for prediction deviates from the state s₃ at actual execution (Δ=2 steps); VLASH (bottom) conditions the model on the rolled-forward future state s_{t+Δ}, removing the temporal misalignment.
17.4×最大反应延迟降低幅度（RTX 5090）Largest reduction in reaction latency (RTX 5090)
2.03×实际机器人最大速度提升（q=2 量化）Largest speedup on the real robot (q=2 quantization)
94%真实环境任务平均得分（vs 同步 83%）Average task score in the real world (vs 83% for synchronous)
5×高效微调时有效训练轨迹扩增倍数Expansion factor of effective training trajectories under efficient fine-tuning
02方法Method
VLASH 由三个互补模块组成：(1) 未来状态滚动（Future State Rollforward），在推理前估算执行时刻的机器人状态；(2) 时序偏移增强（Temporal-Offset Augmentation），让模型在微调阶段学会应对不同推理延迟；(3) 动作量化（Action Quantization），将细粒度动作聚合为粗粒度宏动作，进一步加速执行。VLASH consists of three complementary modules: (1) Future State Rollforward, which estimates the robot state at execution time before inference; (2) Temporal-Offset Augmentation, which teaches the model during fine-tuning to cope with different inference latencies; (3) Action Quantization, which aggregates fine-grained actions into coarse-grained macro-actions to further accelerate execution.
图2：四种部署方式对比。从左到右：同步推理（机器人等待）、朴素异步推理（状态错位）、VLASH 异步推理（未来状态感知）、VLASH + 动作量化（进一步提速）。VLASH 通过预测执行时刻状态，使动作预测与实际执行完全对齐。Figure 2: Comparison of four deployment modes. From left to right: synchronous inference (the robot waits), naive asynchronous inference (state misalignment), VLASH asynchronous inference (future-state-aware), and VLASH + action quantization (further speedup). By predicting the state at execution time, VLASH aligns action prediction exactly with actual execution.
未来状态滚动（Future State Rollforward）Future State Rollforward
对于推理延迟为 Δ 步的情形，VLASH 利用已生成的动作序列将当前状态向前滚动，估算执行开始时刻的机器人状态：For an inference delay of Δ steps, VLASH uses the already generated action sequence to roll the current state forward and estimate the robot state at the moment execution begins:
"The robot state at the beginning of the execution interval st+Δ is determined by the current robot state st and the actions executed during the inference delay at:t+Δ−1."
例如当 Δ=2 时，有 s₃ = s₁ + a₁ + a₂。由于机器人关节运动学可精确建模，未来机器人状态可以精确预测（尽管环境状态仍不可知）。For example, when Δ=2 we have s₃ = s₁ + a₁ + a₂. Because the robot's joint kinematics can be modeled precisely, the future robot state can be predicted exactly (although the environment state remains unknown).
图3：时序偏移增强（Temporal-Offset Augmentation）。微调时，对同一时刻的视觉观测固定不变，随机采样偏移量 δ∈{0,…,Δmax}，构造训练对 (ot, st+δ, a(t+δ):(t+δ+H−1))。这迫使模型在预测时真正"关注状态输入"而非仅依赖视觉特征。Figure 3: Temporal-Offset Augmentation. During fine-tuning the visual observation of a given moment is held fixed while an offset δ∈{0,…,Δmax} is sampled at random, forming the training pair (ot, st+δ, a(t+δ):(t+δ+H−1)). This forces the model to genuinely "attend to the state input" when predicting, rather than relying on visual features alone.
块稀疏注意力高效微调Efficient fine-tuning with block-sparse attention
为提升微调效率，VLASH 将多个偏移分支打包进同一序列并使用块稀疏注意力掩码（block-sparse attention masking）：每个偏移分支的状态-动作 token 可以看到所有观测 token，但不同偏移分支之间相互隔离。对于 π0.5 而言，单次前向传播中约 700 个观测 token 搭配多个偏移（每个约 50 token），序列长度仅增加约 20%，但有效训练轨迹数量扩大 5×，微调速度提升 3.26×。To improve fine-tuning efficiency, VLASH packs several offset branches into a single sequence and applies block-sparse attention masking: the state-action tokens of each offset branch can see all observation tokens, while different offset branches remain isolated from one another. For π0.5, a single forward pass pairs about 700 observation tokens with several offsets (about 50 tokens each), so the sequence length grows by only about 20%, while the number of effective training trajectories is enlarged 5× and fine-tuning speed improves 3.26×.
图4：动作量化（Action Quantization）。将连续 q 个细粒度动作聚合为一个宏动作：âi = aiq + aiq+1 + … + a(i+1)q−1。量化因子 q=2 时可在不损失精度的前提下实现 2.03× 的整体加速。Figure 4: Action Quantization. Consecutive q fine-grained actions are aggregated into one macro-action: âi = aiq + aiq+1 + … + a(i+1)q−1. With a quantization factor of q=2, an overall speedup of 2.03× is obtained without loss of accuracy.
块稀疏注意力结构Block-sparse attention structure
图5：块稀疏注意力掩码。观测 token 之间正常互相关注；不同偏移的状态-动作 token 块均可关注观测，但彼此之间互不可见，确保多偏移独立性。Figure 5: The block-sparse attention mask. Observation tokens attend to one another normally; the state-action token blocks of different offsets can all attend to the observations but stay mutually invisible, which guarantees independence across offsets.
03实验Experiments
实验在仿真（LIBERO、Kinetix）和真实双臂机器人平台上进行，基线模型包括 π0.5 和 SmolVLA-450M，对比方案包括同步推理、朴素异步推理。评估指标为任务成功率、执行时间、反应延迟。Experiments are run in simulation (LIBERO, Kinetix) and on a real bimanual robot platform. The baseline models include π0.5 and SmolVLA-450M, and the compared schemes include synchronous inference and naive asynchronous inference. The evaluation metrics are task success rate, execution time and reaction latency.
图6：Kinetix 仿真基准结果。在不同执行 horizon 下，VLASH 的成功率（蓝色）紧跟同步基线（绿色），而朴素异步（橙色）随 horizon 增大性能急剧下降。延迟 4 步时，VLASH 以 81.7% 对比朴素异步的 51.2%，提升 30.5%。Figure 6: Kinetix simulation benchmark results. Across different execution horizons the success rate of VLASH (blue) closely tracks the synchronous baseline (green), whereas naive asynchronous inference (orange) degrades sharply as the horizon grows. At a delay of 4 steps VLASH reaches 81.7% against 51.2% for naive asynchronous inference, an improvement of 30.5%.
消融实验要点Key ablation findings
时序偏移增强是核心——去除后模型在较大延迟下性能大幅下滑；Temporal-offset augmentation is the core component — removing it makes performance drop sharply at larger delays;
块稀疏注意力微调使单步训练速度提升 3.26×，最终精度与标准微调相当（30K 步时 96.6%）；Block-sparse attention fine-tuning raises per-step training speed by 3.26×, with final accuracy on par with standard fine-tuning (96.6% at 30K steps);
SmolVLA-450M 在延迟 3 步时达到 79.06% 成功率、1.35× 加速，验证了方法的跨模型泛化性；SmolVLA-450M attains a 79.06% success rate and a 1.35× speedup at a delay of 3 steps, confirming that the method generalizes across models;
VLASH 还解锁了此前不可能完成的任务：乒乓球对打和"打地鼠"等高速反应场景。VLASH also unlocks tasks that were previously impossible: high-speed reactive scenarios such as table-tennis rallies and "whack-a-mole".
04局限性Limitations
说明：论文在结论与方法部分对以下局限性有所提及；标注"（推断）"的条目为根据设计逻辑推断，非作者明确陈述。Note: The paper touches on the following limitations in its conclusion and method sections; entries marked "(inferred)" are deduced from the design logic rather than explicitly stated by the authors.
较大推理延迟下精度有所下降（已陈述）Accuracy drops somewhat at larger inference delays (stated)
在 LIBERO 仿真中，延迟 3~4 步时成功率从 97% 降至约 93~94%；SmolVLA-450M 的方差更大，表明不同架构对时序偏移的鲁棒性存在差异。In the LIBERO simulation the success rate falls from 97% to about 93~94% at delays of 3~4 steps; SmolVLA-450M shows larger variance, indicating that different architectures differ in their robustness to temporal offsets.
动作量化存在速度–精度权衡（已陈述）Action quantization involves a speed–accuracy trade-off (stated)
量化因子 q=2 时无明显精度损失（2.03× 加速），但 q=3 时精度下降约 4.7%（2.67× 加速）。量化超参数需要针对具体任务单独调整。With a quantization factor of q=2 there is no noticeable accuracy loss (2.03× speedup), but at q=3 accuracy drops by about 4.7% (2.67× speedup). The quantization hyperparameter has to be tuned separately for each specific task.
仅能预测机器人自身状态，环境状态不可知（已陈述）Only the robot's own state can be predicted; the environment state is unknown (stated)
论文明确指出，未来机器人状态可通过运动学精确滚动，但"未来环境状态仍不可知"。对于物体频繁被外力扰动或场景动态变化激烈的任务，效果可能受限。The paper states explicitly that the future robot state can be rolled forward exactly through kinematics, but that "the future environment state remains unknown". For tasks in which objects are frequently disturbed by external forces or the scene changes dynamically, the benefit may be limited.
依赖精确的机器人运动学模型（推断）Relies on an accurate robot kinematics model (inferred)
状态滚动基于 s_{t+Δ} = s_t + a_{t:t+Δ−1} 的关节运动学。若机器人存在系统误差、关节柔性或滑动，滚动预测误差可能累积，影响较大延迟下的对齐精度。State rollforward rests on the joint kinematics s_{t+Δ} = s_t + a_{t:t+Δ−1}. If the robot suffers from systematic errors, joint compliance or slippage, rollforward prediction errors may accumulate and degrade alignment accuracy at larger delays.
微调仍需一定计算成本（推断）Fine-tuning still carries a certain computational cost (inferred)
尽管块稀疏注意力将微调速度提升 3.26×，VLASH 仍需对预训练 VLA 进行专项微调（实验中收敛略慢于标准微调）。对于资源有限的场景，计算成本仍是考量因素。Although block-sparse attention raises fine-tuning speed by 3.26×, VLASH still requires dedicated fine-tuning of the pre-trained VLA (in the experiments it converged slightly more slowly than standard fine-tuning). For resource-constrained settings the computational cost remains a consideration.
