现有 VLA 模型擅长静态任务，却在需要快速感知、时序预判与持续控制的动态物体操控场景中严重失效。
DynamicVLA 提出仅 0.4B 参数的轻量架构，配合 Continuous Inference（CI） 与
Latent-aware Action Streaming（LAAS），在推理与执行之间消除等待，实现
对运动物体的高频闭环控制。在自建 Dynamic Object Manipulation（DOM）基准上，
整体成功率达 47.06%，超越最强基线 3.46×。Existing VLA models excel at static tasks, yet fail badly in dynamic object manipulation scenarios that demand rapid perception, temporal anticipation and continuous control.
DynamicVLA proposes a lightweight architecture with only 0.4B parameters, coupled with Continuous Inference (CI) and
Latent-aware Action Streaming (LAAS), which removes the waiting between inference and execution and achieves
high-frequency closed-loop control of moving objects. On the self-built Dynamic Object Manipulation (DOM) benchmark,
the overall success rate reaches 47.06%, surpassing the strongest baseline by 3.46×.
机器人操控领域中，绝大多数 VLA 模型都假设目标物体静止不动。然而在真实场景中，物体往往处于运动状态——传送带上的工件、被推动的容器、弹跳的球体。
此时，推理延迟直接决定成败：即便数十毫秒的延迟，也会导致感知与执行之间的时序错位，令机器人抓取"幽灵位置"而非实际位置。In robotic manipulation, the vast majority of VLA models assume the target object stays still. In real scenarios, however, objects are often in motion - workpieces on a conveyor belt, containers being pushed, bouncing balls.
Here inference latency directly decides success or failure: even a delay of tens of milliseconds induces a temporal mismatch between perception and execution, making the robot grasp a "ghost position" instead of the actual one.
"Dynamic object manipulation remains an open challenge for Vision-Language-Action (VLA) models, which…struggle in dynamic scenarios requiring rapid perception, temporal anticipation, and continuous control."
Figure 1: DynamicVLA 框架概览。
(a) 0.4B 参数的轻量 VLA 架构，将轻量 backbone 与 action expert 耦合，支持高频闭环控制。
(b) Continuous Inference：通过流水线推理窗口将推理与执行并行化，消除 action chunk 之间的等待。
(c) Latent-aware Action Streaming：丢弃已过时的 action、优先执行最新 chunk 的预测，保证时序一致性。Figure 1: Overview of the DynamicVLA framework.
(a) A lightweight 0.4B-parameter VLA architecture that couples a lightweight backbone with an action expert, supporting high-frequency closed-loop control.
(b) Continuous Inference: pipelining the inference windows parallelizes inference and execution, removing the waiting between action chunks.
(c) Latent-aware Action Streaming: outdated actions are discarded and the predictions of the latest chunk are executed first, preserving temporal consistency.
47.06%整体成功率（仿真 DOM 基准）Overall success rate (simulated DOM benchmark)
3.46×超越最强基线的成功率倍数Success-rate multiple over the strongest baseline
88 Hz推理频率（RTX A6000）Inference frequency (RTX A6000)
0.4B模型参数量Model parameter count
02方法 MethodMethod
DynamicVLA 围绕三个核心设计：紧凑 VLA 架构、Continuous Inference 与
Latent-aware Action Streaming。架构追求极低延迟而非最大模型容量；推理策略则从根本上消除 chunk 间等待，并解决时序漂移问题。DynamicVLA is built around three core designs: a compact VLA architecture, Continuous Inference and
Latent-aware Action Streaming. The architecture pursues extremely low latency rather than maximum model capacity; the inference strategy fundamentally removes inter-chunk waiting and resolves the temporal-drift problem.
Figure 2: 仿真与真实世界数据自动采集流程。
仿真与真实场景共享多样化物体、桌面场景与同步多视角相机。仿真直接使用 Isaac Sim 的 ground-truth 6D 物体状态；真实场景通过多视角 RGB + EfficientTAM 分割 + 几何三角化，无需遥操作即可自动采集动态操控数据。共享的四阶段状态机控制器（接近→抓取→放置→复位）统一驱动两个场景的 episode 生成。Figure 2: Automatic data collection pipeline for simulation and the real world.
Simulated and real settings share diverse objects, tabletop scenes and synchronized multi-view cameras. Simulation directly uses the ground-truth 6D object states of Isaac Sim; real settings automatically collect dynamic manipulation data without teleoperation, through multi-view RGB + EfficientTAM segmentation + geometric triangulation. A shared four-stage state-machine controller (approach → grasp → place → reset) uniformly drives episode generation in both settings.
语言 backbone 选用 SmolLM2-360M，并截断至 16 层（原 32 层），在多模态推理能力与推理速度之间取最优平衡（16 层：47.06% SR，0.226s；24 层：48.44% SR，0.317s）。
视觉编码器采用 FastViT 卷积架构，而非 Transformer 方案：
"FastViT outperforms transformer-based encoders by lowering encoding latency through reduced tokenization"（FastViT：47.06% SR，0.226s 推理；Transformer 编码器：28.89% SR，0.233s 推理）。
Action expert 基于 Flow Matching Transformer，采用扩散式动作生成。
时序视觉输入选用稀疏双帧 {ot-2, ot}（以 25 FPS 采样，间隔 0.08s），兼顾速度感知与推理开销。The language backbone is SmolLM2-360M, truncated to 16 layers (originally 32 layers), which strikes the best balance between multimodal reasoning ability and inference speed (16 layers: 47.06% SR, 0.226s; 24 layers: 48.44% SR, 0.317s).
The visual encoder adopts the FastViT convolutional architecture rather than a Transformer design:
"FastViT outperforms transformer-based encoders by lowering encoding latency through reduced tokenization" (FastViT: 47.06% SR, 0.226s inference; Transformer encoder: 28.89% SR, 0.233s inference).
The action expert is based on a Flow Matching Transformer and adopts diffusion-style action generation.
The temporal visual input uses sparse dual frames {ot-2, ot} (sampled at 25 FPS, 0.08s apart), balancing velocity perception against inference overhead.
Continuous Inference (CI)
传统方案在执行完一个 action chunk 后才触发下一次推理，造成不可避免的等待间隙。CI 改为：
"triggers inference cycles as soon as the previous inference finishes, independent of whether the previously predicted action sequence has been exhausted."
推理与执行全程流水线并行，消除 inter-chunk waiting，使机器人始终在执行动作的同时完成下一轮感知与预测。
消融实验显示，CI 将成功率从 36.11% 提升至 47.06%（+30.4%）。Conventional schemes trigger the next inference only after one action chunk has been fully executed, causing an unavoidable idle gap. CI instead:
"triggers inference cycles as soon as the previous inference finishes, independent of whether the previously predicted action sequence has been exhausted."
Inference and execution are pipelined in parallel throughout, removing inter-chunk waiting so that the robot always completes the next round of perception and prediction while executing actions.
Ablations show that CI raises the success rate from 36.11% to 47.06% (+30.4%).
Latent-aware Action Streaming (LAAS)
CI 引入新的时序问题：某一推理周期预测的 action 序列，在执行时目标物体已移动到新位置，导致旧 action 作用于"幽灵轨迹"。
LAAS 的解决方案："Actions in At corresponding to timesteps earlier than t+m are discarded as outdated"，
始终优先执行最新 chunk 中的动作。消融实验显示，在 CI 基础上叠加 LAAS 进一步将成功率从 39.72% 提升至 47.06%（+18.5%）。CI introduces a new temporal problem: by the time the action sequence predicted in one inference cycle is executed, the target object has already moved to a new position, so the stale actions act on a "ghost trajectory".
The LAAS remedy: "Actions in At corresponding to timesteps earlier than t+m are discarded as outdated",
always prioritizing the actions of the newest chunk. Ablations show that stacking LAAS on top of CI further raises the success rate from 39.72% to 47.06% (+18.5%).
在 DOM 仿真基准上，每种方法评估 1,800 次（10 场景 × 9 维度 × 20 次试验）。
所有 baseline 均在 DOM 数据集上 fine-tune，使用其官方实现与预训练权重。
真实世界评估使用次级机器臂驱动物体运动，每项任务 20 次试验 × 3 种配对运动-位置组合。On the DOM simulation benchmark, each method is evaluated 1,800 times (10 scenes × 9 dimensions × 20 trials).
All baselines are fine-tuned on the DOM dataset using their official implementations and pretrained weights.
Real-world evaluation uses a secondary robot arm to drive object motion, with 20 trials per task × 3 paired motion-position configurations.
仿真基准（Table I）Simulation benchmark (Table I)
方法Method
Interaction 均值Interaction mean
Perception 均值Perception mean
Generalization 均值Generalization mean
整体 SR (%)Overall SR (%)
Diffusion Policy
12.0
9.8
22.1
13.61（最强基线）13.61 (strongest baseline)
π0
~18
~10
~20
~16
π0.5
21.0（最高）21.0 (highest)
—
—
~15
SmolVLA
—
—
—
~12
DynamicVLA（本文）DynamicVLA (ours)
47.2
41.6
52.3
47.06
具体子维度亮点（均为 DynamicVLA vs. 最强 baseline）：Highlights on individual sub-dimensions (all DynamicVLA vs. the strongest baseline):
Closed-loop Reactivity：60.5% vs. 21.0%（+188.1%）Closed-loop Reactivity: 60.5% vs. 21.0% (+188.1%)
Dynamic Adaptation：38.5% vs. 20.5%（+87.8%）Dynamic Adaptation: 38.5% vs. 20.5% (+87.8%)
Long-horizon Sequencing：40.5% vs. 7.5%（+440.0%）Long-horizon Sequencing: 40.5% vs. 7.5% (+440.0%)
路径长度：2.50m（DynamicVLA）vs. 1.51m（baseline），任务完成时间：8.53s vs. 9.98sPath length: 2.50m (DynamicVLA) vs. 1.51m (baseline); task completion time: 8.53s vs. 9.98s
Figure 3: 真实世界 Interaction 评估。
在 Franka 与 PiPER 两款机器人上，跨六项动态操控任务，DynamicVLA 在"闭环反应性""动态适应性""长时序列"三个子维度全面领先 π0.5 和 SmolVLA，每项任务均值来自 3 组运动-位置配对 × 20 次试验。Figure 3: Real-world Interaction evaluation.
On the Franka and PiPER robots, across six dynamic manipulation tasks, DynamicVLA comprehensively leads π0.5 and SmolVLA on all three sub-dimensions - closed-loop reactivity, dynamic adaptation and long-horizon sequencing - with each task mean coming from 3 motion-position pairings × 20 trials.Figure 4: 真实世界 Perception 评估。
空间时序推理任务（需理解物体形状、纹理、运动速度/方向）中，DynamicVLA 整体成功率达 51.9%，而最强基线仅 11.7%，差距在"运动感知"子任务上尤为显著。Figure 4: Real-world Perception evaluation.
On spatio-temporal reasoning tasks (which require understanding object shape, texture and motion speed/direction), DynamicVLA reaches an overall success rate of 51.9% whereas the strongest baseline reaches only 11.7%; the gap is especially pronounced on the "motion perception" sub-task.Figure 5: 真实世界 Generalization 评估。
在未见物体（视觉泛化 59.5%）、新运动模式（运动泛化 65.0%）和外部干扰（干扰鲁棒性 26.5%）场景下，DynamicVLA 均显著优于基线，其中干扰鲁棒性最具挑战，反映未控扰动处理的固有难度。Figure 5: Real-world Generalization evaluation.
On unseen objects (visual generalization 59.5%), novel motion patterns (motion generalization 65.0%) and external disturbances (disturbance robustness 26.5%), DynamicVLA significantly outperforms the baselines; disturbance robustness is the most challenging among them, reflecting the intrinsic difficulty of handling uncontrolled perturbations.
消融研究（Table II）Ablation study (Table II)
配置Configuration
SR (%)
关键结论Key finding
135M backbone
26.67
参数过少，表征能力不足Too few parameters; insufficient representational capacity
360M backbone（本文）360M backbone (ours)
47.06
效率与鲁棒性最优平衡Best balance of efficiency and robustness
1.7B backbone
24.33
延迟过高，动态任务性能反而下降Latency too high; performance on dynamic tasks drops instead
将 CI 与 LAAS 嵌入现有模型（无需重训练）的效果：
π0.5 + CI/LAAS = 15.89% SR；SmolVLA + CI/LAAS = 25.56% SR；DynamicVLA = 47.06% SR。
说明大模型因推理延迟过高，无法充分发挥 CI/LAAS 的优势——低延迟架构是前提。The effect of embedding CI and LAAS into existing models (without retraining):
π0.5 + CI/LAAS = 15.89% SR; SmolVLA + CI/LAAS = 25.56% SR; DynamicVLA = 47.06% SR.
This shows that large models cannot fully exploit CI/LAAS because their inference latency is too high - a low-latency architecture is the prerequisite.
04局限性 LimitationsLimitations
Note: 以下局限性部分由作者在论文末尾明确陈述（标注 stated），部分由设计中推断（标注 inferred）。Note: Some of the limitations below are explicitly stated by the authors at the end of the paper (marked stated), others are inferred from the design (marked inferred).
作者明确指出："dynamic tasks tightly couple perception, reasoning, and execution, demanding architectures that preserve understanding under strict latency budgets."
截断 backbone（16 层而非 32 层）带来约 1.4% 绝对成功率的让步（24 层达 48.44%），是工程权衡而非最优精度。The authors state explicitly: "dynamic tasks tightly couple perception, reasoning, and execution, demanding architectures that preserve understanding under strict latency budgets."
Truncating the backbone (16 layers instead of 32) concedes about 1.4% absolute success rate (24 layers reach 48.44%); this is an engineering trade-off rather than peak accuracy.
短中时域聚焦，缺乏持续长时序动态处理（stated）Focus on short-to-medium horizons, lacking sustained long-horizon dynamics (stated)
当前方案强调"短至中等时域的反应性交互"，对于物体持续运动的长时序列场景（Long-horizon Sequencing 仅 40.5%）仍存在明显差距，尚未解决持续运动状态的长期跟踪问题。The current scheme emphasizes "reactive interaction over short to medium horizons"; a clear gap remains for long-horizon sequencing scenarios with continuously moving objects (Long-horizon Sequencing only 40.5%), and long-term tracking of sustained motion states is not yet solved.
刚体假设，不适用于非刚体/流体动力学（stated）Rigid-body assumption, not applicable to non-rigid or fluid dynamics (stated)
数据采集流程依赖刚体 6D 位姿估计："non-rigid or fluid dynamics with continuously evolving states…remain an open challenge."
现有管线无法处理布料、液体或形变物体。The data collection pipeline relies on rigid-body 6D pose estimation: "non-rigid or fluid dynamics with continuously evolving states…remain an open challenge."
The existing pipeline cannot handle cloth, liquids or deformable objects.
干扰鲁棒性依然有限（inferred from results）Disturbance robustness remains limited (inferred from results)
即便是 DynamicVLA，在"干扰鲁棒性"子维度的成功率也仅为 26.5%，远低于其他维度，反映系统对不可控外部扰动的处理能力仍然有限。Even for DynamicVLA, the success rate on the "disturbance robustness" sub-dimension is only 26.5%, far below the other dimensions, reflecting that the system's ability to handle uncontrollable external perturbations is still limited.
大模型无法受益于 CI/LAAS（inferred from Table V）Large models cannot benefit from CI/LAAS (inferred from Table V)
将 CI 与 LAAS 迁移到较大模型（π0.5、SmolVLA 等）时收益有限，说明方法的有效性与低延迟架构深度绑定，
不能简单地将动态操控能力通过推理策略"插件式"叠加到高延迟大模型上。Transferring CI and LAAS to larger models (π0.5, SmolVLA, etc.) yields limited gains, showing that the effectiveness of the method is deeply tied to a low-latency architecture;
dynamic manipulation ability cannot simply be stacked onto high-latency large models as a "plug-in" inference strategy.
