Uncertainty Quantification for Flow-Based Vision-Language-Action Models

01 Motivation

现代 VLA 把大规模视觉语言预训练带入物理世界：一个预训练 vision-language backbone 负责语义理解，接一个通过 flow matching 训练的 action expert 生成连续动作。但预训练数据集终究是静态、有限的，而部署环境是非平稳的——任务会变、物体外观会变、场景会变。Modern VLAs bring large-scale vision-language pre-training into the physical world: a pre-trained vision-language backbone handles semantic understanding, followed by an action expert trained with flow matching that generates continuous actions. But pre-training datasets are ultimately static and finite, whereas deployment environments are non-stationary — tasks change, object appearances change, scenes change.

"VLAs lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable." 面对分布外场景时，"VLAs confidently execute erratic actions in out-of-distribution scenarios ... rather than abstaining or asking for help when uncertain, raising significant safety concerns.""VLAs lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable." When facing out-of-distribution scenes, "VLAs confidently execute erratic actions in out-of-distribution scenarios ... rather than abstaining or asking for help when uncertain, raising significant safety concerns."

更棘手的是，把 VLA 适配到新任务目前依赖收集大量人类专家演示，而这非常昂贵（"prohibitively expensive"）。本文因此聚焦两个问题：如何高效量化 flow-based VLA 的认知不确定性(epistemic uncertainty)；以及如何用这个不确定性信号，用更少的专家数据把模型微调到新任务上，同时在部署时及早发现失败。Worse still, adapting a VLA to a new task currently relies on collecting a large number of human expert demonstrations, which is "prohibitively expensive". This paper therefore focuses on two questions: how to efficiently quantify the epistemic uncertainty of a flow-based VLA; and how to use that uncertainty signal to fine-tune the model to new tasks with fewer expert data while detecting failures early at deployment time.

teaser — Figure 1. 上：VFD 通过测量集成模型 velocity field 之间的缩放差异来量化认知不确定性。下：SAVE 依据各任务的平均 VFD 不确定性排序，并在每个采样任务里为最不确定的初始观测请求一条专家演示；模型随后用新数据与回放数据一起微调，实现数据高效的多任务适配。Figure 1. Top: VFD quantifies epistemic uncertainty by measuring the scaled discrepancy between the velocity fields of the ensemble members. Bottom: SAVE ranks tasks by their mean VFD uncertainty and, within each sampled task, requests one expert demonstration for the most uncertain initial observation; the model is then fine-tuned on the new data together with replay data, achieving data-efficient multi-task adaptation.

0.71VFD 校准：negative Spearman ρ（次优 GU 为 0.62）VFD calibration: negative Spearman ρ (second-best GU: 0.62)

67.1%SAVE w/ VFD 最终多任务成功率（Random 为 54.6%）Final multi-task success rate of SAVE w/ VFD (Random: 54.6%)

≥22%比其它不确定性引导采集策略节省的演示数据Fewer demonstrations than other uncertainty-guided acquisition strategies

67% / 79%失败检测 accuracy / TPR（VFD）Failure detection accuracy / TPR (VFD)

02 Method

方法分两层：先推导 VFD 这个可计算的认知不确定性估计量，再在此基础上构建 SAVE，用它来指导主动多任务微调中"该收集哪些数据"的决策。The method has two layers: first derive VFD, a tractable estimator of epistemic uncertainty; then build SAVE on top of it to guide the decision of "which data to collect" in active multi-task fine-tuning.

Figure 6. VFD 计算过程可视化：对两个不同认知不确定性水平的 conditioning input，比较集成中各模型 velocity field 在 ODE 积分轨迹上的差异。Figure 6. Visualization of the VFD computation: for two conditioning inputs with different levels of epistemic uncertainty, the velocity fields of the ensemble members are compared along the ODE integration trajectory.

Velocity Field Disagreement (VFD)

贝叶斯视角下，认知不确定性等于 θ ~ p(θ|D) 下模型自身分布与后验预测分布之间 KL 散度的期望（式 2），可以用一组从后验采样的参数 θ₁,…,θ_M（本文用 ensembling 获得）做蒙特卡洛近似。作者利用 KL 散度对第二个参数的凸性，把这个量进一步用 Jensen 不等式放缩为集成成员两两之间 KL 散度的平均，作为认知不确定性的一个上界（式 4c）。From a Bayesian perspective, epistemic uncertainty equals the expectation under θ ~ p(θ|D) of the KL divergence between the model's own distribution and the posterior predictive distribution (Eq. 2), which can be approximated by Monte Carlo with a set of parameters θ₁,…,θ_M drawn from the posterior (obtained here by ensembling). Exploiting the convexity of the KL divergence in its second argument, the authors use Jensen's inequality to relax this quantity into the average pairwise KL divergence between ensemble members, an upper bound on epistemic uncertainty (Eq. 4c).

直接计算两个 flow-matching 模型之间的 KL 散度需要通过增广 ODE 求 likelihood，在高维空间中代价过高。本文的核心定理（Theorem 4.1）证明：在 OT Gaussian conditional path 下，两个 flow-matching 模型之间的 KL 散度可以精确写成沿 ODE 积分路径、按时间加权 κ_s = s/(1-s) 的 velocity field L2 差异积分（式 5）。这意味着只需在 Euler 积分的中间状态上比较集成成员的 velocity 预测，就能高效近似 KL 散度——即 velocity field disagreement (VFD)。时间越接近 s=1（样本噪声越小），权重 κ_s 越大，因为此时的 velocity 差异更能反映认知不确定性。Computing the KL divergence between two flow-matching models directly requires obtaining the likelihood through an augmented ODE, which is prohibitively costly in high dimensions. The paper's central theorem (Theorem 4.1) proves that, under the OT Gaussian conditional path, the KL divergence between two flow-matching models can be written exactly as an integral of the L2 velocity field difference along the ODE integration path, weighted in time by κ_s = s/(1-s) (Eq. 5). This means that comparing the velocity predictions of the ensemble members at the intermediate states of Euler integration suffices to approximate the KL divergence efficiently — that is, velocity field disagreement (VFD). The closer the time is to s=1 (the less noisy the sample), the larger the weight κ_s, because the velocity difference there is more indicative of epistemic uncertainty.

SAVE: 不确定性引导的主动多任务微调SAVE: Uncertainty-Guided Active Multi-Task Fine-Tuning

由于从零训练多个 VLA 成本过高，SAVE 用一个预训练 base VLA π_b 在多样化数据集 D_pre 上微调 M 次（随机打乱数据顺序）得到一个轻量集成 Π₀ 用于不确定性估计。实验发现 两成员集成 (M=2) 就已经校准良好，因此后续实验全部采用这一轻量配置。Since training multiple VLAs from scratch is too costly, SAVE fine-tunes a single pre-trained base VLA π_b on the diverse dataset D_pre M times (with shuffled data order) to obtain a lightweight ensemble Π₀ for uncertainty estimation. Experiments find that a two-member ensemble (M=2) is already well calibrated, so all subsequent experiments adopt this lightweight configuration.

每一轮主动学习中，SAVE 先对每个任务 k 的候选初始观测计算 VFD 分数，取均值得到任务级不确定性 U_k(式 8)；再用带温度参数 τ 的 categorical 分布对任务采样（式 9），τ 越大越偏向高不确定性任务，τ=0 即均匀采样。采样到任务后，在该任务内选择 VFD 最高的初始观测 o*（式 10），向专家请求一条从 o* 出发的演示；新数据与回放数据混合，用于微调下一轮的集成。In each round of active learning, SAVE first computes VFD scores for the candidate initial observations of every task k and averages them into a task-level uncertainty U_k (Eq. 8); tasks are then sampled from a categorical distribution with temperature parameter τ (Eq. 9), where a larger τ biases sampling toward high-uncertainty tasks and τ=0 gives uniform sampling. Once a task is sampled, the initial observation o* with the highest VFD within that task is selected (Eq. 10) and one expert demonstration starting from o* is requested; the new data are mixed with replay data and used to fine-tune the ensemble for the next round.

03 Experiments

实验环境为 LIBERO 模拟的 Franka 机械臂家居场景。主动微调阶段用 LIBERO-10（最难的长时程任务套件，K=10 个任务）；初始集成 Π₀ 在 LIBERO-Goal/Spatial/Object 共 30 个任务（另加 3 个主动学习任务）上预训练。策略骨干为 SmolVLA（预训练 SmolVLM-2 + flow-matching action expert，双摄像头+本体状态+语言指令输入，动作块长 H=50）。所有实验重复 3 个随机种子，报告均值±标准差。The experimental setting consists of the LIBERO-simulated household scenes with a Franka arm. Active fine-tuning uses LIBERO-10 (the hardest long-horizon task suite, K=10 tasks); the initial ensemble Π₀ is pre-trained on the 30 tasks of LIBERO-Goal/Spatial/Object (plus the 3 active-learning tasks). The policy backbone is SmolVLA (pre-trained SmolVLM-2 + flow-matching action expert, with dual camera views, proprioceptive state and language instruction as input, action chunk length H=50). All experiments are repeated over 3 random seeds, reporting mean ± standard deviation.

校准 (Calibration)：VFD 是否与任务表现负相关Calibration: does VFD correlate negatively with task performance

作者对比 6 个不确定性估计基线：Action-L2（集成动作块两两 L2 距离）、ACE（动作分布条件熵）、DECU（在 DDIM "分叉" ODE 步做成对距离估计，迁移到 flow matching）、Generative Uncertainty/GU（从相同初始噪声出发采样近似后验预测分布的熵）、以及基于 VLM 前缀的 Entropy 与 Perplexity。The authors compare 6 uncertainty estimation baselines: Action-L2 (pairwise L2 distance between the action chunks of the ensemble), ACE (conditional entropy of the action distribution), DECU (pairwise distance estimation at "branching" ODE steps in DDIM, transferred to flow matching), Generative Uncertainty/GU (entropy of the posterior predictive distribution approximated by sampling from the same initial noise), and Entropy and Perplexity over the VLM prefix.

Metric	Action-L2	ACE	DECU	GU	Entropy	Perplexity	VFD (ours)
−Spearman ↑	0.50±0.13	0.31±0.12	0.31±0.13	0.62±0.00	0.10±0.12	−0.04±0.09	0.71±0.03
−Pearson ↑	0.48±0.09	0.36±0.08	0.23±0.15	0.65±0.02	0.23±0.21	0.02±0.15	0.71±0.02

"Across all tasks and rounds, VFD is better calibrated than the baselines, achieving a 0.09 higher negative Spearman correlation than the second-best method, GU."

ensemble size calibration — Figure 2. 集成规模对校准的影响很小：即便只用 2 个成员，VFD 的校准依然稳定，因此后续实验统一采用轻量的两成员集成。（Figure 3 进一步表明，仅改变语言指令时 VFD 同样保持良好校准。）Figure 2. Ensemble size has little effect on calibration: even with only 2 members, VFD stays stably calibrated, so all subsequent experiments uniformly adopt the lightweight two-member ensemble. (Figure 3 further shows that VFD also remains well calibrated when only the language instruction is varied.)

主动微调 (Active Fine-Tuning)：SAVE 能否更快提升多任务表现Active Fine-Tuning: can SAVE improve multi-task performance faster

对比 Random、Diversity（视觉多样性贪心基线）、以及分别用 Action-L2 / GU / VFD 做采集信号的 SAVE。The comparison covers Random, Diversity (a greedy visual-diversity baseline), and SAVE using Action-L2 / GU / VFD respectively as the acquisition signal.

Final SR	Random	Diversity	SAVE w/ Action-L2	SAVE w/ GU	SAVE w/ VFD
成功率 (%) ↑Success rate (%) ↑	54.6±0.9	54.9±1.3	56.8±7.6	64.0±2.6	67.1±3.2

"SAVE w/ VFD surpasses the performance of random and diversity-based selection in the last three rounds with 50% and 45% fewer costly expert demonstrations, respectively. Compared to other approaches for uncertainty-guided data acquisition with SAVE, VFD requires at least 22% less data to achieve similar performance." 最终 SAVE w/VFD 在相同演示预算下比其它方法高出 3–12 个百分点的成功率。"SAVE w/ VFD surpasses the performance of random and diversity-based selection in the last three rounds with 50% and 45% fewer costly expert demonstrations, respectively. Compared to other approaches for uncertainty-guided data acquisition with SAVE, VFD requires at least 22% less data to achieve similar performance." In the end, at the same demonstration budget, SAVE w/VFD reaches a success rate 3–12 percentage points higher than the other methods.

temperature ablation — Figure 4. 任务采样温度 τ 的影响：左图显示不确定性引导在任务选择和初始观测选择两个层面都有增益，且任务级选择贡献了大部分收益；中图显示不确定性引导的采样能更快降低"不确定性集中在单一最难任务"的程度；右图显示 τ≤2.5 时可以在任务覆盖多样性与最终成功率之间权衡探索-利用。Figure 4. Effect of the task sampling temperature τ: the left panel shows that uncertainty guidance brings gains at both the task selection and the initial observation selection level, with task-level selection contributing most of the benefit; the middle panel shows that uncertainty-guided sampling reduces faster the degree to which "uncertainty concentrates on a single hardest task"; the right panel shows that for τ≤2.5 one can trade off exploration and exploitation between task coverage diversity and final success rate.

失败检测 (Failure Detection)Failure Detection

用主动微调后的最终策略每个任务滚动 30 次 rollout，在每个动作生成时间步计算两成员集成的 VFD，并用 10 次成功 rollout 通过 conformal prediction 校准任务专属阈值。对比基线为 ACE（动作分布条件熵）、STAC（比较相邻时间步动作分布）、RND-OE（检测 OOD 观测）。The final policy after active fine-tuning is rolled out 30 times per task; at every action generation timestep the VFD of the two-member ensemble is computed, and a task-specific threshold is calibrated by conformal prediction using 10 successful rollouts. The baselines compared against are ACE (conditional entropy of the action distribution), STAC (comparing action distributions at adjacent timesteps) and RND-OE (detecting OOD observations).

04 Limitations

Note: 以下三点均为论文 Conclusion 中作者明确列出的 limitations（stated），并非本文推断。All three points below are limitations explicitly stated by the authors in the paper's Conclusion; they are not inferences made here.

依赖维护一个独立的模型集成Reliance on maintaining a separate model ensemble

"While we have shown that VFD requires only a two-member ensemble to be well-calibrated, the reliance on training and maintaining a separate set of model weights is a limitation of our method." 即便只需 2 个成员，训练与维护额外一份模型权重仍带来额外开销。"While we have shown that VFD requires only a two-member ensemble to be well-calibrated, the reliance on training and maintaining a separate set of model weights is a limitation of our method." Even though only 2 members are needed, training and maintaining an extra set of model weights still incurs additional overhead.

用初始观测的不确定性作为任务难度的代理Using initial-observation uncertainty as a proxy for task difficulty

"We use the uncertainty for the initial observation as a proxy for task difficulty. Our experiments empirically confirm this correlation, but there may be certain scenarios that 'look simple' but are challenging for the policy (e.g., due to complex contact dynamics)."

未建模任务间的信息依赖关系Informational dependencies across tasks are not modeled

"Treating uncertainty as the primary signal for determining which data to collect does not account for informational dependencies across tasks. Developing tractable methods to quantify these dependencies and maximizing expected information gain are interesting avenues for future work."