Quantifying Epistemic Uncertainty in Diffusion Models

01 动机Motivation

扩散模型在图像合成、时间序列预测等领域取得了显著成效，但其不确定性估计机制至今仍不完善。核心问题在于：现有方法将两种性质截然不同的不确定性混在一起处理，导致对生成样本质量的判断失去可信度。Diffusion models have achieved remarkable results in image synthesis, time-series forecasting and related areas, yet their uncertainty estimation machinery is still incomplete. The core problem: existing methods lump together two kinds of uncertainty of entirely different nature, which undermines the credibility of any judgement about the quality of generated samples.

"Common proxies such as sample variance conflate epistemic and aleatoric effects." — 当模型反复采样，输出的方差既包含了采样随机性（aleatoric），也混入了参数不确定性（epistemic），二者无法拆分。"Common proxies such as sample variance conflate epistemic and aleatoric effects." — When a model is sampled repeatedly, the variance of its outputs contains both sampling randomness (aleatoric) and parameter uncertainty (epistemic), and the two cannot be pulled apart.

现有方法的两大缺陷：Two major shortcomings of existing methods:

基于样本方差的方法（BayesDiff 等）：将预测方差整体用作不确定性代理，无法区分"模型不知道"和"数据本身随机"这两种本质不同的情形。Sample-variance-based methods (BayesDiff and others): they use the predictive variance as a whole as an uncertainty proxy, and cannot distinguish "the model does not know" from "the data itself is random" — two situations of fundamentally different nature.
仅限最后一层的 Laplace 近似（LLLA）：只考察最后一层参数，忽略了网络前层对输出的敏感性，产生结构性偏差，甚至在某些数据集上出现 Gap-Closure 为负的反效果（Damped Sines 数据集：LLLA 达 −18.75%）。Last-layer-only Laplace approximation (LLLA): it examines only the final-layer parameters and ignores the sensitivity of earlier layers to the output, which introduces a structural bias and on some datasets even backfires into a negative Gap-Closure (Damped Sines dataset: LLLA reaches −18.75%).

本文的核心问题：当模型本身由于参数不确定性而"不确定"时，如何可靠地识别并过滤掉低可信度的生成样本？The central question of this paper: when the model itself is "uncertain" because of parameter uncertainty, how can low-credibility generated samples be reliably identified and filtered out?

93.1%Gap-Closure (Sines)
FLARE vs 47.2% LLLA

85.0%Gap-Closure (Damped)
FLARE vs −18.75% LLLA

0.5003ROC-AUC (Sines)
近似随机基准 0.5ROC-AUC (Sines)
near-random baseline 0.5

O(1/√m)随机子网络近似
收敛率理论保证Randomized subnetwork approximation
theoretical guarantee on the convergence rate

02 方法Method

FLARE 基于 Fisher–Laplace 框架，将参数后验协方差通过去噪器的 Jacobian 矩阵逐步传播到生成轨迹中，从而精确刻画每条轨迹的认知不确定性，同时通过随机子网络采样保持计算可行性。FLARE builds on the Fisher-Laplace framework: it propagates the parameter posterior covariance step by step into the generative trajectory through the Jacobian of the denoiser, thereby characterizing the epistemic uncertainty of each trajectory precisely, while randomized subnetwork sampling keeps the computation tractable.

四路认知不确定性比较 — **图：四路 epistemic UQ 全曲率对比。**从左到右分别为：BayesDiff（last-layer Laplace，total recursion）、Fuller-Hessian Laplace（BayesDiff recursion）、BayesDiff（epistemic-only recursion）、以及 FLARE（Fisher–Laplace projection）。在可精确计算全曲率的受控环境下，BayesDiff 产生的不确定性图谱缺乏认知含义，而 FLARE 能紧密追踪基于曲率的基准，同时避免了完整 Hessian 计算。**Figure: four-way full-curvature comparison of epistemic UQ.** From left to right: BayesDiff (last-layer Laplace, total recursion), Fuller-Hessian Laplace (BayesDiff recursion), BayesDiff (epistemic-only recursion), and FLARE (Fisher-Laplace projection). In a controlled setting where the full curvature can be computed exactly, the uncertainty maps produced by BayesDiff lack epistemic meaning, whereas FLARE tracks the curvature-based reference closely while avoiding the full Hessian computation.

A. Fisher 信息引导的参数不确定性投影A. Fisher-Information-Guided Projection of Parameter Uncertainty

在 Laplace 近似下，参数后验为高斯分布 θ ~ N(θ̂, Σ_θ)。FLARE 通过去噪器在每一步的 Jacobian J_t 将参数空间的不确定性投影到样本空间，得到认知协方差：Under the Laplace approximation the parameter posterior is a Gaussian θ ~ N(θ̂, Σ_θ). FLARE projects the uncertainty of parameter space into sample space through the Jacobian J_t of the denoiser at every step, which yields the epistemic covariance:

Σ^ep_t-1|t(η) = b²_t · J_t · Σ_θ · J_t^⊤

这一公式将认知方差从偶然噪声中显式隔离出来，沿整条反向扩散轨迹产生"认知视角"。This formula explicitly isolates the epistemic variance from the aleatoric noise, producing an "epistemic view" along the entire reverse diffusion trajectory.

B. 多步认知递推传播B. Multi-Step Epistemic Recursive Propagation

不确定性通过去噪步骤逐层向前递推，形成轨迹级的累积认知协方差：Uncertainty is propagated forward layer by layer through the denoising steps, forming a trajectory-level cumulative epistemic covariance:

Σ^ep_t-1(η) = a²_t · Σ^ep_t(η) + b²_t · J_t · Σ_θ · J_t^⊤

与仅考虑最终步骤的 LLLA 不同，这种递推式传播能捕捉整个去噪链条上每一步的参数敏感性。Unlike LLLA, which considers only the final step, this recursive propagation captures the parameter sensitivity of every step along the whole denoising chain.

C. 随机子网络近似（Randomized Subnetwork Approximation）C. Randomized Subnetwork Approximation

完整 Fisher 信息矩阵的计算代价与网络参数量的平方成正比，对大型扩散模型不可行。FLARE 的解决方案：The cost of computing the full Fisher information matrix scales with the square of the number of network parameters, which is infeasible for large diffusion models. FLARE's solution:

从全部网络层中均匀随机采样 m 个参数；sample m parameters uniformly at random from across all network layers;
将 GGN（Generalized Gauss-Newton）矩阵限制在所选坐标上；restrict the GGN (Generalized Gauss-Newton) matrix to the selected coordinates;
保留全网络的敏感性结构，同时大幅降低计算成本；preserve the sensitivity structure of the whole network while sharply reducing the computational cost;
避免了 LLLA 仅限最后一层所引入的"结构性偏差"。avoid the "structural bias" introduced by LLLA's restriction to the last layer.

Theorem 1 证明了该随机近似的近似误差以 O(1/√m) 速率收敛，随参数采样量 m 的增加而降低，提供了严格的理论保证。Theorem 1 proves that the approximation error of this randomized approximation converges at a rate of O(1/√m), decreasing as the number of sampled parameters m grows, which provides a rigorous theoretical guarantee.

参数预算消融实验（Bimodal） — **图：参数预算消融（FullSubnetLaplace，Bimodal Sinusoidal 数据集上半部分）。**保留参数比例依次为 {1%, 5%, 10%, 30%, 50%}。随着保留比例增大，轨迹逐渐收紧到数据流形附近，认知协方差平滑收缩；10–30% 时已能观察到稳定的 mode coverage，30–50% 后收益递减。**Figure: parameter-budget ablation (FullSubnetLaplace, upper half of the Bimodal Sinusoidal dataset).** The retained-parameter ratios are, in order, {1%, 5%, 10%, 30%, 50%}. As the retained ratio grows, the trajectories tighten toward the data manifold and the epistemic covariance contracts smoothly; stable mode coverage is already observable at 10–30%, and returns diminish beyond 30–50%.

参数预算消融实验（Chirp） — **图：参数预算消融（Chirp 数据集下半部分）。**同样规律：少量参数（10–30%）已足以捕捉关键的不确定性结构，计算效率与精度达到良好平衡。**Figure: parameter-budget ablation (lower half of the Chirp dataset).** The same pattern holds: a small fraction of the parameters (10–30%) already suffices to capture the key uncertainty structure, striking a good balance between computational efficiency and accuracy.

03 实验Experiments

在三个合成时间序列基准数据集上，与 BayesDiff 和 LLLA 两种基线进行对比，评估指标包括 Gap-Closure（越高越好，衡量高质量样本判别能力）和 ROC-AUC（越低越好，0.5 为随机水平）。所有改进均达到统计显著性（p < 0.005）。On three synthetic time-series benchmark datasets, FLARE is compared against the two baselines BayesDiff and LLLA. The evaluation metrics are Gap-Closure (higher is better, measuring the ability to discriminate high-quality samples) and ROC-AUC (lower is better, 0.5 being the random level). All improvements reach statistical significance (p < 0.005).

数据集Datasets

三个合成时间序列基准，各自对应一种建模挑战：Three synthetic time-series benchmarks, each corresponding to one modeling challenge:

Bimodal Sinusoids：双峰正弦波，测试模型对多峰分布的识别能力；Bimodal Sinusoids: bimodal sine waves, testing the model's ability to recognize multimodal distributions;
Chirped Sinusoids：频率渐变的正弦波，测试外推区域的不确定性估计；Chirped Sinusoids: sine waves with gradually changing frequency, testing uncertainty estimation in extrapolation regions;
Damped Sinusoids：幅度衰减的正弦波，测试振荡抑制效果。Damped Sinusoids: sine waves with decaying amplitude, testing the suppression of oscillations.

数据集Dataset	方法Method	Gap-Closure (%)	ROC-AUC	p 值 (bootstrap)p-value (bootstrap)
Sines	BayesDiff	+13.37	0.6153	0.0001
	LLLA	+47.19	0.5814	0.0201
	FLARE	+93.08	0.5003	0.0012
Chirp	BayesDiff	+41.73	0.6616	0.0030
	LLLA	+59.06	0.5891	0.0116
	FLARE	+74.31	0.5345	0.0002
Damped Sines	BayesDiff	+10.40	0.6861	0.0001
	LLLA	−18.75	0.7754	0.0066
	FLARE	+85.00	0.5085	0.0002

Bimodal Sinusoidal 数据集可视化 — **(a) Bimodal Sinusoidal。**训练集示例：双峰正弦波，代表典型的多峰分布挑战，需要模型在两个模式间正确估计不确定性。**(a) Bimodal Sinusoidal.** A training-set example: bimodal sine waves, representing the typical multimodal-distribution challenge, which requires the model to estimate uncertainty correctly between the two modes.

Chirp 数据集可视化 — **(b) Chirp。**频率随时间变化的正弦波，测试模型在外推区域（数据稀疏处）的 epistemic 不确定性估计能力。**(b) Chirp.** Sine waves whose frequency varies over time, testing the model's ability to estimate epistemic uncertainty in extrapolation regions (where the data is sparse).

Damped Sinusoidal 数据集可视化 — **(c) Damped Sinusoidal。**幅度衰减的正弦波，FLARE 过滤后成功抑制伪振荡，保留符合衰减规律的轨迹。LLLA 在此数据集上 Gap-Closure 为负（−18.75%），而 FLARE 达到 +85.00%，差距最为悬殊。**(c) Damped Sinusoidal.** Sine waves with decaying amplitude. After FLARE filtering, spurious oscillations are successfully suppressed and the trajectories that follow the decay law are retained. On this dataset the Gap-Closure of LLLA is negative (−18.75%), whereas FLARE reaches +85.00% — the widest gap of all.

定性结论Qualitative Findings

通过 FLARE 过滤后的样本呈现出三个一致特征：Samples filtered by FLARE exhibit three consistent characteristics:

在多峰设置中，样本集中在各模式中心附近，减少了模式之间的低密度漫游；in multimodal settings, samples concentrate near the center of each mode, reducing low-density wandering between modes;
在外推区域，消除了虚假的频率漂移；in extrapolation regions, spurious frequency drift is eliminated;
在幅度衰减场景下，抑制了伪振荡，同时保留了分布内的多样性。in amplitude-decay scenarios, spurious oscillations are suppressed while in-distribution diversity is preserved.

"The largest gains occur in settings requiring extrapolation or mode selection, where separating epistemic from aleatoric uncertainty is most critical."

04 局限性Limitations

Note: 以下局限性均为作者在论文中明确承认的内容。All limitations below are explicitly acknowledged by the authors in the paper.

评估范围局限于合成时间序列Evaluation is confined to synthetic time series

本文的实验仅在一维合成时间序列基准（Bimodal Sinusoids、Chirp、Damped Sinusoids）上进行。作者指出，将方法扩展到"更高维度领域，如多变量时间序列或图像生成，仍是一个重要的未来方向"（"extension to higher-dimensional domains such as multivariate time series or image generation remains an important direction"）。能否在真实世界数据或图像扩散模型上取得同等效果尚待验证。The experiments of this paper are carried out only on one-dimensional synthetic time-series benchmarks (Bimodal Sinusoids, Chirp, Damped Sinusoids). The authors note that the "extension to higher-dimensional domains such as multivariate time series or image generation remains an important direction". Whether comparable results can be obtained on real-world data or on image diffusion models remains to be verified.

大规模扩散模型的可扩展性挑战Scalability challenges for large-scale diffusion models

尽管随机子网络采样显著降低了计算成本，但作者承认"在非常大的扩散模型中近似曲率信息仍然代价高昂"（"approximating curvature information in very large diffusion models is still demanding"）。对于包含数十亿参数的现代扩散模型（如 Stable Diffusion），即便是随机子网络方案也面临内存和时间上的严峻挑战。Although randomized subnetwork sampling markedly lowers the computational cost, the authors concede that "approximating curvature information in very large diffusion models is still demanding". For modern diffusion models with billions of parameters (such as Stable Diffusion), even the randomized subnetwork scheme faces severe challenges in memory and time.

近似策略仍有改进空间The approximation strategy still leaves room for improvement

作者建议探索"结构化或自适应子采样策略、低秩曲率近似"（"structured or adaptive subsampling strategies, low-rank curvature approximations"）以进一步提升大型模型的适用性。现有的均匀随机采样方案在参数重要性差异悬殊时可能并非最优选择。The authors suggest exploring "structured or adaptive subsampling strategies, low-rank curvature approximations" in order to further improve applicability to large models. The existing uniform random sampling scheme may not be the optimal choice when parameter importance varies widely.