Improved Mean Flows：加速生成模型的挑战与改进Improved Mean Flows: On the Challenges of Fastforward Generative Models
单步生成 ImageNet 256×256，FID 达到 1.72，无需预训练或蒸馏One-step generation on ImageNet 256×256, reaching 1.72 FID with no pre-training and no distillation
Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, Kaiming He
本文系统分析了 MeanFlow (MF) 框架在快速前向生成模型中存在的两大核心挑战：训练目标依赖于网络自身（非标准回归问题），以及推理时 Classifier-Free Guidance (CFG) 尺度固定缺乏灵活性。作者提出了改进版本 iMF，通过将训练目标重构为瞬时速度（instantaneous velocity）损失、引入灵活 guidance 条件化，以及设计轻量级 in-context conditioning，在 ImageNet 256×256 上以单次函数评估（1-NFE）达到 1.72 FID，相比原始 MF 的 3.43 FID 提升约 50%，且无需任何预训练或蒸馏。This paper systematically analyzes two core challenges of the MeanFlow (MF) framework for fastforward generative models: its training target depends on the network itself (so it is not a standard regression problem), and the Classifier-Free Guidance (CFG) scale is fixed at inference, which lacks flexibility. The authors propose an improved version, iMF, that recasts the training target as an instantaneous velocity loss, introduces flexible guidance conditioning, and designs a lightweight in-context conditioning. On ImageNet 256×256 it reaches 1.72 FID with a single function evaluation (1-NFE), about a 50% improvement over the 3.43 FID of the original MF, and requires no pre-training or distillation.
快速前向生成模型（fastforward generative models）旨在用极少的函数评估次数（NFE）生成高质量图像。MeanFlow 是这一方向的代表性方法，但原始 MF 存在两个根本性挑战，限制了其性能和使用灵活性。Fastforward generative models aim to generate high-quality images with an extremely small number of function evaluations (NFE). MeanFlow is a representative method in this direction, but the original MF suffers from two fundamental challenges that limit its performance and its flexibility in use.
"原始 MF 的训练目标依赖于网络本身，而非构成一个标准的回归问题。" — 作者对第一个核心挑战的描述"The training target of the original MF depends on the network itself, rather than constituting a standard regression problem." — the authors' description of the first core challenge
挑战一：训练目标的网络依赖性Challenge 1: the training target depends on the network
原始 MeanFlow 的目标中含有网络自身的输出，这使得训练目标并非固定的"监督信号"，而是随网络参数变化而变化的移动靶。实验表明，原始 MF 的训练损失不仅存在较大方差，甚至出现不下降的现象（non-decreasing loss）。The objective of the original MeanFlow contains the output of the network itself, which makes the training target not a fixed "supervisory signal" but a moving target that shifts as the network parameters change. Experiments show that the training loss of the original MF not only has large variance but can even be non-decreasing (non-decreasing loss).
挑战二：固定 CFG 尺度缺乏灵活性Challenge 2: a fixed CFG scale lacks flexibility
原始 MF 在训练时将 Classifier-Free Guidance 的尺度 ω 固定，但实验发现最优 CFG 尺度随模型容量和训练进度而变化（图 4）。固定 ω 导致同一模型无法在推理时灵活调整 guidance 强度，降低了实用性。The original MF fixes the Classifier-Free Guidance scale ω during training, yet experiments find that the optimal CFG scale varies with model capacity and training progress (Figure 4). A fixed ω prevents a single model from adjusting the guidance strength flexibly at inference, which reduces its practicality.
图 1： 原始 MeanFlow（左）通过网络自我代入预测平均速度，存在训练目标依赖网络的问题；改进方法（右）将训练重构为以瞬时速度为目标的合法回归问题，训练目标仅依赖于输入 z_t，与网络参数无关。Figure 1: The original MeanFlow (left) predicts the average velocity by substituting the network into its own target, so the training target depends on the network; the improved method (right) recasts training as a valid regression problem onto the instantaneous velocity, where the training target depends only on the input z_t and is independent of the network parameters.
1-NFE单次函数评估 超越多步蒸馏方法single function evaluation surpasses multi-step distillation
02方法Method
iMF 在三个层面对原始 MeanFlow 进行了系统改进：将训练目标重构为 v-loss（velocity loss）、引入灵活的 guidance 条件化、以及用 in-context conditioning 替换参数密集的 adaLN-zero。iMF systematically improves the original MeanFlow along three axes: recasting the training target as a v-loss (velocity loss), introducing flexible guidance conditioning, and replacing the parameter-heavy adaLN-zero with in-context conditioning.
改进一：MeanFlow as v-loss（速度损失重构）Improvement 1: MeanFlow as v-loss (velocity loss reformulation)
原始 MF 的训练目标可以分解为：对平均速度 u(z_t) 的预测，加上一个关于时间导数的修正项。通过引入 MeanFlow 恒等式，可以将目标重写为对瞬时速度 v(z_t) 的回归：The training target of the original MF can be decomposed into a prediction of the average velocity u(z_t) plus a correction term involving the time derivative. By invoking the MeanFlow identity, the target can be rewritten as a regression onto the instantaneous velocity v(z_t):
V_θ(z_t) ≜ u_θ(z_t) + (t−r) · JVP_sg(u_θ; v_θ)
其中 JVP（Jacobian-Vector Product）在 stop-gradient（sg）下计算，确保训练目标仅依赖当前输入 z_t，而与网络参数无关，构成合法的监督回归。Here the JVP (Jacobian-Vector Product) is computed under a stop-gradient (sg), which guarantees that the training target depends only on the current input z_t and not on the network parameters, forming a valid supervised regression.
论文提出两种实现方式：Boundary condition（令 v_θ = u_θ(z_t, t, t)，无额外参数）和 Auxiliary head（独立的 v-head，共享主干参数，效果更优）。The paper proposes two implementations: Boundary condition (setting v_θ = u_θ(z_t, t, t), with no extra parameters) and Auxiliary head (a separate v-head that shares the backbone parameters, which performs better).
图 2： MeanFlow 可以被重构为以平均速度预测为参数化形式的速度损失（v-loss），推导基于 MeanFlow 恒等式关系。这一重构赋予了训练目标与网络无关的合法回归性质。Figure 2: MeanFlow can be recast as a velocity loss (v-loss) parameterized by the average-velocity prediction, with the derivation resting on the MeanFlow identity. This reformulation gives the training target the property of a valid, network-independent regression.图 3： 训练损失曲线对比。原始 MF 表现出高方差和不下降的损失曲线（非标准回归的典型症状）；而改进后的 iMF 收敛更加稳定平滑。Figure 3: Comparison of training loss curves. The original MF shows high variance and non-decreasing loss curves (a typical symptom of a non-standard regression), whereas the improved iMF converges more stably and smoothly.
实验发现，最优 CFG 尺度随模型大小和训练轮次显著变化（图 4），固定 ω 使得原始 MF 在不同推理场景下性能受限。Experiments show that the optimal CFG scale varies markedly with model size and training epochs (Figure 4), so a fixed ω limits the performance of the original MF across different inference settings.
解决方案：将 guidance 尺度 ω 作为显式条件变量，在训练时从分布中采样（偏向较小值以稳定训练），使单个模型在推理时支持任意 CFG 尺度。论文进一步扩展为 Ω = {ω, t_min, t_max}，额外支持 CFG 的应用区间控制。Solution: treat the guidance scale ω as an explicit conditioning variable, sampled from a distribution during training (biased toward smaller values to stabilize training), so that a single model supports an arbitrary CFG scale at inference. The paper further extends this to Ω = {ω, t_min, t_max}, which additionally supports control over the interval in which CFG is applied.
图 4： 不同训练 epoch 和推理步数下的最优 CFG 尺度（ω）差异显著。固定 ω 会导致在多数场景下使用次优的 guidance 强度，验证了灵活 guidance 设计的必要性。Figure 4: The optimal CFG scale (ω) differs markedly across training epochs and inference step counts. A fixed ω therefore leads to a suboptimal guidance strength in most settings, confirming the necessity of the flexible guidance design.
原始 DiT 使用参数密集的 adaLN-zero 来融合条件信息。iMF 改用多 token 的 in-context conditioning：每种条件（时间步 r, t；类别 c；guidance 因子 Ω）转化为若干可学习 token，与图像 token 拼接后统一送入 Transformer。The original DiT uses the parameter-heavy adaLN-zero to fuse conditioning information. iMF instead adopts a multi-token in-context conditioning: each condition (timesteps r, t; class c; guidance factor Ω) is turned into several learnable tokens, which are concatenated with the image tokens and fed into the Transformer together.
配置：类别条件用 8 个 token，其余条件各用 4 个 token。这一设计在 Base 规模上将参数量从 133M 减至 89M（减少 33%），同时 FID 进一步提升。Configuration: 8 tokens for the class condition and 4 tokens for each remaining condition. At the Base scale this design cuts the parameter count from 133M to 89M (a 33% reduction) while further improving FID.
图 5（对应原文图 6）： 改进的 in-context conditioning 架构。将每种条件类型（时间步、类别、guidance 因子）编码为多个 token 并拼接至图像 token 序列，取代了原来参数量较大的自适应层归一化（adaLN-zero），在减少参数的同时提升了性能。Figure 5 (Figure 6 in the paper): The improved in-context conditioning architecture. Each condition type (timestep, class, guidance factor) is encoded as multiple tokens and concatenated to the image token sequence, replacing the parameter-heavy adaptive layer normalization (adaLN-zero) and improving performance while reducing parameters.
03实验Experiments
所有实验在 ImageNet 256×256 上进行，使用 FID 作为主要评价指标，NFE（Number of Function Evaluations）衡量生成效率。基线为原始 MF 及其他快速生成模型。All experiments are conducted on ImageNet 256×256, using FID as the main evaluation metric and NFE (Number of Function Evaluations) to measure generation efficiency. The baselines are the original MF and other fastforward generative models.
消融实验（Table 1）Ablation study (Table 1)
通过逐步叠加各项改进，验证每个组件的独立贡献：Stacking the improvements one by one verifies the independent contribution of each component:
iMF-XL/2 以 1.72 FID 超越了所有现有的蒸馏方法（FACM-XL/2 为 1.76），且无需任何预训练或蒸馏，同时参数量更少（610M vs 675M）。甚至超越了多步方法 DiT-XL/2（250 NFE，2.27 FID）。With 1.72 FID, iMF-XL/2 surpasses every existing distillation method (FACM-XL/2 at 1.76) while requiring no pre-training or distillation and using fewer parameters (610M vs 675M). It even outperforms the multi-step DiT-XL/2 (250 NFE, 2.27 FID).
图 6（对应原文图 7）： FID 训练曲线展示了各改进组件叠加的累积效果：从基线 6.17，经过 v-loss 重构（5.68）、灵活 guidance（4.57）、in-context conditioning（4.09），逐步改进至最终的 3.39（iMF-B/2）。Figure 6 (Figure 7 in the paper): The FID training curves show the cumulative effect of stacking the improvements: from the 6.17 baseline, through the v-loss reformulation (5.68), flexible guidance (4.57) and in-context conditioning (4.09), improving step by step to a final 3.39 (iMF-B/2).
生成样本质量Quality of generated samples
图 7（对应原文图 8）： iMF-XL/2 在 ImageNet 上的 1-NFE 非精选生成样本，展示了单步生成的高视觉质量，涵盖多个不同类别。Figure 7 (Figure 8 in the paper): Uncurated 1-NFE samples generated by iMF-XL/2 on ImageNet, showing the high visual quality of one-step generation across a range of classes.图 8（对应原文附录图）： iMF-XL/2 在更多 ImageNet 类别上的非精选 1-NFE 生成样本，进一步验证了模型的多样性和生成质量。Figure 8 (appendix figure in the paper): Uncurated 1-NFE samples generated by iMF-XL/2 on further ImageNet classes, giving additional evidence for the diversity and generation quality of the model.
消融：灵活 Guidance 的额外收益Ablation: the extra benefit of flexible guidance
在 Ω-conditioning 训练完成后，若将 guidance 尺度固定为 ω=1.0（即不使用 CFG），灵活 guidance 模型的 FID 为 20.95，而未经 Ω-conditioning 训练的对应模型为 30.76。这说明灵活 guidance 的训练机制本身对学习表示质量有正向迁移效果，即便在推理时不使用 CFG 也受益。After Ω-conditioning training, fixing the guidance scale at ω=1.0 (i.e. using no CFG) gives the flexible-guidance model an FID of 20.95, whereas the corresponding model trained without Ω-conditioning reaches 30.76. This shows that the flexible guidance training scheme itself transfers positively to the quality of the learned representation, so the model benefits even when CFG is not used at inference.
04局限性Limitations
注： 论文在 Conclusion 部分明确提出了一项局限性，其余为根据论文设计推断（inferred）的潜在限制。Note: The paper explicitly states one limitation in its Conclusion; the remaining items are potential limitations inferred from its design.
Tokenizer 推理开销（stated — 作者明确指出）Tokenizer overhead at inference (stated — explicitly noted by the authors)
论文原文指出："随着 1-NFE 生成的显著进步，tokenizer 在推理时产生的开销变得不可忽视（the use of a tokenizer begins to incur a non-negligible cost at inference time）"。在单步生成成为主要瓶颈被攻克后，图像编解码器的耗时占比相对上升。作者期待未来研究探索更高效的 tokenizer 或直接在像素空间进行生成。The paper states: "with the significant progress of 1-NFE generation, the use of a tokenizer begins to incur a non-negligible cost at inference time". Once one-step generation has removed the main bottleneck, the share of runtime spent in the image codec rises correspondingly. The authors look forward to future work on more efficient tokenizers or on generating directly in pixel space.
训练成本仍然较高（inferred — 推断）Training cost is still high (inferred)
iMF-XL/2 模型有 610M 参数，需要在 ImageNet 上训练数百个 epoch。虽然无需预训练，但完整的从零训练过程对计算资源要求依然较高。论文未提供训练所需 GPU 小时数等具体信息。The iMF-XL/2 model has 610M parameters and must be trained for hundreds of epochs on ImageNet. Although no pre-training is needed, the full from-scratch training procedure still demands considerable compute. The paper does not report details such as the GPU hours required.
评估局限于 ImageNet 256×256（inferred — 推断）Evaluation limited to ImageNet 256×256 (inferred)
所有定量实验均在 ImageNet 256×256 类别条件生成上进行。对于文本条件生成（text-to-image）、更高分辨率（如 512×512 或更高）以及其他数据集上的泛化能力，论文未作评估，方法在这些设置下的适用性有待验证。All quantitative experiments are run on class-conditional generation on ImageNet 256×256. Generalization to text-conditional generation (text-to-image), higher resolutions (such as 512×512 or beyond) and other datasets is not evaluated, so the applicability of the method in those settings remains to be verified.
JVP 计算的额外开销（inferred — 推断）Extra cost of the JVP computation (inferred)
v-loss 重构中使用了 Jacobian-Vector Product (JVP) 来计算时间导数，这在训练时引入了额外的计算开销（相当于一次额外的前向传播），可能增加每步训练时间。论文未明确量化这一开销的影响。The v-loss reformulation uses a Jacobian-Vector Product (JVP) to compute the time derivative, which adds computation during training (roughly one extra forward pass) and may increase the per-step training time. The paper does not explicitly quantify the impact of this overhead.
