Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR

01 动机 · MotivationMotivation

Muon 是近期兴起的高效矩阵优化器，其核心操作是对动量矩阵做"矩阵符号"（matrix sign，即将所有奇异值统一归一到 1）。这一设计在 LLM 预训练中促进了各向同性探索（isotropic exploration），效果显著。然而，当 Muon 被移植到两种日益重要的后训练场景时，均匀谱白化暴露出根本性的失效模式：Muon is a recently emerging efficient matrix optimizer whose core operation applies the "matrix sign" to the momentum matrix (matrix sign, i.e. normalizing every singular value uniformly to 1). This design promotes isotropic exploration in LLM pretraining and works remarkably well. Yet when Muon is transplanted to two increasingly important post-training regimes, uniform spectral whitening exposes fundamental failure modes:

VLA 跨模态训练的低秩梯度问题Low-rank gradients in cross-modality VLA training

在视觉-语言-动作（VLA）微调中，动作模块（action module）的梯度本质上是低秩的（low-rank）。Muon 的均匀白化会将这些低秩噪声方向同等放大，导致有效秩（effective rank, erank）在训练过程中急剧下降，最终降低动作预测精度。In vision-language-action (VLA) fine-tuning, the gradients of the action module are intrinsically low-rank. Muon's uniform whitening amplifies these low-rank noise directions equally, causing the effective rank (erank) to fall sharply during training and ultimately degrading action-prediction accuracy.

RLVR 低信噪比梯度的模型崩塌Model collapse under low-SNR gradients in RLVR

在带可验证奖励的强化学习（RLVR）中，梯度信噪比（SNR）远低于 SFT 场景（接近 1:1）。Muon 均匀放大所有奇异方向，将噪声信号等同对待，导致模型精度从初始检查点骤降至接近零："accuracy drops from the initial checkpoint and converges to near zero."In reinforcement learning with verifiable rewards (RLVR), the gradient signal-to-noise ratio (SNR) is far lower than in the SFT regime (close to 1:1). Muon amplifies all singular directions uniformly, treating noise and signal alike, so model accuracy plunges from the initial checkpoint to near zero: "accuracy drops from the initial checkpoint and converges to near zero."

梯度 erank 变化曲线 — **Figure 1：**Muon 在 VLA 训练（VLA-Adapter on LIBERO Object）中的局限性。(a) 训练过程中各模块（V/L/A）梯度的平均有效秩（erank），每 900 步记录一次。(b) AdamW、Muon、Pion 的测试成功率曲线。(c) 总训练时间对比，Pion 的计算开销与 Muon 几乎相同。**Figure 1:** Limitations of Muon in VLA training (VLA-Adapter on LIBERO Object). (a) Mean effective rank (erank) of the gradients of each module (V/L/A) during training, logged every 900 steps. (b) Test success-rate curves of AdamW, Muon and Pion. (c) Comparison of total training time; Pion's computational overhead is almost identical to Muon's.

优化器成功率对比 — **Figure 1：**Muon 在 VLA 训练（VLA-Adapter on LIBERO Object）中的局限性。(a) 训练过程中各模块（V/L/A）梯度的平均有效秩（erank），每 900 步记录一次。(b) AdamW、Muon、Pion 的测试成功率曲线。(c) 总训练时间对比，Pion 的计算开销与 Muon 几乎相同。**Figure 1:** Limitations of Muon in VLA training (VLA-Adapter on LIBERO Object). (a) Mean effective rank (erank) of the gradients of each module (V/L/A) during training, logged every 900 steps. (b) Test success-rate curves of AdamW, Muon and Pion. (c) Comparison of total training time; Pion's computational overhead is almost identical to Muon's.

100%Pion — LIBERO Object 成功率Pion — LIBERO Object success rate

97.0%Muon — LIBERO Object 成功率Muon — LIBERO Object success rate

32.2%AdamW — LIBERO Object 成功率AdamW — LIBERO Object success rate

85.6%Pion — 真实机器人平均成功率Pion — mean success rate on a real robot

"Muon's uniform spectral whitening enhances exploration during pretraining but leads to fundamental limitations beyond that setting in two increasingly important regimes: cross-modality vision-language-action training, where low-rank action gradients amplify noise, and reinforcement learning with verifiable rewards, where low-SNR gradients and per-head specialization requirements cause instability."

02 方法 · MethodMethod

Pion 保留 Muon 的动量更新框架（Θₜ = Θₜ₋₁ - η · msign(Mₜ)），但用两阶段高通滤波替代均匀归一（matrix sign）：先通过 Promotion 多项式放大所有奇异值，再通过 Suppression 多项式将大奇异值锚定在 1 并将小奇异值压制趋零。整个过程通过多项式迭代近似，无需显式 SVD，保持与 Muon 相同的计算效率。Pion keeps Muon's momentum-update framework (Θₜ = Θₜ₋₁ - η · msign(Mₜ)) but replaces uniform normalization (matrix sign) with a two-stage high-pass filter: a Promotion polynomial first amplifies all singular values, then a Suppression polynomial anchors the large singular values at 1 and drives the small ones toward zero. The whole process is approximated by polynomial iterations, needs no explicit SVD, and retains the same computational efficiency as Muon.

Muon 的 Newton-Schulz 近似Muon's Newton-Schulz approximation

Muon 用 Newton-Schulz 多项式迭代近似矩阵符号函数（msign）：每次迭代对每个奇异值 σ 施加映射 f_NS(σ) = aσ + bσ³ + cσ⁵，系数为 (a,b,c) = (3.4445, -4.7750, 2.0315)。经过 5 步迭代后，所有奇异值均被推向 1（均匀白化），无论其原始幅度大小。这对低秩梯度或低 SNR 场景是有害的，因为噪声方向与信号方向被等同对待。Muon approximates the matrix sign function (msign) with Newton-Schulz polynomial iterations: each iteration applies the map f_NS(σ) = aσ + bσ³ + cσ⁵ to every singular value σ, with coefficients (a,b,c) = (3.4445, -4.7750, 2.0315). After 5 iterations all singular values are pushed toward 1 (uniform whitening), regardless of their original magnitude. This is harmful for low-rank gradients or low-SNR regimes, because noise directions and signal directions are treated alike.

Pion 的两阶段 Promotion + Suppression 机制Pion's two-stage Promotion + Suppression mechanism

Pion 将 5 步 Newton-Schulz 迭代分成两个子阶段：Pion splits the 5-step Newton-Schulz iteration into two sub-stages:

Promotion 阶段：多项式系数 (aₚ, bₚ, cₚ) = (1.875, -1.25, 0.375)，单调放大所有奇异值，作用类似"预增强"。Promotion stage: polynomial coefficients (aₚ, bₚ, cₚ) = (1.875, -1.25, 0.375) monotonically amplify all singular values, acting like a "pre-emphasis".
Suppression 阶段：多项式系数 (aₛ, bₛ, cₛ) = (0, 2.5, -1.5)，将已被放大到接近 1 的奇异值锚定在 1，同时将仍然较小的奇异值压制趋零。Suppression stage: polynomial coefficients (aₛ, bₛ, cₛ) = (0, 2.5, -1.5) anchor at 1 the singular values already amplified close to 1, while driving the still-small singular values toward zero.

整体映射为 f = fₛ^{kₛ} ∘ fₚ^{kₚ}，其中 kₚ + kₛ = 5。这种高通（high-pass）特性使 Pion 能区分主要奇异方向（信号）与噪声尾部（tail），实现选择性放大。The overall map is f = fₛ^{kₛ} ∘ fₚ^{kₚ}, where kₚ + kₛ = 5. This high-pass property lets Pion separate the leading singular directions (signal) from the noise tail, achieving selective amplification.

Muon NS iteration — **Figure 3：**奇异值映射函数 f(σ) 在 σ∈[0,1] 上的可视化，虚线为恒等参考。(a) Muon 的 NS 迭代逐步将所有奇异值均匀推向 1（均匀白化）。(b) Pion 的 Promotion 多项式 fₚ 单调放大。(c) Suppression 多项式 fₛ 实现高通效果。(d) Pion 最终组合，对主奇异值保持高响应，对尾部有效抑制。**Figure 3:** Visualization of the singular-value map f(σ) over σ∈[0,1], with the dashed line as the identity reference. (a) Muon's NS iteration progressively pushes all singular values uniformly toward 1 (uniform whitening). (b) Pion's Promotion polynomial fₚ amplifies monotonically. (c) The Suppression polynomial fₛ realizes the high-pass effect. (d) Pion's final composition keeps a high response for the leading singular values and effectively suppresses the tail.

**Figure 3：**奇异值映射函数 f(σ) 在 σ∈[0,1] 上的可视化，虚线为恒等参考。(a) Muon 的 NS 迭代逐步将所有奇异值均匀推向 1（均匀白化）。(b) Pion 的 Promotion 多项式 fₚ 单调放大。(c) Suppression 多项式 fₛ 实现高通效果。(d) Pion 最终组合，对主奇异值保持高响应，对尾部有效抑制。**Figure 3:** Visualization of the singular-value map f(σ) over σ∈[0,1], with the dashed line as the identity reference. (a) Muon's NS iteration progressively pushes all singular values uniformly toward 1 (uniform whitening). (b) Pion's Promotion polynomial fₚ amplifies monotonically. (c) The Suppression polynomial fₛ realizes the high-pass effect. (d) Pion's final composition keeps a high response for the leading singular values and effectively suppresses the tail.

Per-Head 模式（RLVR 专用）Per-head mode (RLVR-specific)

在 RLVR 场景中，注意力投影矩阵（Q/K/V projection）的不同 head 在预训练后已形成异质性特化（pretrained heterogeneity）。Pion 的 per-head 模式沿 head 维度对注意力投影进行重塑，对每个 head 独立施加高通 NS 迭代，从而在更新时保留各 head 的特化差异，而不引入额外的计算开销。In the RLVR regime, the different heads of the attention projection matrices (Q/K/V projection) have already developed heterogeneous specialization after pretraining (pretrained heterogeneity). Pion's per-head mode reshapes the attention projections along the head dimension and applies the high-pass NS iteration to each head independently, thereby preserving each head's specialization during updates without adding computational overhead.

MATH500 per-head accuracy — **Figure 4：**Per-head 高通 NS 对 RLVR 的影响（Qwen3-1.7B，GRPO on MATH levels 3–5）。(a) AdamW、Muon (default/per-head)、Pion (default/per-head) 在 MATH500 上的准确率曲线。(b) 预训练前后 Q-projection 跨 head 方差：Pion per-head 模式在更新后保留了更大的跨 head 异质性，而 default 模式会均匀化各 head。**Figure 4:** Effect of per-head high-pass NS on RLVR (Qwen3-1.7B, GRPO on MATH levels 3–5). (a) Accuracy curves on MATH500 for AdamW, Muon (default/per-head) and Pion (default/per-head). (b) Q-projection cross-head variance before and after pretraining: the Pion per-head mode retains greater cross-head heterogeneity after updates, whereas the default mode homogenizes the heads.

Cross-head Q-projection variance — **Figure 4：**Per-head 高通 NS 对 RLVR 的影响（Qwen3-1.7B，GRPO on MATH levels 3–5）。(a) AdamW、Muon (default/per-head)、Pion (default/per-head) 在 MATH500 上的准确率曲线。(b) 预训练前后 Q-projection 跨 head 方差：Pion per-head 模式在更新后保留了更大的跨 head 异质性，而 default 模式会均匀化各 head。**Figure 4:** Effect of per-head high-pass NS on RLVR (Qwen3-1.7B, GRPO on MATH levels 3–5). (a) Accuracy curves on MATH500 for AdamW, Muon (default/per-head) and Pion (default/per-head). (b) Q-projection cross-head variance before and after pretraining: the Pion per-head mode retains greater cross-head heterogeneity after updates, whereas the default mode homogenizes the heads.

03 实验 · ExperimentsExperiments

实验覆盖两大后训练场景：(1) VLA 训练，在 LIBERO 基准和真实机器人上评估 VLA-Adapter 与 VLANeXt；(2) RLVR 后训练，在 MATH 和 GSM8K 数据集上用 GRPO/GMPO 算法评估 Qwen3-1.7B 与 Qwen3-4B。三个优化器（AdamW、Muon、Pion）共享其余所有超参数，仅优化器配置不同。The experiments cover two major post-training regimes: (1) VLA training, evaluating VLA-Adapter and VLANeXt on the LIBERO benchmark and on a real robot; (2) RLVR post-training, evaluating Qwen3-1.7B and Qwen3-4B with the GRPO/GMPO algorithms on the MATH and GSM8K datasets. The three optimizers (AdamW, Muon, Pion) share all remaining hyperparameters and differ only in optimizer configuration.

VLA 实验：LIBERO 基准VLA experiments: the LIBERO benchmark

模型 / 数据集Model / dataset	AdamW	Muon	Pion
VLA-Adapter — LIBERO Object (1,500 steps)	32.2%	97.0%	100%
VLANeXt — LIBERO 平均VLANeXt — LIBERO average	79.45%	93.65%	96.35%
VLANeXt — LIBERO-Plus 平均VLANeXt — LIBERO-Plus average	64.57%	72.34%	75.93%

VLA-Adapter LIBERO success rates — **Figure 5：**VLA-Adapter 在 LIBERO 四个任务套件（Object、Spatial、Goal、Long）上的测试成功率（a）以及 Object 任务的训练曲线（b）。在相同训练预算下，Pion 在所有任务上均优于或持平 Muon，且大幅超越 AdamW。**Figure 5:** Test success rates of VLA-Adapter on the four LIBERO task suites (Object, Spatial, Goal, Long) (a) and the training curve of the Object task (b). Under the same training budget Pion matches or beats Muon on every task and far surpasses AdamW.

VLA-Adapter Object training curve — **Figure 5：**VLA-Adapter 在 LIBERO 四个任务套件（Object、Spatial、Goal、Long）上的测试成功率（a）以及 Object 任务的训练曲线（b）。在相同训练预算下，Pion 在所有任务上均优于或持平 Muon，且大幅超越 AdamW。**Figure 5:** Test success rates of VLA-Adapter on the four LIBERO task suites (Object, Spatial, Goal, Long) (a) and the training curve of the Object task (b). Under the same training budget Pion matches or beats Muon on every task and far surpasses AdamW.

真实机器人实验（π₀.₅ · Franka Research 3）Real-robot experiments (π₀.₅ · Franka Research 3)

在 DROID 配置下使用 π₀.₅ 作为 backbone，对三项抓取放置任务各进行 30 次随机初始化测试：Using π₀.₅ as the backbone under the DROID configuration, three pick-and-place tasks were each tested with 30 random initializations:

优化器Optimizer	平均成功率（3 个任务）Mean success rate (3 tasks)
AdamW	31.1%
Muon	38.9%
Pion	85.6%

RLVR 实验：数学推理后训练RLVR experiments: post-training for mathematical reasoning

在 8 种配置（GRPO/GMPO × Qwen3-1.7B/4B × MATH/GSM8K）下评估，Muon 在所有 RLVR 配置中均出现崩塌，而 Pion 在全部 8 种配置下超越 AdamW：Evaluated under 8 configurations (GRPO/GMPO × Qwen3-1.7B/4B × MATH/GSM8K), Muon collapses in every RLVR configuration, whereas Pion beats AdamW in all 8 configurations:

GRPO Qwen3-1.7B MATH — **Figure 6（部分）：**RLVR 场景下验证准确率 vs. 训练步数（GRPO，Qwen3-1.7B/4B，MATH/GSM8K）。Muon 在所有配置中均崩塌至接近零准确率；Pion 在收敛速度和最终精度上均优于 AdamW。**Figure 6 (partial):** Validation accuracy vs. training steps in the RLVR regime (GRPO, Qwen3-1.7B/4B, MATH/GSM8K). Muon collapses to near-zero accuracy in every configuration; Pion beats AdamW in both convergence speed and final accuracy.

GRPO Qwen3-1.7B GSM8K — **Figure 6（部分）：**RLVR 场景下验证准确率 vs. 训练步数（GRPO，Qwen3-1.7B/4B，MATH/GSM8K）。Muon 在所有配置中均崩塌至接近零准确率；Pion 在收敛速度和最终精度上均优于 AdamW。**Figure 6 (partial):** Validation accuracy vs. training steps in the RLVR regime (GRPO, Qwen3-1.7B/4B, MATH/GSM8K). Muon collapses to near-zero accuracy in every configuration; Pion beats AdamW in both convergence speed and final accuracy.

消融实验Ablation studies

论文通过"低通 Muon（LPMuon）"消融验证高通滤波的必要性：将 Pion 的高通映射反转为低通（抑制大奇异值、放大小奇异值），结果在 RLVR 上退化至与 Muon 相似的崩塌行为，确认高通特性是 Pion 性能提升的关键。此外，per-head 模式消融表明，对 RLVR 场景中注意力投影的逐 head 独立处理是维持预训练异质性、避免 head 间均匀化的必要条件。The paper verifies the necessity of high-pass filtering through a "low-pass Muon (LPMuon)" ablation: reversing Pion's high-pass map into a low-pass one (suppressing large singular values, amplifying small ones) degrades RLVR to collapse behavior similar to Muon, confirming that the high-pass property is the key to Pion's gains. In addition, the per-head ablation shows that handling the attention projections of the RLVR regime independently per head is a necessary condition for maintaining pretrained heterogeneity and avoiding homogenization across heads.

04 局限性 · LimitationsLimitations

Note：以下限制来源于论文附录 M（Appendix M），为作者明确陈述（stated）的局限性。Note: The following limitations come from Appendix M of the paper and are limitations explicitly stated by the authors.

Per-head 模式在超大模型中的计算开销Computational overhead of per-head mode in very large models

Pion 的 per-head 模式需要对注意力投影矩阵沿 head 维度进行重塑，并对每个 head 独立执行多项式迭代。虽然对当前规模的模型（如 Qwen3-1.7B/4B）开销可忽略不计，但对于拥有数千 head 的超大规模模型，这一操作的内存和计算代价可能成为瓶颈。Pion's per-head mode must reshape the attention projection matrices along the head dimension and run the polynomial iteration independently for each head. Although the overhead is negligible for models at the current scale (such as Qwen3-1.7B/4B), for very large models with thousands of heads the memory and compute cost of this operation may become a bottleneck.

多项式系数缺乏理论保证Polynomial coefficients lack theoretical guarantees

Pion 的 Promotion 和 Suppression 多项式系数（(aₚ, bₚ, cₚ) 和 (aₛ, bₛ, cₛ)）目前主要通过实验调优得到，缺乏严格的理论最优性证明。在不同的训练分布和模型结构下，最优系数可能不同。Pion's Promotion and Suppression polynomial coefficients ((aₚ, bₚ, cₚ) and (aₛ, bₛ, cₛ)) are currently obtained mainly through empirical tuning and lack a rigorous proof of theoretical optimality. Under different training distributions and model architectures the optimal coefficients may differ.

超参数 kₚ 的选择缺乏自适应机制No adaptive mechanism for choosing the hyperparameter kₚ

Pion 中控制 Promotion 与 Suppression 步数比例的超参数 kₚ（其中 kₛ = 5 − kₚ）对不同下游任务可能需要手动调整，论文目前未提供跨任务的通用自适应选择策略。The hyperparameter kₚ in Pion, which controls the ratio of Promotion to Suppression steps (where kₛ = 5 − kₚ), may need manual tuning for different downstream tasks, and the paper currently offers no general cross-task adaptive selection strategy.