Legato: Learning Native Continuation for Action Chunking Flow Policies
将平滑续接能力内嵌到流匹配 VLA 策略中，从根本上消除 chunk 边界的不连续性Embedding smooth continuation natively into flow-matching VLA policies to eliminate the discontinuity at chunk boundaries at its root
Yufeng Liu, Hang Yu, Juntu Zhao, Bocheng Li, Di Zhang, Mingzhu Li, Wenxuan Wu, Yingdong Hu, Junyuan Xie, Junliang Guo, Dequan Wang†, Yang Gao† · Spirit AI / Shanghai Jiao Tong University / Tsinghua University / Tongji University / USTC
Action chunking 让 VLA 模型得以实时运行，但 chunk 边界处的不连续性会导致机器人犹豫、抖动与模态切换。Legato 通过训练时的 schedule-shaped 引导与速度场重塑，使策略在推理时自然延续，无需外部补丁。在五项真实双臂操作任务上，Legato 相比 RTC 将轨迹平滑度与完成时间均提升约 10%。Action chunking is what lets VLA models run in real time, but the discontinuity at chunk boundaries makes robots hesitate, jitter and switch modes. Legato uses schedule-shaped guidance and velocity-field reshaping during training so that the policy continues naturally at inference, with no external patch. On five real-world bimanual manipulation tasks, Legato improves both trajectory smoothness and completion time by about 10% over RTC.
Action chunking 是让大型 VLA 模型在机器人上实时运行的关键技术——每次推理预测一段动作序列（chunk），避免逐帧调用昂贵模型。然而，当前一个 chunk 执行完毕、新的 chunk 接续时，两段轨迹之间往往出现明显的跳变与不连续。Action chunking is the key technique that lets large VLA models run in real time on robots: each inference predicts a segment of the action sequence (a chunk), avoiding a costly model call per frame. However, when one chunk finishes executing and a new chunk takes over, an obvious jump and discontinuity often appears between the two trajectory segments.
问题一：chunk 边界不连续Problem 1: discontinuity at chunk boundaries
原始 action chunking 的执行在 chunk 边界处会产生速度/加速度跳变，机械臂抖动明显，影响任务成功率。Vanilla action chunking produces velocity/acceleration jumps at chunk boundaries; the arm jitters visibly, which hurts the task success rate.
问题二：RTC 存在外部 patch 缺陷Problem 2: RTC is an external patch with flaws
Real-Time Chunking (RTC) 在推理时做 inpainting，将已执行动作作为约束注入去噪过程。但它与训练目标不一致，导致虚假模态切换（spurious multimodal switching）——机械臂在不同抓取目标和执行臂之间反复横跳，产生大量犹豫动作。Real-Time Chunking (RTC) performs inpainting at inference time, injecting the already-executed actions into the denoising process as constraints. But this is inconsistent with the training objective and causes spurious multimodal switching — the arm flip-flops between different grasp targets and executing arms, producing a lot of hesitant motion.
"RTC applies inference-time inpainting... leading to spurious multimodal switching and trajectories that are not intrinsically smooth."
图 1：五项真实操作任务上，Legato 与 RTC 的完成时间（横轴，越低越好）和 NSPARC 平滑度（纵轴，越低越好）对比。Legato 在所有任务上均更快、更平滑，整体右下方向偏移明显（箭头所示）。Figure 1: Completion time (horizontal axis, lower is better) versus NSPARC smoothness (vertical axis, lower is better) for Legato and RTC on five real-world manipulation tasks. Legato is faster and smoother on every task, with a clear shift toward the lower right (arrows).
~10%轨迹平滑度提升（NSPARC↓）trajectory smoothness gain (NSPARC↓)
图 3：单次（one-shot）prefix 引导无法在整个去噪过程中保持约束。随着去噪步骤推进（颜色从蓝→红），overlap 动作逐渐偏离参考轨迹（GT），说明必须在每步去噪中持续施加引导。Figure 3: One-shot prefix guidance cannot maintain the constraint throughout denoising. As the denoising steps progress (color from blue to red), the overlap actions drift away from the reference trajectory (GT), showing that guidance must be applied continuously at every denoising step.
02方法 MethodMethod
Legato 的核心思路是将 continuation 约束直接编码进 flow matching 的训练目标，使网络在推理时能够自然地延续已执行的前缀动作，而无需任何推理时 patch。The core idea of Legato is to encode the continuation constraint directly into the flow matching training objective, so that at inference the network naturally continues the already-executed prefix actions without any inference-time patch.
图 2：Legato 的整体框架。调度参数定义如下：s 为每个周期执行的动作长度；d 为完全引导的前缀长度（inference delay）；r 控制引导强度在剩余 horizon 上的线性衰减长度。训练时随机采样 (d, s, r) 并条件化，使单一模型适应不同推理延迟。Figure 2: The overall framework of Legato. The schedule parameters are defined as follows: s is the action length executed per cycle; d is the fully guided prefix length (inference delay); r controls the linear decay length of the guidance strength over the remaining horizon. During training, (d, s, r) is sampled at random and conditioned on, so that a single model adapts to different inference delays.
去噪的起点不再是纯噪声，而是混合了已知前缀动作 A 与噪声 ε 的有效噪声：The starting point of denoising is no longer pure noise, but an effective noise that mixes the known prefix actions A with noise ε:
ε_eff = (1 − ω) ⊙ ε + ω ⊙ A
其中 ω 是 schedule-shaped 的引导权重向量，前缀区域权重接近 1（强约束），远端逐渐降为 0（自由生成）。这使模型在去噪起点便已感知到部分动作信息。Here ω is the schedule-shaped guidance weight vector: the weight is close to 1 in the prefix region (strong constraint) and decays to 0 further out (free generation). The model therefore already perceives part of the action information at the start of denoising.
训练目标被重新加权，前缀区域的速度场目标受到抑制：The training objective is reweighted so that the velocity-field target is suppressed in the prefix region:
v_target = (1 − κ ⊙ (1 − t)) ⊙ (A − ε)
这使得网络在高引导区域倾向于"停留"，而非切换到另一种模态，从根本上抑制了虚假模态切换。This makes the network tend to "stay" in strongly guided regions rather than switch to another mode, suppressing spurious multimodal switching at its root.
3. Per-Step Guidance ODE（每步引导的精确动力学）3. Per-Step Guidance ODE (the exact dynamics of per-step guidance)
在推理时，每步去噪后都会重新施加引导，等效于如下 ODE：At inference, guidance is re-applied after every denoising step, which is equivalent to the following ODE:
Ẏ(t) = (1 − ω) ⊙ f_θ(Y(t), t) − κ ⊙ (Y(t) − A)
Legato 通过推导网络目标使训练动力学与上述推理 ODE 精确一致，消除训练-推理的不对齐（training-inference inconsistency）。By deriving the network target, Legato makes the training dynamics match the inference ODE above exactly, removing the training-inference inconsistency.
训练时随机采样 (d, s, r) 三元组，并将其作为额外条件输入策略网络。这使得单一模型即可适应不同推理延迟，无需为每种延迟单独训练。During training, (d, s, r) triples are sampled at random and fed to the policy network as an additional condition. This lets a single model adapt to different inference delays, with no need to train a separate model per delay.
图 4：在双臂机器人上评估的五项操作任务：叠碗（Bowl）、倒水（Pour）、拾放（PickPlace）、折毛巾（Towel）、开抽屉（Drawer），覆盖多种运动模式与多模态选择场景。Figure 4: The five manipulation tasks evaluated on a bimanual robot: stacking bowls (Bowl), pouring water (Pour), pick-and-place (PickPlace), folding a towel (Towel) and opening a drawer (Drawer), covering diverse motion patterns and multimodal choice scenarios.
03实验 ExperimentsExperiments
在真实双臂机器人上评估五项操作任务，每个任务 30–50 次 trials，与 RTC 及其训练时版本进行对比。评估指标涵盖任务完成得分、完成时间、NSPARC（频域平滑度）、NLDLJ（冲击积分）和 Overlap RMSE（chunk 边界一致性）。Five manipulation tasks are evaluated on a real bimanual robot with 30–50 trials each, compared against RTC and its training-time variant. The metrics cover task completion score, completion time, NSPARC (frequency-domain smoothness), NLDLJ (jerk integral) and Overlap RMSE (chunk-boundary consistency).
Table I：五项任务主要结果（vs. RTC）Table I: main results on five tasks (vs. RTC)
图 5：叠碗任务的定性对比。RTC 在连续 chunk 间反复在不同抓取目标与执行臂之间切换，产生明显犹豫和冗余纠正动作。Legato 始终保持一致的抓取目标与臂的选择，执行流畅。Figure 5: Qualitative comparison on the bowl-stacking task. Across consecutive chunks, RTC repeatedly switches between different grasp targets and executing arms, producing visible hesitation and redundant corrective motion. Legato keeps a consistent choice of grasp target and arm, and executes fluently.
消融实验（Ablations）Ablations
通过改变 (d, s, r) 三元组进行调度参数消融（Table III）：减小 stride 可提升 overlap 一致性（Overlap RMSE↓），但可能轻微牺牲全局平滑度；schedule conditioning（条件化调度参数）对两个指标均有稳定改善（Table IV）。π₀ 主干模型上也复现了 Legato 的优势（Table V，完成时间 88.30s vs RTC 92.93s），证明方法的通用性。Schedule-parameter ablations vary the (d, s, r) triple (Table III): a smaller stride improves overlap consistency (Overlap RMSE↓) but may slightly sacrifice global smoothness; schedule conditioning improves both metrics consistently (Table IV). The advantage of Legato also reproduces on a π₀ backbone (Table V, completion time 88.30s vs RTC 92.93s), showing that the method generalizes.
图 6：调度参数消融揭示 overlap 一致性与全局平滑度之间存在可控权衡。在所有配置下，Legato 均优于 RTC，且随 stride 减小改善更显著。Figure 6: The schedule-parameter ablation reveals a controllable trade-off between overlap consistency and global smoothness. Legato outperforms RTC under every configuration, and the improvement grows as the stride shrinks.
04局限性 LimitationsLimitations
说明：以下局限性结合论文明确陈述与设计层面推断，已标注来源。Note: The limitations below combine statements made explicitly in the paper with inferences at the design level; the source of each is marked.
去噪步数在训练时固定，推理时无法动态调整（论文明确陈述）The number of denoising steps is fixed at training time and cannot be adjusted dynamically at inference (stated explicitly in the paper)
论文指出："the denoise step is specified at training time, limiting the ability to adjust it during inference." 一旦训练完成，不同推理延迟只能通过 schedule conditioning 来适配，无法完全自由地改变去噪步数。The paper notes: "the denoise step is specified at training time, limiting the ability to adjust it during inference." Once training is finished, different inference delays can only be accommodated through schedule conditioning, and the number of denoising steps cannot be changed entirely freely.
续接方案的灵活性仍有提升空间（论文明确陈述）There is still room to make the continuation scheme more flexible (stated explicitly in the paper)
论文将 "more flexible native continuation schemes" 列为未来工作方向，暗示当前 schedule-shaped 线性衰减的引导形式尚有局限，更复杂的引导曲线或自适应调度仍待探索。The paper lists "more flexible native continuation schemes" as a direction for future work, implying that the current schedule-shaped linear-decay form of guidance still has limits, and that more elaborate guidance curves or adaptive schedules remain to be explored.
实验仅覆盖流匹配（flow matching）框架（设计推断）Experiments cover only the flow matching framework (inferred from the design)
Legato 的速度场重塑与 per-step guidance ODE 专为 flow-based 策略设计。能否推广到扩散（diffusion）或自回归动作策略尚未验证。The velocity-field reshaping and per-step guidance ODE of Legato are designed specifically for flow-based policies. Whether they carry over to diffusion or autoregressive action policies has not been verified.
评估局限于双臂桌面操作任务（设计推断）Evaluation is limited to bimanual tabletop manipulation tasks (inferred from the design)
所有实验均在固定桌面操作场景下进行，对移动操作、腿式机器人或高动态任务的泛化能力尚未验证。All experiments are conducted in a fixed tabletop manipulation setting; generalization to mobile manipulation, legged robots or highly dynamic tasks has not been verified.
