BPP: Long-Context Robot Imitation Learning by Focusing on Key History Frames
用 VLM 检测的语义关键帧代替全历史条件，从根源上解决“历史覆盖不足”问题Replacing full-history conditioning with semantic keyframes detected by a VLM, solving the "insufficient history coverage" problem at its root
Max Sobol Mark, Jacky Liang, Maria Attarian, Chuyuan Fu, Debidatta Dwibedi, Dhruv Shah, Aviral Kumar · Google DeepMind / CMU AIRe Lab
很多机器人任务需要“记住过去发生了什么”——找抽屉里的钥匙要记住哪些抽屉已经翻过，加两勺糖要记住已经加过一勺。但主流机器人策略只看当前观测。直接把历史观测拼进策略输入常常越训越差：策略会学到训练历史中的偶然特征（spurious correlations），这些特征在部署时的分布外历史上完全失效。本文提出 Big Picture Policies (BPP)：不再条件化于原始历史序列，而是用视觉语言模型（VLM）挑出少数“行为关键帧”（keyframes），把庞大的历史空间压缩成一小撮任务相关事件，从而大幅提升训练/部署时输入分布的重合度。Many robot tasks require "remembering what has already happened": finding the keys in a drawer requires remembering which drawers have already been searched, and adding two spoonfuls of sugar requires remembering that one spoonful has already gone in. Yet mainstream robot policies only look at the current observation. Naively concatenating past observations into the policy input often makes training worse: the policy latches onto incidental features of the training histories (spurious correlations), and those features fail completely on the out-of-distribution histories seen at deployment. This paper proposes Big Picture Policies (BPP): instead of conditioning on the raw history sequence, it uses a vision-language model (VLM) to pick out a handful of "behaviorally salient keyframes", compressing the vast history space into a small set of task-relevant events and thereby greatly increasing the overlap between the input distributions at training and deployment time.
当前最强的模仿学习策略普遍是“无记忆”的——只条件化于当前观测，或最多一个很短的时间窗口。这在需要历史推理的任务上（在杂乱场景里找东西、精确加入若干份配料、执行多阶段流程）注定失败：仅凭当前观测无法判断正确动作。而直接把过去观测历史拼进策略输入，看似自然，却被以往工作反复证实会“伤害”性能。The strongest imitation learning policies today are generally "memoryless": conditioned only on the current observation, or at most on a very short time window. On tasks that require reasoning over history (searching for an object in a cluttered scene, adding a precise number of portions of an ingredient, executing a multi-stage procedure) this is bound to fail, because the correct action cannot be determined from the current observation alone. Concatenating the past observation history into the policy input looks natural, yet prior work has repeatedly shown that it "hurts" performance.
“Naïvely conditioning on past observations often fails due to spurious correlations: policies latch onto incidental features of training histories that do not generalize to out-of-distribution trajectories upon deployment.”
作者进一步分析了根因：可能的历史观测空间随时间跨度呈指数增长，现有远程操作数据采集范式（人类专家近最优演示）根本无法覆盖这个空间。历史条件化策略虽然在训练集上能准确预测底层任务状态，但在部署遇到的分布外历史上却灾难性失败。现有的正则化、辅助目标、架构约束等方法只是修改了损失函数，并未触及“覆盖不足”这个根本问题——效果因任务而异、不稳定。The authors go on to analyze the root cause: the space of possible observation histories grows exponentially with the time horizon, and the existing teleoperation data collection paradigm (near-optimal demonstrations by human experts) simply cannot cover that space. History-conditioned policies can predict the underlying task state accurately on the training set, yet fail catastrophically on the out-of-distribution histories encountered at deployment. Existing remedies such as regularization, auxiliary objectives and architectural constraints merely modify the loss function and never touch the fundamental problem of insufficient coverage, so their effect varies from task to task and is unstable.
Figure 1. Big Picture Policies (BPP)：在一系列非马尔可夫任务上，BPP（左）用现成的 VLM 从历史中识别任务相关关键帧，相比无记忆策略和朴素历史条件化基线（右）表现更鲁棒。Figure 1. Big Picture Policies (BPP): on a range of non-Markovian tasks, BPP (left) uses an off-the-shelf VLM to identify task-relevant keyframes from the history, and behaves more robustly than memoryless policies and naive history-conditioning baselines (right).
+70%真实机器人平均成功率相对最强基线 PTP 的提升（"nearly 70% higher"）Improvement in average real-robot success rate over the strongest baseline PTP ("nearly 70% higher")
53.6%BPP 真机 4 任务平均成功率 vs. PTP 31.8% / 朴素历史 12.8%BPP average success rate over 4 real-robot tasks vs. PTP 31.8% / naive history 12.8%
−41%Drawer Search 上训练时间相对朴素历史条件化的降幅Reduction in training time on Drawer Search relative to naive history conditioning
4+34 个真实双臂操作任务 + 3 个仿真任务，均需历史条件化才能成功4 real-robot bimanual manipulation tasks + 3 simulation tasks, all of which need history conditioning to succeed
02 Method
BPP 的核心思路不是改学习算法或加正则项，而是改“历史的表征方式”：与其条件化于完整观测序列，不如条件化于一个极小的任务相关关键帧集合。以“往咖啡里加两勺糖”为例，机器人执行时可能多次没舀到糖，产生的观测历史和演示数据差异很大，即使任务状态其实没变；而关键帧只关心“糖成功掉进杯子”这一行为显著事件——细节各异的轨迹会坍缩成同一个任务相关表征，使策略训练/测试所面对的输入分布窄得多、覆盖也更充分。BPP's core idea is not to change the learning algorithm or add a regularizer, but to change "the way history is represented": rather than conditioning on the full observation sequence, condition on a minimal set of task-relevant keyframes. Take "adding two spoonfuls of sugar to the coffee": while executing, the robot may fail to scoop sugar many times, producing an observation history very different from the demonstration data even though the task state has not actually changed. A keyframe, by contrast, cares only about the behaviorally salient event "sugar successfully falls into the cup", so trajectories that differ in detail collapse into the same task-relevant representation, making the input distribution the policy faces at training and test time far narrower and far better covered.
Figure 6. BPP system architecture：在标准 diffusion transformer 策略架构基础上，额外条件化于一小组由 VLM 检测出的历史关键帧；同时对近期历史做延迟遮蔽（latency masking）以应对检测延迟。Figure 6. BPP system architecture: on top of a standard diffusion transformer policy architecture, it additionally conditions on a small set of history keyframes detected by a VLM, and applies latency masking to the recent history to cope with detection delay.
设 φ(o_t) ∈ {0,1} 为一个二值关键帧检测器（例如 VLM），标记观测 o_t 是否对应某个显著事件（如“按下按钮”“抽屉已打开且可见内部”）。为避免连续帧被重复标记，关键帧集合定义为检测信号的“上升沿”：K = {t : φ(o_t)=1 ∧ φ(o_{t-1})=0}，即只保留一次事件的首帧。策略最终形式为 π_θ(a_t | o_t, {o_k}_{k∈K_t})，只条件化于当前帧和迄今检测到的关键帧。Let φ(o_t) ∈ {0,1} be a binary keyframe detector (a VLM, for example) marking whether observation o_t corresponds to some salient event (such as "the button has been pressed" or "the drawer is open and its interior is visible"). To avoid marking consecutive frames repeatedly, the keyframe set is defined as the "rising edges" of the detection signal: K = {t : φ(o_t)=1 ∧ φ(o_{t-1})=0}, i.e. only the first frame of each event is kept. The policy therefore takes the form π_θ(a_t | o_t, {o_k}_{k∈K_t}), conditioning only on the current frame and the keyframes detected so far.
真实任务中用现成的 VLM（Gemini 3 Pro）作为二值分类器实现 φ，以 1 Hz 频率查询当前腕部相机图像和上一次查询图像，平均推理延迟 3–5 秒。为了让策略适应部署时的检测延迟，训练时引入延迟遮蔽：定义 K_t^Δ = {k ∈ K_t : k ≤ t − Δ}（训练时 Δ=3 秒），只用“在延迟约束下应当已经可见”的关键帧训练策略，推理时则直接使用所有已检测到的关键帧。由于处理的上下文帧数大幅减少，BPP 在 Drawer Search 上还将训练时间相较朴素历史条件化降低 41%。On the real tasks φ is implemented with an off-the-shelf VLM (Gemini 3 Pro) used as a binary classifier, queried at 1 Hz on the current wrist-camera image together with the image from the previous query, with an average inference latency of 3–5 seconds. To make the policy tolerate the detection delay at deployment, latency masking is introduced during training: define K_t^Δ = {k ∈ K_t : k ≤ t − Δ} (Δ=3 seconds during training), so the policy is trained only on the keyframes that "should already be visible under the latency constraint", while at inference time all keyframes detected so far are used directly. Because far fewer context frames have to be processed, BPP also cuts training time on Drawer Search by 41% relative to naive history conditioning.
03 Experiments
真机实验基于 ALOHA 2 双臂平台（4 路 RGB 相机 + 本体感知，50 Hz 控制），四个任务：Mug Replacement（换杯子）、Marshmallows（舀两把棉花糖后按按钮）、Drawer Search（在两组抽屉柜中找钥匙）、Stacking Puzzle（拆解并按颜色重新堆叠拼图块）。所有方法共用同一 Diffusion Transformer 架构与 DDPM 训练目标（action chunk = 50），对比 Current Observation、Naïve History、Past-Token Prediction (PTP)（论文复现版本，全部时间步共享梯度训练图像编码器）和 BPP。The real-robot experiments run on the ALOHA 2 bimanual platform (4 RGB cameras + proprioception, 50 Hz control) with four tasks: Mug Replacement, Marshmallows (scoop two servings of marshmallows, then press the button), Drawer Search (find the keys among two sets of drawer cabinets) and Stacking Puzzle (take the puzzle pieces apart and restack them by color). All methods share the same Diffusion Transformer architecture and DDPM training objective (action chunk = 50), comparing Current Observation, Naïve History, Past-Token Prediction (PTP) (the paper's reimplementation, training the image encoder with gradients shared across all timesteps) and BPP.
Task
Current Obs
Naïve History
PTP
BPP (Ours)
Drawer Search
11.1%
0.0%
0.0%
33.3%
Marshmallows
40.0%
25.0%
35.0%
65.0%
Mug Replacement
0.0%
5.0%
40.0%
60.0%
Stacking Puzzle
6.5%
21.0%
52.0%
56.0%
Average
14.4%
12.8%
31.8%
53.6%
论文原话总结：“BPP significantly outperforms all baselines, achieving nearly 70% higher performance than the best prior approach (PTP) on average.” 值得注意的是，朴素历史条件化在两个真实任务上甚至比 Current Observation 更差——作者归因于真实数据中还存在背景变化、传感器/执行器噪声等额外的伪相关来源。In the paper’s own words: "BPP significantly outperforms all baselines, achieving nearly 70% higher performance than the best prior approach (PTP) on average." Notably, naive history conditioning is even worse than Current Observation on two of the real tasks; the authors attribute this to additional sources of spurious correlation in real data, such as background changes and sensor/actuator noise.
Figure 7. Simulation results：BPP 在所有非 oracle 方法中表现最好，在 Variable-Password 任务上甚至超过了拥有特权状态信息的 Oracle 基线，说明关键帧表征可能比 one-hot 状态编码更利于学习。Figure 7. Simulation results: BPP performs best among all non-oracle methods, and on the Variable-Password task it even surpasses the Oracle baseline that has privileged state information, suggesting that a keyframe representation may be easier to learn from than a one-hot state encoding.Figure 8. Drawer Search 上的定性行为：BPP 系统性地依次搜索各个抽屉、不重复翻找，展现出稳健的长时程进度追踪能力（相较之下 PTP 在抓取失败或空抽屉后容易陷入原地打转的循环，见 Figure 9）。Figure 8. Qualitative behavior on Drawer Search: BPP systematically searches the drawers one after another without going back over the same ones, showing robust long-horizon progress tracking (by contrast, PTP easily falls into a loop of spinning in place after a failed grasp or an empty drawer, see Figure 9).
Ablations
动作分块（action chunking）与架构：更长的预测动作块（chunk 50 vs. 10）能显著抑制朴素历史条件化对伪相关的依赖（history-state 分类误差增幅 2.9× vs. 7.2×），联合训练（而非冻结）图像编码器同样重要；但即便如此，历史条件化策略在自身 rollout 上的“历史理解”仍会显著退化，说明这不是架构问题。Action chunking and architecture: longer predicted action chunks (chunk 50 vs. 10) markedly suppress naive history conditioning’s reliance on spurious correlations (history-state classification error increase of 2.9× vs. 7.2×), and jointly training (rather than freezing) the image encoder matters just as much; even so, the "history understanding" of a history-conditioned policy still degrades markedly on its own rollouts, showing that this is not an architectural problem.
“黄金”正则化实验（反直觉结果）：用 ground-truth 历史状态（如已按按钮数）直接正则化视觉编码器，虽提升了同分布验证集上的准确率，却让 out-of-distribution rollout 上的状态预测准确率从 86% 降到 18%，成功率从 55.5%±3.3% 降到 19.0%±3.3%。这证明瓶颈是覆盖（coverage）而非特征学习目标或架构。The "golden" regularization experiment (a counter-intuitive result): directly regularizing the vision encoder with ground-truth history state (such as the number of button presses so far) does improve accuracy on the in-distribution validation set, yet it drops state-prediction accuracy on out-of-distribution rollouts from 86% to 18% and the success rate from 55.5%±3.3% to 19.0%±3.3%. This proves that the bottleneck is coverage rather than the feature-learning objective or the architecture.
数据效率（Mug Replacement）：朴素历史条件化的表现随演示量呈非单调变化——数据少时因暴露更多失败模式反而更差，但数据足够多时能追平 BPP；BPP 用远更少的数据就能达到高性能。Data efficiency (Mug Replacement): the performance of naive history conditioning varies non-monotonically with the number of demonstrations: with little data it is worse because more failure modes are exposed, but with enough data it catches up with BPP, whereas BPP reaches high performance with far less data.
VLM 关键帧 vs. Oracle 关键帧：用首帧作为额外上下文构造的 oracle 关键帧基线达到 70% 成功率，VLM 检测的关键帧为 60%，"indicating that keyframe recognition errors explain only a small portion of the performance gap"——VLM 检测误差和延迟仍显著优于 Naïve History（5%）与 PTP（40%）。VLM keyframes vs. oracle keyframes: an oracle keyframe baseline built by using the first frame as additional context reaches 70% success, while VLM-detected keyframes reach 60%, "indicating that keyframe recognition errors explain only a small portion of the performance gap"; even with VLM detection errors and latency this remains markedly better than Naïve History (5%) and PTP (40%).
04 Limitations
Note:以下均为论文 Discussion and Conclusion 部分作者明确陈述（stated）的局限性。All of the following are limitations explicitly stated by the authors in the paper's Discussion and Conclusion section.
依赖外部 VLM 的推理速度Dependence on the inference speed of an external VLM
“BPP relies on the inference speed of an external VLM; while we mitigate this during training via latency masking, real-time deployment remains constrained by VLM query latency, which can be critical for highly dynamic tasks (e.g., parts of Mug Replacement).” 训练时的延迟遮蔽只是缓解，部署时仍受 VLM 查询延迟（3–5 秒/次）约束，对高动态子任务影响较大。"BPP relies on the inference speed of an external VLM; while we mitigate this during training via latency masking, real-time deployment remains constrained by VLM query latency, which can be critical for highly dynamic tasks (e.g., parts of Mug Replacement)." Latency masking during training only mitigates the problem; at deployment the system is still constrained by VLM query latency (3–5 seconds per query), which weighs heavily on highly dynamic subtasks.
依赖关键帧检测的准确性Dependence on the accuracy of keyframe detection
“Although our ablations show robustness to some errors, systematic false positives, such as misclassifying an empty grasp, can trigger premature state transitions.” 例如在 Marshmallows 任务中把空手抓取误判为成功舀取，会触发过早的完成按钮判断。"Although our ablations show robustness to some errors, systematic false positives, such as misclassifying an empty grasp, can trigger premature state transitions." In the Marshmallows task, for instance, mistaking an empty grasp for a successful scoop triggers a premature decision that the task is done and the button should be pressed.
关键帧定义仍需人工设计Keyframe definitions still have to be designed by hand
论文承认目前 BPP 依赖任务特定的 VLM prompt 来定义关键帧（“BPP currently relies on task-specific VLM prompting”），作者将“用 LLM 自动生成关键帧定义”列为未来方向，而非本文已解决的问题。The paper concedes that BPP currently depends on task-specific VLM prompts to define keyframes ("BPP currently relies on task-specific VLM prompting"); the authors list "automatically generating keyframe definitions with an LLM" as a future direction rather than a problem solved here.
关键帧粒度对某些任务可能不够Keyframe granularity may be too coarse for some tasks
论文在 Future Directions 中指出（inferred 延伸讨论）：单一时间点的关键帧可能不足以捕捉某些事件（如一次失败抓取为何失败），并提出将关键帧推广为“关键片段”（key segments）作为后续工作方向，说明当前方法在此类场景下的表达力有限。In its Future Directions the paper notes (an inferred extended discussion) that a keyframe at a single point in time may not be enough to capture certain events (such as why a given grasp failed), and proposes generalizing keyframes into "key segments" as a direction for follow-up work, which shows that the current method has limited expressiveness in such scenarios.
