FAST：面向视觉-语言-动作模型的高效动作分词

01 动机Motivation

现有 VLA 模型（如 OpenVLA）使用朴素的 per-dimension per-timestep binning 对连续动作离散化。当机器人控制频率升高（例如折叠衣物需要 50 Hz）或任务需要精细灵巧操作时，这种方案产生数百个高度相关 token，使得自回归 next-token prediction 形同虚设——模型退化为简单地复制最近一个动作 token。Existing VLA models (such as OpenVLA) discretize continuous actions with naive per-dimension per-timestep binning. As robot control frequency rises (folding clothes, for instance, requires 50 Hz) or as tasks demand fine-grained dexterous manipulation, this scheme produces hundreds of highly correlated tokens, rendering autoregressive next-token prediction useless — the model degenerates into simply copying the most recent action token.

"Highly correlated action tokens diminish the effectiveness of the next token prediction objective used in autoregressive VLAs."

FAST 对比朴素 binning 的效果对比 — **Figure 2（原文）**：FAST 分词（右）与朴素 binning 分词（左）的对比。在高频灵巧任务上，binning 方案完全失效，而 FAST 能持续取得良好成果。FAST "outperforms popular binning tokenization schemes"。**Figure 2 (original paper)**: FAST tokenization (right) compared with naive binning tokenization (left). On high-frequency dexterous tasks binning fails completely, whereas FAST keeps achieving good results. FAST "outperforms popular binning tokenization schemes".

13.2×Shirt Fold (50 Hz) 上的 token 压缩率Token compression ratio on Shirt Fold (50 Hz)

5×相对扩散基线训练提速倍数Training speedup over the diffusion baseline

1MFAST+ 预训练所用真实机器人轨迹数Real robot trajectories used to pretrain FAST+

5 Hz→50 HzFAST 适用的控制频率范围Control frequency range FAST applies to

案例研究：采样率对 binning 的致命影响Case study: the fatal effect of sampling rate on binning

采样率与分词策略的关系 — **Figure 1（原文）**：训练收敛曲线对比。FAST 使基于自回归 Transformer 的灵巧机器人控制成为可能——"FAST enables training of autoregressive Transformers for dexterous robot control via simple next token prediction"，且训练速度比扩散 VLA 快 5 倍。**Figure 1 (original paper)**: comparison of training convergence curves. FAST makes dexterous robot control with autoregressive Transformers possible — "FAST enables training of autoregressive Transformers for dexterous robot control via simple next token prediction" — and trains 5 times faster than diffusion VLAs.

研究者在一个简单的 table-top manipulation 任务上系统研究了采样率对不同分词策略的影响。随着控制频率升高，使用 binning 分词训练的策略"produce increasingly poor predictions as we increase the sampling frequency"，而基于 DCT 的 FAST 在各频率下均保持稳定表现。The authors systematically study the effect of sampling rate on different tokenization strategies in a simple table-top manipulation task. As control frequency rises, policies trained with binning tokenization "produce increasingly poor predictions as we increase the sampling frequency", while DCT-based FAST stays stable across all frequencies.

02 方法Method

FAST 的核心思想是将动作序列视为信号而非独立离散点——先用 DCT 转换到频域分离低频与高频信息，再用 BPE 压缩稀疏系数，得到少量、低相关性、语义丰富的 token。整个流程完全不需要神经网络，可离线预处理，也可通用化迁移到新机器人。The core idea of FAST is to treat an action sequence as a signal rather than a set of independent discrete points: a DCT first maps it into the frequency domain to separate low- from high-frequency information, then BPE compresses the sparse coefficients, yielding few, weakly correlated, semantically rich tokens. The whole pipeline needs no neural network at all, can be run offline as preprocessing, and can be transferred to new robots as a universal tokenizer.

**Figure 4（原文）**：FAST 分词流水线全览。输入动作序列经过三步处理：(1) 基于分位数的归一化至 [−1, 1]；(2) 对每个动作维度独立施加 DCT，转换到频域；(3) 对 DCT 系数量化并按频率优先顺序排列后，用 BPE 进行无损压缩。**Figure 4 (original paper)**: overview of the FAST tokenization pipeline. The input action sequence goes through three steps: (1) quantile-based normalization to [−1, 1]; (2) a DCT applied independently to each action dimension, mapping it into the frequency domain; (3) quantization of the DCT coefficients, which are ordered frequency-first and then losslessly compressed with BPE.

步骤 1：归一化Step 1: normalization

使用 1st 和 99th 百分位数对动作进行分位数归一化，映射到 [−1, 1]，有效处理大规模数据集中的异常值，保证 DCT 系数的数值稳定性。Actions are quantile-normalized with the 1st and 99th percentiles and mapped to [−1, 1], which handles outliers in large-scale datasets effectively and keeps the DCT coefficients numerically stable.

步骤 2：离散余弦变换（DCT）Step 2: discrete cosine transform (DCT)

对每个动作维度独立做 DCT，将时域动作序列转换到频域。低频系数捕捉运动的整体形状，高频系数捕捉细节变化。自回归解码时先预测低频成分，"leads to more stable policy rollouts"。A DCT is applied independently to each action dimension, converting the time-domain action sequence into the frequency domain. Low-frequency coefficients capture the overall shape of the motion, high-frequency coefficients capture fine detail. During autoregressive decoding the low-frequency components are predicted first, which "leads to more stable policy rollouts".

步骤 3：量化与 BPE 压缩Step 3: quantization and BPE compression

用比例参数 γ 对 DCT 系数取整，再按"频率优先"顺序展开成一维序列。高频系数通常接近零，展开后形成大量重复的零值串，BPE 算法可以高效压缩这些稀疏模式，大幅减少 token 数量。The DCT coefficients are rounded with a scale parameter γ and then flattened into a one-dimensional sequence in "frequency-first" order. High-frequency coefficients are usually close to zero, so flattening produces long repeated runs of zeros; the BPE algorithm compresses these sparse patterns efficiently and sharply reduces the token count.

FAST+ 通用分词器FAST+ universal tokenizer

在约 100 万条来自多种机器人（单臂、双臂、移动操作臂）的真实 1 秒动作序列上离线训练 BPE 词表，得到通用分词器 FAST+。在未见机器人形态和控制频率上可实现 "2× reduction across all datasets"。The BPE vocabulary is trained offline on roughly one million real 1-second action sequences from a variety of robots (single-arm, bi-manual, mobile manipulators), giving the universal tokenizer FAST+. On unseen robot morphologies and control frequencies it still achieves "2× reduction across all datasets".

**Figure 5（原文）**：七个评测环境，涵盖桌面收拾（Table Bussing, 20 Hz）、T 恤折叠（Shirt Folding, 50 Hz）、洗衣折叠（Laundry Folding）、杂货装袋（Grocery Bagging）、烤面包（Toast）等多样化真实机器人任务。**Figure 5 (original paper)**: seven evaluation environments covering diverse real-robot tasks such as Table Bussing (20 Hz), Shirt Folding (50 Hz), Laundry Folding, Grocery Bagging and Toast.

与 π₀ 的集成：π₀-FASTIntegration with π₀: π₀-FAST

将 FAST 分词器接入物理智能公司的 π₀ VLA 模型（基于预训练视觉-语言模型的自回归策略），替换原有 binning 分词，无需改变模型架构。π₀-FAST 在大规模数据集上训练时收敛显著更快，且能处理此前扩散版本才能胜任的高频灵巧任务。The FAST tokenizer is plugged into Physical Intelligence's π₀ VLA model (an autoregressive policy built on a pretrained vision-language model), replacing the original binning tokenizer without any change to the model architecture. π₀-FAST converges markedly faster when trained on large-scale datasets and handles high-frequency dexterous tasks that previously only the diffusion variant could manage.

03 实验Experiments

实验涵盖：(1) token 压缩率对比；(2) 在多个真实机器人任务上与朴素 binning 和扩散基线的性能对比；(3) FAST+ 通用分词器验证；(4) 大规模泛化策略训练；(5) OpenVLA 主干上的消融实验。评测在 Physical Intelligence 实验室真实机器人硬件上进行。The experiments cover: (1) token compression ratios; (2) performance against naive binning and diffusion baselines on several real-robot tasks; (3) validation of the FAST+ universal tokenizer; (4) training of large-scale generalist policies; (5) ablations on an OpenVLA backbone. All evaluations are run on real robot hardware in the Physical Intelligence lab.

Token 压缩率（Table I 原文数据）Token compression ratio (data from Table I of the paper)

数据集Dataset	控制频率Control frequency	朴素 token 数Naive token count	FAST token 数FAST token count	压缩比Compression ratio
BridgeV2	5 Hz	35	20	1.75×
DROID	15 Hz	105	29	3.6×
Table Bussing	20 Hz	140	28	5.0×
Shirt Folding	50 Hz	700	53	13.2×

策略性能对比Policy performance comparison

不同分词方案下的策略性能 — **Figure 6（原文）**：在全部评测任务上比较三种方案——朴素 binning、FSQ（学习型向量量化基线）和 FAST。朴素 binning 在 Table Bussing（20 Hz）和 T-Shirt Folding（50 Hz）上**完全失效**；FAST 在所有任务上均取得最强表现，且特别在灵巧真实机器人任务上超越 FSQ。FAST+ 与数据集专用 FAST 性能相当，验证了通用性。**Figure 6 (original paper)**: three schemes compared across all evaluation tasks — naive binning, FSQ (a learned vector-quantization baseline) and FAST. Naive binning **fails completely** on Table Bussing (20 Hz) and T-Shirt Folding (50 Hz); FAST achieves the strongest performance on every task and beats FSQ especially on dexterous real-robot tasks. FAST+ performs on par with dataset-specific FAST, confirming its universality.

与扩散 π₀ 对比（训练效率）Comparison with diffusion π₀ (training efficiency)

FAST vs 扩散 π₀ 对比 — **Figure 9（原文）**：在小数据集上，FAST 与扩散 π₀ 性能相当；在大数据集（Table Bussing）上，"FAST converges faster"，约快 3 倍训练步数。两种方案最终性能持平，但 FAST 所需 GPU 计算量大幅更少。**Figure 9 (original paper)**: on small datasets FAST performs on par with diffusion π₀; on a large dataset (Table Bussing) "FAST converges faster", taking about 3 times fewer training steps. The two schemes reach the same final performance, but FAST requires far less GPU compute.

DROID 零样本泛化Zero-shot generalization on DROID

DROID 策略跨环境泛化 — **Figure 7（原文）**：首次成功在 DROID 数据集上训练可在完全未见环境中**零样本**执行的 VLA 策略。实验跨三所大学校园进行桌面操作测试，证明 FAST 使语言条件泛化策略在此前以 binning 为分词时无法实现的高频任务上成为可能。**Figure 7 (original paper)**: the first successful training on the DROID dataset of a VLA policy that executes **zero-shot** in entirely unseen environments. Table-top manipulation was tested across three university campuses, showing that FAST makes language-conditioned generalist policies possible on high-frequency tasks that were unattainable when binning was used for tokenization.

大规模泛化策略（π₀-FAST）Large-scale generalist policy (π₀-FAST)

在 10,000 小时多样化机器人数据上训练的 π₀-FAST 与扩散版 π₀ 在洗衣折叠、T 恤折叠、杂货装袋、烤面包任务上性能相当，即"matches the performance of diffusion π₀"，同时训练 GPU 时间仅为扩散基线的 1/5（"5x fewer GPU hours for training than the π₀ model"）。Trained on 10,000 hours of diverse robot data, π₀-FAST performs on par with diffusion π₀ on laundry folding, shirt folding, grocery bagging and toasting, that is, it "matches the performance of diffusion π₀", while requiring only 1/5 of the training GPU time of the diffusion baseline ("5x fewer GPU hours for training than the π₀ model").

消融实验Ablations

主干无关性：在 OpenVLA 上替换分词器后，FAST 同样显著提升高频任务性能，证明方法不依赖特定 VLA 架构。Backbone-agnostic: swapping in the tokenizer on OpenVLA likewise improves high-frequency task performance markedly, proving that the method does not depend on a particular VLA architecture.
BPE 压缩的必要性：仅用 DCT 而不加 BPE 时，性能大幅下降——原因在于 DCT 产生的大量零值 token 未经压缩会导致解码步数爆炸，推理 token 数增加数量级。Necessity of BPE compression: using DCT alone without BPE degrades performance sharply, because the many zero-valued tokens produced by the DCT, left uncompressed, blow up the number of decoding steps and raise the inference token count by an order of magnitude.

04 局限性Limitations

说明：第一条限制由论文作者在 Section VI-E 明确陈述；其余各点为根据设计与实验范围的推断（inferred）。Note: the first limitation is stated explicitly by the authors in Section VI-E; the remaining points are inferred from the design and the scope of the experiments.

推断 (inferred) 推理速度慢——自回归解码延迟Inferred: slow inference — autoregressive decoding latency

论文 Section VI-E 明确指出："One current limitation of the autoregressive VLA is its inference speed...the π₀ model with FAST tokenization needs approximately 750ms of inference time per chunk, since it must perform more autoregressive decoding steps (typically 30-60 action tokens...vs. 10 diffusion steps)."（750 ms vs. 扩散版的 100 ms on NVIDIA 4090 GPU，约慢 7.5 倍）。这使得实时、高速机器人控制场景目前难以满足要求，实验中也不得不降低评测吞吐量。Section VI-E of the paper states explicitly: "One current limitation of the autoregressive VLA is its inference speed...the π₀ model with FAST tokenization needs approximately 750ms of inference time per chunk, since it must perform more autoregressive decoding steps (typically 30-60 action tokens...vs. 10 diffusion steps)." (750 ms vs. 100 ms for the diffusion version on an NVIDIA 4090 GPU, roughly 7.5 times slower). This makes real-time, high-speed robot control hard to serve for now, and the experiments also had to lower evaluation throughput.

推断 (inferred) 有损压缩权衡——精度与压缩率的 trade-offInferred: lossy compression trade-off — precision versus compression ratio

DCT 量化步骤受比例参数 γ 控制，量化并非完全无损——较高的压缩率会牺牲动作精度。具体在精密操作任务中此权衡的影响程度，论文未做系统分析。The DCT quantization step is governed by the scale parameter γ, and quantization is not entirely lossless: a higher compression ratio sacrifices action precision. How much this trade-off matters on precision manipulation tasks is not analyzed systematically in the paper.

推断 (inferred) 评测任务范围受限——以静态桌面操作为主Inferred: limited scope of evaluation tasks — mostly static table-top manipulation

所有在线真实机器人评测均集中于固定基座的桌面操作任务；移动机器人、灵巧手、类人机器人等形态仅做了离线数据分析，未在实机上验证策略部署效果。All online real-robot evaluations concentrate on table-top manipulation with a fixed base; mobile robots, dexterous hands, humanoids and other morphologies are only analyzed offline on data, with no policy deployment verified on real hardware.

推断 (inferred) VLA 架构选择尚不明朗Inferred: the choice of VLA architecture is still unsettled

论文自述"the jury on the best VLA architecture is still out"——自回归 vs. 扩散架构各有优劣，FAST 主要解决了分词瓶颈，但更深层的架构设计问题（如何最优融合视觉、语言、动作模态）仍属开放问题。The paper itself notes that "the jury on the best VLA architecture is still out" — autoregressive and diffusion architectures each have strengths and weaknesses; FAST mainly resolves the tokenization bottleneck, while the deeper architectural design question (how best to fuse the visual, language and action modalities) remains open.