This note takes the chain "continuous action → discrete token → continuous action" and unpacks the background knowledge each link requires. The main thread is the π₀-FAST style discrete autoregressive route^[1], held up against FSQ, Binning, and the continuous alternative, flow matching^[2].

Three labels appear throughout:

• DIRECT — pure mathematical transform (deterministic, no training needed)
• LEARNED — fitted from data (statistical fit or neural network)
• LOSSY — information is lost

§1The big picture: what problem is actually being solved

What a robot policy has to do: given an observation $o$ (images + proprioceptive state), output a stretch of future actions $a_{1:T}$. Actions are continuous real-valued vectors (joint angles, end-effector pose, gripper opening).

There are two completely different routes to modelling $p(a_{1:T}\mid o)$:

This note focuses on the second route. The core magic: "translate" continuous actions into a string of integer tokens, and robot control becomes a language modelling problem — the whole LLM machinery (Transformer, cross-entropy, sampling) can be reused as is.

Concept dependency graph:

robot action (sequence of continuous vectors)
   │  why not regress it directly? → the mode averaging problem (§3.1)
   ▼
motivation for discretization ─┐
   │                           │
   ├─ FAST route:              ├─ signal processing: DCT (§5) compacts energy
   │   DCT→quantize→BPE        ├─ quantization/rounding (§6) the one lossy step
   │                           └─ BPE (§7) lossless compression vocabulary
   │
   ├─ FSQ route: neural autoencoder + VQ/FSQ codebook (§9, §10)
   │
   └─ Binning route: uniform bucketing (§11, the most naive baseline)
                ▼
        all of them feed an autoregressive Transformer (§12)
        trained with cross-entropy → naturally multimodal (§13)
        sampling + temperature (§13.1) pulls out different solutions
        attention masking (§14) separates prompt from action

§2Prerequisites: the core concept triples

A few concepts recur throughout the note; here they are as English term = Chinese term = minimal definition. The Chinese names are kept as a bridge back to the Chinese literature, and the wording in the body matches these entries.

• tokenization = 词元化 = mapping text or a continuous signal onto a sequence of integer IDs drawn from a finite vocabulary, so a Transformer can treat them as categories^[3].
• mode averaging = 模式平均 = the phenomenon where L2/MSE regression averages several mutually exclusive correct solutions into one wrong solution; it is the central motivation for this entire discretization route (§3.1).
• DCT (discrete cosine transform) = 离散余弦变换 = an orthogonal transform that invertibly decomposes a discrete signal into the weights (coefficients) of a set of cosine bases at different frequencies; for a smooth signal the energy concentrates in the low-frequency coefficients^[4].
• quantization = 量化 = the operation of mapping continuous values onto a finite set of grid points (e.g. multiply by a scale, then round)^[5]; it is the only lossy step in the "continuous → discrete" chain (§6).
• BPE (byte pair encoding) = 字节对编码 = a lossless compression algorithm that repeatedly merges the most frequent adjacent pair in a sequence to shorten it; the merge rules come from data statistics and involve no neural network^[3].
• prefix-LM = 前缀语言模型 = a mixed-mask Transformer in which the prefix (prompt/condition) uses bidirectional attention while the suffix (generation target) uses causal attention^[6].

§3What a robot action is (the object being tokenized)

One action sample is a two-dimensional matrix:

actions.shape == (T, D)
            ┌─ T = action_horizon: how many future steps one prediction covers (e.g. 50 steps ≈ 1 second @ 50Hz)
            └─ D = action_dim: degrees of freedom per step (e.g. 7 = 6 joints + 1 gripper, or 14 = dual arm)

Two key properties:

• ① Smooth in time: consecutive actions differ little (robots do not teleport) → this is the precondition that lets DCT compress.
• ② Different units per dimension: joint angles may span ±3 rad, the gripper 0~1 → they must first be normalized onto a common range.

Normalization: the first preprocessing step LEARNED (statistical quantiles, approximately lossless)

Scale every action dimension to roughly $[-1,1]$. The usual choice is quantile normalization rather than plain min-max: take the 1% and 99% quantiles as the bounds, which is more robust to outliers.

# Idea: use quantiles computed on the training set to squeeze actions into [-1,1]
a_norm = 2 * (a - q01) / (q99 - q01) - 1   # anything outside gets clipped

§4Why discretize continuous actions at all

The most natural idea is to have the network output real-valued actions directly, supervised by L2/MSE regression. So why go to the trouble of discretizing into tokens? Three reasons:

1. Reuse the entire LLM infrastructure: once actions are tokens, you can plug straight into a pretrained VLM's (e.g. PaliGemma) Transformer, vocabulary and cross-entropy, and skip designing a new output head.
2. Multimodality comes for free (detailed in §13): a discrete distribution can assign high probability to "go left" and "go right" simultaneously; regression cannot.
3. Compression + efficient autoregression: FAST uses DCT+BPE to squeeze a stretch of actions into a very short token sequence, making autoregressive inference faster.

The core motivation: the mode averaging disaster

This is the single most important intuition for understanding the whole discretization route. Picture an obstacle in front of the robot with two equally correct ways around it: go around the left or go around the right. Both appear in the training data.

The wider the gap, the deeper the L2 "optimal prediction" (the mean of the two clusters) sinks into no-man's-land — an action nobody ever demonstrated, one that may drive straight into a wall. A discrete distribution, by contrast, can light up both the left and the right bin with high probability, preserving both modes perfectly. That is why continuous actions get sliced into discrete tokens and trained with classification cross-entropy.

§5The tokenization concept (borrowed from NLP)

In NLP, tokenization = cutting a continuous stream of characters into integer IDs from a finite vocabulary:

"the robot is cute"  →  ["the","robot","is","cute"]  →  [8123, 442, 19, 5601]
       text                    subwords                    integer token ids

These integers then index an embedding table and become vectors fed into the Transformer. The key points:

• The vocabulary is finite (PaliGemma, for instance, has 257,152 tokens^[6]).
• Each token id corresponds to one row of embedding vector in the vocabulary.
• The model's output is a probability distribution over the vocabulary (softmax over 257,152 classes).

§6DCT, the discrete cosine transform DIRECT

This is FAST's first step, and the piece most people are least familiar with.

Intuition: decompose a curve into a sum of cosine waves at different frequencies

Any discrete signal (say the 50 samples of one joint angle over time) can be represented exactly as a weighted sum of cosine waves at various frequencies. DCT is precisely the computation of "how much weight each frequency carries"^[4].

The formula (DCT-II; knowing what it looks like is enough)

Turning a length-$N$ signal $x_0,\dots,x_{N-1}$ into coefficients $X_0,\dots,X_{N-1}$:

$k=0$ is the DC component (the overall mean); larger $k$ means higher frequency. The inverse transform IDCT adds the coefficients back up using the same cosine bases and is fully invertible (up to floating-point error). The DCT is taken along the time axis T, independently for each action dimension D.

§7Quantization / rounding DIRECT LOSSY

This is the only step in the whole FAST chain that genuinely loses information, and the place where "continuous → discrete" actually happens.

A continuous value can take infinitely many values; a token vocabulary can only express finitely many. Quantization = mapping a continuous value onto the nearest "grid point"^[5]. In its simplest form: multiply by a scaling factor, then round.

q = round(x * scale)        # continuous x → integer q   (encode, lossy)
x_hat = q / scale           # integer q → approximate continuous x̂ (decode, the original value is gone)

The error $|x-\hat x|$ is at most about $\frac{1}{2\,\text{scale}}$. A larger scale → a denser grid → more precision, but a wider integer range and more tokens. This is the precision vs compression ratio trade-off.

§8BPE, byte pair encoding LEARNED (statistical fit, not an NN, lossless)

Rounding yields a long string of integers (with plenty of repetition, especially 0s). BPE is a lossless compression algorithm^[3] that merges "frequently repeated patterns" into a single new token, shortening the sequence.

Repeat the following: "find the most frequent adjacent pair in the current sequence and merge it into a new symbol", until the target vocabulary size is reached.

initial: a a b a a b a a b       (the pair a a occurs often)
merge (a,a)→Z:  Z b Z b Z b      (now the pair Z b occurs often)
merge (Z,b)→Y:  Y Y Y             ← 9 symbols squeezed into 3, fully recoverable

"Which pairs to merge, and in what order" = the BPE vocabulary / merge rules, which are derived from statistics over the training data (hence learned), but are only lookup-style statistical rules, containing no neural network or gradients whatsoever. Given the vocabulary, both encoding and decoding are entirely lossless.

§9Putting FAST together (the panorama)

The encoding direction (continuous → token):

normalized action matrix A (T, D), roughly [-1,1]
   │
   ① DCT (along the time axis)     DIRECT lossless, energy pushed into low frequencies
   ▼
frequency coefficient matrix C (T, D), high frequencies ≈0
   │
   ② Scale + Round                 DIRECT · LOSSY ← discretization happens here
   ▼
sparse integer matrix (mostly 0)
   │
   ③ Flatten to 1D                 DIRECT
   │
   ④ BPE merges repeated patterns  LEARNED vocabulary · lossless
   ▼
final discrete token sequence (variable length, usually far shorter than T×D)
   │
   ⑤ map onto the last 128 slots of the PaliGemma vocabulary   DIRECT pure arithmetic

The decoding direction is simply the exact reverse: BPE decode → reshape → divide by the scale → IDCT → denormalize.

A complete numerical example: every step's shape and data made visible

Many people have no concrete feel for "what shape actually goes into and comes out of the DCT, and how scale+round plays out". Below is a walk-through with genuinely computed numbers (for clarity we take D=1, a single dimension, and T=8 time steps; with more dimensions each column independently repeats the same process).

Input: the normalized action matrix A, of shape (T, D) = (8, 1), values already normalized to $[-1,1]$:

A = [ 1.000, 0.816, 0.494, 0.111, -0.258, -0.555, -0.755, -0.853 ]   # shape (8,1)
     t=0    t=1    t=2    t=3    t=4     t=5     t=6     t=7         ← time axis
This is a smoothly descending curve (what a robot action typically looks like).

① DCT — transform along the time axis; input and output shapes are identical ((T,)→(T,), and for (T,D) it is one transform per column):

C = DCT(A) = [ 0.00, 1.889, 0.159, -0.00, -0.00, 0.00, 0.00, -0.00 ]   # still (8,1)
              k=0   k=1    k=2    k=3    k=4   k=5   k=6   k=7         ← frequency axis
              DC    low ────────────→                     high
Nearly all the energy sits at k=1 (=1.889)! The high-frequency coefficients from k≥3 are already ≈0.

② Scale + Round — discretization happens here (with scale = 8):

C × scale = [ 0.00, 15.115, 1.273, -0.00, -0.00, 0.00, 0.00, -0.00 ]
round(·)  = [   0,    15,     1,     0,     0,    0,    0,    0    ]   = Cq (integers!)
                                                                       only 2 of the 8 are non-zero

This step: ① turns continuous decimals into integers (the real "discretization"); ② crushes those ≈0 high-frequency coefficients into exactly 0. The loss originates here (the rounding of 15.115→15 and 1.273→1).

③ Flatten flattens (T,D) into the 1D integer sequence [0, 15, 1, 0, 0, 0, 0, 0] (for D>1 it is straightened out along T×D).

④ BPE merges frequently repeated patterns (such as "5 consecutive 0s") into single tokens, shortening the sequence, and finally ⑤ maps them onto the ids of the top 128 slots of the vocabulary.

Decoding: retrace the path and see how small the loss is

token → BPE decode → [0,15,1,0,0,0,0,0] → reshape(8,1) → ÷scale → IDCT → denormalize
reconstructed A_rec = [0.977, 0.803, 0.497, 0.125, -0.241, -0.545, -0.756, -0.862]
original A          = [1.000, 0.816, 0.494, 0.111, -0.258, -0.555, -0.755, -0.853]
reconstruction MSE ≈ 1.7e-4   ← 8 values recovered almost perfectly from just 2 non-zero coefficients!

§10VQ-VAE and codebooks (the prerequisite for understanding FSQ) LEARNED (a genuine neural network)

The FSQ route is a genuine neural network approach. To understand FSQ, first understand the "codebook" idea of its predecessor, VQ-VAE.

The idea of VQ-VAE (Vector Quantized VAE)^[7]: maintain a learnable "codebook" — say 256 vectors, each called a codeword, numbered 0~255.

encoder (NN) → continuous vector z
         ↓ find the nearest codeword in the codebook
index of the nearest codeword = token (e.g. #42)      ← discretization
         ↓
decoder (NN) reconstructs the action from the codeword vector of #42

The encoder, the decoder and the 256 vectors of the codebook themselves are all obtained by gradient training (with a reconstruction MSE loss). This is a genuine "end-to-end learned discrete representation".

The straight-through estimator

"Nearest neighbour / rounding" is a staircase function whose derivative is 0 everywhere, so gradients never make it back to the encoder. The trick: use the discrete value in the forward pass, and pretend it is the identity function in the backward pass (let the gradient pass straight through)^[8].

z_q = z + stop_gradient(quantize(z) - z)
#   forward:  z_q == quantize(z) (discrete)
#   backward: d z_q / d z == 1   (the gradient flows straight through as if nothing were quantized)

§11FSQ, finite scalar quantization LEARNED

FSQ (Finite Scalar Quantization)^[9] is the elegant replacement for VQ-VAE: drop the learnable codebook and instead round onto a fixed grid, independently in each of a handful of dimensions. It needs no codebook learning, yet is inherently collapse-free.

The core trick:

1. The encoder projects the action into a very low-dimensional space (say only 3 dimensions).
2. Each dimension is squashed to $[-1,1]$ with $\tanh$, then rounded to the few fixed grid points of that dimension (e.g. 8 grid points for dimension 1, 6 for dimension 2, 5 for dimension 3).
3. The combination of grid points across dimensions is encoded into a single integer token using mixed-radix arithmetic ($8\times6\times5=240\approx256$ codewords).

# FSQ encode (conceptual)
x = proj_down(z)                      # project down to ~3 dims        [LEARNED]
zc = tanh(x)                          # squash to [-1,1]
digits = round((zc+1)*(bases-1)/2)    # round each dim to a grid point [DIRECT, LOSSY]
token = undigitize(digits)            # multi-dim digits → one integer (mixed radix) [DIRECT]

§12Binning, uniform bucketing (the most naive baseline) DIRECT

The RT-2^[10] / OpenVLA^[11] style, with zero learning: for every dimension and every time step independently, cut $[-1,1]$ into 256 uniform buckets and round to the bucket index.

token = round((a+1)/2 * n_bins)     # encode
a_hat = token / n_bins * 2 - 1      # decode

• Upside: simple, needs no training at all, fully invertible (up to rounding).
• Downside: the token count = T×D (no compression, very long sequences); precision is capped at 256 levels and it exploits no temporal correlation. It is the control baseline in the papers.

Seeing all three side by side:

§13Autoregressive modelling + cross-entropy

Whichever tokenizer is used, once the actions are tokens the training objective is exactly GPT's: predict the next token.

What the sequence looks like:

Task: pick up the cup, State: <discretized proprioceptive state>;
Action: <a₁><a₂>...<aₙ>|
└──────── prefix (the condition, no loss) ────────┘└─ postfix (loss here) ─┘

Note: even the proprioceptive state is discretized (cut into 256 buckets) and stuffed into the prompt.

The training objective = next-token cross-entropy (negative log-likelihood):

targets = one_hot(tokens[:, 1:], vocab=257152)   # shift right by one to form the target
logp    = log_softmax(logits)
token_logp = sum(targets * logp, axis=-1)        # log probability of the correct token
loss = -sum(token_logp * loss_mask) / sum(loss_mask)  # loss only over the action segment

§14How multimodality is obtained "for free"

Back to the mode averaging problem of §3.1. Discrete tokens + cross-entropy solve it on three levels:

1. A discrete categorical is inherently multimodal: at each token position the model emits a softmax over the whole vocabulary, and can assign high probability to "the token corresponding to left" and "the token corresponding to right" at the same time. No more forced averaging.
2. The autoregressive factorization expresses joint multimodality: $p(a_{1:n}\mid o)=\prod_t p(a_t\mid a_{<t},o)$. A product of categoricals can express complex multimodal joint distributions — "left" is one coherent token sequence, "right" is another.
3. At inference, sampling pulls out the different solutions (next section).

Sampling and temperature

Given a multimodal distribution, how do you extract an action? Sample each token's logits with a temperature:

• $\tau=0$: take the argmax → deterministic, the single most likely solution.
• $\tau>0$: sample randomly from the distribution → different RNG yields different feasible solutions; the larger $\tau$, the more diverse.

§15Attention masking: bidirectional prefix vs causal action

Within the sequence, the prompt/state (prefix) and the action (postfix) play different roles and get different attention masks (ar_mask):

• Prefix = bidirectional attention (like an encoder): the prompt + state are known conditions and can all see each other (BERT style), letting the model fully absorb the task context.
• Postfix = causal attention (like a decoder): an action token may only see what lies to its left (GPT style), which guarantees no peeking at the future during autoregressive generation.

This "bidirectional prefix + causal postfix" mixed mask is the prefix-LM structure, used by both PaliGemma^[6] and π₀-FAST^[1].

§16A contrast: the other route, flow matching (π₀)

To place the discrete route on a map, let us glance at the continuous one. π₀^[2] does not tokenize; it uses flow matching^[12] (of the diffusion family) to generate continuous actions directly: starting from Gaussian noise, it integrates along a learned "velocity field", gradually "flowing" the noise into an action.

§17Reconstruction quality of the three tokenizers compared

"Continuous → token → continuous" is necessarily lossy (because of the quantization rounding). The three tokenizers strike different balances between reconstruction error and token count. First a measured comparison, then a chance to turn the knobs yourself.

§18The full π₀-FAST model: base model / inputs / outputs

Once the tokenizer has turned actions into tokens, who consumes those tokens? This section lays out the π₀-FAST model skeleton.

The base model: PaliGemma (a Gemma-2B decoder + SigLIP vision)

The backbone of π₀-FAST is PaliGemma^[6] — a vision-language model (VLM) made of two parts:

Inputs and the multimodal embedding order

The model input contains: images from several cameras, their respective masks, the proprioceptive state, and the tokenized text prompt. They are concatenated into one token sequence, in this order:

• Images first, text second: each image is passed through SigLIP to obtain a run of image tokens, and then the embeddings of the tokenized text prompt are appended.
• The state lives in the text: the proprioceptive state is discretized into 256 buckets, turned into a string and spliced into "State: …;", so it travels through the text embedding rather than a separate channel.
• Images share the same Transformer with text/action, but image tokens attend to each other bidirectionally.

Output: one token predicted per step, autoregressively until the end

The model's output is a probability distribution over the vocabulary (257,152 classes), i.e. "who the next token is". Generation is autoregressive, token by token:

while EOS not reached and steps < limit(256):
    token = sample from last_logit          # each step produces exactly [1] token (argmax or temperature sampling)
    embed the token back in, predict the next  # append to the input and continue
stop on EOS or when the limit is reached

Is the decoded result (T, D)? — yes

The generated run of variable-length action tokens is restored: take the tokens between "Action: " and "|" → map them back into the FAST token space → decode with time_horizon=T, action_dim=D and reshape into (T, D).

§19Glossary (quick reference)

§20Further reading

• The Fourier transform and DCT — §5's DCT in full: energy compaction, why it is near-optimal (KLT), and the shared recipe running from JPEG to FAST
• Autoregressive models · BERT / GPT — the expansion of §12/§14: the three paradigms, BERT bidirectional vs GPT causal, and why VLAs choose prefix-LM

§21References

1. K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, S. Levine. FAST: Efficient Action Tokenization for Vision-Language-Action Models. arXiv:2501.09747, 2025.
2. K. Black et al. π₀: A Vision-Language-Action Flow Model for General Robot Control. RSS 2025. arXiv:2410.24164.
3. R. Sennrich, B. Haddow, A. Birch. Neural Machine Translation of Rare Words with Subword Units. ACL 2016. arXiv:1508.07909.
4. N. Ahmed, T. Natarajan, K. R. Rao. Discrete Cosine Transform. IEEE Transactions on Computers, 1974.
5. A. Gersho, R. M. Gray. Vector Quantization and Signal Compression. Kluwer Academic Publishers, 1992.
6. L. Beyer et al. PaliGemma: A versatile 3B VLM for transfer. arXiv:2407.07726, 2024.
7. A. van den Oord, O. Vinyals, K. Kavukcuoglu. Neural Discrete Representation Learning. NeurIPS 2017. arXiv:1711.00937.
8. Y. Bengio, N. Léonard, A. Courville. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv:1308.3432, 2013.
9. F. Mentzer, D. Minnen, E. Agustsson, M. Tschannen. Finite Scalar Quantization: VQ-VAE Made Simple. ICLR 2024. arXiv:2309.15505.
10. A. Brohan et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. arXiv:2307.15818, 2023.
11. M. J. Kim et al. OpenVLA: An Open-Source Vision-Language-Action Model. arXiv:2406.09246, 2024.
12. Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, M. Le. Flow Matching for Generative Modeling. ICLR 2023. arXiv:2210.02747.
13. X. Zhai, B. Mustafa, A. Kolesnikov, L. Beyer. Sigmoid Loss for Language Image Pre-Training. ICCV 2023. arXiv:2303.15343.