NoteMathematicsRoboticsEvergreenupdated 2026.0811 min read

Starting from Fourier series and running all the way to the energy-compaction property of the DCT, this note builds a genuine understanding of why applying a DCT to a smooth signal (a robot motion, say) makes its energy pile up almost miraculously onto a handful of low-frequency coefficients. It is the expanded version of the step that Action Tokenization treats as a black box.

§1Why move a signal into the "frequency domain"

The same stretch of signal admits two views:

  • Time domain: "what is the value at each instant." A robot motion is natively of this kind: the joint angles at step 0, step 1, and so on.
  • Frequency domain: "which fast and slow waves make up the signal, and in what proportion." Another coordinate system for the same information.

The payoff of switching to the frequency domain: many signals that look densely packed in time are extremely sparse in frequency (only a few frequencies carry any value). Smooth = slowly varying = low frequencies only = sparse in frequency → compressible.

§2Prerequisites

The main text leans on the following concepts repeatedly; each is written as an "English term = Chinese name = minimal definition" triple:

  • periodic function = 周期函数 = a function for which there exists a positive number TT with f(t+T)=f(t)f(t+T)=f(t) for all tt; TT is called the period[1].
  • harmonic = 谐波 = a sine/cosine component whose frequency is an integer multiple of the fundamental; the nn-th harmonic has nn times the fundamental frequency[1].
  • complex exponential = 复指数 = a complex-valued function of the form eiωte^{i\omega t}; by Euler's formula eiωt=cos⁡ωt+isin⁡ωte^{i\omega t}=\cos\omega t + i\sin\omega t, it packs a sine and a cosine into a single complex quantity[1].
  • basis function = 基函数 = one of a fixed set of functions/vectors used for expansion; a signal is written as their weighted sum, and the weights are the transform coefficients[1].
  • energy compaction = 能量压缩 = the property that a transform concentrates most of a signal's energy onto a few coefficients; the better the concentration, the more compressible the signal[2].
  • quantization = 量化 = the lossy operation of mapping continuous or high-precision values onto a finite set of discrete levels (rounding, for instance)[2].

§3Fourier series: a periodic signal = a sum of sines and cosines

Fourier's insight (1807): any periodic function can be written as a weighted sum of sines and cosines at different frequencies[3]:

f(t)=a02+∑n=1∞(ancos⁡(nωt)+bnsin⁡(nωt))f(t)=\frac{a_0}{2}+\sum_{n=1}^{\infty}\Big(a_n\cos(n\omega t)+b_n\sin(n\omega t)\Big)

The higher the frequency nωn\omega, the faster that term varies. Below we approximate a square wave by superposing odd harmonics — the classic example, and one in which you can also watch the Gibbs phenomenon: the more terms you add the better the fit, yet the corners always overshoot.

Figure 1 · Fourier series approximating a square wave
A square wave = the sum of infinitely many odd sine harmonics. The grey curve is the target square wave, the red curve is the superposition of the first few harmonics; drag the slider to add harmonics and watch the sum (red) close in on the square wave (grey). What to look for: more harmonics means a tighter overall fit, yet the overshoot of the red curve near the jumps never disappears — that is the Gibbs phenomenon discussed in the text.

§4The continuous Fourier transform: from periodic to arbitrary signals

Push the period to infinity and the series' "sum over discrete frequencies" becomes an "integral over continuous frequency," which gives the Fourier transform:

F(ω)=∫−∞∞f(t) e−iωt dtF(\omega)=\int_{-\infty}^{\infty} f(t)\,e^{-i\omega t}\,dt

Here e−iωt=cos⁡ωt−isin⁡ωte^{-i\omega t}=\cos\omega t - i\sin\omega t (Euler's formula) packs sine and cosine into a single complex exponential. F(ω)F(\omega) is complex: its magnitude gives the strength of that frequency, its argument the phase.

§5DFT and FFT: Fourier in the discrete world

Inside a computer a signal is a finite set of samples x0,…,xN−1x_0,\dots,x_{N-1}, which calls for the discrete Fourier transform (DFT):

Xk=∑n=0N−1xn e−i2πkn/N,k=0,…,N−1X_k=\sum_{n=0}^{N-1}x_n\,e^{-i2\pi kn/N},\quad k=0,\dots,N-1

Computed directly this costs O(N2)O(N^2). The FFT (fast Fourier transform) exploits symmetries to bring it down to O(Nlog⁡N)O(N\log N)[4] — one of the most important algorithms of the 20th century, and what made real-time audio and video processing possible.

NameInputOutputComplexity
Fourier seriescontinuous periodic functiondiscrete coefficients an,bna_n,b_n—
Fourier transform (FT)continuous, non-periodiccontinuous spectrum F(ω)F(\omega)—
DFTNN discrete samplesNN complex coefficientsO(N2)O(N^2)
FFTsame as DFT (algorithmic speedup)same as DFTO(Nlog⁡N)O(N\log N)
DCTNN real samplesNN real coefficientsO(Nlog⁡N)O(N\log N)

§6From DFT to DCT: dropping the complex numbers and the boundary jump

The DFT has two features that are unfriendly to compression, and the DCT exists precisely to fix them:

  1. DFT coefficients are complex (real and imaginary parts), which is redundant for a real-valued signal.
  2. The DFT implicitly extends the signal periodically, so if the first and last values differ, an artificial jump appears at the boundary → a flood of high frequencies → bad for compression.

The workhorse is DCT-II (the one JPEG and FAST use)[5]:

Xk=∑n=0N−1xncos⁡ ⁣[πN(n+12)k]X_k=\sum_{n=0}^{N-1}x_n\cos\!\Big[\frac{\pi}{N}\big(n+\tfrac12\big)k\Big]

Note that the basis is pure cosine and the coefficients XkX_k are real. k=0k=0 is the DC term (the mean), and larger kk means higher frequency. The inverse transform, the IDCT (i.e. DCT-III), reconstructs using the same cosine basis and is exactly invertible.

§7The four DCT variants (knowing they differ is enough)

Depending on how the symmetric extension is performed at the two ends, there are four types, DCT-I through IV[2]. In practice:

VariantUse
DCT-IIThe most common. JPEG, MPEG, and FAST action tokenization all use it
DCT-IIIThe inverse of DCT-II (i.e. the IDCT)
DCT-IDifferent endpoint handling, rarely used
DCT-IVUsed in the MDCT (audio, e.g. the lapped transform in MP3/AAC)

The DCT decomposes a signal onto this set of fixed cosine basis vectors. The kk-th basis vector = a sampled cosine of frequency kk. Any signal is a weighted sum of these bases, and the weights are the DCT coefficients. Here are the first 8 bases (N=32N=32):

Figure 2 · DCT-II basis functions (k = 0…7)
In each panel the horizontal axis is the sample index n and the vertical axis the value of that basis vector: k=0 is constant (DC / mean), and the larger k is the faster the oscillation (the higher the frequency). Hover over any panel to highlight it. What to look for: all the DCT does is project the signal onto this fixed set of cosine waves ordered from slow to fast, and each projection is a coefficient.
Each panel is one cosine basis vector cos⁡[π/N (n+0.5) k]\cos[\pi/N\,(n+0.5)\,k]. The DCT coefficient XkX_k = the projection of the signal onto the k-th basis vector.

§9Energy compaction: the DCT's killer feature (verify it yourself)

This is the single most important concept on the page, and the fundamental reason FAST works. Energy compaction: for a smooth signal, the DCT concentrates the overwhelming majority of the energy onto a very small number of low-frequency coefficients, leaving the rest close to 0.

So we need only keep the first few large coefficients and discard the great many that sit close to 0 to reconstruct the signal almost perfectly from very little data. Verify it for yourself below:

Figure 3 · Reconstruction quality when keeping only the first k DCT coefficients
Top: the original signal (blue) vs the reconstruction from only the first k DCT coefficients (red). Bottom: the energy of the DCT coefficients (the retained ones highlighted). Drag k and see how few coefficients a smooth signal needs before the curves nearly coincide; then switch to the "with high frequency / noise" signal for comparison. What to look for: for the smooth signal almost all the energy is crammed into the leftmost few low-frequency coefficients, so red and blue overlap at a very small k, whereas for the signal with high frequency content the coefficient energy is spread out and a far larger k is needed — that is exactly what energy compaction means, made visible.

§10Why the DCT of all transforms (it is close to "optimal")

In theory, for a given signal statistic the transform with optimal energy compaction is the KLT (Karhunen–Loève transform, i.e. PCA) — it projects the signal onto the eigenvectors of the covariance matrix. But the KLT depends on the data statistics and requires computing eigenvectors on the spot: expensive and not general-purpose.

§11Applications: from JPEG to FAST

SystemHow the DCT is used
JPEGCut the image into 8×8 blocks → 2D DCT → quantize (drop the high frequencies) → entropy coding[7]. Nearly every photo you have ever seen has been through a DCT
MPEG / H.26xDCT + quantization on video frames (residuals)
MP3 / AACCompress audio with the MDCT (the lapped version of DCT-IV)
FAST (robotics)DCT of the action array (T,D) along the time axis → quantize by rounding → BPE[6]. Smooth motions → concentrated energy → many 0s after rounding → short token sequences

§12Further reading


§13References

  1. A. V. Oppenheim, R. W. Schafer. Discrete-Time Signal Processing, 3rd ed. Prentice Hall, 2010.
  2. K. R. Rao, P. Yip. Discrete Cosine Transform: Algorithms, Advantages, Applications. Academic Press, 1990.
  3. J. B. J. Fourier. Théorie analytique de la chaleur. Firmin Didot, Paris, 1822. (The book-length expansion of the heat-conduction memoir submitted to the Paris Academy of Sciences in 1807.)
  4. J. W. Cooley, J. W. Tukey. "An Algorithm for the Machine Calculation of Complex Fourier Series." Mathematics of Computation, 19(90): 297–301, 1965.
  5. N. Ahmed, T. Natarajan, K. R. Rao. "Discrete Cosine Transform." IEEE Transactions on Computers, C-23(1): 90–93, 1974.
  6. K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, S. Levine. "FAST: Efficient Action Tokenization for Vision-Language-Action Models." arXiv:2501.09747, 2025.
  7. G. K. Wallace. "The JPEG Still Picture Compression Standard." IEEE Transactions on Consumer Electronics, 38(1): xviii–xxxiv, 1992.
Titles, sections and body text, in this language.
    ↑↓ · Enter · Escastro-inkstone