ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

Paper

ActionPiece: Rethinking Action Tokenization Beyond Reconstruction Error, via Physical Ranking

TL;DR — In autoregressive VLA (vision-language-action) models, the action tokenizer is the key interface that compresses continuous robot commands into discrete tokens and then reconstructs predicted tokens back into executable commands. Existing tokenizers only reduce pointwise reconstruction error such as MSE; they do not check whether the “fine adjustments” between demonstrations survive compression. This paper newly defines PRC (Physical Rank Consistency), which measures the degree to which the ranking of physical distances is preserved in the reconstructed action space, and proposes ActionPiece, which supervises representation learning and quantization jointly to preserve that relation. Under the same Qwen3-VL-4B policy training setup, it achieves state-of-the-art results of 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus (source: Abstract, Table 1).

Key Idea

The action tokenizer simultaneously determines the “prediction target” for policy learning and the “decoder” at execution time (source: §1). Prior work improved compression efficiency through coordinate discretization (RT-2, OpenVLA), frequency-domain compression (FAST), and learned vector quantization (FASTer, OAT, ActionCodec, X-Tokenizer) (source: §2.2), but did not sufficiently address the fact that compression itself changes the actions the robot ultimately executes (source: §1).

ActionPiece’s core insight boils down to three points.

1. MSE only reduces the error of individual actions; it does not coordinate the “direction” of reconstruction error across demonstrations (source: §3.1).
2. After compression, similar actions cluster around a representative motion, but the adjustments needed for different contexts can be weakened, distorted, or inverted (source: §1, §3.1).
3. Therefore, in addition to pointwise accuracy, one must evaluate preservation of the ranking of physical distances (PRC) and directly supervise that ranking itself as a training objective.

The authors call this perspective “relational fidelity,” and to achieve it they apply rank supervision to both the representation learning stage and the codeword assignment stage (source: §1, Fig. 1).

Background: The Problem They Solved

The “adjustments” the tokenizer silently erases

Across demonstrations of the same task, repeated motions appear with subtle differences. For example, in “pick up the red can and put it in the box,” the grasp position varies slightly depending on object position, robot state, and execution stage (source: §3.1). In tasks requiring precise alignment, grasping, and contact, these small differences determine success or failure (source: §1, §3.1).

The tokenizer compresses these demonstrations into a discrete representation of finite capacity and then reconstructs them back into executable commands. This lossy compression inevitably produces reconstruction error $e_i = \hat x_i - x_i$. The reconstructed difference between two demonstration chunks $x_i$, $x_j$ decomposes as follows (source: Eq. 1):

The first term is the “action change” that actually exists in the demonstrations, and the second is the “extra change” that tokenization inserts. The problem is that MSE only reduces the magnitude of this second term and does not control its direction. As a result, the extra term can attenuate or amplify the original difference and can even flip its direction (source: §3.1).

Crucially, this distortion does not disappear even when the policy predicts the tokens “exactly.” Even if the VLA correctly reproduces the token sequence encoding a demonstration action, the decoder will still execute the “reconstructed version” of that action (source: §1, §3.1).

The landscape of existing approaches

The authors organize existing action interfaces into three branches (source: §2.1).

• Discrete autoregressive: RT-2/OpenVLA discretize coordinates into bins, and FAST compresses action chunks into short token sequences. The advantage is that they can reuse the next-token prediction objective of a pretrained VLM as is (source: §2.1).
• Continuous prediction: OpenVLA-OFT ($\ell_1$ regression) and the RDT/$\pi_0$/GR00T family (diffusion, flow matching) directly predict real-valued action chunks (source: §2.1).
• Hybrid: $\pi_{0.5}$, HybridVLA, Fast-in-Slow, etc. mix discrete supervision with continuous execution (source: §2.1).

Among the learned vector quantization family are FASTer (residual vector quantization + structural patching), OAT (nested dropout + coarse-to-fine alignment), ActionCodec (token budget, alignment, residual grammar), and X-Tokenizer (semantic supervision from a vision-language teacher) (source: §2.2). All of these focused on compression, ordering, and multimodal alignment, but none had a mechanism to measure or supervise “how compression changes the relations between demonstrations” (source: §2.2).

New Approach: ActionPiece

PRC: Looking at “distance ranking,” not “distance values”

The authors evaluate relation preservation through the relative ordering of physical distances. First they define the physical distance $d_{\text{phys}}$. This expresses the difference between two end-effector commands $a=(p, R, g)$ (position, rotation, gripper) as a weighted sum of normalized components (source: §4.2, Eq. 5):

The rotation distance is measured as the shortest geodesic angle on $\mathrm{SO}(3)$ (source: Eq. 4):

The scales $s_p, s_R$ are fit to the training actions, and the group weights $\alpha_p, \alpha_R, \alpha_g$ are fixed within an embodiment (source: §4.2). The chunk-level distance averages the per-timestep distances.

Based on this distance, PRC@k measures, for each anchor $A_i$, how well the distance ranking to the $k$ nearest neighbors $N_i^k$ in the original action space agrees with the ranking to the same neighbors of the reconstruction $\hat A_i$, using the Spearman correlation $\rho_S$ ($k=32$) (source: §3.2, Eq. 2):

The key design choice is that the ranking is measured in the decoded action space. This lets tokenizers be compared on the same physical basis regardless of token vocabulary, sequence length, or decoder architecture (source: §3.2). In short, MSE handles the fidelity of individual reconstructions, and PRC handles preservation of the physical distance ranking.

Two physical rank losses

ActionPiece consists of a shallow Transformer encoder/decoder and residual vector quantization (RVQ). An action chunk $A\in\mathbb{R}^{H\times D}$ with $H$ steps is turned by the encoder $E_\phi$ into $S$ latent slots, and an RVQ of depth $Q$ and codebook size $K$ represents each slot as a sum of $Q$ codewords $q_s = \sum_{r=1}^{Q} e^{(r)}[z_s^{(r)}]$ (source: §4.1, Eq. 3). The total number of tokens is $T = SQ$. The decoder $D_\omega$ reconstructs the entire chunk at once from these $T$ tokens.

In addition to the reconstruction loss, the authors add two rank supervisions.

① PRP (Physical Rank Preservation) — After LayerNorm and unit $\ell_2$ normalization of the encoder features $\bar h_{i,s}$ and the selected codeword features $\bar e_{i,s}$ respectively, the slot-mean squared distances $d^2_{\text{enc}}$, $d^2_{\text{quant}}$ are computed and mixed $0.25:0.75$ to form the representation distance (source: §4.3, Eq. 6–7):

Then, for each anchor, the remaining actions in the batch are sorted by physical distance, and the near action at the 5th percentile ($q_{0.05}$) is selected as the positive (near) $j_i^+$, while the far action at the 95th percentile ($q_{0.95}$) is selected as the negative (far) $j_i^-$ (source: §4.3, Appx. B.3). The ranking loss imposes the near–far order (source: Eq. 8):

② QR (Quantization Regularization) — Rather than the representation distance, it imposes the same order on the mean Jensen–Shannon divergence between the codeword assignment probability distributions $\pi_{i,s}$ of each slot (source: §4.4, Eq. 9–10):

The two losses target two different stages of compression. PRP aligns the geometry of learned features to the physical order, and QR aligns the assignment into the discrete vocabulary (source: §4.4). The overall objective is as follows (source: §4.5, Eq. 12):

The weights are $\lambda_{vq}=0.25$ and $\lambda_r=\lambda_q=6.25\times10^{-4}$ (source: §4.5, Table 6). After training, the tokenizer is frozen; the encoder and quantizer supply the policy’s discrete targets, and the frozen decoder turns predicted tokens back into execution chunks. The VLA is trained only with standard next-token prediction $\mathcal{L}_{\text{VLA}} = -\sum_t \log p_\theta(z_t | I, \ell, z_{\lt t})$ (source: §4.5, Eq. 13).

  flowchart TD
    A["Select anchor / near / far from demonstrations<br/>(based on physical distance d_phys)"] --> T["ActionPiece tokenizer"]
    T --> E["Encoder: 8-step chunk → 16 slots"]
    E --> Q["RVQ: 1 codeword per slot<br/>codebook size 512"]
    Q --> D["Decoder: 16 tokens → 8-step action reconstruction"]
    E --> P["PRP: d_rep(anchor,near) < d_rep(anchor,far)"]
    Q --> R["QR: JS(anchor,near) < JS(anchor,far)"]
    D --> M["MSE reconstruction loss"]

How It Works: A Concrete Example

To aid understanding, let us give an illustrative example (we note that this is not an actual experimental result from the paper).

Suppose we have three action chunks in the “pick up the red can and put it in the box” task.

The original physical distances are $d_{\text{phys}}(A,n)=0.01$ (small) and $d_{\text{phys}}(A,f)=0.10$ (large), so the ranking is $n < f$. This ranking must be preserved for the meaning of “fine adjustment” to survive.

Suppose that a tokenizer trained with MSE alone reconstructs them, yielding distances as follows (illustrative numbers).

After reconstruction, $d(\hat A,\hat f) < d(\hat A,\hat n)$, so the ranking is flipped. The individual reconstruction errors were small, but the direction of those errors collapsed the boundary between “fine adjustment” and “entirely different motion” (source: consistent with the interpretation of Eq. 1 in §3.1). ActionPiece’s PRP and QR enforce this $n < f$ ranking in both the representation distance and the codeword assignment, training so that the ranking is preserved after reconstruction.

The quantitative evidence the paper presents is the correlation analysis in Fig. 2. Across 55 tokenizer-benchmark evaluations, PRC showed a Spearman correlation of $\rho_S = 0.681$ with policy success, while reconstruction fidelity (MSE ranking) showed $\rho_S = 0.544$ (source: Fig. 2, §5.2). In other words, “how well the distance ranking is preserved” is more strongly associated with downstream success than “how similar the distance values are”, which is the empirical basis for this design.

Performance Validation: Key Results

Summary of experimental setup

Four benchmarks cover three kinds of action data sources and different generalization conditions (source: §5.1).

• LIBERO: In-distribution baseline. Simulated teleoperation demonstrations (20 Hz sampling).
• LIBERO-Plus: Evaluates a LIBERO-trained policy under 7 kinds of unseen perturbations (camera, robot, language, lighting, background, noise, layout).
• SimplerEnv: Real-to-sim transfer of a WidowX policy trained on BridgeData V2 real-robot demonstrations (5 Hz).
• VLA-Arena: Train on L0 (keyboard teleoperation, 10 Hz) and evaluate on L0–L2.

The tokenizer is trained with AdamW, lr $1\times10^{-4}$, batch 128, for 100K steps, and the policy is trained with the StarVLA protocol on 8×NVIDIA RTX PRO 6000 GPUs using AdamW (lr $1\times10^{-5}$, cosine schedule) + DeepSpeed ZeRO-2 (source: §5.1). In all comparisons, the backbone, prompt, optimizer, batch, training budget, 8-step prediction/execution horizon, seed, and evaluation protocol are held fixed (source: §5.1).

Matched comparison: LIBERO / LIBERO-Plus

Table 1 compares results while changing only the action tokenizer on the same Qwen3-VL-4B policy (source: Table 1).

ActionPiece leads the strongest baseline (ActionCodec 93.7%) by +1.1p on LIBERO and the strongest baseline (64.3%) by +4.5p on unseen LIBERO-Plus (source: §5.3, Table 1). It ranks first in 6 of the 7 perturbation categories of LIBERO-Plus, supporting the claim that preserving adjustments at the execution stage is more advantageous when going outside the training distribution (source: §5.3).

Transfer evaluation: SimplerEnv / VLA-Arena

On SimplerEnv (WidowX), ActionPiece achieved 71.9% on average, the best among the 15 compared models. It placed first on block stacking (58.3%), which requires precise placement, and was also strong on spoon (83.3%) and branch (91.7%) (source: §5.4, Table 2). This shows that physical relation supervision is also effective for tokenizers trained on real robot actions.

On VLA-Arena, the L0/L1/L2 success rates were 82.2% / 42.7% / 29.5%, with a 33-cell equal-weighted average of 51.5%, the best among the compared models (source: §5.4, Table 3). Despite training only on L0, it led the L1 and L2 averages by +7.0p and +5.0p respectively, and showed an advantage in L2 performance on static obstacles, dynamic obstacles, and unseen objects (source: §5.4).

Component ablation: The role of the two losses

Table 4 and Table 5 decompose the effect by removing each of the two physical rank losses (source: §5.5, Table 4–5).

Three observations emerge. First, PRP contributes more individually (+PRP gives +5.0p on LIBERO-Plus). Second, adding QR raises LIBERO-Plus by a further +3.4p, so the two losses are complementary. Third, improvements in policy success move together with improvements in PRC (0.902→0.953) (source: §5.5). In particular, on camera perturbation, PRP alone jumps from 35.5% to 45.1% when combined, and the combination beats the individual losses in 6 perturbation categories (source: §5.5).

Appendix A also provides comparisons with alternative objectives. SIGReg, Neighborhood (pulling nearby actions together), and Temporal (matching temporal changes) can also raise policy success, but Neighborhood actually lowers PRC (0.889). ActionPiece simultaneously achieves the lowest MSE, the highest PRC, and the highest success rate on both benchmarks among them (source: Appx. A, Table 5).

Our Perspective: Strengths, Limitations, and Why This Work Matters

Strengths

1. The problem definition is sharp. The fact that “the tokenizer changes the executed action” has long been known in robotics, but this work elevates it into a measurable framework called “relational fidelity.” PRC is a clear contribution in that it lets tokenizers with different vocabularies, sequences, and decoders be compared on a single common physical basis (source: §3.2).
2. The placement of supervision is convincing. The design that supervises the representation distance (PRP) and the codeword probabilities (QR) separately corresponds to the two stages of compression, “feature learning” and “discrete assignment,” and is conceptually clean and justified by ablations (source: §4.4, §5.5).
3. The experiments are rigorous. In the matched comparison, everything from backbone and data to prompt, budget, and seed is held fixed, and the coverage spans in-distribution (LIBERO), out-of-distribution (LIBERO-Plus), real-to-sim (SimplerEnv), and difficulty generalization (VLA-Arena) (source: §5.1).

Limitations

We first cover the limitations the authors acknowledge or reveal.

• For RVQ depth, $Q=1$ and $Q=2$ give similar downstream performance, so $Q=1$ was chosen as the final configuration. This means the depth benefit of residual quantization is limited on this task (source: Appx. B.2).
• The group weights $\alpha_p,\alpha_R,\alpha_g$ of the physical distance and the scales are fixed within an embodiment, and their specific values are not reported in the paper (source: §4.2, “not reported in the original”).
• Behind the overall advantage, there is per-task variance. For example, there are cases where it is not first, such as “place carrot on plate” (54.2%) on SimplerEnv or some categories on VLA-Arena (source: Table 2, Table 3).

The potential limitations inferred from a reviewer’s perspective are as follows.

• $\lambda_r=\lambda_q=6.25\times10^{-4}$ is a very small value, yet there is no sensitivity analysis for these weights (source: Table 6). The balance between losses may be sensitive to the data distribution.
• The effect on results of PRC’s hyperparameters ($k=32$, Spearman correlation) and the near/far percentiles (5%, 95%) was not ablated (source: §3.2, Appx. B.3).
• Table 1 compares tokenizers, whereas Tables 2 and 3 compare against full policies such as $\pi_0$, $\pi_{0.5}$, and GR00T. There is no comparable apple-to-apple point of comparison that directly bridges these two kinds of comparisons with different conditions (source: Table 1 vs Table 2–3).
• Although SimplerEnv trained on real-robot demonstrations is real-to-sim, physical real-robot deployment performance is not reported (source: §5.4).
• Total computation (FLOPs/PF-days) and energy cost are not reported (“not reported in the original”).

Why this work matters

As long as VLA keeps the framework of “predicting discrete tokens,” the tokenizer is the bottleneck that sets the policy’s ceiling. ActionPiece redefines this bottleneck from the perspective of “preserving relations between actions” rather than “reconstructing individual actions,” and showed that this perspective actually yields gains in out-of-distribution generalization (source: §5.3, §6). The result that merely fixing the tokenizer yields a +4.5p (LIBERO-Plus) improvement in transfer performance under the same policy suggests that “representation design” is still an underexplored lever in robotics foundation models.

What’s Next?: The Road Ahead

The paper does not list explicit future research directions, but one can propose paths that naturally follow from the results and limitations.

1. Revisiting multi-stage RVQ ($Q>1$). It is worth checking whether the current result that $Q=1$ and $Q=2$ are similar still holds once physical rank supervision is added (source: Appx. B.2).
2. Generalizing the physical distance weights. Learning $\alpha_p,\alpha_R,\alpha_g$ adaptively from the task or data, rather than fixing them per embodiment, is key to extending this method to various robot morphologies (source: §4.2).
3. Making PRC a standard diagnostic metric. PRC can be used as an independent quality metric for choosing a tokenizer. If reporting PRC alongside MSE when selecting a tokenizer becomes standard practice, the impact of this paper could exceed that of ActionPiece itself (source: the correlation analysis in Fig. 2).
4. Combining with continuous and diffusion policies. PRP/QR is supervision for discrete tokenizers, but the principle of “preserving the direction of adjustments between demonstrations” is a general principle transplantable to flow/diffusion policies of the $\pi_0$ family and to hybrid designs (source: connected to the hybrid discussion in §2.1).

Key Numbers (per the global rules)

• Tokenizer configuration: chunk length $H=8$, output tokens $T=16$, latent slots $S=16$, RVQ depth $Q=1$, codebook $K=512$ (source: Table 6)
• Physical distance: normalized weighted sum of translation + $\mathrm{SO}(3)$ geodesic rotation + gripper (source: Eq. 5)
• Rank loss: near (5th percentile)/far (95th percentile), margins $m_r=m_q=0.1$, softplus (source: Eq. 8, Eq. 10)
• Loss weights: $\lambda_{vq}=0.25$, $\lambda_r=\lambda_q=6.25\times10^{-4}$ (source: Table 6)
• Policy: Qwen3-VL-4B, 8×RTX PRO 6000, DeepSpeed ZeRO-2 (source: §5.1)
• Performance: LIBERO 94.8% · LIBERO-Plus 68.8% · SimplerEnv 71.9% · VLA-Arena L0–L2 51.5% (source: Table 1–3)
• PRC: Standard RVQ 0.902 → ActionPiece 0.953 (source: Table 5)
• Compute/energy/cost: not reported in the original

Tables from the paper

Tables converted mechanically from the arXiv e-print LaTeX source. The numbers are the paper’s own and did not pass through a model.

Table 1. Matched action-tokenizer comparison on LIBERO and LIBERO-Plus. All policies use the same Qwen3-VL-4B backbone, training data, prompt, vision–language co-training procedure, global batch size 128, an 8-step prediction and execution horizon, and maximum policy-training budget. LIBERO-Plus is evaluated out of distribution: no LIBERO-Plus demonstrations are used for policy training. FASTerVQ$^*$ denotes our implementation following the published method. Best results are bold.

Table 2. SimplerEnv WidowX results. We follow the official SimplerEnv evaluation protocol . All values are success rates (%). Best and second-best averages are bold and underlined, respectively.

Table 3. VLA-Arena results . SR is success rate (%); CC is cumulative cost (lower is better). Avg equally weights the 11 suites. For each task and difficulty level, the highest SR across the six models is bold, including ties; the same rule applies to Avg. CC values are not bold. Baselines include $\pi_{0.5}$ , GR00T-N1.6 , Evo-Depth , and Motus . Qwen-GR00T denotes Qwen3-VL with the GR00T N1 action head .

Table 4. Component ablation on LIBERO and LIBERO-Plus. Starting from standard RVQ, we add physical rank preservation (PRP), quantization regularization (QR), or both. Success rates are reported in percent. Best results are bold.

Table 5. Training-objective variants of standard RVQ. Each ``+’’ row adds only the named objective to standard RVQ. PRP and QR denote physical rank preservation and quantization regularization. Policies use Qwen3-VL-4B with 30K training steps. Success rates are in percent; MSE is scaled by $10^4$. Best values are bold, determined before rounding.

Table 6. ActionPiece configuration and objective weights.