ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation

01 动机Motivation

现有 VLA 模型在语义理解方面表现出色，却难以应对接触密集型操作（contact-rich manipulation）——如插拔、工具使用、装配——这类任务要求精细的力控制，尤其在视觉遮挡或动态不确定性场景下。机器人系统普遍忽视了力感知这一关键模态，而人类却自然地整合触觉与本体感知来适应操作策略。Existing VLA models excel at semantic understanding, yet struggle with contact-rich manipulation (such as insertion and unplugging, tool use, assembly) — tasks that demand fine-grained force control, especially under visual occlusion or dynamic uncertainty. Robotic systems have largely neglected force sensing as a key modality, whereas humans naturally integrate tactile and proprioceptive cues to adapt their manipulation strategies.

"6D external force sensed at the robot's end-effector should be treated as a first-class modality"，而非事后追加，这与人类操作中触觉与本体感知的协同机制一致。"6D external force sensed at the robot's end-effector should be treated as a first-class modality", rather than an afterthought bolted on later; this mirrors the interplay of touch and proprioception in human manipulation.

ForceVLA Teaser — **图 1：动机示意。**以 USB 插入任务为例：纯视觉 VLA（左）因接触遮挡导致插入失败；ForceVLA（右）借助实时 6D 力反馈持续调整末端姿态，最终成功完成插入。力信号在视觉信息缺失时充当关键的感知替代通道。**Figure 1: Motivation.** Take USB insertion as an example: the vision-only VLA (left) fails because contact occludes the view; ForceVLA (right) uses real-time 6D force feedback to keep adjusting the end-effector pose and finally completes the insertion. The force signal acts as a critical perceptual substitute when visual information is missing.

23.2%平均成功率提升（vs. π₀-base w/o Force）Average success-rate gain (vs. π₀-base w/o Force)

80%单任务最高成功率（对象泛化测试）Best single-task success rate (object generalization test)

90%视觉遮挡条件下成功率Success rate under visual occlusion

5接触密集型任务 · 244 条轨迹 · 140K 同步帧contact-rich tasks · 244 trajectories · 140K synchronized frames

02 方法Method

ForceVLA 以 π₀ 为骨干，融合 RGB 视觉、自然语言指令、本体感知状态与 6 轴力-扭矩数据，通过 FVLMoE 模块在 VLM 编码之后动态整合力特征，最终由基于 conditional flow matching 的动作头输出连续轨迹。ForceVLA takes π₀ as its backbone and fuses RGB vision, natural-language instructions, proprioceptive state and 6-axis force-torque data. The FVLMoE module dynamically integrates force features after VLM encoding, and an action head based on conditional flow matching outputs continuous trajectories.

FVLMoE Architecture — **图 2：整体架构。**系统管线分三个阶段：① 预训练 PaliGemma SigLIP 对 RGB 图像与语言指令编码，输出视觉-语言嵌入 E_VL；② FVLMoE 模块将 6D 原始力数据线性投影为力 token E_F，与 E_VL 拼接后经 Transformer 编码器（multi-head self-attention）和稀疏 MoE 层融合，得到 G_FVLMoE；③ 融合特征以 element-wise addition 注入状态与动作投影层，再经 flow matching 去噪生成动作轨迹。**Figure 2: Overall architecture.** The pipeline has three stages: ① a pretrained PaliGemma SigLIP encodes the RGB images and the language instruction into vision-language embeddings E_VL; ② the FVLMoE module linearly projects the raw 6D force data into force tokens E_F, concatenates them with E_VL, and fuses them through a Transformer encoder (multi-head self-attention) and a sparse MoE layer, yielding G_FVLMoE; ③ the fused feature is injected into the state and action projection layers by element-wise addition, and flow matching then denoises it into an action trajectory.

FVLMoE：力感知混合专家融合模块FVLMoE: A Force-aware Mixture-of-Experts Fusion Module

FVLMoE 的核心设计分三步：The core design of FVLMoE comprises three steps:

Input Mapping：原始 6D 力数据 f_raw ∈ ℝ⁶ 经线性投影生成力 token E_F，与 VLM 输出的视觉-语言嵌入 E_VL 拼接为多模态序列。Input Mapping: The raw 6D force data f_raw ∈ ℝ⁶ is linearly projected into a force token E_F and concatenated with the vision-language embedding E_VL produced by the VLM into a multimodal sequence.
Multimodal Routing & Fusion：拼接序列通过 Transformer 编码器进行多头自注意力建模，再路由至含 E=4 个专家网络（MLP）的稀疏 MoE 层，动态门控选择 top-k=1 个专家处理每个 token，残差连接整合 MoE 输出。Multimodal Routing & Fusion: The concatenated sequence is modeled by a Transformer encoder with multi-head self-attention, then routed to a sparse MoE layer holding E=4 expert networks (MLPs); a dynamic gate selects the top-k=1 expert for each token, and a residual connection integrates the MoE output.
Action Injection：融合特征 G_FVLMoE ∈ ℝ^(H_action × D_a) 以 element-wise addition 注入状态与动作投影，再由 flow matching 动作头解码为机器人轨迹。Action Injection: The fused feature G_FVLMoE ∈ ℝ^(H_action × D_a) is injected into the state and action projections by element-wise addition, and the flow matching action head decodes it into a robot trajectory.

关键设计选择：Late-stage Force FusionA Key Design Choice: Late-stage Force Fusion

力信息在 VLM 编码之后引入，而非之前。消融实验表明，若在 VLM 之前融合力特征（MoE-before-VLM 变体），预训练视觉-语言表征会被破坏，导致成功率降至 0%。"晚融合"策略在保留 VLM 语义能力的同时，赋予模型实时力自适应能力。Force information enters after VLM encoding rather than before it. Ablations show that fusing force features ahead of the VLM (the MoE-before-VLM variant) destroys the pretrained vision-language representation and drives the success rate down to 0%. The "late fusion" strategy preserves the semantic capability of the VLM while granting the model real-time force adaptivity.

ForceVLA-Data：同步多模态数据集ForceVLA-Data: A Synchronized Multimodal Dataset

为支撑训练，团队采集了专用数据集：244 条轨迹，140K 个同步时间步，覆盖 5 种接触密集型任务（Bottle Pumping、Plug Insertion、USB Drive Insertion、Whiteboard Wiping、Cucumber Peeling），由 5 名专家操作员在 Flexiv Rizon 7-DOF 机械臂上完成，配备双 RGB-D 相机与 6 轴力-扭矩传感器。To support training, the team collected a dedicated dataset: 244 trajectories and 140K synchronized time steps, covering 5 contact-rich tasks (Bottle Pumping, Plug Insertion, USB Drive Insertion, Whiteboard Wiping, Cucumber Peeling), recorded by 5 expert operators on a Flexiv Rizon 7-DOF arm fitted with dual RGB-D cameras and a 6-axis force-torque sensor.

Task Setup — **图 3：五种接触密集型任务场景。**从左至右：Bottle Pumping（瓶泵按压）、Plug Insertion（插头插入）、USB Drive Insertion（USB 插入）、Whiteboard Wiping（白板擦拭）、Cucumber Peeling（黄瓜削皮）。每类任务均需精细力控制，且部分任务存在视觉遮挡。**Figure 3: The five contact-rich task scenarios.** From left to right: Bottle Pumping, Plug Insertion, USB Drive Insertion, Whiteboard Wiping, Cucumber Peeling. Every task calls for fine-grained force control, and some of them involve visual occlusion.

03 实验Experiments

在 Flexiv Rizon 机械臂上评估三种方法：π₀-base w/o Force（纯视觉基线）、π₀-base w/ Force（朴素力融合）、ForceVLA（FVLMoE 完整方法）。评估维度包括 5 任务成功率、泛化鲁棒性与多任务联合训练。Three methods are evaluated on the Flexiv Rizon arm: π₀-base w/o Force (vision-only baseline), π₀-base w/ Force (naive force fusion) and ForceVLA (the complete FVLMoE method). The evaluation covers success rates on the 5 tasks, generalization robustness and multi-task joint training.

主要结果：五任务成功率Main Results: Success Rate on the Five Tasks

方法Method	平均成功率Avg. success rate	vs. w/o Force
π₀-base w/o Force	37.3%	—
π₀-base w/ Force	40.2%	+2.9 pp
ForceVLA（ours）ForceVLA (ours)	60.5%	+23.2 pp

Success Rate Comparison — **图 4：各任务成功率对比。**ForceVLA 在所有五个接触密集型任务上均优于两条基线，在 Plug Insertion 等任务上个别条件可达约 80% 成功率。**Figure 4: Per-task success rate comparison.** ForceVLA outperforms both baselines on all five contact-rich tasks, and on tasks such as Plug Insertion it reaches roughly 80% success under individual conditions.

泛化测试（Plug Insertion，Table 2）Generalization Tests (Plug Insertion, Table 2)

测试条件Test condition	π₀-base w/o Force	ForceVLA
Object generalization（换插座）Object generalization (different socket)	—	80.00%
Height variation（高度变化）Height variation	—	88.89%
Visual occlusion（视觉遮挡）Visual occlusion	—	90.00%
Unstable socket（插座不稳定）Unstable socket	—	20.00%
平均Average	38.93%	63.78%

特别地，视觉遮挡条件下 ForceVLA 仍达 90.00%，充分证明力信号在视觉失效时的补偿作用。Notably, ForceVLA still reaches 90.00% under visual occlusion, ample evidence that force signals compensate when vision fails.

多任务联合训练（Table 5，4 任务）Multi-task Joint Training (Table 5, 4 tasks)

方法Method	平均成功率（4 任务）Avg. success rate (4 tasks)
π₀-base w/o Force	5%
π₀-base w/ Force	42.5%
ForceVLA（ours）ForceVLA (ours)	67.5%

消融实验（Table 3，Plug Insertion 单任务）Ablation Study (Table 3, Plug Insertion single task)

变体Variant	成功率Success rate
Linear projection before VLM	55%
MoE before VLM	0%
Concatenate after VLM	60%
ForceVLA（FVLMoE, ours）ForceVLA (FVLMoE, ours)	80%

结果验证：① 力的融合位置至关重要，VLM 之后融合是最优策略；② 朴素拼接（Concatenate after VLM）虽优于早融合，但相比 FVLMoE 还差 20 个百分点；③ MoE 的动态路由带来显著收益。The results confirm that ① where force is fused matters greatly, and fusing after the VLM is the optimal strategy; ② naive concatenation (Concatenate after VLM) beats early fusion but still trails FVLMoE by 20 percentage points; ③ the dynamic routing of the MoE yields a substantial gain.

Router 分析：专家负载与任务专化Router Analysis: Expert Load and Task Specialization

Expert Router Analysis — **图 5：MoE Router 分析。**Expert 0 作为"通用专家"主导约 50% 的 token 分配；Plug Insertion 与 Cucumber Peeling 等任务在不同操作阶段呈现出不同专家的时序激活模式（phase-specific expert activation）；Whiteboard Wiping 则始终由单一专家主导，反映该任务的相对均匀性。**Figure 5: MoE router analysis.** Expert 0 acts as a "generalist expert" and dominates about 50% of the token assignments; tasks such as Plug Insertion and Cucumber Peeling exhibit phase-specific expert activation across manipulation stages; Whiteboard Wiping is consistently dominated by a single expert, reflecting the relative uniformity of that task.

轨迹可视化Trajectory Visualization

04 局限性Limitations

Note：以下局限性均为论文作者明确陈述（stated by the authors）。Note:All limitations below are explicitly stated by the authors of the paper.

力估计精度的局限（Force Estimation Accuracy）Force Estimation Accuracy

系统使用的是估计的外力旋量（estimated external wrench），而非高保真传感器的直接测量值。对于需要极高触觉灵敏度的任务，这可能带来感知误差，影响精细力控的可靠性。The system relies on an estimated external wrench rather than a direct measurement from a high-fidelity sensor. For tasks that demand extremely high tactile sensitivity this may introduce perception error and undermine the reliability of fine force control.

硬件可及性的制约（Hardware Accessibility）Hardware Accessibility

当前评估局限于配备集成 6 轴力-扭矩传感器的高端平台（Flexiv Rizon）。作者承认，能否迁移至"lower-cost platforms equipped with external or retrofitted force sensors"尚不确定，这可能限制该方法的"broader accessibility"。The current evaluation is confined to a high-end platform with an integrated 6-axis force-torque sensor (Flexiv Rizon). The authors admit it is unclear whether the approach transfers to "lower-cost platforms equipped with external or retrofitted force sensors", which may limit its "broader accessibility".

不稳定场景的鲁棒性不足（Unstable Socket Condition）Insufficient Robustness in Unstable Scenarios (Unstable Socket Condition)

泛化测试中，在"Unstable socket"（插座不稳定）条件下 ForceVLA 成功率仅 20.00%，明显低于其他泛化条件（80%–90%）。这表明对于动态基座或随机扰动场景，当前方法仍存在明显短板。In the generalization tests, ForceVLA reaches only 20.00% under the "Unstable socket" condition, clearly below the other generalization conditions (80%–90%). This shows the method still falls short on dynamic bases or randomly perturbed scenes.