RLBench: The Robot Learning Benchmark & Learning Environment

01 动机 MotivationMotivation

机器人操作算法的研究面临严峻的评估碎片化问题：现有基准要么任务数量有限（如 RoboTurk 仅 3 个任务）、要么缺乏视觉观测、要么演示数据极难获取。研究者往往各自定义私有评估集，导致方法间横向对比几乎不可能。与此同时，小样本学习、多任务学习等新兴范式也缺乏专门的机器人操作测试平台。 Research on robot manipulation algorithms suffers from severe evaluation fragmentation: existing benchmarks either contain very few tasks (RoboTurk, for instance, offers only 3), lack visual observations, or make demonstration data extremely hard to obtain. Researchers therefore tend to define their own private evaluation sets, which makes cross-method comparison nearly impossible. At the same time, emerging paradigms such as few-shot learning and multi-task learning have no dedicated robot manipulation testbed.

"We present RLBench, an ambitious large-scale benchmark and learning environment designed to facilitate research in a number of areas, including: reinforcement learning, imitation learning, multi-task learning, geometric computer vision, and in particular, few-shot learning. We believe it is important to find the potential and limits of these methods in a controlled, reproducible environment."

RLBench 100 tasks overview — 图 1：RLBench 包含 100 个完全独特的手工设计任务，展示了从简单的金字塔堆叠、插入方块，到复杂的植物浇水等任务样本。任务跨越难度谱系，使基准对新手和有经验的学习算法都具有挑战性。 Figure 1: RLBench contains 100 completely unique hand-designed tasks; shown here are samples ranging from simple pyramid stacking and block insertion to more complex tasks such as watering a plant. The tasks span a spectrum of difficulty, making the benchmark challenging for both novice and experienced learning algorithms.

100完全独特的手工设计任务Completely unique hand-designed tasks

∞运动规划器生成的专家演示Expert demonstrations generated by the motion planner

5支持的传感器模态（RGB/深度/分割等）Supported sensor modalities (RGB / depth / segmentation, etc.)

1st机器人领域首个大规模小样本挑战First large-scale few-shot challenge in robotics

现有基准的不足Shortcomings of Existing Benchmarks

任务多样性不足：OpenAI Gym、DeepMind Control Suite 等主流基准任务数量极少，以连续控制为主，缺乏视觉驱动的操作场景。Insufficient task diversity: mainstream benchmarks such as OpenAI Gym and DeepMind Control Suite offer very few tasks, focus on continuous control, and lack vision-driven manipulation scenarios.
演示获取困难：Similate [29]（Sim）等混合真实方案需要动作捕捉校准，耗时耗力；纯仿真方案如 RoboTurk 仅有 3 个任务且需要众包操作员。Demonstrations are hard to collect: mixed-reality solutions such as Similate [29] (Sim) require motion-capture calibration, which is time-consuming and laborious; purely simulated alternatives such as RoboTurk provide only 3 tasks and depend on crowdsourced operators.
缺乏小样本评估：没有现成平台能在机器人场景下公平测试 N-way K-shot 元学习方法。No few-shot evaluation: no off-the-shelf platform allows a fair test of N-way K-shot meta-learning methods in a robotics setting.
评估标准不统一：各研究组自定义评估集，跨论文横向比较几乎不可能。Inconsistent evaluation protocols: each research group defines its own evaluation set, making cross-paper comparison nearly impossible.

02 方法 MethodMethod

RLBench 以 V-REP 仿真器与 PyRep Python API 为基础，构建了统一的机器人操作基准平台。 Franka Panda 机械臂被固定在中央工作台上，配备两路视觉传感器，并通过运动规划器自动生成高质量专家演示。整个系统围绕三个核心概念展开：Task（任务定义）、Variation（参数变体）、Episode（执行片段）。 Built on the V-REP simulator and the PyRep Python API, RLBench provides a unified robot manipulation benchmark platform. A Franka Panda arm is mounted at the centre of a workspace table, equipped with two visual sensors, and high-quality expert demonstrations are generated automatically by a motion planner. The whole system revolves around three core concepts: Task (the task definition), Variation (parameter variants) and Episode (an execution rollout).

V-REP scene and visual observations — 图 2&3：左图为 V-REP 场景示意，Franka Panda 机械臂配有俯视立体摄像头（over-the-shoulder stereo camera）和手眼单目摄像头（eye-in-hand monocular camera）。右图展示了两路摄像头提供的 RGB、深度（Depth）及分割掩码（Segmentation Mask）等多种视觉观测，可按需组合使用。 Figures 2 & 3: on the left, the V-REP scene, in which the Franka Panda arm carries an over-the-shoulder stereo camera and an eye-in-hand monocular camera. On the right, the visual observations provided by the two cameras — RGB, depth and segmentation mask — which can be combined as needed.

场景设计（Scene）Scene Design

V-REP 场景包含 Franka Panda 机械臂，固定于工作台中央。视觉观测来自两路摄像头： over-the-shoulder stereo camera（俯视立体摄像头，提供全局视角）和 eye-in-hand monocular camera（手眼摄像头，提供局部精细视角）。每帧观测可包含 RGB 图像、深度图、分割掩码，并附带本体感知数据（joint angles、velocities、torques、 gripper pose、end-effector pose）。机械臂末端执行器可快速替换，适配不同抓取需求。 The V-REP scene contains a Franka Panda arm fixed at the centre of the table. Visual observations come from two cameras: an over-the-shoulder stereo camera (a global viewpoint) and an eye-in-hand monocular camera (a close-up, fine-grained viewpoint). Each observation may include an RGB image, a depth map and a segmentation mask, together with proprioceptive data (joint angles, velocities, torques, gripper pose, end-effector pose). The end effector can be swapped quickly to accommodate different grasping requirements.

Task / Variation / Episode 三层结构The Task / Variation / Episode Hierarchy

Task variation and episode example — 图 4：以 "stack_blocks" 任务为例，展示 Task（任务）、Variation（变体）与 Episode（片段）三层概念。不同 Variation 改变颜色或数量配置，Episode 则是具体执行中物体初始位置与颜色的随机化实例。跨 Variation 时任务目标类型改变，而 Episode 之间仅目标颜色或位置不同。 Figure 4: the "stack_blocks" task illustrates the three-level notion of Task, Variation and Episode. Different Variations change the colour or count configuration, while an Episode is a concrete rollout with randomised initial object positions and colours. Across Variations the type of task goal changes, whereas Episodes differ only in target colour or position.

RLBench 将每个任务分为三个层级： RLBench decomposes every task into three levels:

Task：定义任务的成功条件与运动规划 Waypoint，通过 Python 文件（.ttt 场景 + .py 脚本）描述。Task: defines the success condition and the motion-planning waypoints, specified through Python files (a .ttt scene plus a .py script).
Variation：同一任务的语义变体，由文字描述列表表征（如"把红色方块放在蓝色方块上"），可通过 Variation Generator 自动枚举。Variation: a semantic variant of the same task, characterised by a list of textual descriptions (e.g. "put the red block on the blue block"), which can be enumerated automatically by a Variation Generator.
Episode：具体执行片段，随机化物体初始位置与颜色等属性。成功奖励为稀疏 +1，失败为 0。Episode: a concrete rollout in which attributes such as initial object positions and colours are randomised. The reward is sparse: +1 on success and 0 on failure.

演示生成与 Environment APIDemonstration Generation and the Environment API

RLBench 通过内置运动规划器（Open Motion Planning Library，OMPL）可以无限生成专家演示，无需人工遥控。每个演示（Demo）由一系列（观测, 动作）对组成，训练时用户可按需抽取任意数量的演示。 API 遵循强化学习标准接口：env.reset()、env.step(action)，动作空间支持绝对或相对的关节速度、末端执行器速度与位姿。任务构建工具（Task Building Tool）允许社区用户通过简单的 Python 文件贡献新任务，有望持续扩展基准规模。 Through a built-in motion planner (the Open Motion Planning Library, OMPL), RLBench can generate expert demonstrations without limit, requiring no human teleoperation. Each Demo consists of a sequence of (observation, action) pairs, and users may draw as many demonstrations as they wish for training. The API follows the standard reinforcement learning interface — env.reset() and env.step(action) — and the action space supports absolute or relative joint velocities as well as end-effector velocities and poses. A Task Building Tool lets community users contribute new tasks through simple Python files, so the benchmark can keep growing.

Task word frequency and language description stats — 图 7：100 个任务对应的 Variation 描述文本中词频分布（去除停用词后）。大多数任务的 Variation 描述长度在 100 到 1000 个 token 之间，复杂任务（如 "empty_dishwasher"）描述极长，涵盖多个子步骤（开门、取碗、放托盘、关门等）。 Figure 7: word-frequency distribution over the Variation description texts of the 100 tasks (after stop-word removal). For most tasks the Variation descriptions are between 100 and 1000 tokens long; complex tasks such as "empty_dishwasher" have extremely long descriptions covering multiple sub-steps (opening the door, taking out the bowl, placing it on the tray, closing the door, and so on).

03 实验 ExperimentsExperiments

论文将 RLBench 定位为评估平台，本身不提出新算法，而是在基准上运行若干代表性基线，为模仿学习、强化学习与小样本学习研究提供参照点。 The paper positions RLBench as an evaluation platform: it proposes no new algorithm, but instead runs a set of representative baselines on the benchmark to provide reference points for imitation learning, reinforcement learning and few-shot learning research.

模仿学习基线（Imitation Learning）Imitation Learning Baselines

论文在多个任务上评估了以下基线方法，使用 K 个演示进行训练（K=1、5、20），并报告测试时 25 个 episode 的成功率： The following baselines are evaluated across several tasks, trained with K demonstrations (K = 1, 5, 20), with success rates reported over 25 test episodes:

方法Method	观测类型Observation type	代表性任务成功率Success rate on representative tasks	说明Notes
BC（Behavioral Cloning）BC (Behavioral Cloning)	RGB + proprioception	部分简单任务可达到较高成功率Reaches a fairly high success rate on some simple tasks	直接监督克隆，复合型任务成功率低Direct supervised cloning; low success rate on compound tasks
LSTM-BC	RGB + proprioception	序列任务略优于 BCSlightly better than BC on sequential tasks	循环网络捕捉时序依赖The recurrent network captures temporal dependencies
Imitation（state-based）Imitation (state-based)	关节状态Joint states	state-based 成功率明显高于 RGBState-based success rate is markedly higher than RGB	揭示视觉感知仍是主要瓶颈Shows that visual perception remains the main bottleneck

实验结果表明：训练演示数量（K）对成功率影响显著——K=20 时多数任务成功率明显优于 K=1；复杂多步任务（如 "put_groceries_in_cupboard"）对所有基线仍极具挑战性。 The results show that the number of training demonstrations (K) has a substantial effect on success rate — with K = 20 most tasks clearly outperform K = 1 — while complex multi-step tasks such as "put_groceries_in_cupboard" remain extremely challenging for every baseline.

小样本学习挑战（Few-Shot Learning Challenge，v1.0）Few-Shot Learning Challenge (v1.0)

论文提出了 RLBench Few-Shot Challenge v1.0，构建机器人领域首个大规模小样本基准： The paper introduces the RLBench Few-Shot Challenge v1.0, the first large-scale few-shot benchmark in robotics:

从 100 个任务中随机划分：训练集 70 个任务，测试集 30 个未见任务。The 100 tasks are split at random into 70 training tasks and 30 unseen test tasks.
测试时每个任务提供 K 个演示（K=1、5、20），系统需在 25 个新 episode 上完成任务。At test time each task comes with K demonstrations (K = 1, 5, 20), and the system must complete the task over 25 new episodes.
用户报告 1-shot、5-shot、20-shot 成功率，评估泛化到未见任务的能力。Users report 1-shot, 5-shot and 20-shot success rates, measuring generalisation to unseen tasks.

评估 MAML [25]、ProtoNets [26] 等元学习基线，发现现有方法在机器人小样本操作上成功率普遍较低，说明该挑战仍具有巨大研究空间。 Evaluating meta-learning baselines such as MAML [25] and ProtoNets [26] reveals that existing methods achieve generally low success rates on few-shot robot manipulation, indicating that the challenge leaves ample room for research.

多任务学习（Multi-Task Learning）Multi-Task Learning

论文提供了多任务学习的评估框架：系统同时在 M 个训练任务上学习，测试时需在全部 100 个任务（含未见任务）上达到目标成功率。实验显示现有多任务学习方法在任务数量增大后性能下降明显，为未来研究提供了清晰的性能参考基线。 The paper also supplies an evaluation framework for multi-task learning: the system learns on M training tasks simultaneously and must reach a target success rate on all 100 tasks (including unseen ones) at test time. Experiments show that existing multi-task learning methods degrade noticeably as the number of tasks grows, which provides a clear performance reference point for future work.

其他应用方向Other Application Directions

强化学习（RL）：内置稀疏奖励（成功 +1），支持连续控制 RL 算法评估。Reinforcement learning (RL): a built-in sparse reward (+1 on success) supports the evaluation of continuous-control RL algorithms.
SLAM & 视觉几何（Geometric Vision）：多模态观测（RGB-D + 分割）支持感知算法研究，如目标检测、深度估计、语义分割在机器人场景下的评估。SLAM & geometric vision: multi-modal observations (RGB-D plus segmentation) support perception research, such as evaluating object detection, depth estimation and semantic segmentation in robotic settings.
Sim-to-Real 迁移：仿真平台便于测试域随机化（domain randomization）方法，为真实机器人部署提供受控评估环境。Sim-to-real transfer: the simulation platform makes it easy to test domain randomization methods, offering a controlled evaluation environment for real-robot deployment.

04 局限性 LimitationsLimitations

Note：以下局限性部分由论文作者在 "Other Applications & Challenges" 节明确指出，部分为从系统设计中推断（标注为 inferred）。Note: some of the limitations below are stated explicitly by the authors in the "Other Applications & Challenges" section, while others are inferred from the system design (marked as inferred).

仿真与真实世界的感知差距（Sim-to-Real Gap）Perceptual gap between simulation and the real world (Sim-to-Real Gap)

论文明确指出，RLBench 虽可生成照片级渲染，但仿真渲染与真实相机图像之间仍存在域差距。作者提及使用高质量渲染系统并鼓励研究 sim-to-real 迁移，但 gap 本身无法在仿真内消除。 (stated by authors) The authors note explicitly that although RLBench can produce photorealistic renderings, a domain gap remains between simulated renderings and real camera images. They mention the use of a high-quality rendering system and encourage research on sim-to-real transfer, but the gap itself cannot be eliminated inside simulation. (stated by authors)

单一机械臂构型，缺乏双臂与移动场景（inferred）A single-arm configuration, with no bimanual or mobile settings (inferred)

当前平台仅支持固定底座的 Franka Panda 单臂机器人，不支持双臂协作或移动机器人场景。所有任务都假设工作台固定、物体在工作台范围内，排除了导航、全身运动控制等更广泛的操作场景。 (inferred from design) The platform currently supports only a fixed-base, single-arm Franka Panda robot, with no bimanual collaboration or mobile-robot scenarios. Every task assumes a fixed table with objects within its reach, excluding broader settings such as navigation and whole-body motion control. (inferred from design)

运动规划器演示质量依赖场景可规划性（inferred）Demonstration quality depends on the planability of the scene (inferred)

RLBench 的无限演示依赖 OMPL 运动规划器成功找到路径。对于极度复杂或狭窄空间的任务，规划器可能失败或生成不自然的轨迹，影响演示质量和后续模仿学习的上界。 (inferred from design) RLBench's unlimited demonstrations rely on the OMPL motion planner successfully finding a path. For extremely complex tasks or tasks in confined spaces, the planner may fail or produce unnatural trajectories, degrading demonstration quality and hence the upper bound of downstream imitation learning. (inferred from design)

稀疏奖励对强化学习构成挑战（stated）Sparse rewards pose a challenge for reinforcement learning (stated)

论文明确指出，稀疏奖励（成功 +1，失败 0）对纯 RL 方法极具挑战性，尤其是多步任务完成率极低，需要大量样本。当前基线的 RL 结果普遍较差，说明该设置远超现有 RL 算法的能力边界。(stated by authors) The authors state explicitly that the sparse reward (+1 on success, 0 on failure) is extremely challenging for pure RL methods; completion rates on multi-step tasks in particular are very low and require large numbers of samples. The RL results of the current baselines are generally poor, indicating that this setting lies well beyond the capabilities of existing RL algorithms. (stated by authors)