← 论文海报合集← Paper Notes|
机器人 · Robotics · ICRA 2024Robotics · ICRA 2024

GELLO: 通用、低成本、直觉式机器人臂遥操作框架GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators

General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators
Philipp Wu, Yide Shentu, Zhongke Yi, Xingyu Lin, Pieter Abbeel  ·  UC Berkeley

GELLO 通过构建目标机械臂的缩小版运动等效结构,以不到 $300 的成本提供直觉式遥操作体验。相比 VR 控制器和 SpaceMouse,GELLO 在五项双臂操作任务中平均成功率达到 0.92,显著超越其他低成本方案,并已为 Franka、UR5、xArm 三款机器人平台开源了全部硬件设计与代码。By building a scaled kinematically equivalent structure of the target manipulator, GELLO delivers an intuitive teleoperation experience for under $300. Compared with VR controllers and the SpaceMouse, GELLO reaches an average success rate of 0.92 across five bimanual manipulation tasks, clearly outperforming other low-cost options, and it open-sources all hardware designs and code for three robot platforms: Franka, UR5 and xArm.

arXiv 2023-09 DYNAMIXEL XL330 硬件全开源Fully open-source hardware 📄 arXiv:2309.13037 🌐 Project Page
teleoperation robot manipulation imitation learning data collection DYNAMIXEL kinematically equivalent low-cost hardware 遥操作teleoperation 模仿学习imitation learning 数据采集data collection

01 动机Motivation

机器人模仿学习的性能随数据集规模提升而提升,但高质量演示数据的采集仍是主要瓶颈。传统双边遥操作设备(如力反馈外骨骼)成本高昂;而 VR 控制器、SpaceMouse 等低成本替代方案将机器人运动学抽象化,使操作者难以感知奇异点和碰撞风险,操作效率低下。The performance of robot imitation learning improves as dataset scale grows, yet collecting high-quality demonstration data remains the main bottleneck. Conventional bilateral teleoperation devices (such as force-feedback exoskeletons) are expensive, while low-cost alternatives such as VR controllers and the SpaceMouse abstract away the robot's kinematics, making it hard for the operator to perceive singularities and collision risks and leaving teleoperation inefficient.

"Systems that abstract away kinematic constraints prevent operators from managing singularities and collisions effectively."
GELLO 设计图
图 1:GELLO 硬件设计概览。左侧展示三款 GELLO 实例(针对 Franka Panda、UR5、xArm),右侧展示对应的全尺寸机器人臂。GELLO 采用缩小比例(α = 0.5)的运动等效结构,由 DYNAMIXEL XL330 舵机和 3D 打印零件构成。Figure 1:Overview of the GELLO hardware design. The left side shows three GELLO instances (for Franka Panda, UR5 and xArm), the right side the corresponding full-size robot arms. GELLO adopts a kinematically equivalent structure at a reduced scale (α = 0.5), built from DYNAMIXEL XL330 servos and 3D-printed parts.
<$300GELLO 全套硬件成本Total hardware cost of GELLO
0.92GELLO 平均任务成功率Average task success rate of GELLO
3支持的机器人平台(Franka / UR5 / xArm)Supported robot platforms (Franka / UR5 / xArm)
5用户研究任务数(12 名参与者)Tasks in the user study (12 participants)

现有方案的局限Limitations of existing solutions

高成本方案High-cost solutions

  • 力反馈外骨骼(如 Omega7):约 $40,000Force-feedback exoskeleton (e.g. Omega7): about $40,000
  • Robot-to-robot 遥操作(UR5):约 $30,000Robot-to-robot teleoperation (UR5): about $30,000
  • ALOHA 等方案需额外完整机器人臂,成本高且移植性差Approaches such as ALOHA need an extra complete robot arm, which is costly and ports poorly

低成本方案的痛点Pain points of low-cost solutions

  • 3D Mouse(SpaceMouse):$150,抽象末端执行器,忽略运动学3D Mouse (SpaceMouse): $150, abstracts the end effector and ignores kinematics
  • VR 控制器(Meta Quest 2):$300,手腕方向不自然映射VR controller (Meta Quest 2): $300, unnatural mapping of wrist orientation
  • 均无法感知奇异点与自碰撞,导致频繁失败Neither can sense singularities or self-collisions, which causes frequent failures

02 方法Method

GELLO 的核心思路是构建目标机器人臂的缩小版运动等效结构(scaled kinematically equivalent structure):操作者直接操纵各关节,无需计算逆运动学,同时通过关节阻力天然感知奇异点。配合弹簧/橡皮筋关节正则化和 DYNAMIXEL 高精度编码器,GELLO 以极低成本实现直觉式操作。The core idea of GELLO is to build a scaled kinematically equivalent structure of the target robot arm: the operator moves each joint directly, without computing inverse kinematics, and senses singularities naturally through joint resistance. Together with spring/rubber-band joint regularization and the high-precision DYNAMIXEL encoders, GELLO achieves intuitive teleoperation at extremely low cost.

多平台任务展示
图 2:GELLO 在三个机器人平台(Franka Panda、UR5、xArm)上执行的多样化操作任务,包括双臂协作、接触丰富型任务(插 USB、折叠毛巾)以及长时程任务(倒水)。Figure 2:Diverse manipulation tasks performed with GELLO on three robot platforms (Franka Panda, UR5, xArm), including bimanual coordination, contact-rich tasks (plugging in USB, folding a towel) and long-horizon tasks (pouring water).

硬件设计原则Hardware design principles

舵机选型:DYNAMIXEL XL330Servo choice: DYNAMIXEL XL330

GELLO 采用 DYNAMIXEL XL330 系列舵机。该舵机配备 12-bit 高精度编码器,关节测量精度达 0.088 mechanical degree 以内。选用 XL-330-288T 型号(最大传动比),提供较大关节阻力,作为天然阻尼,帮助操作者感知力矩和奇异点。舵机自带编码器和通信协议,构成"off-the-shelf, self-contained solution"。GELLO uses the DYNAMIXEL XL330 servo series. These servos carry a 12-bit high-precision encoder, giving joint measurement accuracy within 0.088 mechanical degree. The XL-330-288T model (the largest gear ratio) is chosen for its higher joint resistance, which acts as natural damping and helps the operator feel torques and singularities. The servos come with their own encoders and communication protocol, forming an "off-the-shelf, self-contained solution".

缩小版运动等效结构Scaled kinematically equivalent structure

GELLO 将目标机械臂按比例因子 α = 0.5 缩小,保留完全相同的运动学结构。操作者直接控制各关节角度,无需求解逆运动学(IK),并可通过关节阻力感受奇异点临近。小尺寸设计提升便携性,同时降低操作者对环境碰撞的视觉遮挡。GELLO scales the target manipulator down by a factor of α = 0.5 while keeping an identical kinematic structure. The operator commands each joint angle directly, without solving inverse kinematics (IK), and feels an approaching singularity through joint resistance. The small size improves portability and reduces the visual occlusion of environment collisions for the operator.

关节正则化实物
图 3a:关节正则化机构实物照片。弹簧/橡皮筋安装在关节 2、3 处,为 UR5 版本的 GELLO 提供被动力反馈。Figure 3a:Photograph of the joint regularization mechanism. Springs/rubber bands are mounted at joints 2 and 3, providing passive force feedback for the UR5 version of GELLO.
关节正则化力矩分析
图 3b:关节正则化前后的维持姿态所需力对比。无正则化时需 ~1.9 N;有正则化后,力随高度变化,辅助操作者规避奇异构型。Figure 3b:Comparison of the force needed to hold a pose with and without joint regularization. Without regularization about 1.9 N is required; with regularization the force varies with height, helping the operator avoid singular configurations.

关节正则化(Joint Regularization)Joint Regularization

重力会使手臂自然下垂至不理想构型。GELLO 使用弹簧或橡皮筋等简单机械元件进行"关节正则化",产生被动力反馈:
• 无正则化:维持关节姿态需约 1.9 N 的外力。
• 有正则化:所需力随高度/关节角度变化,自然引导操作者远离奇异构型。
Gravity makes the arm sag into undesirable configurations. GELLO uses simple mechanical elements such as springs or rubber bands for "joint regularization", producing passive force feedback:
• Without regularization: holding a joint pose needs about 1.9 N of external force.
• With regularization: the required force varies with height/joint angle, naturally guiding the operator away from singular configurations.

软件架构Software architecture

软件栈基于 Python,利用 DYNAMIXEL SDK 读取各关节编码器值,通过 ZMQ 消息传递将关节角度流式发送给机器人控制器,实现直接关节空间控制,无需 IK 求解。设计高度模块化,支持快速扩展至新机器人平台。The software stack is built on Python: the DYNAMIXEL SDK reads the encoder value of each joint, and joint angles are streamed to the robot controller over ZMQ messaging, giving direct joint-space control without IK solving. The design is highly modular and extends quickly to new robot platforms.

03 实验Experiments

用户研究共招募 12 名无专业遥操作训练经历的大学志愿者,使用双臂 UR5 系统对 GELLO、VR(Meta Quest 2)、3D Mice(SpaceMouse)三种设备进行对比。每位参与者先进行 6 分钟通用介绍,再对每种设备进行 5 分钟练习,然后执行 5 项任务,设备顺序随机化(6 种排列各重复 2 次)。The user study recruited 12 university volunteers without professional teleoperation training and compared three devices — GELLO, VR (Meta Quest 2) and 3D Mice (SpaceMouse) — on a bimanual UR5 system. Each participant first received a 6-minute general introduction, then practiced 5 minutes with each device and performed 5 tasks, with the device order randomized (each of the 6 permutations repeated 2 times).

五项测试任务The five test tasks

任务成功率对比(Table II)Task success rate comparison (Table II)

设备DeviceHatMaskBananaTowelUSB平均Average
GELLO0.920.921.00.920.830.92
VR (Meta Quest 2)0.920.830.750.580.500.72
3D Mice (SpaceMouse)0.750.580.670.580.580.63

失败模式分析(Table III)Failure mode analysis (Table III)

失败原因Failure causeGELLO3D MiceVR
超时 (Timeout)Timeout195
自碰撞 (Self Collision)Self Collision155
环境碰撞 (Env Collision)Env Collision366
其他 (Other)Other21

GELLO 超时次数仅为 1 次(SpaceMouse 为 9 次),说明运动等效设计显著降低了操作难度。论文指出:"self collisions were a common problem for spacemouse and VR controllers due to teleoperator paying attention only to end effector",而 GELLO 操作者能感知整臂构型,有效规避自碰撞。GELLO timed out only 1 time (against 9 times for the SpaceMouse), showing that the kinematically equivalent design substantially lowers the difficulty of teleoperation. The paper notes: "self collisions were a common problem for spacemouse and VR controllers due to teleoperator paying attention only to end effector", whereas GELLO operators perceive the whole-arm configuration and avoid self-collisions effectively.

用户研究场景
图 4:用户研究场景。左图展示双臂 UR5 系统配置及 GELLO 操作姿态;右图展示用户研究中的五项任务(帽子、眼罩、香蕉、毛巾、USB)。Figure 4:The user study setting. Left: the bimanual UR5 system configuration and the GELLO operating posture; right: the five tasks of the user study (hat, eye mask, banana, towel, USB).
任务完成时间对比
图 5:各任务成功完成时间的箱线图对比(仅统计成功试次)。GELLO 在全部五项任务上均展现更短的中位完成时间,说明其操作效率更高。Figure 5:Box-plot comparison of the successful completion time per task (successful trials only). GELLO shows a shorter median completion time on all five tasks, indicating higher teleoperation efficiency.

系统能力展示System capability demonstrations

GELLO 成功完成了多类复杂任务:接触丰富型任务(插 USB、折叠毛巾)、长时程任务(倒水至水瓶,需承担较大负载)、双臂协作任务,以及 7-DOF 臂(Franka Panda、xArm)的零空间管理操作,验证了框架的通用性。GELLO successfully completed many kinds of complex tasks: contact-rich tasks (plugging in USB, folding a towel), long-horizon tasks (pouring water into a bottle, which carries a heavier load), bimanual coordination tasks, and null-space management on 7-DOF arms (Franka Panda, xArm), validating the generality of the framework.

成本对比Cost comparison

设备Device成本Cost
3D Mouse (SpaceMouse)$150
GELLO(本文)GELLO (ours)$300
VR (Meta Quest 2)$300
Robot-to-robot(UR5)Robot-to-robot (UR5)$30,000
Haptic Device (Omega7)$40,000

04 局限性Limitations

注:以下局限性均为作者在论文 Discussion 章节中明确陈述(stated by the authors)。Note:all limitations below are explicitly stated by the authors in the Discussion section of the paper.
缺乏力反馈(No Force Feedback)No Force Feedback

由于 DYNAMIXEL XL330 舵机输出力矩有限,GELLO 不具备力反馈功能。论文原文指出:"GELLO lacks force feedback due to limited output torque of motors, compromising contact-rich task performance. This sacrifice prioritized affordability and accessibility."
作者表示希望"将来作为可选功能加入力反馈,以支持更高级的使用场景"。
Because the output torque of the DYNAMIXEL XL330 servos is limited, GELLO has no force feedback. The paper states: "GELLO lacks force feedback due to limited output torque of motors, compromising contact-rich task performance. This sacrifice prioritized affordability and accessibility."
The authors say they hope to "add force feedback as an optional feature in the future, to support more advanced use cases".

用户研究的局限性(Limited Practice Time)Limited Practice Time in the User Study

参与用户均为缺乏专业遥操作训练经验的志愿者,且每种设备的练习时间仅为 5 分钟。论文坦承:"User study involved inexperienced users with limited practice time. Extended training could significantly improve proficiency with alternative devices."
换言之,VR 控制器和 SpaceMouse 在更长时间训练后的性能差距可能缩小。
The participants were all volunteers lacking professional teleoperation training, and the practice time per device was only 5 minutes. The paper concedes: "User study involved inexperienced users with limited practice time. Extended training could significantly improve proficiency with alternative devices."
In other words, the performance gap of VR controllers and the SpaceMouse may narrow after longer training.

设计依赖目标机器人运动学(Kinematic-Specific Hardware)Kinematic-Specific Hardware

(作者隐含,inferred from design)每套 GELLO 均针对特定机器人臂运动学定制 3D 打印件,扩展至新平台需重新设计并打印结构件。尽管开源了三款平台(Franka、UR5、xArm)的设计,但快速适配任意新型机器人仍需一定工程工作量。(Implied by the authors, inferred from design) Every GELLO uses 3D-printed parts customized to the kinematics of one specific robot arm, so extending to a new platform requires redesigning and reprinting the structural parts. Although the designs for three platforms (Franka, UR5, xArm) are open-sourced, quickly adapting to an arbitrary new robot still takes some engineering effort.