ComPose: 统一补全-姿态框架用于鲁棒类别级物体姿态估计

01 动机Motivation

类别级物体姿态估计旨在预测特定类别内任意物体的 6D 姿态与 3D 尺寸，无需实例级 CAD 模型。Category-level object pose estimation aims to predict the 6D pose and 3D size of any object within a given category, without instance-level CAD models. 现有方法的核心瓶颈：观测点云的固有不完整性——深度相机因自遮挡只能捕获物体正面，导致网络无法感知完整形状。The core bottleneck of existing methods is the inherent incompleteness of the observed point cloud — because of self-occlusion, a depth camera captures only the front surface of an object, leaving the network unable to perceive the complete shape.

**图1：三种几何表示策略对比。Figure 1: Comparison of three geometric representation strategies.** (a) 经典方法直接从残缺点云提取几何特征，无法感知完整物体结构；(a) Classical methods extract geometric features directly from the partial point cloud and cannot perceive the complete object structure; (b) Prior-based 方法借助类别形状先验（SPD [31]）在特征层面补充形状信息，但仍在不完整几何上操作；(b) Prior-based methods use a category shape prior (SPD [31]) to supplement shape information at the feature level, but still operate on incomplete geometry; (c) **本文方法（ComPose）**显式集成形状补全，在观测空间恢复完整几何，实现更全面鲁棒的姿态推断。(c) **our method (ComPose)** explicitly integrates shape completion and recovers the complete geometry in the observation space, enabling more comprehensive and robust pose inference.

"how can we effectively and efficiently integrate the complete geometric cues recovered from point cloud completion to enhance object pose estimation?"

作者通过 oracle 实验量化了完整几何的上界价值：将 AG-Pose（当前最优 depth-only 方法）的输入替换为 ground-truth 完整点云（网络架构不变），10°2cm 精度从 68.5% 跃升至 91.7%，增幅高达 23.2 个百分点。The authors quantify the upper-bound value of complete geometry with an oracle experiment: replacing the input of AG-Pose (the current best depth-only method) with the ground-truth complete point cloud (network architecture unchanged) lifts 10°2cm accuracy from 68.5% to 91.7%, a gain of as much as 23.2 percentage points. 但朴素的"先补全再估计"两阶段流水线仅能达到 71.0%，且推理速度从 33.5 FPS 骤降至 21.5 FPS——说明简单级联补全与估计网络无法充分挖掘完整几何的潜力。A naive two-stage "complete first, then estimate" pipeline, however, reaches only 71.0% and its inference speed drops sharply from 33.5 FPS to 21.5 FPS — showing that simply cascading a completion network with an estimation network cannot fully exploit the potential of complete geometry.

精度与推理速度对比散点图 — **图2：depth-only 各版本精度与推理速度对比。Figure 2: Accuracy versus inference speed for the depth-only variants.** 虚线圆圈为使用 ground-truth 完整点云的性能上界（91.7%, 33.5 FPS）。The dashed circle marks the performance upper bound obtained with ground-truth complete point clouds (91.7%, 33.5 FPS). ComPose 以 **77.8% 精度、38.4 FPS 速度**同时超越两阶段方案（71.0%, 21.5 FPS）和原始 AG-Pose（68.5%, 33.5 FPS），实现"高精度且高速度"的最优平衡。With **77.8% accuracy at 38.4 FPS**, ComPose surpasses both the two-stage pipeline (71.0%, 21.5 FPS) and the original AG-Pose (68.5%, 33.5 FPS), striking the best "high accuracy and high speed" balance.

+9.3%10°2cm vs AG-Pose
(depth-only, REAL275)

38.4 FPS推理速度
RTX3090TiInference speed
RTX3090Ti

91.7%Oracle 上界
完整点云输入Oracle upper bound
complete point cloud input

无先验Prior-free不依赖
类别形状先验No reliance on
category shape priors

02 方法Method

ComPose 由四个模块串联构成：(1) 残缺特征提取，(2) 基于关键点的渐进补全，(3) 几何关系编码，(4) 基于对应关系的姿态估计。ComPose is a cascade of four modules: (1) partial feature extraction, (2) keypoint-based progressive completion, (3) geometric relation encoding, and (4) correspondence-based pose estimation. 输入为残缺点云 P^part（和可选 RGB 图像），输出 6D 旋转 R ∈ SO(3)、平移 t ∈ ℝ³、尺寸 s ∈ ℝ³。The input is a partial point cloud P^part (plus an optional RGB image); the outputs are a 6D rotation R ∈ SO(3), a translation t ∈ ℝ³ and a size s ∈ ℝ³.

**图3：ComPose 框架总览。Figure 3: Overview of the ComPose framework.** (a) 框架整体流程，支持 RGB-D 与 depth-only 双模式。(a) Overall pipeline, supporting both RGB-D and depth-only modes. (b) 关键点渐进补全：从可见候选 C^vis 和缺失候选 C^miss 自适应选取粗关键点，经 Transformer decoder 与残缺特征交互，渐进恢复精细关键点 P^kpt 和稠密完整点云 P^com。(b) Keypoint-based progressive completion: coarse keypoints are adaptively selected from the visible candidates C^vis and the missing candidates C^miss, then interact with the partial features through a Transformer decoder to progressively recover refined keypoints P^kpt and a dense complete point cloud P^com. (c) 几何关系建模：增强关键点特征后预测 NOCS 坐标，并通过成对几何关系一致性约束保证结构对齐。(c) Geometric relation modeling: the keypoint features are enhanced before predicting NOCS coordinates, and a pairwise geometric relation consistency constraint enforces structural alignment.

① 残缺特征提取① Partial feature extraction

对残缺点云 P^part 使用 PointNet++ 提取逐点几何特征 F^pn。PointNet++ extracts per-point geometric features F^pn from the partial point cloud P^part. 在 RGB-D 模式下，还引入 DINOv2 提取姿态一致的语义特征 F^dino，与几何特征拼接并投影到 D 维。In RGB-D mode, DINOv2 additionally extracts pose-consistent semantic features F^dino, which are concatenated with the geometric features and projected to D dimensions. 随后通过 Self-Attention (SA) 层捕获全局上下文，得到残缺表示 F^part。A Self-Attention (SA) layer then captures the global context, yielding the partial representation F^part.

② 关键点渐进补全② Keypoint-based progressive completion

粗关键点生成：全局 max pooling 得到 f^global，MLP 预测缺失关键点候选 C^miss；FPS 采样可见关键点候选 C^vis；二者合并后由 scoring MLP 自适应选出 N^kpt=64 个代表关键点 C^kpt。Coarse keypoint generation: global max pooling gives f^global, from which an MLP predicts the missing keypoint candidates C^miss; FPS samples the visible keypoint candidates C^vis; after merging the two sets, a scoring MLP adaptively selects N^kpt=64 representative keypoints C^kpt.
渐进精化：以 f^global+PE(C^kpt) 构造查询 Q^kpt，通过 Cross-Attention + Self-Attention 与 F^part 交互，输出精化关键点坐标 P^kpt 和稠密完整点云 P^com（N^com=1024 点）。Progressive refinement: queries Q^kpt are built from f^global+PE(C^kpt) and interact with F^part through Cross-Attention + Self-Attention, producing refined keypoint coordinates P^kpt and a dense complete point cloud P^com (N^com=1024 points). 训练时以 Chamfer Distance 对 {C^miss, P^kpt, P^com} 分别监督。During training, Chamfer Distance supervises {C^miss, P^kpt, P^com} separately.

③ 几何关系编码③ Geometric relation encoding

对每个关键点 P^kpt_n，从 P^part 检索 N^knn 个近邻点及特征，计算：For each keypoint P^kpt_n, N^knn nearest-neighbour points and their features are retrieved from P^part to compute:
• 局部关系嵌入Local relation embedding E^l_n = MLP(P^kpt_n − P^knn_n)
• 全局关系嵌入Global relation embedding E^g_n = MLP(P^kpt_n − P^kpt)
交替通过 Cross-Attention 和 AvgPool 增强关键点特征，得到几何感知表示 F^geo。Cross-Attention and AvgPool are applied alternately to enhance the keypoint features, giving the geometry-aware representation F^geo.

④ 几何关系一致性约束④ Geometric relation consistency constraint

经典 point-to-point 损失无法捕获全局结构——两组 NOCS 坐标可能逐点误差相近却整体形状迥异。The classical point-to-point loss cannot capture global structure — two sets of NOCS coordinates may have similar per-point errors yet very different overall shapes. 本文提出几何关系一致性损失：计算关键点缩放坐标 P^kpt/‖s^gt‖₂ 的成对 L₂ 距离矩阵 G^kpt，以及预测 NOCS 坐标的对应矩阵 G^nocs，强制二者一致：The paper proposes a geometric relation consistency loss: it computes the pairwise L₂ distance matrix G^kpt of the scaled keypoint coordinates P^kpt/‖s^gt‖₂ together with the corresponding matrix G^nocs of the predicted NOCS coordinates, and forces the two to agree:
L^geo = (1/N²_kpt) Σ_n,m (G^kpt_n,m − G^nocs_n,m)²
总损失：L^all = λ^comL^com + λ^scoreL^score + λ^corrL^corr + λ^geoL^geo（λ^com=15, λ^score=1, λ^corr=2, λ^geo=1）Total loss: L^all = λ^comL^com + λ^scoreL^score + λ^corrL^corr + λ^geoL^geo (λ^com=15, λ^score=1, λ^corr=2, λ^geo=1)

关键点渐进补全可视化Visualization of keypoint-based progressive completion

03 实验Experiments

在三个基准数据集上评估：CAMERA25（27.5万合成图像，6类）、REAL275（真实世界，4.3K训练/2.75K测试，6类）、HouseCat6D（20K训练/3K测试，10类，含严重遮挡）。Evaluated on three benchmarks: CAMERA25 (275000 synthetic images, 6 categories), REAL275 (real world, 4.3K training / 2.75K test, 6 categories) and HouseCat6D (20K training / 3K test, 10 categories, with severe occlusion). 评估指标：n°m cm 精度（旋转误差 <n°且平移误差 <m cm 的预测比例）与 IoU_x 3D 尺寸精度。Metrics: n°m cm accuracy (the proportion of predictions whose rotation error is <n° and whose translation error is <m cm) and the IoU_x 3D size accuracy. 实例分割掩码与 AG-Pose 相同（Mask R-CNN）。The instance segmentation masks are the same as in AG-Pose (Mask R-CNN).

REAL275 数据集对比（depth-only setting）Comparison on REAL275 (depth-only setting)

方法Method	形状先验Shape prior	IoU₅₀	IoU₇₅	5°2cm	5°5cm	10°2cm	10°5cm
SAR-Net [11]	✓	79.3	62.4	31.6	42.3	50.3	68.3
RBP-Pose [43]	✓	—	67.8	38.2	48.1	63.1	79.2
DR-Pose [45]	✓	78.9	68.2	41.7	46.0	67.7	76.3
GPV-Pose [5]	✗	—	64.4	32.0	42.9	—	73.3
HS-Pose [44]	✗	82.1	74.7	46.5	55.2	68.6	82.7
Query6DoF [37]	✗	82.5	76.1	49.0	58.9	68.7	83.0
AG-Pose* [15]	✗	83.2	75.6	48.8	58.8	68.5	80.8
ComPose（本文）ComPose (ours)	✗	82.1	77.0	55.6	61.3	77.8	85.0

REAL275 数据集对比（RGB-D setting）Comparison on REAL275 (RGB-D setting)

方法Method	形状先验Shape prior	IoU₅₀	IoU₇₅	5°2cm	5°5cm	10°2cm	10°5cm
SPD [31]	✓	77.3	53.2	19.3	21.4	43.2	54.1
GCE-Pose [10]	✓	84.1	79.8	57.0	65.1	75.6	86.3
AG-Pose [15]	✗	84.1	80.1	57.0	64.6	75.1	84.7
SpotPose [29]	✗	84.1	81.2	59.7	64.8	81.5	88.2
CleanPose [16]	✗	—	—	61.5	67.4	78.3	86.2
ComPose（本文）ComPose (ours)	✗	84.0	81.4	62.1	68.0	81.8	89.2

HouseCat6D 数据集对比Comparison on HouseCat6D

方法Method	Setting	IoU₂₅	IoU₅₀	5°2cm	5°5cm	10°2cm	10°5cm
AG-Pose* [15]	D-only	81.4	59.9	9.7	10.6	25.9	29.7
ComPose	D-only	81.6	65.1	11.8	12.7	34.8	38.9
GCE-Pose [10]	RGB-D	—	79.2	24.8	25.7	55.4	58.4
AG-Pose [15]	RGB-D	88.1	76.9	21.3	22.1	51.3	54.3
ComPose	RGB-D	90.3	80.6	25.8	27.6	57.8	61.5

定性对比：ComPose vs AG-Pose — **图5：ComPose 与 AG-Pose 的定性对比（REAL275）。Figure 5: Qualitative comparison between ComPose and AG-Pose (REAL275).** 红色为预测结果，绿色为 GT。ComPose 受益于对完整物体几何的更好理解，在各类物体上均取得更准确、更稳定的姿态预测。Red denotes the prediction and green the ground truth. Benefiting from a better understanding of the complete object geometry, ComPose obtains more accurate and more stable pose predictions on every object category.

消融实验Ablation studies

形状补全策略消融Ablation on the shape completion strategy

将完整形状恢复替换为 AG-Pose 的局部实例重建（仅可见区域），5°2cm 下降 6%（55.6→49.6），证明完整几何对精确姿态估计的关键作用。Replacing complete shape recovery with the partial instance reconstruction of AG-Pose (visible region only) lowers 5°2cm by 6% (55.6→49.6), which proves how critical complete geometry is for accurate pose estimation. 去除稠密完整形状 P^com，10°5cm 下降 1.7%（85.0→83.3）。Removing the dense complete shape P^com lowers 10°5cm by 1.7% (85.0→83.3).

几何关系建模消融Ablation on geometric relation modeling

无编码、无一致性约束的基线：5°2cm 仅 49.5%。Baseline without encoding and without the consistency constraint: 5°2cm is only 49.5%. 加入几何关系编码 +4.3%（→53.8%），再加入几何关系一致性约束 +1.8%（→55.6%），两者均显著有益。Adding geometric relation encoding gives +4.3% (→53.8%), and further adding the geometric relation consistency constraint gives +1.8% (→55.6%); both are clearly beneficial.

形状补全质量（REAL275 camera 类）Shape completion quality (REAL275 camera category)

方法Method	设置Setting	形状先验Shape prior	CD^unit ↓	CD ↓
SPD [31]	RGB-D	✓	8.89	—
SGPA [1]	RGB-D	✓	5.51	—
DR-Pose [45]	D	✓	5.26	—
ComPose	RGB-D	✗	4.20	0.17
ComPose	D	✗	6.09	0.23

Chamfer Distance (×10⁻³)。ComPose 在无形状先验条件下于观测空间直接重建完整形状，RGB-D 版本 CD^unit=4.20，优于所有使用形状先验的 canonical space 重建方法。Chamfer Distance (×10⁻³). ComPose reconstructs the complete shape directly in the observation space without any shape prior; its RGB-D version reaches CD^unit=4.20, better than all canonical-space reconstruction methods that rely on shape priors.

遮挡鲁棒性测试（REAL275 + 25% 遮挡增强）Occlusion robustness test (REAL275 + 25% occlusion augmentation)

方法Method	是否加遮挡Occlusion added	5°2cm	5°5cm	10°2cm	10°5cm
AG-Pose* [15]	×	48.8	58.8	68.5	80.8
AG-Pose* [15]	✓	37.1	49.1	54.3	72.6
AG-Pose 下降幅度AG-Pose relative drop		24.0%	16.5%	20.7%	10.1%
ComPose	×	55.6	61.3	77.8	85.0
ComPose	✓	42.7	53.6	62.9	77.7
ComPose 下降幅度ComPose relative drop		23.2%	12.6%	19.2%	8.6%

在 5°5cm 指标上，AG-Pose 精度下降 16.5%，而 ComPose 仅下降 12.6%，表明完整形状补全带来了更强的遮挡鲁棒性。On the 5°5cm metric the accuracy of AG-Pose drops by 16.5% while ComPose drops by only 12.6%, indicating that complete shape completion brings stronger occlusion robustness.

04 局限性Limitations

Note：论文中无独立 Limitations 章节。以下各点均为从设计和实验中推断（inferred from the design），非作者明确陈述。Note: the paper has no separate Limitations section. All points below are inferred from the design and the experiments, and are not stated explicitly by the authors.

仍依赖实例分割掩码Still depends on instance segmentation masks

ComPose 的输入为预先经过 Mask R-CNN 分割的单实例残缺点云。The input to ComPose is a single-instance partial point cloud that has been segmented in advance by Mask R-CNN. 因此，分割质量（掩码精度、类别召回）直接影响最终姿态精度，对复杂遮挡或多实例密集场景的鲁棒性仍受制于上游分割模块。Segmentation quality (mask precision, category recall) therefore directly affects the final pose accuracy, and robustness under complex occlusion or in densely multi-instance scenes remains constrained by the upstream segmentation module. 尽管作者指出补全模块可过滤不准确分割引入的离群点，但根本问题未被消除。Although the authors point out that the completion module can filter out the outliers introduced by inaccurate segmentation, the underlying problem is not removed.

完整形状上界与实际精度之间仍有差距A gap remains between the complete-shape upper bound and the actual accuracy

Oracle 实验表明完整点云输入可将 10°2cm 提升至 91.7%，而 ComPose 实际仅达到 77.8%，差距约 14 个百分点。The oracle experiment shows that complete point cloud input can raise 10°2cm to 91.7%, whereas ComPose actually reaches only 77.8%, a gap of about 14 percentage points. 这意味着补全质量（尤其是高遮挡情形下）仍是瓶颈，进一步提升补全精度有望带来可观的收益空间。This means that completion quality (especially under heavy occlusion) is still the bottleneck, and further improving completion accuracy promises considerable room for gains.

在 observation space 补全，训练需要 CAD 模型变换Completion in observation space requires CAD model transformation during training

为监督观测空间的形状补全（L^com），训练时需要将 CAD 模型 M^cad 通过 ground-truth 姿态 {R^gt, t^gt, s^gt} 变换到观测空间作为监督信号。To supervise shape completion in the observation space (L^com), training requires transforming the CAD model M^cad into the observation space with the ground-truth pose {R^gt, t^gt, s^gt} to serve as the supervision signal. 这要求训练数据集提供完整的 CAD 模型和精确的 GT 姿态标注，限制了在纯真实数据（无 CAD 模型）上的训练泛化性。This requires the training dataset to provide complete CAD models and accurate GT pose annotations, which limits the training generality on purely real data (where no CAD model is available).

仅在 6 / 10 个室内物体类别上验证Validated only on 6 / 10 indoor object categories

实验仅覆盖 REAL275（6 类）和 HouseCat6D（10 类）等室内桌面物体，均为相对规则的刚体。The experiments cover only indoor tabletop objects such as REAL275 (6 categories) and HouseCat6D (10 categories), all of which are relatively regular rigid bodies. 对于形状更不规则、类内变化极大（如衣物、食物）或铰接物体，方法的泛化能力尚未被验证。For objects with more irregular shapes and extremely large intra-category variation (such as clothing or food), or for articulated objects, the generalization ability of the method has not yet been verified.