Global Structure-from-Motion Meets Feedforward Reconstruction

01 动机Motivation

经典 SfM（以 COLMAP 为代表）在标准重建场景表现优异，但在低纹理、有限重叠度、对称结构等场景下系统性失效。前馈神经网络方法（如 π³）在这些困难场景中展现出惊人的鲁棒性，却在可扩展性、精度和鲁棒性上远不及经典方法。本文提出 GlueMap，系统性地整合两类方法的互补优势。Classical SfM (represented by COLMAP) performs excellently on standard reconstruction scenes, but fails systematically under low texture, limited overlap and symmetric structures. Feedforward neural network methods (such as π³) show remarkable robustness in these hard scenes, yet lag far behind classical methods in scalability, accuracy and robustness. This paper proposes GlueMap, which systematically integrates the complementary strengths of the two families.

"We systematically analyze these limitations and propose a new Structure-from-Motion pipeline by combining the respective strengths of classical and feedforward methods."

各方法在5个数据集上的综合排名雷达图 — **图1：5 个数据集上的平均排名雷达图。** 每个轴代表一个评测数据集，轴向外表示排名越高（越好）。经典方法（SIFT、ALIKED+LG）在标准场景好但在困难场景失效；前馈方法（π³）在对称/低纹理场景强但在精度和大规模场景差； **GlueMap（ours）**在所有场景上均保持最优或接近最优。**Figure 1: radar chart of average ranking over the 5 datasets.** Each axis is one evaluation dataset, and points farther out mean a higher (better) ranking. Classical methods (SIFT, ALIKED+LG) do well on standard scenes but fail on hard ones; the feedforward method (π³) is strong on symmetric / low-texture scenes but weak in accuracy and on large-scale scenes; **GlueMap (ours)** stays optimal or near-optimal on every scene.

83.6ETH3D AUC@5
（ALIKED+LG: 67.4）ETH3D AUC@5
(ALIKED+LG: 67.4)

92.4SMERF AUC@20
低重叠室内场景SMERF AUC@20
low-overlap indoor scenes

37.3LaMAR HGE AUC@3
（SIFT: 2.6, π³: OOM）LaMAR HGE AUC@3
(SIFT: 2.6, π³: OOM)

58.7CO3Dv2 40img AUC@3
（π³: 47.1）CO3Dv2 40img AUC@3
(π³: 47.1)

02 方法Method

GlueMap 将整个 SfM 流程分解为四个阶段：首先构建可靠的视图图，然后利用前馈网络进行局部推断，再通过全局运动平均整合局部结果，最后用增强型 Bundle Adjustment 精化位姿与三维结构。GlueMap decomposes the whole SfM pipeline into four stages: it first builds a reliable view graph, then performs local inference with a feedforward network, integrates the local results through global motion averaging, and finally refines poses and 3D structure with an augmented Bundle Adjustment.

GlueMap 四阶段 pipeline 示意图 — **图2：GlueMap pipeline 总览。** 四个阶段依次为：(1) View Graph Initialization 利用 SALAD 图像检索 + Doppelganger++ 过滤构建视图图； (2) Feedforward Local Inference 以每张图为中心构建局部 star graph，用 π³ 推断局部三维结构； (3) Global Motion Averaging 融合局部重建，经由中位焦距、旋转同步、相似变换平均恢复全局位姿； (4) Augmented Bundle Adjustment 联合 SIFT 匹配、deep tracks 与合成 virtual tracks 精化全局结果。**Figure 2: overview of the GlueMap pipeline.** The four stages are: (1) View Graph Initialization builds the view graph with SALAD image retrieval plus Doppelganger++ filtering; (2) Feedforward Local Inference builds a local star graph centered on each image and infers local 3D structure with π³; (3) Global Motion Averaging fuses the local reconstructions and recovers global poses via median focal length, rotation synchronization and similarity averaging; (4) Augmented Bundle Adjustment jointly refines the global result with SIFT matches, deep tracks and synthesized virtual tracks.

Step 1

View Graph Initialization

使用 SALAD 进行可扩展图像检索，通过 Doppelganger++ 过滤掉对称结构带来的虚假对，并用动态阈值保证图的连通性。这一步骤确保视图图既覆盖所有图像，又不受对称陷阱干扰。Scalable image retrieval is performed with SALAD, and Doppelganger++ filters out the spurious pairs caused by symmetric structures, while a dynamic threshold guarantees the connectivity of the graph. This step ensures the view graph covers all images yet stays free of symmetry traps.

Step 2

Feedforward Local Inference

将视图图分解为以每张图为中心的局部 star graph，对每个子图独立调用 π³ 前馈网络获得局部三维重建。随后通过"在 1 像素半径内 snap 到 SIFT 关键点"将不同子图的轨迹对齐，并用前向-后向深度一致性检验剔除误匹配。The view graph is decomposed into local star graphs centered on each image, and the π³ feedforward network is run independently on every subgraph to obtain a local 3D reconstruction. Tracks from different subgraphs are then aligned by "snapping to SIFT keypoints within a 1-pixel radius", and forward-backward depth consistency checks discard wrong matches.

Step 3

Global Motion Averaging

依次执行三步融合：(a) 内参平均：以中位数焦距作为初始化； (b) 旋转平均（rotation synchronization）：最小化测地误差； (c) 相似变换平均（similarity averaging）：统一各局部重建的尺度，恢复全局相机中心。Fusion runs in three steps: (a) intrinsics averaging: the median focal length is used as initialization; (b) rotation averaging (rotation synchronization): minimizes the geodesic error; (c) similarity averaging: unifies the scales of the local reconstructions and recovers the global camera centers.

Step 4

Augmented Bundle Adjustment

引入三类 track 同时参与 BA 优化：经典 SIFT 匹配提供高精度约束； π³ 输出的 deep tracks 在困难场景提供额外覆盖；通过对相邻视图重投影采样像素构造 virtual tracks（虚拟轨迹） 填补稀疏区域的约束空洞。Three kinds of tracks take part in the BA optimization at the same time: classical SIFT matches provide high-accuracy constraints; the deep tracks output by π³ add extra coverage in hard scenes; and virtual tracks, built by reprojecting sampled pixels into neighboring views, fill the constraint gaps in sparse regions.

view graph radius 与重建精度关系分析 — **图3：View Graph Radius 与重建精度的关系。** "radius"衡量图中任意两张图交换信息所需的最少跳数，反映场景的内在重建难度。随着 radius 增大，前馈方法性能急剧下降；而 GlueMap 因引入全局优化和多种 track 类型，对 radius 增大更加鲁棒，且对更高的视图图密度有明显增益。**Figure 3: relation between view graph radius and reconstruction accuracy.** The "radius" measures the minimum number of hops required for any two images in the graph to exchange information, and reflects the intrinsic reconstruction difficulty of a scene. As the radius grows, the performance of feedforward methods drops sharply; GlueMap, thanks to global optimization and multiple track types, is far more robust to a growing radius and gains clearly from a denser view graph.

03 实验Experiments

在 5 个基准数据集（ETH3D、IMC2021、CO3Dv2、SMERF、LaMAR）上与 π³、π³+BA、SIFT、ALIKED+LG 对比，覆盖高精度室内外、互联网照片、低纹理物体、低重叠室内和大规模场景等多样化挑战。Comparison with π³, π³+BA, SIFT and ALIKED+LG on 5 benchmark datasets (ETH3D, IMC2021, CO3Dv2, SMERF, LaMAR), covering diverse challenges: high-accuracy indoor and outdoor scenes, internet photos, low-texture objects, low-overlap indoor scenes and large-scale scenes.

ETH3D — 高精度室内外场景（AUC，越高越好）ETH3D — high-accuracy indoor and outdoor scenes (AUC, higher is better)

方法Method	AUC@1°	AUC@3°	AUC@5°
SIFT	45.6	62.2	66.7
ALIKED+LG	42.9	62.1	67.4
π³	13.2	36.1	48.9
π³ + BA	30.6	55.1	65.1
GlueMap	53.0	76.9	83.6

SMERF — 低重叠室内多房间场景（AUC@20°，越高越好）SMERF — low-overlap indoor multi-room scenes (AUC@20°, higher is better)

方法Method	Minimal Overlap	Low Overlap
SIFT	1.8	1.8
π³	51.7	49.8
GlueMap	82.0	92.4

LaMAR — 大规模室内外场景（6k–9k 张图像，AUC@3°）LaMAR — large-scale indoor and outdoor scenes (6k–9k images, AUC@3°)

场景Scene	图像数# images	SIFT	π³	GlueMap
CAB	6,587	0.6	OOM	4.5
HGE	7,553	2.6	OOM	37.3
LIN	9,319	4.6	OOM	37.3

OOM = Out of Memory（RTX 4090 24GB 显存耗尽）；GlueMap 是唯一能处理大规模场景的方法。OOM = Out of Memory (the 24GB of an RTX 4090 is exhausted); GlueMap is the only method able to handle large-scale scenes.

CO3Dv2 — 低纹理物体中心场景（AUC@3°，40 张图像）CO3Dv2 — low-texture object-centric scenes (AUC@3°, 40 images)

方法Method	10 imgs	20 imgs	40 imgs
SIFT	25.8	35.1	50.0
π³	48.2	46.4	47.1
GlueMap	54.8	56.7	58.7

GlueMap 重建结果（展览大厅） — **GlueMap 重建结果**（LaMAR Exhibition Hall）— 成功恢复完整点云与位姿。**GlueMap reconstruction** (LaMAR Exhibition Hall) — the complete point cloud and poses are successfully recovered.

π³ 重建结果（展览大厅） — **π³ 重建结果**（同一场景）— 因内存溢出或局部失败导致重建不完整。**π³ reconstruction** (same scene) — the reconstruction is incomplete because of out-of-memory or local failures.

消融实验（Augmented BA 组件贡献，ETH3D AUC@1°）Ablation study (contribution of the Augmented BA components, ETH3D AUC@1°)

组合Combination	ETH3D AUC@1°
Deep Tracks + SIFT + Virtual Tracks（完整）Deep Tracks + SIFT + Virtual Tracks (full)	52.6
Deep Tracks + Virtual Tracks（去掉 SIFT）Deep Tracks + Virtual Tracks (w/o SIFT)	45.4
SIFT + Virtual Tracks（去掉 Deep Tracks）SIFT + Virtual Tracks (w/o Deep Tracks)	46.6
Deep Tracks + SIFT（去掉 Virtual Tracks）Deep Tracks + SIFT (w/o Virtual Tracks)	54.8

三类 track 各有贡献，在低重叠场景（SMERF）中 virtual tracks 尤为关键，完整组合在不同场景下取得最佳综合表现。All three kinds of tracks contribute; virtual tracks are especially critical in low-overlap scenes (SMERF), and the full combination gives the best overall performance across scenes.

04 局限性Limitations

Note: 以下局限性由作者在论文中明确陈述（stated）。The limitations below are explicitly stated by the authors in the paper.

依赖前馈模型质量，暂不支持鱼眼相机Depends on the quality of the feedforward model, and fisheye cameras are not yet supported

GlueMap 将 π³ 作为前馈推断骨干，其性能上限受限于 π³ 本身的覆盖范围。当前实现不支持鱼眼（fisheye）图像，这限制了在自动驾驶和机器人领域的直接应用。GlueMap uses π³ as its feedforward inference backbone, so its performance ceiling is bounded by the coverage of π³ itself. The current implementation does not support fisheye images, which limits direct application in autonomous driving and robotics.

纯旋转运动场景不适用Not applicable to purely rotational motion

Augmented Bundle Adjustment 的公式化设计依赖视差信息；对于相机原地旋转（purely rotational motion）的图像序列，该方法无法正确工作。The formulation of Augmented Bundle Adjustment relies on parallax; for image sequences taken under purely rotational motion, the method cannot work correctly.

并非单一端到端架构，需要多组件协同Not a single end-to-end architecture, several components must work together

GlueMap 本质上是多个模块（SALAD、Doppelganger++、π³、SIFT、global averaging）的工程组合，而非单一的可端到端训练模型。随着前馈基础模型（如 MASt3R、π³）持续迭代，组件替换可带来直接性能提升，但也引入了维护成本。GlueMap is essentially an engineering combination of several modules (SALAD, Doppelganger++, π³, SIFT, global averaging) rather than a single end-to-end trainable model. As feedforward foundation models (such as MASt3R and π³) keep improving, swapping in new components brings direct performance gains, but also introduces maintenance cost.