现代前馈式三维重建方法（如 VGGT）在相机坐标系中预测像素对齐的点图，但这种坐标系选择并非最优。
G3T 提出在直立的重力对齐坐标系中预测点图，利用真实场景中普遍存在的结构先验，
将跨视角的旋转自由度从 3 DoF 降低至 1 DoF（仅剩 yaw 旋转）。
在此基础上，G3T-Long 通过子图拼接的增量式重建流程，显著减少垂直漂移并提升重建精度。Modern feed-forward 3D reconstruction methods (such as VGGT) predict pixel-aligned pointmaps in the camera coordinate frame, but this choice of frame is not optimal. G3T instead predicts pointmaps in an upright, gravity-aligned frame, exploiting structural priors that are ubiquitous in real-world scenes, and reduces the cross-view rotational degrees of freedom from 3 DoF to 1 DoF (only yaw remains). Building on this, G3T-Long uses an incremental reconstruction pipeline based on chunk stitching, markedly reducing vertical drift and improving reconstruction accuracy.
训练：8×A100，约 1 周Training: 8×A100, about 1 week基础模型：VGGT（微调）Base model: VGGT (fine-tuned)评测数据集：7Scenes / NRGBD / ETH3D / TUM RGBDBenchmarks: 7Scenes / NRGBD / ETH3D / TUM RGBDarXiv 论文arXiv paper
当前主流的前馈式三维重建模型（DUSt3R、VGGT 等）在相机坐标系中预测点图，
要求 7 自由度（DoF）的刚体变换来对齐不同视角的点图，导致优化困难、累积误差大。
真实场景中"地面是水平的、墙壁是竖直的"这一结构先验几乎无处不在，
却被相机中心坐标系所忽视。Mainstream feed-forward 3D reconstruction models (DUSt3R, VGGT and others) predict pointmaps in the camera coordinate frame, which requires a 7-degree-of-freedom (DoF) rigid transform to align pointmaps from different viewpoints, making optimization hard and letting errors accumulate. The structural prior that "the floor is horizontal and the walls are vertical" holds almost everywhere in real scenes, yet camera-centric frames ignore it.
"We propose instead to predict pointmaps in upright, gravity-aligned frames that exploit strong
structural cues present in many real-world scenes. Unlike camera-centric frames,
gravity-aligned frames share a common vertical axis across viewpoints,
reducing the rotational degrees of freedom needed to relate pointmaps to one another."
图 1（论文第 2 页）：
左：传统的 VGGT 在相机坐标系中预测点图，不同视角的点图需要 7 DoF 变换（3 旋转 + 3 平移 + 1 缩放）才能对齐。
右：G3T 在重力对齐坐标系中预测点图，不同视角共享同一垂直轴，对齐只需 1 DoF 的 yaw 旋转，极大简化了拼接问题。
底部颜色可视化：蓝色 = 直立方向预测良好，橙色 = 偏转较大。Figure 1 (paper page 2): Left: conventional VGGT predicts pointmaps in the camera frame, so pointmaps from different viewpoints need a 7 DoF transform (3 rotation + 3 translation + 1 scale) to be aligned. Right: G3T predicts pointmaps in a gravity-aligned frame, where different viewpoints share a common vertical axis, so alignment needs only a 1 DoF yaw rotation, which greatly simplifies stitching. Bottom color visualization: blue = upright direction well predicted, orange = large tilt.
论文首页 Teaser：G3T 预测直立、重力对齐的点图（绿色/蓝色表示方向正确），
而 VGGT 在相机坐标系中产生倾斜的点图（橙/红色）。
G3T-Long 基于子图拼接的增量式重建，显著改善了长序列重建的精度与垂直一致性。Paper teaser (first page): G3T predicts upright, gravity-aligned pointmaps (green/blue indicates correct orientation), while VGGT produces tilted pointmaps in the camera frame (orange/red). G3T-Long, built on chunk-stitched incremental reconstruction, markedly improves the accuracy and vertical consistency of long-sequence reconstruction.
02方法Method
G3T 在 VGGT 的基础上进行微调，核心改动是将点图预测头和相机参数预测头适配到重力对齐坐标系，
并引入 GA-Procrustes（重力感知 Procrustes 对齐）算法约束子图拼接中的旋转方向。G3T fine-tunes VGGT; the core changes adapt the pointmap head and the camera-parameter head to a gravity-aligned frame, and introduce GA-Procrustes (gravity-aware Procrustes alignment) to constrain the rotation direction during chunk stitching.
图 2（论文第 3 页）：G3T 模型架构。
左图：相机坐标系（VGGT）与重力对齐坐标系（G3T）的坐标系对比示意。
右图：G3T 在 VGGT 架构上做了两处关键修改：
(1) Point Head 改为输出重力对齐点图 𝒳G；
(2) 原 Camera Head 被替换为两个新头：
Local Camera Head（输出重力到相机的旋转 RG→C 及内参）和
Relative Camera Head（仅输出 1 DoF 的 yaw 旋转及平移 t）。Figure 2 (paper page 3): the G3T architecture. Left: schematic comparison between the camera frame (VGGT) and the gravity-aligned frame (G3T). Right: G3T makes two key modifications to the VGGT architecture: (1) the Point Head now outputs gravity-aligned pointmaps 𝒳G; (2) the original Camera Head is replaced by two new heads: Local Camera Head (outputs the gravity-to-camera rotation RG→C and the intrinsics) and Relative Camera Head (outputs only a 1 DoF yaw rotation and a translation t).
重力对齐点图预测Gravity-aligned pointmap prediction
传统 VGGT 的点图在相机坐标系 C1 中表示，G3T 改为在重力坐标系 G1（第一帧的重力对齐坐标系）中表示。
重力坐标系保持 y 轴朝上（与重力方向对齐），不同视角的点图可共享同一垂直轴。
Point Head 在训练时监督于由 COLMAP model_orientation_aligner 提取的真实重力方向标注。Conventional VGGT expresses pointmaps in the camera frame C1, whereas G3T expresses them in the gravity frame G1 (the gravity-aligned frame of the first view). The gravity frame keeps the y axis pointing up (aligned with gravity), so pointmaps from different viewpoints can share a common vertical axis. The Point Head is supervised during training by ground-truth gravity directions extracted with COLMAP model_orientation_aligner.
对于相机姿态预测：Local Camera Head 预测每帧的 gravity-to-camera 旋转矩阵（将重力坐标系旋转至相机坐标系），
以及相机内参（焦距、主点）。Relative Camera Head 仅预测 1 DoF 的 yaw 角和平移向量，
大幅简化了相对姿态估计问题。For camera pose prediction: the Local Camera Head predicts the per-frame gravity-to-camera rotation matrix (rotating the gravity frame into the camera frame) together with the camera intrinsics (focal length, principal point). The Relative Camera Head predicts only a 1 DoF yaw angle and a translation vector, greatly simplifying relative pose estimation.
标准 Procrustes 对齐需要 7 DoF 变换（包含完整的 3D 旋转）；
由于所有子图均在重力对齐坐标系中，对齐旋转只需绕 y 轴旋转（即 xz 平面上的 2D 旋转），
从而将对齐问题简化为 5 DoF（1 DoF yaw + 3D 平移 + 1 缩放）。Standard Procrustes alignment requires a 7 DoF transform (including a full 3D rotation); since all chunks live in gravity-aligned frames, the alignment rotation only has to be about the y axis (that is, a 2D rotation in the xz plane), which reduces alignment to 5 DoF (1 DoF yaw + 3D translation + 1 scale).
在 G3T-Long 增量重建流程中：将输入长视频序列分成若干 overlapping chunk（子图），
对每个子图独立运行 G3T 获得点图预测，再用 GA-Procrustes 将相邻子图对齐并融合，
最终输出全局一致的三维重建结果。环路闭合检测通过回环检测模块进一步提升全局一致性。In the G3T-Long incremental reconstruction pipeline the long input video sequence is split into several overlapping chunks; G3T is run independently on each chunk to obtain pointmap predictions, and GA-Procrustes then aligns and fuses neighboring chunks, finally producing a globally consistent 3D reconstruction. A loop-closure detection module further improves global consistency.
图 3（论文第 4 页）：GA-Procrustes 算法伪代码（Algorithm 1、2）与增量重建流程。
左侧 Algorithm 1 为标准 Procrustes 对齐，用于 VGGT-Long；
右侧 Algorithm 2（GA-Procrustes）将旋转约束到 y 轴方向，适用于 G3T-Long。
两个算法的差异体现了重力对齐坐标系带来的简化：旋转搜索空间从 SO(3) 降低至 SO(2)。Figure 3 (paper page 4): pseudo-code of the GA-Procrustes algorithm (Algorithms 1 and 2) and the incremental reconstruction pipeline. On the left, Algorithm 1 is standard Procrustes alignment, used by VGGT-Long; on the right, Algorithm 2 (GA-Procrustes) constrains the rotation to the y axis and is used by G3T-Long. The difference between the two algorithms embodies the simplification brought by gravity-aligned frames: the rotation search space shrinks from SO(3) to SO(2).
训练细节Training details
基础模型：VGGT（pretrained checkpoint）微调 40 个 epoch，约 1 周Base model: VGGT (pretrained checkpoint), fine-tuned for 40 epochs, about 1 week
硬件：8× A100 GPU；batch size 每 GPU 2–12 视角，最多 96 张图Hardware: 8× A100 GPUs; batch size of 2–12 views per GPU, at most 96 images
所有评测均在训练集不含的 unseen 数据集上进行（7Scenes、NRGBD、ETH3D、TUM RGBD）All evaluations are run on unseen datasets that are not part of the training set (7Scenes, NRGBD, ETH3D, TUM RGBD)
03实验Experiments
实验分两部分：(1) 评估 G3T 的直立预测精度（camera-to-gravity 旋转估计）；
(2) 评估 G3T-Long 的增量式三维重建质量（姿态误差与结构指标）。
所有测试数据集均为训练时未见过的数据。The experiments have two parts: (1) evaluating the upright prediction accuracy of G3T (camera-to-gravity rotation estimation); (2) evaluating the incremental 3D reconstruction quality of G3T-Long (pose errors and structural metrics). All test datasets are unseen during training.
在 7Scenes、NRGBD、ETH3D 上测试 1/4/8 视角输入时的旋转误差（Rerr，°）和准确率（Racc@5°）。
与 GeoCalib（后处理重力估计）和 G3T-Procrustes（从点图回归）对比。Rotation error (Rerr, °) and accuracy (Racc@5°) on 7Scenes, NRGBD and ETH3D with 1/4/8 input views.
Compared against GeoCalib (post-hoc gravity estimation) and G3T-Procrustes (regression from pointmaps).
数据集Dataset
方法Method
1V Rerr↓
1V Racc@5°↑
4V Rerr↓
4V Racc@5°↑
8V Rerr↓
8V Racc@5°↑
7Scenes
GeoCalib
6.78°
74.80%
6.60°
76.80%
6.56°
77.60%
G3T-Procrustes
2.00°
96.40%
1.87°
97.20%
1.88°
97.20%
G3T Local Head
1.92°
96.80%
1.78°
98.00%
1.78°
98.40%
NRGBD
GeoCalib
2.61°
89.60%
2.28°
92.00%
2.19°
91.60%
G3T-Procrustes
1.47°
93.60%
1.32°
95.60%
1.26°
95.20%
G3T Local Head
1.33°
94.00%
1.21°
95.20%
1.13°
94.00%
ETH3D
GeoCalib
2.24°
97.20%
2.22°
97.60%
2.21°
98.00%
G3T-Procrustes
1.96°
98.00%
1.95°
95.60%
1.85°
96.00%
G3T Local Head
1.62°
98.40%
1.11°
98.80%
1.05°
98.80%
结论（Observation 1）："G3T can estimate high-quality camera-to-gravity rotation estimates,
reducing mean errors by more than half compared to post-hoc gravity alignment with GeoCalib."Conclusion (Observation 1): "G3T can estimate high-quality camera-to-gravity rotation estimates, reducing mean errors by more than half compared to post-hoc gravity alignment with GeoCalib."
表 2：多视角点图结构质量（Table 2）Table 2: structural quality of multi-view pointmaps
对比 VGGT 与 G3T 在三个数据集上的 ACC（精度）↓、COMP（完整度）↓、NC（法向一致性）↑。Comparison of VGGT and G3T on three datasets in terms of ACC (accuracy) ↓, COMP (completeness) ↓ and NC (normal consistency) ↑.
数据集Dataset
模型Model
ACC↓
COMP↓
NC↑
7Scenes
VGGT-P
0.029
0.034
0.796
VGGT-D
0.031
0.032
0.753
G3T-P
0.028
0.032
0.793
G3T-D
0.026
0.029
0.780
NRGBD
VGGT-P
0.024
0.019
0.921
VGGT-D
0.022
0.018
0.913
G3T-P
0.026
0.021
0.907
G3T-D
0.026
0.021
0.900
ETH3D
VGGT-P
0.191
0.191
0.890
VGGT-D
0.209
0.174
0.880
G3T-P
0.188
0.181
0.892
G3T-D
0.194
0.165
0.882
结论（Observation 2）："Pointmaps predicted by G3T have stronger gravity-alignment while retaining
comparable pointmap quality." 结构质量指标与 VGGT 基本持平，说明重力对齐微调未损坏三维重建能力。Conclusion (Observation 2): "Pointmaps predicted by G3T have stronger gravity-alignment while retaining comparable pointmap quality." The structural quality metrics stay essentially on par with VGGT, showing that gravity-aligned fine-tuning does not damage 3D reconstruction ability.
图 4（论文第 6 页）：定性结果对比。
每组左侧为 VGGT 预测的点图（蓝色/绿色渲染，带有明显倾斜），
右侧为 G3T 预测的点图（颜色编码更均匀，整体方向更直立）。
G3T 在各种室内场景（桌子、书架、地板等）均能保持点图直立，
与地面真值（ground truth, GT）更接近。Figure 4 (paper page 6): qualitative comparison. In each group the left side shows pointmaps predicted by VGGT (rendered in blue/green, with visible tilt), and the right side shows pointmaps predicted by G3T (more uniform color coding and a more upright overall orientation). G3T keeps pointmaps upright across a variety of indoor scenes (tables, bookshelves, floors and so on) and stays closer to the ground truth (GT).
表 3 & 4：TUM RGBD 增量式三维重建（G3T-Long vs VGGT-Long）Tables 3 & 4: incremental 3D reconstruction on TUM RGBD (G3T-Long vs VGGT-Long)
论文第 8 页：Tables 3 & 4 详细数值，含 TUM RGBD 10 个序列上的姿态误差（APER、APEt）、
垂直漂移（δy）及结构指标（ACC、COMP、NC）。Paper page 8: detailed numbers for Tables 3 & 4, covering pose errors (APER, APEt), vertical drift (δy) and structural metrics (ACC, COMP, NC) on 10 TUM RGBD sequences.
以下为 Table 3 关键数据（TUM RGBD 各序列，Absolute Pose Error）：Key numbers from Table 3 below (per-sequence results on TUM RGBD, Absolute Pose Error):
方法Method
指标Metric
fr1/desk
fr1/room
fr1/plant
fr2/ps
fr2/ps2
fr2/ps3
VGGT-Long
APER (°)↓
2.31
4.28
2.96
5.92
13.85
15.23
APEt (m)↓
0.025
0.179
0.053
0.444
0.553
0.947
δy (m)↓
0.005
0.029
0.018
0.224
0.358
0.368
G3T-Long
APER (°)↓
1.43
3.50
1.43
3.48
3.51
6.38
APEt (m)↓
0.012
0.178
0.036
0.255
0.235
0.220
δy (m)↓
0.008
0.033
0.016
0.032
0.032
0.029
注：fr1/360 序列 G3T-Long 的 APER（19.31°）略高于 VGGT-Long（16.32°），为本文中未击败基线的情形，原文如实呈现。
除此之外，G3T-Long 在 9/10 个序列上均优于 VGGT-Long，特别是垂直漂移 δy 的改善尤为显著
（fr2/ps3：VGGT-Long δy=0.368m vs G3T-Long δy=0.029m，改善约 12×）。Note: on the fr1/360 sequence the APER of G3T-Long (19.31°) is slightly higher than that of VGGT-Long (16.32°); this is the one case in the paper where the baseline is not beaten, and it is reported faithfully. Otherwise G3T-Long outperforms VGGT-Long on 9 of the 10 sequences, and the improvement in vertical drift δy is especially striking (fr2/ps3: VGGT-Long δy=0.368m vs G3T-Long δy=0.029m, an improvement of roughly 12×).
消融分析（Ablation）Ablation analysis
仅使用 Local Head 或仅使用 Procrustes 均能改善旋转估计，两者联合效果最佳。Using the Local Head alone or Procrustes alone both improve rotation estimation, and combining them works best.
GA-Procrustes 对比标准 Procrustes：在旋转误差上提供了一致的改善，尤其是在长序列重建中。GA-Procrustes versus standard Procrustes: it gives a consistent improvement in rotation error, especially for long-sequence reconstruction.
多视角输入（4V、8V）相比单视角（1V）进一步提升重力估计精度，体现了跨帧信息融合的有效性。Multi-view input (4V, 8V) further improves gravity estimation accuracy over single-view input (1V), demonstrating the effectiveness of cross-frame information fusion.
04局限性Limitations
说明：以下第 1、2 条为论文中明确陈述的限制（stated by authors）；第 3、4 条为从方法设计中推断的限制（inferred from design）。Note: Items 1 and 2 below are limitations explicitly stated by the authors; items 3 and 4 are inferred from the design of the method.
（stated）场景结构先验不明确时预测退化(stated) Predictions degrade when structural cues in the scene are ambiguous
"G3T may not produce good upright-aware predictions in scenes with ambiguous structural cues."
例如，在缺少上下文的情况下，近距离拍摄地板或墙壁时，
G3T 难以正确估计直立方向，产生倾斜的点图（论文 Figure 5 的失败案例）。"G3T may not produce good upright-aware predictions in scenes with ambiguous structural cues." For example, without enough context, close-up shots of floors or walls make it hard for G3T to estimate the upright direction correctly and yield tilted pointmaps (the failure cases in Figure 5 of the paper).
（stated）水平旋转视角导致方向混淆(stated) Horizontally rotated viewpoints cause orientation confusion
对于竖向物体（如橱柜）的水平旋转图像，模型可能产生方向错误的点图。
"G3T can struggle to estimate upright pointmaps from close-up images of floors and walls
if additional unambiguous context is not present."For horizontally rotated images of vertical objects such as cabinets, the model may produce pointmaps with the wrong orientation. "G3T can struggle to estimate upright pointmaps from close-up images of floors and walls if additional unambiguous context is not present."
（inferred）依赖重力先验，不适用于非重力场景(inferred) Relies on a gravity prior, so it does not apply to gravity-free settings
G3T 的整个设计假设重力是场景中的主要结构先验。
在室外无参考平面场景、航拍图像、或水下/太空环境中，重力对齐假设可能不成立，
此时 G3T 的优势将不适用，可能退化为普通坐标系预测。The whole design of G3T assumes that gravity is the dominant structural prior of the scene. In outdoor scenes without reference planes, in aerial imagery, or in underwater and space environments, the gravity-alignment assumption may not hold; the advantages of G3T then no longer apply and it may degrade to ordinary frame prediction.
（inferred）训练数据需额外重力标注，增加数据准备成本(inferred) Training data needs extra gravity annotations, raising data preparation cost
训练 G3T 需要对所有点图数据集使用 COLMAP 的 model_orientation_aligner 提取重力方向真值，
这对没有预计算 COLMAP 重建的数据集来说增加了数据准备成本，
可能限制训练数据的规模和多样性。Training G3T requires extracting ground-truth gravity directions for every pointmap dataset with COLMAP's model_orientation_aligner, which adds data preparation cost for datasets without precomputed COLMAP reconstructions and may limit the scale and diversity of the training data.
论文第 7 页：Tables 1 & 2 的完整数据，以及论文对两个主要观察结论的文字阐述。
可见 G3T 在重力旋转估计上全面超越 GeoCalib，而结构质量指标（ACC/COMP/NC）与 VGGT 保持在同等水平。Paper page 7: the complete data of Tables 1 & 2, together with the paper's written discussion of its two main observations. G3T clearly surpasses GeoCalib on gravity rotation estimation, while its structural quality metrics (ACC/COMP/NC) stay on par with VGGT.
