3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object-relation-object graphs, providing a compact relational abstraction for embodied spatial understanding. In embodied applications, the same scene may be observed from viewpoints that differ by yaw rotations. However, current 3DSGG models often fail to produce relation predictions that follow the expected transformation behavior under such viewpoint shifts. We attribute this limitation to predicate-level transformation heterogeneity: directional predicates such as left, front, right, and behind should transform with the observation frame, whereas most contact, support, and semantic predicates such as standing on and attached to should remain stable. To address this conflict, we propose Transformation-Aware Decoupling (TAD), a viewpoint-robust 3DSGG framework that decouples relation reasoning according to predicate transformation behavior and is supported by viewpoint-stable object representations. TAD decomposes relation reasoning into two parts: one learns cues that should stay stable across viewpoints, while the other learns directional cues that should change with the observation frame. The two parts are then merged for standard multi-label predicate prediction. Transformation-specific descriptors and group-aware auxiliary supervision encourage the two branches to capture complementary relation cues. Extensive experiments on 3DSSG show that TAD achieves state-of-the-art robustness under yaw viewpoint changes without training-time rotation augmentation, while maintaining competitive performance under the standard benchmark.
Not all relations should respond to rotation in the same way. Under yaw viewpoint changes, directional predicates such as left, right, front, and behind should change with the observation frame, whereas most support, contact, and semantic predicates should remain stable.
TAD addresses the conflict between relation types with different transformation behavior under yaw rotation.
TAD combines viewpoint-stable object representations with decoupled relation reasoning for invariant and direction-sensitive predicates.
The framework separates relation reasoning through transformation-specific descriptors, independent branches, and group-aware training objectives.
A rotation-stable object encoder provides reliable object representations for downstream relation reasoning.
Invariant and directional predicates are processed with non-shared relation branches instead of one entangled relation space.
The two relation factors are merged to preserve the original 26-class multi-label predicate prediction setting.
Each animation pairs a continuously rotating input scene with the corresponding predicted 3D scene graph throughout the viewpoint change.
The input 3D scene rotates continuously, while the scene graph below is updated in sync with the current observation frame.
A second rotating scene with its graph prediction shown throughout the motion, illustrating the viewpoint-dependent evolution of relation edges.
Strong canonical performance, stable rotated-view predictions.
Main takeaway
The model maintains high directional and invariant relation accuracy when the observation frame undergoes the challenging 90° axis exchange.
Overall R@50
86.0 → 84.5
Directional mR@50
87.9 → 86.8
| Method | 0° Overall R@50 | 90° Overall R@50 | 0° → 90° Change | 90° Directional mR@50 | 90° Invariant mR@50 |
|---|---|---|---|---|---|
| VL-SAT | 79.9 | 68.2 | −11.7 | 67.8 | 51.1 |
| OCRL | 82.0 | 69.9 | −12.1 | 67.4 | 53.1 |
| TAD (Ours) | 86.0 | 84.5 | −1.5 | 86.8 | 63.3 |
| Configuration | Triplet R@50 | SGCls mR@50 | PredCls R@50 | PredCls mR@50 | 90° R@50 | 90° mR@50 |
|---|---|---|---|---|---|---|
| Baseline | 89.83 | 23.6 | 79.2 | 57.8 | 59.8 | 36.48 |
| + VSOE | 90.37 | 32.1 | 83.6 | 63.6 | 68.4 | 51.5 |
| + VSOE + TAD | 90.95 | 33.1 | 85.0 | 64.3 | 83.3 | 62.7 |
| + VSOE + TAD + TSD (Ours) | 91.03 | 34.6 | 86.0 | 66.7 | 84.5 | 66.9 |
@misc{tad_3dsgg_2026,
title = {Not All Relations Rotate Alike: Transformation-Aware Decoupling for Viewpoint-Robust 3D Scene Graph Generation},
author = {Sun, Jingjun and Wang, Chaowei and Liu, Zhirui and Tian, Jiaxu and Yang, Ming and Wang, Yaoxing and Gao, Shan},
year = {2026},
note = {Under review}
}