SkySplat-OV: Generalizable Language Gaussian Splatting for Open-Vocabulary Scene Understanding from Sparse Satellite Views

Abstract. Open-vocabulary scene understanding aims to recognize arbitrary semantic concepts in 3D beyond a fixed set of categories. Recent methods built upon 3D Gaussian Splatting (3DGS) enable scene understanding by distilling features from vision–language models, but most rely on per-scene optimization and dense observations, which limit scalability. More recent generalizable 3DGS methods show promise, but they perform poorly on sparse-view satellite imagery due to the pushbroom imaging mode of satellites, limited geometric constraints, and extreme scale variations. To address these limitations, we propose SkySplat-OV, a feed-forward framework that integrates the rational polynomial coefficient (RPC) model into a generalizable language 3DGS pipeline. A Geometry–Radiance–Semantics Fusion module, together with monocular relative height supervision and multi-view feature distillation, enables accurate Gaussian prediction and view-consistent segmentation results. Compared to per-scene optimization methods, SkySplat-OV achieves up to $1070\times$ speedup over LangSplatV2 with higher accuracy. It also outperforms generalizable 3DGS baselines on two benchmarks, and demonstrates strong cross-dataset generalization on MVS3D.

SkySplat-OV first extracts deep features from sparse views and constructs a cost volume via RPC warping for regularization and height regression, achieving geometry reconstruction (Sec. 3.1). It then employs a GRS-Fusion module to accurately predict Gaussian parameters (Sec. 3.2). Additionally, monocular relative height and language feature extracted from pre-trained 2D vision foundation models (VFMs) are incorporated as auxiliary supervision, combined with radiance supervision for self-supervised learning (Sec. 3.3). Finally, the distilled language features are fed into the FPN structure and decoder of SAM3 to obtain open-vocabulary segmentation results (Sec. 3.4).