🫧 Learn2Splat: Learned Optimizer for 3D Gaussian Splatting

Learn2Splat is a meta-learned optimizer for 3D Gaussian Splatting (3DGS) that replaces hand-designed optimizers (e.g., Adam/SGD) with a learned update rule.

It improves early convergence speed while remaining stable over long optimization horizons without requiring learning-rate schedules or time-step conditioning.

---

🌐 Links

• Project page: https://autonomousvision.github.io/learn2splat/
• Paper: https://arxiv.org/abs/2605.15760
• Code: https://github.com/autonomousvision/learn2splat
• Hugging Face: https://huggingface.co/autonomousvision/learn2splat

---

⚙️ Overview

Learn2Splat learns to optimize Gaussian scene representations by directly predicting structured parameter updates.

Key properties:

• Learned optimizer for 3D Gaussian Splatting
• Faster early convergence compared to standard optimizers
• Stable long-horizon optimization without LR schedules
• Zero-shot generalization to unseen scenes and resolutions

The model is trained across many scenes and applied without fine-tuning at test time.

---

📦 Checkpoints

Checkpoint	Contents	Use it for
joint/	Learn2Splat, one optimizer for sparse and dense views	reproducing the paper (antialiased, eps2d=0.3)
joint_resplat_v2/	Learn2Splat trained with the ReSplat-Base initializer (optimizer weights only)	comparing with ReSplat (antialiased, eps2d=0.1)
resplat_init/	Pre-release ReSplat, the initializer joint was trained with	sparse-view initialization for joint

joint_resplat_v2 uses ReSplat-Base from ReSplat (Xu et al., 2025), loaded from haofeixu/resplat.

See MODEL_ZOO.md and the code README for usage.

---

📊 Results of joint_resplat_v2

Evaluated with ReSplat's renderer settings (antialiased, eps2d=0.1), so these numbers are not comparable with the paper table (eps2d=0.3).

Sparse views, ReSplat-Base initialization. All methods start from the same ReSplat initialization. PSNR / SSIM / LPIPS at Adam's 4000 steps runtime:

Benchmark	Init (of all methods)	Adam	Adam (tuned)	joint_resplat_v2
DL3DV, 8 views, 256x448	27.35	30.30 / 0.916 / 0.093	30.51 / 0.920 / 0.088	30.79 / 0.922 / 0.084
DL3DV, 8 views, 512x960¹	26.14	27.84 / 0.866 / 0.169	27.88 / 0.869 / 0.166	28.26 / 0.874 / 0.156
DL3DV, 32 views, 256x448¹	26.31	30.21 / 0.913 / 0.095	30.37 / 0.918 / 0.091	30.61 / 0.918 / 0.089
RE10K, 8 views	21.30	27.58 / 0.873 / 0.192	27.81 / 0.880 / 0.181	27.91 / 0.878 / 0.182

Against ReSplat's own refinement, at the runtime of ReSplat's 4 refinement steps (about 0.5 s), from the same initialization:

Benchmark	ReSplat (4 steps)	joint_resplat_v2
DL3DV, 8 views, 512x960¹	27.65 / 0.866 / 0.161	27.78 / 0.871 / 0.165
DL3DV, 32 views, 256x448¹	28.31 / 0.889 / 0.114	28.55 / 0.901 / 0.114

Dense views, SfM initialization. All methods start from the same SfM points. PSNR / SSIM / LPIPS at Adam's 4000 steps runtime:

Benchmark	Adam	Adam (tuned)	joint (paper)	joint_resplat_v2
DL3DV	27.78 / 0.880 / 0.183	26.94 / 0.860 / 0.213	28.96 / 0.895 / 0.157	28.95 / 0.895 / 0.157
DTU	28.91 / 0.899 / 0.309	29.32 / 0.892 / 0.328	29.30 / 0.900 / 0.307	29.31 / 0.899 / 0.309
LLFF	22.73 / 0.730 / 0.357	20.97 / 0.659 / 0.446	24.61 / 0.768 / 0.300	24.63 / 0.771 / 0.299
Mip-NeRF 360	25.62 / 0.716 / 0.372	24.38 / 0.663 / 0.432	25.73 / 0.714 / 0.368	25.75 / 0.715 / 0.367

¹ Initialized with ReSplat-Base trained for that setting (512x960 8 views, 256x448 32 views).

The evaluation commands are in the code README.

---

📚 Citation

If you find this project useful, please consider citing:

@article{pearl2026learn2splat,
  title   = {Learn2Splat: Extending the Horizon of Learned 3DGS Optimization},
  author  = {Pearl, Naama and Esposito, Stefano and Xu, Haofei and Peleg, Amit and
             Gschoßmann, Patricia and Porzi, Lorenzo and Kontschieder, Peter and
             Pons-Moll, Gerard and Geiger, Andreas},
  journal = {arXiv preprint arXiv:2605.15760},
  year    = {2026}
}