RGBD GS-ICP SLAM (100FPS Gaussian Splatting SLAM)

Seongbo Ha, Jiung Yeon, Hyeonwoo Yu

ECCV 2024

Paper | Video

This repository is intended to substantiate the results reported in the paper. Additional features including visualization tools will be updated soon!

Environments

Install requirements

conda create -n gsicpslam python==3.9
conda activate gsicpslam
conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt

Also, PCL is needed for fast-gicp submodule.

Install submodules

conda activate gsicpslam
pip install submodules/diff-gaussian-rasterization
pip install submodules/simple-knn

cd submodules/fast_gicp
mkdir build
cd build
cmake ..
make
cd ..
python setup.py install --user

Datasets

• Replica

  • Download

    bash download_replica.sh

  • Configure

    Please modify the directory structure to ours.

    The original structure

    Replica
        - room0
            - results (contain rgbd images)
                - frame000000.jpg
                - depth000000.jpg
                ...
            - traj.txt
        ...

    Our structure

    Replica
        - room0
            - images (contain rgb images)
                - frame000000.jpg
                ...
            - depth_images (contain depth images)
                - depth000000.jpg
                ...
            - traj.txt
        ...

• TUM-RGBD

  • Download

        bash download_tum.sh

Run

• Limited to 30 FPS

      # Replica
      bash replica.sh
      # TUM
      bash tum.sh

• Unlimited tracking speed

      # Replica
      bash replica_unlimit.sh
      # TUM
      bash tum_unlimit.sh

Installing SIBR Viewer

cd SIBR_viewers
cmake -Bbuild . -DCMAKE_BUILD_TYPE=Release
cmake --build build -j24 --target install

Real-time demo

Using rerun.io viewer

Rerun viewer shows the means of trackable Gaussians, and rendered image from reconstructed 3dgs map.

python -W ignore gs_icp_slam.py --rerun_viewer

Using SIBR viewer

python -W ignore gs_icp_slam.py --dataset_path dataset/Replica/office0 --verbose

# In other terminal
cd SIBR_viewers
./install/bin/SIBR_remoteGaussian_app --rendering-size 1280 720

Spiking opacity (Phase A, optional)

Mapping can use a learnable global threshold Vth_opa to gate opacities (SpikingGS-style) during RGB/depth optimization, plus spike-aware pruning every 200 iterations. Tracking (GICP) is unchanged; trackable point selection still uses raw sigmoid opacity.

Enable on the Python entry you use (same flags for gs_icp_slam.py and gs_icp_slam_unlimit.py):

python -W ignore gs_icp_slam_unlimit.py --dataset_path dataset/Replica/room0 \
  --config configs/Replica/caminfo.txt --output_path output/room0 --save_results \
  --use_spike \
  --vth_opa_init 0.001 --vth_opa_lr 0.0002 \
  --spike_min_opacity 1e-3 --max_spike_remove_ratio 0.3

• --use_spike: turn on gating + pruning (default off, baseline unchanged).
• --spike_min_opacity: gated opacity below this can be pruned (with cap).
• --max_spike_remove_ratio: max fraction of Gaussians removed in one prune step for the spike-based low-opacity rule (oversized Gaussians are still removed as before).

Docker

Please see the README.md in the docker_files folder.