Non-Glare-NeRF

To prevent unwanted glare

Seongjun Choi1   Hyoseok Hwang1+  
1KyungHee University   
+corresponding authors
KSC 2024
Paper Video Code

Abstract

This paper proposes two approaches to solve the depth distortion problem that occurs on glare surfaces in the process of 3D reconstruction based on Neural Radiance Fields (NeRF).

First, by introducing a preprocessing process using the Inpainting technique, distortion of texture and depth information due to Glare is alleviated. Second, a dataset that combines high-precision depth information and rich color information is constructed using the iPhone's LiDAR sensor and RGB camera.

It was confirmed through experiments that the proposed method improves the 3D reconstruction performance of NeRF even in scenes where the Glare phenomenon exists.

Video

Non-Glare-NeRF

Non-Glare-NeRF Pipeline

Pipeline of Non-Glare-NeRF: (a): Extract Instant NGP (NeRF) model results. (b): Extract the result of Instant NGP (NeRF) model with pre-processing. Pre-processing includes the process of detecting glare areas and applying interpainting.


Visualization: Here are examples of HuGS on different scenes (datasets). More results can be found in the paper and the data.

(1) Input RGB
(2) Input Depth
(3) Glare Mask
(4) Inpainting
(5) Result RGB
(6) Result Depth
yoda gt
yoda sfm
yoda sfm mask
yoda residual
yoda res mask
yoda final mask
crab gt
crab sfm
crab sfm mask
crab residual
crab res mask
crab final mask

Rendering Results

Quality Evaluation: Our methodology shows that the Glare region is effectively restored, and we obtain much more natural texture and depth information compared to existing methods.


classroom1
Ground Truth
instant NGP
w/ Non-Glare-NeRF (ours)
classroom2

Qualitative Evaluation: The proposed method showed excellent overall performance and the performance improvement was particularly remarkable in the SqRel and RMSELog indicators. Through this, it was confirmed that the proposed method showed better performance in depth generation reflecting the scale.


qualitative

BibTeX

@article{choi2024nonglarenerf,
  author    = {Choi, Seongjun and Hwang, Hyoseok},
  title     = {Non-Glare-NeRF: To prevent unwanted glare},
  journal   = {KSC},
  year      = {2024},
}