Low Light Image Enhancement by Multispectral Fusion of RGB and NIR Images

In low light condition, the captured images suffer from severe noise and loss of textures and details due to low signal-to-noise ratio (SNR). In this paper, we propose low light image enhancement by multispectral fusion of color (RGB) and near infrared (NIR) images. We adopt adaptive weighted total variation to take the multispectral advantages in fusion. Since the NIR imaging depends on the NIR light strength, the NIR values are not reliable out of the range that NIR light can reach. Thus, based on a sigmoid function, we selectively perform total variation regularization according to the NIR values. If the NIR values are reliable, we perform the multispectral fusion with the guidance of the NIR image. If the NIR values are not reliable, i.e. none or too low, we only perform total variation regularization for denoising on the RGB image without NIR gradient transfer. For optimization, we convert the nonconvex total variation regularization into a linear system using iterative reweighted least squares. Experimental results demonstrate that the proposed method successfully transfer the NIR details into the fusion result while removing noise and keeping colors as well as it produces natural looking fusion results even with unreliable NIR values.