Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss
In
ACM Transactions on Graphics (To be presented at SIGGRAPH), 2020
Abstract
Digital cameras can only capture a limited range of real-world scenes' luminance, producing images with saturated pixels. Existing single image high dynamic range (HDR) reconstruction methods attempt to expand the range of luminance, but are not able to hallucinate plausible textures, producing results with artifacts in the saturated areas. In this paper, we present a novel learning-based approach to reconstruct an HDR image by recovering the saturated pixels of an input LDR image in a visually pleasing way. Previous deep learning-based methods apply the same convolutional filters on well-exposed and saturated pixels, creating ambiguity during training and leading to checkerboard and halo artifacts. To overcome this problem, we propose a feature masking mechanism that reduces the contribution of the features from the saturated areas. Moreover, we adapt the VGG-based perceptual loss function to our application to be able to synthesize visually pleasing textures. Since the number of HDR images for training is limited, we propose to train our system in two stages. Specifically, we first train our system on a large number of images for image inpainting task and then fine-tune it on HDR reconstruction. Since most of the HDR examples contain smooth regions that are simple to reconstruct, we propose a sampling strategy to select challenging training patches during the HDR fine-tuning stage. We demonstrate through experimental results that our approach can reconstruct visually pleasing HDR results, better than the current state of the art on a wide range of scenes.
Downloads
Publication
Supplementals
Acknowledgments
We thank the reviewers for their constructive comments. M. Santos is funded by the Brazilian agency CNPQ grant 161268/2018-8. T. Ren is partially supported by FACEPE grant APQ-0192- 1.03/14. N. Kalantari is in part funded by a TAMU T3 grant 246451. The website template was borrowed from Joey Litalien.
Cite
Marcel Santana Santos, Tsang Ing Ren and Nima Khademi Kalantari. Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss. ACM Transactions on Graphics, 39, 4, Article 1 (July 2020).
@article{Marcel:2020:LDRHDR,
author = {Santos, Marcel Santana and Tsang, Ren and Khademi Kalantari, Nima},
title = {Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss},
journal = {ACM Transactions on Graphics},
volume = {39},
number = {4},
year = {2020},
month = {7},
doi = {10.1145/3386569.3392403}
}