Adversarial Single-Image SVBRDF Estimation
with Hybrid Training
Eurographics 2021

Abstract

In this paper, we propose a deep learning approach for estimating the spatially-varying BRDFs (SVBRDF) from a single image.Most existing deep learning techniques use pixel-wise loss functions which limits the flexibility of the networks in handlingthis highly unconstrained problem. Moreover, since obtaining ground truth SVBRDF parameters is difficult, most methodstypically train their networks on synthetic images and, therefore, do not effectively generalize to real examples. To avoid theselimitations, we propose an adversarial framework to handle this application. Specifically, we estimate the material propertiesusing an encoder-decoder convolutional neural network (CNN) and train it through a series of discriminators that distinguishthe output of the network from ground truth. To address the gap in data distribution of synthetic and real images, we train ournetwork on both synthetic and real examples. Specifically, we propose a strategy to train our network on pairs of real imagesof the same object with different lighting. We demonstrate that our approach is able to handle a variety of cases better than thestate-of-the-art methods.

BibTeX

Acknowledgements

We thank the Eurographics 2021 reviewers for their constructive comments. The website template was borrowed from Michael Gharbi.