File(s) stored somewhere else

Please note: Linked content is NOT stored on Open Access Te Herenga Waka-Victoria University of Wellington and we can't guarantee its availability, quality, security or accept any liability.

Adaptive Light Estimation using Dynamic Filtering for Diverse Lighting Conditions

journal contribution
posted on 30.11.2021, 03:30 authored by Junhong ZhaoJunhong Zhao, Andrew ChalmersAndrew Chalmers, Taehyun RheeTaehyun Rhee
High dynamic range (HDR) panoramic environment maps are widely used to illuminate virtual objects to blend with real-world scenes. However, in common applications for augmented and mixed-reality (AR/MR), capturing 360° surroundings to obtain an HDR environment map is often not possible using consumer-level devices. We present a novel light estimation method to predict 360° HDR environment maps from a single photograph with a limited field-of-view (FOV). We introduce the Dynamic Lighting network (DLNet), a convolutional neural network that dynamically generates the convolution filters based on the input photograph sample to adaptively learn the lighting cues within each photograph. We propose novel Spherical Multi-Scale Dynamic (SMD) convolutional modules to dynamically generate sample-specific kernels for decoding features in the spherical domain to predict 360° environment maps. Using DLNet and data augmentations with respect to FOV, an exposure multiplier, and color temperature, our model shows the capability of estimating lighting under diverse input variations. Compared with prior work that fixes the network filters once trained, our method maintains lighting consistency across different exposure multipliers and color temperature, and maintains robust light estimation accuracy as FOV increases. The surrounding lighting information estimated by our method ensures coherent illumination of 3D objects blended with the input photograph, enabling high fidelity augmented and mixed reality supporting a wide range of environmental lighting conditions and device sensors.

History

Preferred citation

Zhao, J., Chalmers, A. & Rhee, T. (2021). Adaptive Light Estimation using Dynamic Filtering for Diverse Lighting Conditions. IEEE Transactions on Visualization and Computer Graphics, 27(11), 4097-4106. https://doi.org/10.1109/TVCG.2021.3106497

Journal title

IEEE Transactions on Visualization and Computer Graphics

Volume

27

Issue

11

Publication date

01/11/2021

Pagination

4097-4106

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Publication status

Published

ISSN

1077-2626

eISSN

1941-0506

Language

en