Efficient and Differentiable Shadow Computation for Inverse Problems

Download Video: HD (MP4, 42 MB)

Abstract

Differentiable rendering has received increasing interest for image-based inverse problems. It can benefit traditional optimization-based solutions to inverse problems, but also allows for self-supervision of learning-based approaches for which training data with ground truth annotation is hard to obtain. However, existing differentiable renderers either do not model visibility of the light sources from the different points in the scene, responsible for shadows in the images, or are too slow for being used to train deep architectures over thousands of iterations. To this end, we propose an accurate yet efficient approach for differentiable visibility and soft shadow computation. Our approach is based on the spherical harmonics approximations of the scene illumination and visibility, where the occluding surface is approximated with spheres. This allows for a significantly more efficient shadow computation compared to methods based on ray tracing. As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and geometric deformation recovery from images using analysis-by-synthesis optimization.

Downloads


  • Paper
    PDF

  • Supplemental document
    PDF

  • Main video
    MP4


Citation

@InProceedings{lyu2021efficient,
      title={Efficient and Differentiable Shadow Computation for Inverse Problems}, 
      author={Linjie Lyu and Marc Habermann and Lingjie Liu and Mallikarjun {B R} and Ayush Tewari and Christian Theobalt},
    booktitle = {{IEEE} International Conference on Computer Vision ({ICCV})},
	year={2021},
}
				

Contact

For questions, clarifications, please get in touch with:
Linjie Lyu
llyu@mpi-inf.mpg.de

This page is Zotero translator friendly. Page last updated Imprint. Data Protection.