Visual Computing and Artificial Intelligence Department

Computer Vision and Machine Learning
for Computer Graphics

Seminar – Summer Semester 2022

Organizers: Christian Theobalt, Marc Habermann, Thomas Leimkühler

Organization | Content | Format | Ressources


EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks ArXiv, 2021	Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation 3DV, 2018

Overview

Computer Vision strives to develop algorithms for understanding, interpreting and reconstructing information about real-world scenes from image and video data.

Computer Graphics focuses on image synthesis: algorithms to build and edit static and dynamic virtual worlds and to display them in photorealistic or stylized ways.

Machine Learning is concerned with studying and developing algorithms which use statistical models to solve problems by analyzing and drawing inference from data.

In recent years, these fields have converged more and more. Both Computer Vision and Computer Graphics create and exploit models describing the visual appearance of objects and scenes, while the most successful models heavily utilize ideas from Machine Learning. In this seminar series, we will cover advanced research topics that cross the boundaries between the fields of Computer Vision, Computer Graphics, and Machine Learning. This seminar will cover research papers from the following topics:

image and video generation, manipulation, and analysis,
multi-view geometry and reconstruction,
computational photography and videography,
shape matching,
pose estimation, tracking, and character animation,
deep learning for computer vision and computer graphics.

Topics

The following table lists more topics than available weeks – this is to allow some flexibility for the interests of the group to change the course structure. Once every participant has submitted their choice of topics, this list will be updated to show the presenter of each topic. Send us an email if you cannot access a paper for some reason.

Click on each topic to show the papers to be discussed or show all papers hide all papers.

Topic and Papers	Presentation
Multiview Synthesis and Reconstruction NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis Mildenhall et al. ECCV 2020 NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction Wang et al. NeurIPS 2021	28 April 2022 Presenter: Chengjiangrong Peng Supervisor: Mohit Mendiratta
Learning Neural Radiance Fields from Few Views pixelNeRF: Neural Radiance Fields from One or Few Images Yu et al. CVPR 2021 RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs Niemeyer et al. ArXiv 2021	5 May 2022 Presenter: Anh Tuan Tran Supervisor: Viktor Rudnev
Multiscale Neural Scene Representation Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields Barron et al. ICCV 2021 BACON: Band-limited coordinate networks for multiscale scene representation Lindell et al. ArXiv 2021	12 May 2022 Presenter: Wanyue Zhang Supervisor: Xingang Pan
Event-based Pose Estimation EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera Xu et al. CVPR 2020 EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream Rudnev et al. ICCV 2021	19 May 2022 Presenter: Pankhuri Vanjani Supervisor: Vladislav Golyanik
Material Editing and Relighting in Neural Rendering NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination Zhang et al. SIGGRAPH Asia 2021 PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting Zhang et al. CVPR 2021	2 June 2022 Presenter: Basavaraj Rajendra Sunagad Supervisor: Linjie Lyu
3D Generative Adversarial Networks pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis Chan et al. CVPR 2021 EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks Chan et al. ArXiv 2021	9 June 2022 Presenter: Devikalyan Das Supervisor: Navami Kairanda
GANs through Space and Time A Good Image Generator Is What You Need for High-Resolution Video Synthesis Tian et al. ICLR 2021 StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN Fox et al. ArXiv 2021	23 June 2022 Presenter: Junaid Supervisor: Gereon Fox
Discrete Neural Representations Zero-Shot Text-to-Image Generation Ramesh et al. PMLR 2021 Taming Transformers for High-resolution Image Synthesis Esser et al. CVPR 2021	30 June 2022 Presenter: Yasin Esfandiari Supervisor: Ikhsanul Habibie
Human Estimation and Tracking in 3D Space Tracking People with 3D Representations Rajasegaran et al. NeurIPS 2021 GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras Yuan et al. ArXiv 2021	7 July 2022 Presenter: Pranay Raj Kamuni Supervisor: Diogo Luvizon
Compositional Models Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering Yang et al. ICCV 2021 GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields Niemeyer & Geiger CVPR 2021	14 July 2022 Presenter: Akshay Mundra Supervisor: Mallikarjun B R
Inverse Graphics with Neuro-explicit Models Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation Omran et al. 3DV 2018 NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation Wang et al. ICLR 2021	21 July 2022 Presenter: Mohammad Amin Parchami Araghi Supervisor: Adam Kortylewski
Human Motion and Language Synthesis of Compositional Animations from Textual Descriptions Ghosh et al. ICCV 2021 Action-Conditioned 3D Human Motion Synthesis with Transformer VAE Petrovich et al. ICCV 2021	28 July 2022 Presenter: Muhammad Hamza Mughal Supervisor: Rishabh Dabral
Physics-based Character Animation Data-Driven Physics for Human Soft Tissue Animation Kim et al. TOG 2017 SuperTrack: Motion Tracking for Physically Simulated Characters using Supervised Learning Fussell et al. TOG 2021	-
Deeper Human Modeling Multi-Segment Foot for Human Modelling and Simulation Park et al. CGF 2020 A Multilinear Model for Bidirectional Craniofacial Reconstruction Achenbach et al. CVBM 2018	-
Reconstructing General Dynamic Scenes with Neural Rendering Nerfies: Deformable Neural Radiance Fields Park et al. ICCV 2021 Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes Li et al. CVPR 2021	-
Taking Transformers from Language to 3D Reconstruction Perceiver IO: A General Architecture for Structured Inputs & Outputs Jaegle et al. ArXiv 2021 Input-level Inductive Biases for 3D Reconstruction Yifan et al. ArXiv 2021	-
Monocular Non-rigid 3D Reconstruction BANMo: Building Animatable 3D Neural Models from Many Casual Videos Yang et al. ArXiv 2021 φ-SfT: Shape-from-Template with a Physics-based Deformation Model Kairanda et al. CVPR 2022	-
Shape Matching with Quantum Algorithms Matching Point Sets with Quantum Circuit Learning Noormandipour & Wang ArXiv 2021 Q-Match: Iterative Shape Matching via Quantum Annealing Benkner et al. ICCV 2021	-
Coherent 3D Human Pose Estimation Body Size and Depth Disambiguation in Multi-Person Reconstruction from Single Images Ugrinovic et al. 3DV 2021 Putting People in their Place: Monocular Regression of 3D People in Depth Sun et al. ArXiv 2021	-
Bayesian Pose Estimation Probabilistic Modeling for Human Mesh Recovery Kolotouros et al. ICCV 2021 Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows Wehrbein et al. ICCV 2021	-
Differentiable Physics Simulation for Inverse Problems GradSim: Differentiable Simulation for System Identification and Visuomotor Control Jatavallabhula et al. ICLR 2020 Inverse Simulation: Reconstructing Dynamic Geometry of Clothed Humans via Optimal Control Guo et al. CVPR 2021	-
3D Human Motion Synthesis and Control Phase-Functioned Neural Networks for Character Control Holden et al. TOG 2017 Character Controllers Using Motion VAEs Ling et al. SIGGRAPH 2020	-
Model-based 3D Reconstruction of Faces MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction Tewari et al. ICCV 2017 Markov Chain Monte Carlo for Automated Face Image Analysis Schönborn et al. IJCV 2017	-
Photorealistic Face Relighting Monocular Reconstruction of Neural Face Reflectance Fields B R et al. CVPR 2021 PhotoApp: Photorealistic Appearance Editing of Head Portraits B R et al. SIGGRAPH 2021	-
Photorealistic Face Editing PIE: Portrait Image Embedding for Semantic Control Tewari et al. SIGGRAPH Asia 2020 Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction Gafni et al. CVPR 2021	-
Placing 3D People in 3D Scenes: Analyzing Human-Scene Interaction in 3D Space Resolving 3D Human Pose Ambiguities with 3D Scene Constraints Hassan et al. ICCV 2019 Generating 3D People in Scenes without People Zhang et al. CVPR 2020	-
Egocentric Human Body Pose Estimation Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera Xu et al. IEEE VR 2019 Estimating Egocentric 3D Human Pose in Global Space Wang et al. ICCV 2021	-
3D Body Models GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models Xu et al. CVPR 2020 imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose Alldieck et al. ICCV 2021	-