We present Generative Relightable Avatars (GRA), a person-specific method for photorealistic free-view rendering and environment-map relighting of full-body humans. We postulate that modeling fine-grained appearance details is inherently a one-to-many problem that can benefit from a generative formulation. In contrast to fully regressive relightable avatar methods, GRA follows a hybrid approach that combines controllable, physics-grounded relighting with probabilistic refinement. Starting from a tracked animated mesh, we optimize material parameters in UV-space and render a coarse relit appearance under a target HDR environment map. Next, we refine the textures with a feed-forward model to capture pose-dependent texture dynamics and illumination effects beyond simplified reflectance assumptions. Finally, a fine-tuned video-to-video diffusion model transforms the physically grounded renderings into temporally coherent, high-detail videos while preserving 3D control, with an error-recycling strategy for generating long videos. Experimental evaluations demonstrate our method's improved perceptual quality over prior relightable avatar baselines.
Given skeletal motion θ, viewpoint κ, and environment illumination E , GRA outputs a photorealistic rendering of the character avatar in three steps: (a) A person-specific animatable mesh predicts pose-dependent geometry M, relit under a microfacet BRDF to produce a coarse appearance, which is refined by (b) RelightNet to capture illumination- and pose-dependent effects. (c) Finally, a finetuned video diffusion model synthesizes fine-scale details conditioned on the coarse renderings and per-frame environment maps. Blue arrows indicate the inference path; red arrows indicate gradient propagation.
We test our method on out-of-distribution lighting conditions, i.e. OLAT environment maps. Notably the model never saw OLAT environment maps during training.
With slight modifications, our method also qualitatively generalizes to near-field relighting without any retraining.
@InProceedings{singh2026gra,
title={Generative Relightable Avatars},
author={Singh, Kunwar Maheep and Theobalt, Christian and Dabral, Rishabh},
booktitle={ECCV},
year={2026}
}