Shading concerned generative models for fine-grained photo-realistic image generation
Abstract
Modern generative models can synthesize highly realistic images, but their internal representation of content, material, illumination, and viewpoint is often implicit. This makes the generated result difficult to analyse and edit in a controlled manner. The central question of this thesis is therefore how to introduce fine-grained control into generative models without sacrificing visual fidelity. This thesis studies that question through a sequence of methods that progressively connect controllable generation with physics-based rendering. First, we investigate controllability inside a style-based generator and introduce generative fields as a way to analyse the spatial extent of channel-wise control in StyleGAN2. This analysis is used to relate different generator layers to coarse and fine facial attributes, and motivates a reference-guided editing framework that preserves identity while transferring pose and expression from a reference image. The resulting study shows that controllability inside a latent generator can be made more explicit when the spatial role of intermediate channels is analysed rather than treated as a purely implicit property of the model. Second, we study illumination control as a rendering problem and propose a physics-based neural deferred shading pipeline for real-world portrait images. The method takes estimated material, geometry, and illumination attributes as input, and learns a scene-agnostic mapping from geomertry buffer representations to…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.