Physics-Informed Guided Disentanglement in Generative Networks
Abstract
Image-to-image translation (i2i) networks suffer from entanglement effects in presence of physics-related phenomena in target domain (such as occlusions, fog, etc), lowering altogether the translation quality, controllability and variability. In this paper, we propose a general framework to disentangle visual traits in target images. Primarily, we build upon collection of simple physics models, guiding the disentanglement with a physical model that renders some of the target traits, and learning the remaining ones. Because physics allows explicit and interpretable outputs, our physical models (optimally regressed on target) allows generating unseen scenarios in a controllable manner. Secondarily, we show the versatility of our framework to neural-guided disentanglement where a generative network is used in place of a physical model in case the latter is not directly accessible. Altogether, we introduce three strategies of disentanglement being guided from either a fully differentiable physics model, a (partially) non-differentiable physics model, or a neural network. The results show our disentanglement strategies dramatically increase performances qualitatively and quantitatively in several challenging scenarios for image translation.
Read the paper
Related studies
- Preliminary Toxicity Analysis of 3DCRT versus IMRT on the High Dose Arm of the RTOG 0126 Prostate Cancer Trial
- Adaptive Radiotherapy for Head and Neck Cancer: Initial Clinical Outcomes from a Prospective Trial
- American Association of Physicists in Medicine Task Group 263: Standardizing Nomenclatures in Radiation Oncology
- Motion Management Strategies and Technical Issues Associated with Stereotactic Body Radiotherapy of Thoracic and Upper Abdominal Tumors: A Review from NRG Oncology
- High-dose-rate interstitial brachytherapy as a monotherapy for localized prostate cancer: treatment description and preliminary results of a phase I/II clinical trial
- Highly efficient training, refinement, and validation of a knowledge-based plan quality control system for radiotherapy clinical trials
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.