A Modular Image Manipulation Localization Framework using a Dual-Stream Classifier and Conditional Diffusion Models
Abstract
Abstract Image manipulation localization (IML), the task of accurately identifying the manipulated regions within an image, remains a significant challenge. Conven tional deep learning methods often struggle with generalization and adapting to new manipulation schemes. To address these limitations, a novel two-stage IML system is proposed. The first stage employs a Dual-Stream Manipulation Classifier, fusing features from the standard RGB domain with low-level foren sic noise artifacts extracted via Steganalysis Rich Model (SRM) filters using a ResNet like 4-stage backbone, to accurately classify the input image into one of four common manipulation types: Copy-Move, Splicing, Removal (Inpainting), or Enhancement. In the second stage, the classified image is directed to a spe cialized Conditional Diffusion Model (CDM), which treats the localization task as an image-to-mask generation problem. These CDMs are conditioned on the input image, and are trained with a modified loss function incorporating both Mean Squared Error (MSE) and Intersection over Union (IoU) to better handle smaller manipulation masks and encourage better spatial structure. Evaluated on the DF2023 dataset, the manipulation classifier achieves an accuracy of 89% and localization CDMs achieve average IoU of 0.70 and F1-Score of 0.77. The system demonstrates competitive performance on benchmark datasets, confirm ing the efficacy of a modular, generative approach for tackling the complexity of image forgeries.
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.