A Calibrated Multi-Dimensional Evaluation Framework for Diffusion-Based Radio Frequency Signal Generation
Abstract
Evaluation of generative models for RF (Radio Frequency) signals remains largely ad hoc, with existing approaches relying on uncalibrated metrics imported from computer vision without systematic justification or baseline establishment. We present a multi-dimensional, distribution-level evaluation framework comprising ten metrics across six layers, calibrated against real-real baselines. A real-real baseline is computed by splitting authentic signals into two independent subsets and measuring the same metric between them; the resulting value sets the achievable ceiling for that metric, against which generated-vs-real scores are normalized. The framework is grounded in four design principles: distribution-level aggregation, real-real baseline normalization, signal-to-noise ratio (SNR)-modulation stratification, and multi-dimensional coverage. Application of the framework reveals two previously unreported phenomena. First, linear Short-Time Fourier Transform preprocessing creates a gradient imbalance between frequency-domain and time-domain loss components that causes catastrophic generation failure for analog amplitude modulation; logarithmic compression resolves this. Second, the widely adopted noise-prediction training objective exhibits systematic gradient suppression for amplitude-modulated signals due to SNR-dependent implicit loss weighting; switching to signal-prediction substantially improves temporal structure fidelity for amplitude-modulated signals, with only modest…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.