A dataset-centric review of IoT and IIoT intrusion detection: realism, evaluation biases, and future research directions
Abstract
The rapid growth of IoT and IIoT expands the cyber-attack surface of interconnected and safety-critical systems, and, as such, IDSs have become a fundamental security mechanism. Although very impressive results have been reported for machine learning and deep learning-based IDS in benchmark datasets, these gains often do not generalize to real-world deployments owing to dataset design limitations, realism deficits, and evaluation biases, rather than inherent flaws in detection algorithms, which can lead to significant vulnerabilities in actual operational environments. This study presents a dataset-centric review of widely used intrusion detection datasets from the IIoT, IoT, and traditional network domains. A unified taxonomy differentiates datasets based on the domain context, traffic representation, protocol semantics, and attack modeling assumptions. Based on a common analytical framework, each dataset was reviewed regarding its realism, coverage of the threats, class imbalance, temporal continuity, and modern ML/DL-based evaluation of the IDS. The cross-dataset analysis conducted in this study shows that, in addition to the fact that model architecture and feature engineering play a major role, several studies indicate that the simplicity of the datasets, the class imbalance, and the repetitive attack patterns as well as the evaluation methods can affect accuracy of the IDS. This work further underlines the remaining gaps, such as zero-day and adaptive attacks, limited…
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Efficient explainable federated learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.