Assessing inter-observer variability in the delineation of structures in radiation oncology: A systematic review
Abstract
INTRODUCTION
The delineation of target volumes and organs at risk is the main source of uncertainty in radiation therapy (RT)(1). Numerous inter-observer variability (IOV) studies have been conducted, often with unclear methodology and non-standardized reporting. We aimed to identify the parameters chosen in conducting delineation IOV studies and assessed their performances and limits.
MATERIEL AND METHODS
We conducted a systematic literature review to highlight major points of heterogeneity and missing in IOV studies published between 2018 and 2021. For the main used metrics, we did in silico analyses to assess their limits in specific clinical situations.
RESULTS
All disease sites were represented in the 66 studies. Organs at risks (OAR) were studied independently of tumor site in 29% of reviewed IOV studies. In 65% of studies, statistical analyses were performed. No gold standard (GS, i.e. reference) was defined in 36% of studies. A single expert was considered as GS in 21% studies without testing intraobserver variability. All studies reported both absolute and relative indices, including the Dice Similarity Coefficient (DSC) in 68% and Hausdorff Distance (HD) in 42%. Limitations were shown in silico for small structures when using DSC and dependence on irregular shapes when using HD. Variations in DSC values were large between studies and their thresholds inconsistent. Most (51%) studies included 1-10 cases. The median number of observers/experts was 7 (range 2-35). The…
How strong is the evidence?
Evidence Hub reads this as a systematic review. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid.
Read the paper
Related studies
- Preliminary Toxicity Analysis of 3DCRT versus IMRT on the High Dose Arm of the RTOG 0126 Prostate Cancer Trial
- Adaptive Radiotherapy for Head and Neck Cancer: Initial Clinical Outcomes from a Prospective Trial
- American Association of Physicists in Medicine Task Group 263: Standardizing Nomenclatures in Radiation Oncology
- Motion Management Strategies and Technical Issues Associated with Stereotactic Body Radiotherapy of Thoracic and Upper Abdominal Tumors: A Review from NRG Oncology
- High-dose-rate interstitial brachytherapy as a monotherapy for localized prostate cancer: treatment description and preliminary results of a phase I/II clinical trial
- Highly efficient training, refinement, and validation of a knowledge-based plan quality control system for radiotherapy clinical trials
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.