Comparison between RoB-1 and RoB-2 tool and impact on network meta-analysis results - a case study from a living Cochrane review on psoriasis
Abstract
OBJECTIVES
This study was conducted within a large Cochrane living systematic review on psoriasis treatments with the aim to evaluate the inter-rater agreement of the Cochrane RoB-2 tool, to compare its risk of bias judgments with the original RoB-1, and to explore the impact of changes in risk of bias judgment between the two tools on the Cochrane network meta-analysis'(NMA) results.
STUDY DESIGN AND SETTING
This study was conducted within the 2025 update of a living Cochrane review on systemic treatments for psoriasis. Four pairs of assessors used RoB-2 to evaluate the risk of bias of 193 randomized controlled trials for two primary outcomes: Psoriasis Area Severity Index (PASI) 90 (reflecting clear or almost clear skin) and Serious Adverse Events (SAE). Inter-rater reliability (IRR) was calculated using Cohen's kappa. RoB-2 judgments for 147 trials (PASI 90) and 154 trials (SAE) were compared to the previous RoB-1 assessments from the Cochrane 2023 update. The impact of using RoB-2 versus RoB-1 judgments on the NMA's results was explored through sensitivity analyses, with calculation of ratio of risk ratios (RRR) between the analyses for each treatment effect.
RESULTS
For the RoB-2 overall judgment, the IRR was fair for PASI 90 (kappa = 0.37) and moderate for SAEs (kappa = 0.46). IRR varied between domains (from kappa = 0.33, to kappa = 0.65), with lower IRR found for domains 2, 3 and 5. Significant discrepancies were found between RoB-1 and RoB-2 judgments. Compared to…
How strong is the evidence?
Evidence Hub reads this as a meta-analysis. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid.
Read the paper
Related studies
- Global, regional, and national burden of hepatitis B, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019
- A meta‐analysis of anal cancer incidence by risk group: Toward a unified anal cancer risk scale
- Global fertility in 204 countries and territories, 1950–2021, with forecasts to 2100: a comprehensive demographic analysis for the Global Burden of Disease Study 2021
- Exercise interventions in Alzheimer's disease: A systematic review and meta-analysis of randomized controlled trials
- Prognostic Significance of Carbonic Anhydrase IX Expression in Cancer Patients: A Meta-Analysis
- Glucocorticoid Resistance: Is It a Requisite for Increased Cytokine Production in Depression? A Systematic Review and Meta-Analysis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.