Evidence Hub

Is meta-analysis really meta-physics?

M. Machtay, Larry R. Kaiser, E. Glatstein

Chest, 1999

10 participants · 22 citations

Abstract

T here is a growing obsession within the medical community and the public for swift and succinct evidence-based decisions. However, the presumption of many of those who advocate this view is that only meta-analyses of prospectively randomized trials— however flawed—provide useful information. The meta-analysis of postoperative radiotherapy (PORT) in non-small cell lung cancer (NSCLC) published recently in Lancet1 well exemplifies this way of thinking. “It should be compulsory reading for CEOs of hospitals. There is a very clear message here,” said the director general of Britain’s Cancer Research Campaign to Associated Press reporters. He congratulated the PORT researchers for “determining” that PORT for NSCLC not only failed to improve survival but actually worsened survival. The publicity generated by the PORT meta-analysis caused unwarranted fear among many lung cancer patients, who already have enough to worry about. The popular and academic media have not adequately addressed a real discussion of the true scientific limitations of this study and of the questionable validity of meta-analyses in general. Although performed with statistical rigor and considerable effort on the part of the researchers to obtain individual, updated patient data, the Lancet metaanalysis is scientifically flawed. This is particularly with respect to the “sensational” conclusion that radiotherapy (XRT) causes more deaths than it prevents in lung cancer patients. The first problem with…

How strong is the evidence?

Evidence Hub reads this as a meta-analysis, done in people. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid. Reported size: 10 participants or records.

How the five levels of evidence work. The level is assigned automatically from the abstract and can be wrong, so check the paper.

Read the paper

Related studies

Go deeper in Evidence Hub

Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.

Open this study Sign up free