Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

Meta-analysis of hemodynamic optimization in high-risk patients.

Kern JW, Shoemaker WC.

Crit Care Med. 2002;30(8):1686-92. [abstract]

Reviewed by Satid Thammasitboon, MD, MHPE, Pediatric Critical Care Medicine, Baylor College of Medicine, Houston, TX, and Supat Thammasitboon, MD, Pulmonary and Critical Care Medicine, Tulane University, New Orleans, LA

Review posted May 4, 2003

---

I. Are the results of the study valid?

A. Primary questions:

1. Did the overview address a focused clinical question?

No. The purpose of the overview was to determine the effect of goal-directed therapy on mortality rates in critically ill patients - this is simply too diffuse a clinical question to be addressed in a meta-analysis without much more specific focus. Indeed, the meta-analysis included studies performed in a wide variety of clinical circumstances, the time that the therapeutic goals were implemented and the presence or absence of organ failures. The subgroup analysis - which was not well specified in advance - diverted focus of the overview in some degree.

2. Were the criteria used to select articles for inclusion appropriate?

Perhaps. The inclusion criteria were stated in general terms; usually such studies list MeSH terms explicitly. It is uncertain whether there was bias in choosing articles to support the authors' prior conclusion.

Three clinical criteria were used to define patient populations, therapeutic goals, and interventions.

1. Patients: Critically ill patients after high-risk elective surgery, severe trauma, and septic shock
2. Therapeutic goals: Cardiac index (CI) of > 4.5 L/min m2, Pulmonary occlusion pressure (PAOP) of < 18 mmHg, oxygen delivery (Do2) of > 600 mL/min m2, or oxygen consumption of > 170 mL/min m2 (the "or" terms makes it unclear if this refers to two different goals of oxygen delivery or oxygen consumption, or whether any one of these goals would be adequate to meet criteria for inclusion)
3. Interventions: Initial intervention with fluid therapy, added inotropes if hemodynamic goals were not achieved.

Study design as a randomized, controlled trial is the only methodological criterion used to select studies.

B. Secondary questions:

3. Is it unlikely that important, relevant studies were missed?

Perhaps. The overview might have missed several relevant studies since the literature search was conducted limited to only English publications in MEDLINE. The authors should also search other databases, i.e., EMBASE, the Cochrane Controlled Trials Register and the Cochrane Medical Editors Trial Amnesty for published and unpublished trials. The authors did not mention if the experts in the area have been contacted. Hence this overview might have missed unpublished studies, studies that are in press or not yet indexed. This omission potentially increases the chances of publication bias. The publication bias usually overestimates efficacy of the therapy since the studies with "positive" results are more likely to be published. However, goal-directed therapy, as we know, has been controversial in critical care medicine. We might assume there are equal amount of published studies with "positive" and "negative" results. Also, only considering trials in English was a limitation. Many good international studies may have been missed.

Of note, had a large randomized, controlled trial by Sandham (1), for instance (published after the paper under evaluation was published), have been included in this meta-analysis the overall outcome of this meta-analysis might have been very different. This study found no benefit to therapy directed by pulmonary artery catheter over standard care in elderly, high-risk surgical patients.

Another concerning aspect of this paper was the breakdown of a study by Yu et al (2) into two separate studies. This appeared to be a study with two subgroups identified by age. Why was this considered two separate trials by Kern and Shoemaker?

4. Was the validity of the included studies appraised?

Yes. The authors classified the trials according to randomization process, concealed randomized group allocation, blinding vs. no blinding and adequacy of withdrawal or dropout analysis. However, they do not perform any sensitivity analysis or other means of determining the effect of study quality on the overall results. A solid systematic review will, for example, report the analysis with lower quality trials excluded. This was not done here. Many people would have weighted the studies according to their quality using an a prior determined scoring system. The average ages were assessed but limited information on other patient characteristics. The authors also avoid redundancy by excluding the study that may contain repeated data.

5. Were assessments of studies reproducible?

Unclear. The authors did not mention how they came up with the agreement between two authors in judging the quality of the literature. No Kappa statistic, scoring system, or any other interobserver variability test performed.

6. Were the results similar from study to study?

Not quite. The authors described important differences in patients, interventions and outcome measures from study to study (Table 1.) but did not use the test of heterogeneity to assess the differences among the treatment effects of individual studies. Readers must decide whether these factors are so different that the study results cannot be combined for analysis. Considering only the differences in outcomes (Figure 1.), the wide range of the results is easily appreciated. Given this reason, the random effects, not the fixed effects, model should have been used for meta-analysis.

II. What are the results?

1. What are the overall results of the review?

The authors use the fixed effects model for meta-analysis. The weight of the individual results (according to sample size or quality of the studies) routinely used for meta-analysis was not noted in this study. The subgroup analysis* was conducted as well as overall analysis. The results are noted to be reported as differences in mortality rates between groups, with confidence intervals. However, in the first sentence of the results they note the confidence intervals are "twice the SD." This is not correct. The 95% CI are ± 1.96 x std error of the mean. It is not clear whether this is just a typographical error. Nowhere else in the results, and certainly not on their primary graphical results report in Figure 1, is there mention of 95% confidence intervals. (Hence the question mark in our table below.)

Subgroup analysis	ARR (% ± 95% CI?)	NNT	p
Control groups with mortality rates greater than 20%
Goals to supranormal value after organ failure	0 ± 7	infinity	>0.05
Goals to supranormal value before organ failure	23 ± 7**	4	<0.05
Control groups with mortality rates less than 15%	1 ± 3	100	>0.05
Overall	4 ± 2.5	25	< 0.05

ARR: Absolute Risk Reduction
NNT: Number Needed to Treat

*It has been advised to interpret the results of subgroup analyses from a meta-analysis skeptically since the treatment effects in subgroups could be only due to chance (3). Furthermore, the subgroups evaluated here - in particular, evaluating studies with different control group mortality rates - was not specified a priori. However, the ARR** found in patients with high mortality and before organ failure is credible due to a big difference in treatment effect, its a priori hypothesis and biological plausibility.

2. How precise were the results?

Uncertain. Given the variable clinical settings of the individual studies, the observed differences among treatment effects and qualification of the studies, it is unlikely we can draw precision well from these results.

III. Will the results help me in caring for my patients?

1. Can the results be applied to my patient care?

No, certainly not directly. The application of the results of a meta-analysis in adult to pediatric population is particularly troublesome for this clinical question. The pathophysiology and outcomes (mortality rates in particular) of critically ill patients is different between pediatric and adult populations.. Hence, even if we wanted to apply these results, our pediatric population would likely fall in subgroup with mortality rates less than 15% in which no significant benefit was found.

2. Were all clinically important outcomes considered?

No. Other outcomes, e.g. pulmonary artery catheter-associated complications and length of ICU stay should be considered. These are common outcomes considered in most of the individual studies. The effort to review them would yield valuable information in determining the efficacy of the intervention.

3. Are the benefits worth the harms and costs?

Unclear. Given the overall NNT of 25 in high risk adult patients, without information on costs and complications, it is difficult to declare the goal-directed therapy an effective intervention. Certainly the benefits of routine pulmonary artery catheter placement in high-risk pediatric patients may not outweigh its harms due to the possible complications.

References

1. Sandham JD, Hull RD, Brant RF, et al. A Randomized , controlled trial of the use of pulmonary-artery catheter in high-risk surgical patients. N Engl J Med 2003;348:5-14. [abstract]
2. Yu M, Burchell S, Hasaniya NW, Takanishi DM, Myers SA, Takiguchi SA. Relationship of mortality to increasing oxygen delivery in patients > or = 50 years of age: a prospective, randomized trial. Crit Care Med. 1998;26(6):1011-9. [abstract]
3. Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA 1991; 266:93-8. [abstract]

Comments

Back to the EB Journal Club Index