Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Specific questions in this review are based on Jaeschke RZ, Meade MO, Guyatt GH, Keenan SP, Cook DJ. How to use diagnostic test articles in the intensive care unit: diagnosing weanability using f/Vt. Crit Care Med. 1997;25(9):1514-2. [abstract]

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

Microscopic examination of intracellular organisms in protected bronchoalveolar mini-lavage fluid for the diagnosis of ventilator-associated pneumonia.

Sirvent JM, Vidaur L, Gonzalez S, et al.

Chest. 2003;123(2):518-23 [abstract]

Reviewed by Curt Kennedy MD, Baylor College of Medicine, Texas Children's Hospital, Houston, TX

Review posted December 4, 2003

I. What is being studied?

Study objective:

The objective of this nonrandomized nonblinded prospective study was to determine if a cutoff percentage of intracellular organisms (ICOs) obtained from mini-protected bronchoalveolar lavage (PBAL) could be used as a rapid method for diagnosing ventilator associated pneumonia (VAP) in patients who met clinical criteria for their first episode of VAP. The authors also wished to determine the accuracy of this diagnostic procedure in patients who had previously received antibiotic therapy.

II. Are the results of the study valid?

Primary questions:

1. Was there an independent, blind comparison with a reference standard?

Besides direct examination for ICOs, the mini-BAL specimen was submitted for culture on blood, chocolate and McConkey agar media. These cultures were used as the reference standard. Cultures were classified as positive if they contained > 10³cfu/mL (i.e., greater than 10 colonies on the culture media). Although lung biopsy culture is regarded as the gold standard, we feel the cultures satisfy the criterion for a reference standard.

2. Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

The cohort of 82 patients studied (65 with VAP) were adults with largely surgical / traumatic illness (less than a third were primarily admitted for a medical indication). Therefore, caution should be used when applying the results to a pediatric intensive care unit that treats children with medical illness predominantly. Incidentally, there was also a predominance of males in the study (73%). We felt it would have been helpful if the authors distinguished between total duration of mechanical ventilation (days), ICU stay (days), and the time of evaluation.

Specifically mentioned in this study was the exclusion of common medical comorbidities, including HIV, malignancies, transplant recipients and patients on hemodialysis. While we understood the rationale for wanting to examine a homogenous cohort, we feel an opportunity was missed in examining these types of patients. The purpose of the study was "prompt" identification of ventilator associated pneumonia; not the comparison of the mini-PBAL culture with standard tracheal aspirate culture. However, the patients with confounding comorbidities are frequently the ones that require lung biopsy for definitive diagnosis. It would have been useful to see data as to whether or not the mini-PBAL culture may have possibly served as a prompt alternative to PBAL culture in this subgroup of high risk patients as well (It's doubtful one center has enough experience with lung biopsy cultures to permit an adequate comparison).

Secondary questions:

3. Did the results of the test being evaluated influence the decision to perform the reference standard?

No - all samples were split, with the same specimen being sent for both Gram stain and culture (reference standard) as well as Giemsa staining (technique being studied).

4. Were the methods for performing the test described in sufficient detail to permit replication?

Yes

Yes - the equipment and techniques were described with sufficient detail as to permit replication of the experiment. Brand names are mentioned in the body of the article, and funding sources for the experiment are not identified. However, Dr. Sirvent has stated in personal correspondence that the study was funded by internal sources, not by industry, thus satisfying this reviewer's concern for potential conflict of interest.

III. What are the results?

1. Are the test's sensitivity, specificity, and likelihood ratios presented (or are the data necessary for their calculation provided)?

First, the 2 x 2 table that results from this study is:

	(+) VAP	(-) VAP	Total
(+) PBAL	52	3	55
(-) PBAL	13	14	27
Total	65	17	82

(Note: +PBAL indicates 2% or more of cells examined contained intracellular organisms.)

Sensitivity: (True PBAL +) / (VAP +) = 52/65 = 0.80
(95% CI = 0.69 - 0.88)
(If the VAP is present, the PBAL is positive in 80% of samples.)
(20% of VAP patients will have false negative PBALs.)

Specificity: (True PBAL-) / (VAP -) = 14/17 = 0.82
(95% CI = 0.59 - 0.94)
(If the VAP is absent, the PBAL is negative in 82% of samples.)
(18% of nondiseased patients will have false positive PBALs.)

Likelihood (+) = Sensitivity / (1 - Specificity) = 0.8 / (1 - 0.82) = 0.8/0.176 = 4.5
(95% CI = 1.61 - 12.75)
(A positive test increases the likelihood of having VAP by 4.5 times)

Likelihood (-) = (1 - Sensitivity) / Specificity = (1 - 0.80) / 0.82 = 0.2/0.824 = 0.24
(95% CI = 0.14 - 0.41)
(A negative test reduces the likelihood you have VAP by 0.24 times)

Pretest probability (prevalence) = (Diagnosis +) / Total = 65/82 = 79%
Pretest odds = Baseline odds of having VAP = prevalence / (1-prevalence) = 0.79 / (1 - 0.79) = 0.79 / 0.21 = 3.8
+ Posttest odds = pretest odds x likelihood ratio = 3.8 x 4.5 = 17
+ Posttest probability = posttest odds / (1 + posttest odds) = 17 / 18 = 94%
(95% CI = 86% - 98%)
(94% of patients with a (+) PBAL will have VAP)

Posttest odds = pretest odds x likelihood ratio = 3.8 x 0.24 = 0.91
Posttest probability = posttest odds / (1 + posttest odds) = 0.91 / 1.91 = 47%
(95% CI = 35% - 61%)
(47% of patients with a (-) PBAL will have VAP)

The results were less accurate in the antibiotic-therapy group (sensitivity 70%, specificity 75%, positive predictive value 87.5%, negative predictive value 39%).

IV. Will the results help me in caring for my patients?

1. Will the reproducibility of the test result and its interpretation be satifactory in my setting?

The population of patients served in a pediatric ICU often requires ETT sizes frequently as small as a 3.0 mm internal diameter (and occasionally even smaller). Representatives for the Combicath product quoted that ETT sizes to 3.0 can be accommodated. Therefore, the techniques described would work in most of our patients. (NOTE: The volume of 20 ml of saline instilled for the purpose of the PBAL may require reduction in smaller patients.)

2. Are the results applicable to my patients?

As stated above, a cohort consisting of adults admitted with predominantly surgical and trauma related diagnoses (and who are predominantly male) is not directly applicable to a pediatric ICU population. Also, since the population studied was relatively deficient in medical cases and specifically excluded comorbidities that are commonplace in a pediatric ICU, the utility of the results in the pediatric ICU environment are only marginal. Finally, where local practice norms are to begin empiric antibiotic therapy on all patients at the time of clinical suspicion and evaluation by tracheal cultures, the utility of this study for its stated purpose no longer holds, since all patients are being treated regardless of microscopic appearance. (Likewise, a sensitivity of 80% is insufficient to justify withholding antibiotic therapy in patients who are clinically suspected to have VAP.)

3. Will the results change my management?

Based on the lack of similarity between the study population and my patients, the local norm of providing empiric broad-spectrum antibiotic coverage at the time of culture evaluation for VAP, and the low sensitivity, it is unlikely that I will use the techniques described to change my clinical practice.

4. Will patients be better off as a result of the test? The techniques described in the study theoretically benefit the patient by prompt identification of a specific causative agent for VAP through direct examination for ICOs. The authors correctly conclude that PBAL for direct examination for ICOs is not a reliable method of discerning which organisms are infecting patients with clinical diagnosis of VAP. If we presume that prompt treatment with appropriate antibiotics improves outcome (1), it is not clear how identification of ICO's will direct one to use the appropriate antibiotics - culture and sensitivity results are still required and still entail a potential 24-72 hour delay.

References:

1. Iregui M, Ward S, Sherman G, Fraser VJ, Kollef MH. Clinical importance of delays in the initiation of appropriate antibiotic treatment for ventilator-associated pneumonia. Chest. 2002;122(1):262-8. [abstract]

 

Comments

Back to the EB Journal Club Index