Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

A prospective study of outcome predictors after severe brain injury in children.

Carter BG, Butt W.

Intensive Care Med. 2005 Jun;31(6):840-845. [abstract]

Reviewed by Linda Thompson MD, Baylor College of Medicine, Texas Children's Hospital, Houston TX

Review posted November 17, 2005

I. What is being studied?

Study objective:

To directly compare the predictive performance of somatosensory evoked potentials (SEPs) to commonly used clinical signs of: 1) motor response to supraorbital pressure and 2) pupillary response to light in severely brain-injured children.

Study design

Prospective clinical trail of 102 children with severe, acute brain injury (traumatic or non-traumatic including hypoxic-ischemic injury) admitted to the ICU at Royal Children's Hospital in Melbourne, Australia. The authors do not mention the years during which patients were recruited, but all were likely recruited before 1999.

The authors aimed to record SEPs, motor responses and pupillary responses for the first 9 days of admission. Data was recorded daily until the patient awoke, was discharged from the hospital, or died. The predictive power of the initial and last SEP was analyzed, but note that the last SEPs were recorded within the first 2 days in 59 of the patients. The predictive power of SEPs obtained at least 48 hours after admission was also analyzed. While SEP results could fall into one of 5 categories, the authors only analyzed data from patients with bilaterally normal SEPs or bilaterally absent SEPs. Similarly, while the motor response could fall into one of 5 categories, patients demonstrating two of the responses (flexion response and extension response) were not analyzed.

Inclusion criteria: Glasgow Coma Scale (GCS) on admission less than 8 and details of the injury consistent with severe injury according to clinical judgment.

Exclusion criteria: children less than 1 month, children brain dead prior to SEP testing, and patients with pre-existing neurological disease

The outcomes assessed:

The outcome for the study is Glasgow Outcome Scale (GOS) assessed 5 years after injury. Outcomes were grouped into two classifications: favorable or unfavorable. Favorable outcomes included patients found by GOS to be "normal" to "independent and moderately disabled." Unfavorable outcomes ranged from "dependent and severely disabled" to "death." At 5 years, 47 patients had a "good" outcome, 10 were found to be "moderately disabled," 2 had "severe disability," 3 were "vegetative," and 40 had died.

II. Are the results of the study valid?

Note: These questions follow from Randolph AG et al. Understanding articles describing clinical prediction tools. Crit Care Med 1998;26:1603-1612. [abstract]

1. Was a representative and well-defined sample of patients at a similar point in the course of the disease? Was follow-up sufficiently long and complete?

Yes. The group of 102 patients included both genders and approximately half the patients were under the age of 6. The etiologies of brain injury included trauma (39%), hypoxic-ischemic encephalopathy (34%), infection (18%), hemorrhage (6%), a case of emboli with global injury, and a case of hemolytic-uremic syndrome. The heterogeneity of the group may be a double-edged sword, however, making application of some of the overall results difficult. For prediction of unfavorable outcome, sensitivity and specificity of the three measures were significantly different in patients suffering traumatic brain injury than in patients with hypoxic-ischemic injury.

2. Was follow-up sufficiently long and complete?

Yes. All 102 patients appeared to have had GOS scores assessed at 5 years after injury. The follow-up is sufficiently long, as no neurological improvement would be expected after this time.

3. Were all potential predictors included?

Yes and No. SEP is the modality of neurological prediction currently showing the most promise, but it is not the only potential predictor of neurological outcome. MRI, EEG, initial GCS score, visual and auditory evoked potentials have also been studied as predictors of neurological outcome and could have been included if the study aimed to include all potential predictors of outcome. Visual and auditory evoked potentials have not proven predictive of outcome in children (1,2), however, and MRI shows only 50% specificity in predicting moderate to severe disability in children (3). Serial EEGs are effective at predicting poor outcome, but SEPs are more powerful than EEG in predicting poor outcome in children (4,5). It is worth noting that the above modalities have been studied well only in children with hypoxic-ischemic brain injury.

Serial neurological examinations have been studied extensively as a means of predicting outcome, again in patients with hypoxic-ischemic coma. In children, GCS less than 5 at 24 hours has a positive predictive outcome for severe disability or death of 100 percent, though sensitivity is considerably less (6). Considering the available modalities for prediction of neurological outcome that were proven most effective and widely available at the time of the study, the authors' choice to compare SEP, pupillary response and motor response is appropriate.

4. Did the investigators test the independent contribution of each predictor variable?

Motor and pupillary responses were each tested against SEPs for ability to predict favorable and unfavorable outcome with the initial examination and the last examination. All three modalities were compared to each other when the tests were performed at or after day 2.

5. Were outcome variables clearly and objectively defined?

Yes and No. All patients were evaluated at 5 years after injury by the Glasgow Outcome Scale. Investigators evaluating the patients were blind to results of SEPs, motor responses and pupillary responses from the initial hospitalization. The GOS is a gross measure of outcome that categorizes patients into 1 of 5 outcome categories. It has largely been replaced by other scales, but is still used to measure outcome after traumatic brain injury. Using a standard format to interview patients and families can significantly increase the interrater reliability of the scale (7). The study does not mention whether the GOS for all patients was assessed by the same investigator or whether any standard format for the interview was used by the investigator(s).

III. What are the results?

1. What is(are) the prediction tool(s)?

The prediction tool studied is SEP on initial exam and on last exam before patient waking, patient discharge or death. SEPs are also combined with motor response and pupillary response using "and" and "or" to evaluate outcome prediction.

2. How well does the model categorize patients into different levels of risk?

SEP versus motor response

SEPs show superior sensitivity and equal specificity to a motor response of at least withdraw to pain on the initial or last exams when predicting a favorable outcome. SEPs show superior sensitivity and specificity when compared to an absent motor response on initial exam for predicting an unfavorable outcome. An absent motor response on the last exam, however, is more sensitive than a bilaterally absent SEP when predicting an unfavorable outcome. ROC curves are provided by the authors to show the superiority of SEP over motor response using the last test result.

It is important to note that patients with paralysis were excluded from the analysis, but not patients who received sedation / analgesia. All patients with absent motor responses who went on to have favorable outcomes had received sedation.

The authors use a diagnostic odds ratio (OR = likelihood ratio of a positive test / likelihood ratio of a negative test) as a single indicator of each test's performance (8). A high OR in this context reveals a strong association between a positive test and the disease, but does not help convert this directly to the likelihood of the disease in a given patient based on a positive test.

Outcome prediction	Sensitivity (%)	Specificity (%)	OR (95% CI)
Favorable outcome
Initial test
Motor response ≥withdraw	33.3	96.4	13.5 (7.3-19.7)
Normal SEP	73.3	96.4	74.3 (67.9-80.6)
Last test
Motor response ≥withdraw	35.0	100	37.2 (28.5-46.0)
Normal SEP	75.0	100	188 (180-197.7)
Unfavorable outcome
Initial test
Motor absent bilaterally	75.0	73.3	8.3 (4.2-12.3)
SEP absent bilaterally	82.1	93.3	64.4 (58.2-70.5)
Last test
Motor absent bilaterally	81.8	65.0	8.4 (4.6-12.1)
SEP absent bilaterally	69.7	100	91.8 (83.2-100)

SEP versus pupillary response to light

SEPs show superior specificity, but inferior sensitivity, when compared to initial or last pupillary response for predicting favorable outcome. SEPs show superior sensitivity and near-equal specificity when compared to initial or last pupillary response for predicting unfavorable outcome. The authors provide ROC curves to show the superiority of SEP over pupillary response to light using the last test result.

Outcome prediction	Sensitivity (%)	Specificity (%)	OR (95% CI)
Favorable outcome
Initial test
Pupil response present	87.0	51.2	7.0 (3.6-10.4)
Normal SEP	67.4	95.1	40.3 (36.0-44.6)
Last test
Pupil response present	97.9	58.5	66.4 (60.7-72.0)
Normal SEP	68.8	100	179.4 (171.0-187.8)
Unfavorable outcome
Initial test
Pupils absent bilaterally	51.2	87.0	7.0 (3.6-10.4)
SEP absent bilaterally	70.7	95.7	53.2 (48.8-57.5)
Last test
Pupils absent bilaterally	58.5	97.9	66.4 (60.7-72.0)
SEP absent bilaterally	75.6	95.8	71.3 (66.9-75.7)

SEP versus motor response versus pupillary response on or after day 2 post-admission

Outcome prediction	Sensitivity (% )	Specificity (%)	OR (95% CI)
Favorable outcome
Motor response ≥withdraw	0	100	2.1 (0-17.8)
Pupil response present	82.3	25.0	1.6 (0-6.5)
Normal SEP	47.1	100	24.2 (15.2-33.1)
Unfavorable outcome
Motor absent bilaterally	33.3	25.0	0.2 (0-7.8)
Pupil absent bilaterally	25.0	82.3	1.6 (0-6.5)
SEP absent bilaterally	38.5	94.1	10.0 (3.6-16.4)

SEP combined with motor response and pupillary response using "and" and "or"

For predicting a favorable outcome, many of the resulting combinations are not superior to the individual tests used alone. The best resulting combination was with the last motor response "or" SEPs which increased the sensitivity/specificity to 85%/100% versus 75%/100% seen with SEPs alone. For predicting an unfavorable outcome, "or" combinations increased the sensitivity, but decreased the specificity to lower than that seen with SEP, motor response or pupillary response alone. All "and" combinations increased the specificities to 100%, but decreased sensitivity to lower than that seen with SEP, motor response or pupillary response alone. The highest sensitivities are achieved by combining SEPs with initial (67.9%) and last motor response (63.6%), but are not higher than the sensitivities of SEPs alone.

3. How confident are you in the estimates of risk?

All methods of outcome prediction evaluated (i.e. SEP, motor response to pain and pupillary response to light) show high odds ratios with 95% confidence intervals significantly over 1. The odds ratios reflect a high degree of confidence that any of the modalities individually, provided that the results are sufficiently normal or abnormal, can predict favorable versus unfavorable outcome using initial or last exam. The study is essentially a comparison of good outcome versus death as 47/57 in the favorable group had a good outcome and 40/45 died in the unfavorable outcome group. The discriminating ability of these tests is significantly enhanced by how different the two groups are.

Odds ratios were significantly higher in SEPs compared to those seen with motor exam without overlap of confidence intervals. Odds ratios were significantly higher in SEPs compared to those seen with pupillary response without overlap of confidence intervals except when attempting to predict unfavorable outcome using the last exam. In this case, 95% confidence intervals overlap considerably and cast doubt that SEPs are superior to pupillary response in this instance.

Results obtained after day 2 are not better than either the initial or last test results. In fact, the predictive ability of motor and pupillary response is quite poor in this group of patients. The results should be interpreted with caution, however, as many patients were not included in the analysis because of paralysis, improvement or death. The authors do not give an "n" for this group, but comment that 42 patients were paralyzed and 59 improved or died within the first 2 days.

IV. Will the results help me in caring for my patients?

1. Does the tool maintain its prediction power in a new sample of patients?

Unknown. The authors did not test the validity of the predictive tool in a new population.

2. Are your patients similar to those patients used in deriving and validating the tool(s)?

Yes and No. The patients included in the study were of both genders, with approximately half the patients under 6 years approximately one third of the patients under 2 years. The etiologies of traumatic injury, hypoxic-ischemic encephalopathy, and infection are also the most common causes of acute brain injury seen in my PICU. The patients in my PICU, however, often have motor and pupillary responses that are less obviously reassuring or obviously ominous than the responses of the patients in this study. My patients may also have SEP results that fall into one of the three categories not included in this study. The authors do not provide the number of patients who were not included because they fall into these "in-between" categories, leaving me uncertain of how to apply this tool to many of my patients.

3. Will the results lead directly to selecting or avoiding therapy?

No. With the wide confidence intervals and the exclusion of patients with intermediate responses on SEP and motor examination, the authors do not clearly demonstrate the superior ability of SEP testing to predict neurological outcome in children with brain-injury. Consider also that evoked potential testing is technically difficult to perform and can have an erroneously absent response in patients with focal lesions, subdural effusions or craniotomies (9,10). For decisions of gravity that often accompany a brain injury (e.g., whether or not to withhold life-sustaining treatment, proceed with tracheostomy, etc.), SEP results may provide additional information, but should not be the sole basis for a decision until better data is available. One obvious practical application is when high levels of sedation and/or neuromuscular blockade is present, SEP's may be helpful.

4. Are the results useful for reassuring or counseling patients?

Not necessarily, although SEP's may provide useful adjunctive information. Parents may put more faith in a machine that can determine their child's prognosis rather than a physician's serial examinations and clinical experience so parents should be counseled on the limitations of SEP testing prior to the testing and should be warned about the possibility of results that are neither normal nor absent for which good predictive data does not exist.

References

1. Shewmon DA. Coma prognosis in children. Part I: definitional and methodological challenges. J Clin Neurophysiol 2000; 17:457. [abstract]
2. Frank LM, Furgiuele TL, Etheridge JE. Prediction of chronic vegetative state in children using evoked potentials. Neurology 1985; 35:931. [abstract]
3. Christophe C, Fonteyne C, Ziereisen F, et al. Value of MR imaging of the brain in children with hypoxic coma. Am J Neuroradiol 2002; 23:716. [abstract]
4. Zandbergen EG, de Haan RJ, Stoutenbeek CP, et al. Systematic review of early prediction of poor outcome in anoxic-ischemic coma. Lancet 1998; 352:1808 [abstract].
5. Chen R, Bolton CF, Young B. Prediction of outcome in patients with anoxic coma: a clinical and electrophysiologic study. Crit Care med 1996; 24:672. [abstract]
6. Mandel R, Martinot A, Delepoulle F, et al. Prediction of outcome after hypoxic-ischemic encephalopathy: a prospective clinical and electrophysiologic study. J Pediatr 2002; 141. [abstract]
7. Wilson JT, Pettigrew LE, Teasdales GM. Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use. J Neurotrauma 1998; 15(8):573. [abstract]
8. Glas AS et al. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 2003; 56:1129. [abstract]
9. Trübel HK, Novotny E, Lister G. Outcome of coma in children. Curr Opinion Pediatr 2003; 15:283. [abstract]
10. Carter BG, Butt W. Review of the use of somatosensory evoked potentials in the prediction of outcome after severe brain injury. Crit Care Med 2001; 29:178. [abstract]

 

Comments

Back to the EB Journal Club Index