PREDICTION TOOL

Criteria abstracted from The Users' Guides to the Medical Literature series in JAMA

Crit Care Med. 2003 Mar;31(3):960-7..[abstract]

Review posted January 21, 2008

1. Are the results valid?
   
   
   1. Did the overview address a focused clinical question?
      
      The objective of this study was, “To improve the level of confidence for prognosis of non-awakening based on SEP results, by systematic evaluation of the medical literature and a search for rare cases in which SEPs might falsely predict non-awakening.� Non-awakening was defined as the occurrence of death or PVS by the time of study completion; awakening was defined as the absence of these.
      
      
   2. What was the study design?
      
      In this systematic review, the literature was searched for studies that evaluated SEPs as a predictive tool for awakening from coma. Included studies were not analyzed separately. Instead, the authors extracted outcome data from these studies, and re-categorized them into 4 groups according to their determinations of coma etiology and age. These 4 groups were adults with hypoxic-ischemic injury (HIE) (n=1,136), adults with intracranial hemorrhage (ICH) (n=157), adults and adolescents with traumatic brain injury (TBI) (n=838), and children and adolescents with coma of any etiology (n=570). The percentage awakening and 95% CIs were then calculated for each group according to the presence of absence of SEPs. Thus, traditional meta-analytical methods were not used.
      
   
   
   3. Were all potential predictors included?
      
      The value of other outcome predictors such as Glasgow Coma Scale scores, hypotension, sedation administration, or pupillary abnormalities were not evaluated in this analysis. Evoked potentials from nerves other than the median were also not considered.
      
      
   4. Were the criteria used to select articles for inclusion appropriate?
      
      The authors included studies that reported initial cortical responses from median nerve stimulation. Study design was not specified nor were patient management strategies, though single case reports, case series with < 4 patients, and non-peer-reviewed articles were excluded. Patients had to be comatose due to brain injury, and have had SEPs performed. Patients of all ages were included but re-categorized for analysis.
      
      12 of 57 studies that met original inclusion criteria did not report data in a fashion that etiologies of coma and/or ages of patients could be extracted. 4 additional articles did not distinguish between abnormal, unilaterally absent and bilaterally absent SEPs, and thus were also excluded. These 16 trials met inclusion criteria and represented at least 1,317 children and adults (or ~ 33% of 4,018 potentially included patients), but were not included in the pooled analysis because their data could not be discriminated. Data was extracted and analyzed from the 41 remaining studies.
      
      While the inclusion criteria are sufficiently general to minimize the introduction of author bias into the selection of studies, the large number of patients excluded from pooled analysis and the absence of a specified study design may confound summary estimates.
      
      
   5. Is it unlikely that important, relevant studies were missed?
      
      It is possible that studies were missed. The terms “somatosensory or evoked potentials, coma or resuscitation, and prognosis or outcome� were used in various combinations to search MEDLINE only, for studies published in English only. Articles recovered were checked for references. Other databases, conference proceedings, and authors were not queried for unpublished or non-English, published trials. Publication bias was not assessed.
      
      
   6. Was the validity of the studies appraised? Were assessments of studies reproducible?
      
      Included studies were not appraised for validity because the studies were not treated separately. Data was extracted by one author and reviewed for accuracy by another author.
      
   
   
   7. Were outcome variables clearly and objectively defined?
      
      Glasgow outcome scale categories were combined to form two outcome groups: death and persistent vegetative state (PVS/d) comprising “non-awakening,� and disability or better comprising “awakening.� However, a pre-defined follow-up period to allow accurate outcome categorization of patients was missing from this analysis. Follow-up periods for outcome assessments ranged from discharge to 4 yrs after injury.
      
   
   
   8. Were the results similar from study to study?
      
      Table 1 in the paper lists the 41 studies included in the analysis, and their respective positive predictive values (PPVs) of bilaterally absent SEPs for non-awakening. According to the authors, heterogeneity and sensitivity testing were not performed because trials were not treated separately. The ranges of PPVs of studies that included adults with HIE and adults with ICH where quite tight. In contrast, the range of PPVs for studies that included adults and adolescents with TBI was broad, and extremely so for studies that included children and adolescents with coma of any etiology. Because of the wide variability in results, it is very difficult to generalize based on the means.
      
   
   
   9. Was the representative group of patients completely followed up?
      
      This is a major limitation of the study. It is unclear if all patients were followed for a long-enough period of time to assess outcome. For example, assessment of awakening in less than one month after injury would likely be an insufficient duration to measure the outcome of interest.
      
      
2. What are the results?
   
   
   1. What are the overall results of the overview? How precise were the results?
      
      The included 41 studies represented a total of 2,701 patients. (Only 2,603 can be accounted for from table 1 of the paper, however.) Children of ages 10 to 18 years were included in either the “children and teens with TBI� or “children with any etiology� groups, but not both. Thus, adolescents were split between groups. % awakenings were re-calculated for simplicity by etiology and presence/absence of SEPs. These are depicted in figure 1 of the paper, tabulated in figure 2, and summarized below.

Rates of non-awakening in patients with normal SEPs noted in column 3 of the above table are derived from Figure 2 of the paper (100 – % awakened with normal SEPs). * SEPs are only denoted as “present� or “absent� in adult patients with ICH; “normal� vs “abnormal� is not discriminated.

2. How well does the model categorize patients into different levels of risk? How confident are you in the estimates of risk?
   

This systematic review categorized patients by etiology of coma and age in an attempt to ascertain different predictive values of SEPs for awakening. While summary statistics were not calculated as in traditional meta-analyses, rates of awakening with 95% CIs were calculated from re-categorized patients (see table above.)

3. Will the results help me in caring for my patients?
   
   
   1. Can the results be applied to my patient care?
      
      Several concerns mitigate application of the reported results, especially to pediatric patients.
      
      It was challenging for the authors to extract data from the studies that met their inclusion criteria and re-categorize them according to their analysis plan. Designations of SEP results, age, etiology, and outcome in the studies are described as sometimes vague. The included studies also appeared to be heterogeneous in several respects. For instance, techniques of SEP measurement varied among studies. Timing of SEP testing ranged from one to several days after injury, and the stimulation frequency ranged from 1 – 9 Hz. Definitions of normal vs abnormal SEPs were also variable. Follow-up times to assess awaking were not pre-defined, and ranged from discharge to 4 yrs after injury. Per the authors, levels of sedation were also not controlled for.
      
      Quite notably, the authors acknowledge that “many studies were retrospective, and thus SEP results could have influenced decisions on life support and ultimately outcome� – a particular methodological weakness for a systematic review of this nature. Other methodological short-comings include a non-comprehensive literature search, omitted evaluation of the quality of included studies, and the absence of evidence that patient data from individual studies could be combined (i.e. that studies lacked clinical, methodological, and statistical heterogeneity.)
      
      Lastly, the re-categorized cohort of children and adolescents represented multiple etiologies of coma which were not differentiated. This cohort had a higher rate of awakening with absent SEPs. Because all children were lumped together, any differences in pediatric outcomes due to etiology of brain injury are indeterminable.
      
      
   2. Were all clinically important outcomes considered?
      
      Awakening or non-awakening from coma was the only clinically important outcome in this study. The authors defined non-awakening as PVS/d according to Glasgow outcome score criteria; they acknowledge that some patients with “severe disability� had to be categorized as “non-awakening,� however.
      
      
   3. Does the tool maintain its prediction power in a new sample of patients? Are your patients similar to those patients used in deriving and validating the tools?
      
      SEPs were not tested or validated prospectively in a new sample of patients in this study – a necessary step for acceptance of a prediction tool.
      
   
   
   4. Are the benefits worth the harms & costs? Will the results lead directly to selecting or avoiding therapy? Are the results useful for reassuring or counseling patients?
      
      In the devastating circumstance of coma due to brain injury, the monetary cost of a predictive tool such as SEP is non-prohibitive, especially given that accurate prediction of non-/awakening from coma may lead to more appropriate allocation of medical resources. Instead, the relevant costs are those emotional and moral ones associated with a patient awakening when predicted not to do so, and with not awakening when predicted to do so.
      
      The quality of this systematic review is insufficient to guide clinical decision-making, particularly in pediatric patients. Absent SEPs were found to have a robust PPV for non-awakening in adults with HIE, and 0% awakening with absent SEPs (95% CI 0 – 1%) in a cohort of over 1,100 patients. However, a 95% CI that includes 1% (11 patients in this instance) may be unacceptably high to influence a clinical decision to withdraw support. Furthermore, the re-calculated rate of awakening should be looked upon with skepticism in light of the concerns discussed above.
      
      In children and adolescents, the PPV of absent SEPs for non-awakening ranged from zero to 1.00 in the included studies. The rate of awakening for children with absent SEPs was 7% (95% CI 4 – 10%). It is unclear whether this is a true value, given that it may have been inappropriate to combine patients from the different studies, that some adolescents were combined with adults, and that search & inclusion criteria were not thorough. It is also unclear what clinical scenario this value is applicable to, given that no etiological differentiation was provided for patients in this cohort. The rate of awakenings is clearly too high to acceptably influence decisions of support withdrawal.

Last Updated: January 21, 2008

Fill out our Feedback Form | Editorial Notes | About Us