Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

Neurodevelopmental outcome of young pediatric intensive care survivors of serious brain injury.

Robertson CMT, Joffe AR, Moore AJ, Watt JM.

Pediatric Critical Care Medicine 2002; 3(4):345-350. [abstract]

Reviewed by Robert Graham MD, Children's Hospital, Boston

Review posted September 15, 2003

I. What is being studied?

Study objective:

The study objective was threefold. First, authors sought to determine the proportion of admissions with severe head injury due to closed head injury (CHI) or hypoxic-ischemic encephalopathy (HIE) in children under 3 years of age at a regional pediatric intensive care unit. Second, they assessed outcome of these children at least six months postinjury. And, last, relationships between measured outcomes and predictors of outcome obtained within the first 24 hours were assessed.

Study design

This was a prospective, descriptive, inception cohort study.

II. Are the results of the study valid?

Note: These questions follow from Randolph AG et al. Understanding articles describing clinical prediction tools. Crit Care Med 1998;26:1603-1612. [abstract]

1. Was a representative group of patients completely followed up?

The investigators screened 1294 consecutive pediatric intensive care admissions under the age of 3 years for severe brain injury and found 53 eligible for inclusion - 4% of all admissions. Inclusion criteria included severe brain injury, defined as any child with either CHI or HIE and a Glasgow Coma Score (GCS) < 8 within the first 24 hours postinjury.

Exclusion criteria included children with preinsult diagnoses of congenital central nervous system malformations, chromosomal or metabolic disorders, cerebral palsy, or primary heart disease. These were established to minimize confounding variable when assessing outcome disability. There was no significant referral filter bias as the study site was reportedly the sole regional center for critically ill children.

Investigators engaged in a multidisciplinary follow-up effort to assess outcomes at least six months after the injury. They achieved 94% (50/53) follow-up despite the long interval. Only three of the 41 survivors were lost to follow-up.

Cohorting of children with CHI and HIE allowed for increased sampling, but may have some limitations when interpreting the findings. These are presumably heterogeneous groups based on mechanism and pattern of injury as well as the potential concurrent, confounding injuries. Even within these categories, there likely exists great variability. External impact injuries differ from nonaccidental trauma in infants (1]). Similarly, the injury and outcome associated with near-miss sudden infant death syndrome cannot necessarily be clustered with those of septic shock or near drowning. All 3 of the children lost to follow-up had suffered HIE. Their outcomes may actually have been important given the small sample size.

2. Was follow-up sufficiently long and complete?

Yes. The study ambitiously attempted to assess neuro-developmental outcome following severe brain injury. Thus, the investigators had to allow for an adequate recovery interval between injury and follow-up as well as recognizing the dynamic nature of toddler development. Three children (6%) were lost to follow-up. All children were seen at 6 month post insult. Those under 6 months of age at the time of the initial injury underwent formal evaluation at 18 months of age to permit more accurate outcomes measurement. The authors acknowledge that longer-term follow-up would be ideal. Yet, they do not agree with the premise of childhood brain plasticity and continued synaptogenesis (2), suggesting that 6 months should be sufficient time to determine residual deficits. Regardless, additional assessment would be informative to determine if there has been adaptation or interruption of future learning and development.

3. Were all potential predictors included?

Predictive indices included gender, age < 1 year, the need for epinephrine, nonreactive pupils, arterial pH ≤ 7.2, and GCS = 3. Investigators also adapted the Pediatric Coma Scale (PCS) to reflect severity of injury in different age groups. The ratio of PCS versus maximum age aggregate was compared to a GCS of 8 versus a maximum score of 15, thus a ratio < 0.6 represented severe injury. This is not an exhaustive list of potential acute care indicators but all are readily available, less subjective, and pertinent to care providers in the field and in the intensive care. Other more global measurements, such as PRISM scores, could have been utilized but data collection may have been incomplete. Radiologic findings or the need for neurosurgical intervention would also have been interesting, but may have reflected difference in injury type rather than serving as a uniform prognostic for brain injury outcome.

4. Did the investigators test the independent contribution of each predictor variable?

No. Investigators tested the contribution of both demographics and primary predictor variables individually and in aggregate but never tested their independent roles using formal regression analysis. Individual acute care variables, including apnea at the accident scene (p < 0.05), the need for epinephrine (p < 0.001), nonreactive pupils (p < 0.001), arterial pH ≤ 7.2 (p < 0.001), GCS = 3 (p < 0.001), and PCS score ratio < 0.6 (p < 0.05), were correlated with Glasgow Outcome Scale (GOS) (3, 4) scores of 1-4, equating with death or disability. The presence of any adverse acute care variable was then tested against GOS and neurologic outcome as assessed using the Mental Developmental Index and the Performance Developmental Index. Comparison was further subdivided based on mechanism of injury, CHI vs. HIE, and sensitivity and specificity were calculated.

5. Were outcome variables clearly and objectively defined?

Yes, mortality and morbidity were determined by a multidisciplinary assessment team, utilizing the GOS and the Bayley Scales of Infant Development-II (BSID-II). Death as a primary outcome index was evaluated in the PICU and, again, when obtaining follow-up for comprehensive neurologic status. A minimum of 6 months post injury and chronologic age of, at least, 18 months were established as the lower boundary of the follow-up interval. This period allowed for recovery, continued developmental progress, and facilitated testing based on the instruments available.

Investigators recognized the limitations of the arsenal of outcome measures in children. While the GOS was initially developed for adult stroke patients, they elected not to utilize the Pediatric Overall Performance Category (POPC) and the Pediatric Cerebral Performance Category (PCPC) suggesting that these tools were not practical for use in younger patients. This supposition may be fallacious, as Fiser (5) devised these indices based upon the GOS and has validated across the pediatric age spectrum. The King's Outcome Scale for Childhood Injury and the Wee-FIM were also excluded because they were too developmentally advanced. The investigators converted the GOS into a dichotomous outcome variable by grouping score of 1-4 as "poor" versus a score of 5 corresponding to a "good recovery". A more detailed neuro-developmental evaluation was performed using the components of the BSID-II (i.e., Mental Developmental Index - MDI and Performance Developmental Index - PDI) as well as ophthalmology and audiology screenings.

III. What are the results?

1. What is(are) the prediction tool(s)?

The investigators did not develop a prediction tool. The investigators demonstrated a statistically significant relationship between each predictive marker, apnea, need for epinephrine, nonreactive pupils, arterial pH ≤ 7.2, GCS = 3, and PCS score ratio < 0.6, and poor outcome as defined as GOS score of 1-4. All of the 8 patients who required more than 2 doses of epinephrine for resuscitation had exceptionally poor outcomes, with 7 deaths and severe disability in the single survivor. The PCS did not correlate any better with outcome than the GCS.

Further calculation of sensitivity, specificity and likelihood ratios for the individual prediction indices suggests that no individual variable is terribly discriminative. While the need for epinephrine and nonreactive pupils yield infinite positive likelihood ratios, the sensitivities are only 0.48 and 0.44, respectively. GCS score of 3 appears to be the best predictor of poor outcome with a likelihood ratio of 5.40 (95% CI: 1.82, 15.94) and sensitivity and specificity of 0.70 and 0.87. Again, this did not hold for the PCS, suggesting that the authors adaptation of a PCS ratio < 0.6 may not equate directly with GCS and may not be valid.

Investigators examined all factors in aggregate, tabulating sensitivity and specificity against dichotomous GOS levels (1-4 vs. 5), GOS levels excluding death (2-4 vs. 5), and the two BSID-II components, MDI and PDI (< 85 vs. ≥ 85). This was further stratified by the nature of the injury, CHI or HIE. The presence of one or more adverse markers were both sensitive and specific for adverse outcomes but the relationship was greatest for patients with HIE as assessed by the BSID-II. For children suffering CHI the likelihood ratio of a MDI < 85 with any adverse acute care predictor was 1.50 whereas those with HIE the likelihood was nearly infinite. Similarly, positive likelihood ratios for PDI < 85 were 2.05 and 3.69 for CHI and HIE, respectively, as the prevalence of performance disabilities was generally higher in both groups.

When interpreting these results, one must consider the small sample size, high prevalence of "poor outcomes", and, most importantly, the extreme heterogeneity of the head injury etiologies. The division of the GOS and BSID-II subindexes into dichotomous outcome variables is also problematic. The investigators allude to the insensitivity of he GOS, but development of receiver operator characteristic (ROC) curves for each GOS level and each standard deviation of MDI and PDI scores would be more informative.

2. How well does the model categorize patients into different levels of risk?

The investigators did not develop a model. Presumably, all of the patients included in the study were "high risk" for poor outcome, as the investigators chose only to study those with GCS ≤ 8. Studying the other variables, while not necessarily independent markers, was meant to further stratify these patients. Analysis suggested that all of the markers, except for PCS ratio < 0.6, were useful for positive prediction of GOS 1-4 (PPV ranged from 0.86 - 1.0) yet they were poor negative predictive markers (NPV ranged from 0.34 - 0.55 with wide confidence intervals). Thus, none of the individual markers had adequate discriminating ability.

When considered as a group, the acute care predictors better categorized patients into high risk groups based upon GOS. PPV was 91.7% and NPV was 80.8%, when including death as a poor outcome. While the investigators could not plot receiver operating characteristic curves, 86% of the patients were correctly classified according to their acute predictors. The relationship between indicator and outcome, however, declines when considering BSID-II performance despite a high prevalence of poor outcome. Appropriate risk identification for MDI and PDI were only 71% and 76% overall. Division of patients by nature of injury showed a showed that this grouping of acute indicators was not helpful when identifying high risk patients following CHI. Sensitivity and specificity for the CHI group and MDI outcome were 67% and 56%.

The investigators questioned their own definition of "good recovery" based on a GOS of 5. Only 8 (15% overall) children classified as "good outcomes" had both MDI and PDI scores at or above the normal range. The high prevalence of poor outcomes (54%) and the lack of sensitive outcome measures for less affected children limits the use of prediction tools. Inclusion of all children with brain injury, regardless of initial GCS, might be interesting.

3. How confident are you in the estimates of risk?

The acute care predictors are very good at predicting poor outcome if there is a positive test. Their absence, however, provides little support when estimating risk. A negative LR reflects the likelihood that a negative test result would be expected in a patient with a good outcome compared to the likelihood that the same test result would be expected in a patient with a poor outcome. The confidence intervals are quite broad limiting the utility of "normal" test. The need for epinephrine, for example, yielded an infinite positive LR but a negative LR of 0.52 (95% CI: 0.36, 0.75). GCS of 3 produced an estimated risk with LR of 5.4 (95% CI: 1.82, 15.94) but a negative LR of 0.34 (95% CI: 0.19, 0.62). Again, the clinical uncertainty in the children with less severe outcomes supports the need for more sensitive outcome indices and would likely facilitate identification of more discriminative and precise predictive tools.

IV. Will the results help me in caring for my patients?

1. Does the tool maintain its prediction power in a new sample of patients?

The investigators did not test the prediction power of their model in a new sample and the small study size would not necessarily allow for resampling to test for accuracy. The catchment area of the study site was reportedly comprehensive, thus there should be little selection bias and a new sample should reveal similar results. The authors do not, however, specify if any critical care management decisions were influenced by the prediction markers or if there were any unit-based changes during the course of the 4 year study. Optimal testing in a new population would control for intervention and management variability. Most importantly, independent studies for children with CHI versus HIE should be pursued. These are conceptually different entities and the trends of MDI and PDI performance in the current study suggest that these are not comparable groups.

2. Are your patients similar to those patients used in deriving and validating the tool(s)?

Yes, the sample population would reflect admissions to a tertiary care facility with pediatric neurosurgical, trauma and intensive care teams.

3. Will the results lead directly to selecting or avoiding therapy?

No, the results are not compelling enough to either escalate or withhold therapy. Concomitant injury patterns were not discussed by the investigators but must also be considered. More definitive treatments and outcome measures are needed for childhood brain injury before a given predictive tool can become truly useful.

4. Are the results useful for reassuring or counseling patients?

The great variability in the outcome following severe brain injury and the subjective nature of quality of life with a disability limit the utility of these markers when discussing a child's status with the parents. If the predictive model suggests a poor outcome, providers should prepare families for that potential. If, however, the tests are negative, there is enough uncertainty that care providers would be foolhardy to provide unconditional reassurances.

The investigators recommendation and efforts to provide uniform, multidisciplinary follow-up are to be applauded. Intensive care providers, and subsequently their patients, can only benefit from a better understanding of the long-term effect and relationship of injury and ICU intervention.

References

1. Beers, S.R. and M.D. De Bellis, Outcomes of child abuse. Neurosurgery Clinics of North America., 2002. 13(2): p. 235-41. [abstract]
2. Chugani, H.T., R.A. Muller, and D.C. Chugani, Functional brain reorganization in children. Brain & Development., 1996. 18(5): p. 347-56. [abstract]
3. Rankin, J., Cerebral vascular accidents in patients over the age of 60: II. prognosis. Journal of Scotish Medicine, 1957. 2: p. 200-215.
4. Jennett, B. and M. Bond, Assessment of outcome after severe brain damage. Lancet., 1975. 1(7905): p. 480-4. [abstract]
5. Fiser, D.H., Assessing the outcome of pediatric intensive care.[comment]. Journal of Pediatrics., 1992. 121(1): p. 68-74. [abstract]

 

Comments

Back to the EB Journal Club Index