Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

Early severe neutropenia and thrombocytopenia identifies the highest risk cases of severe meningococcal disease.

Peters MJ, Ross-Russell RI, White D et al.

Pediatr Crit Care Med 2001; 2:225-231. [abstract]

Reviewed by Anandapadmanaban Gourishankar MD, , Texas Children's Hospital, Houston, TX

Review posted December 12, 2005

I. What is being studied?

Study objective:

The primary aim was to study the performance of extreme neutropenia and thrombocytopenia on presentation as predictors of outcome in meningococcal disease (MD); and to study the performance of these predictors and other established predictors of mortality in MD in a contemporary population.

Study design

Observational study with prospective and retrospective components.

Patients included:

Two populations of pediatric patients with MD were included: (1) a developmental set, and (2) a validation set. The developmental set included all patients with clinical MD admitted to a tertiary PICU (Great Ormond Street Hospital) between 11/97 and 12/99 (N=32, prospective component). The validation set included all patients with clinical MD admitted to two other tertiary PICUs (Cambridge and Manchester) between 1993 and 1999 (N=195, retrospective component). The study populations included patients that died on arrival and died of cerebral herniation. Incomplete data on 26 patients from validation set were assumed normal.

Patients excluded:

Not mentioned

The outcomes assessed:

Mortality and amputation

II. Are the results of the study valid?

Note: These questions follow from Randolph AG et al. Understanding articles describing clinical prediction tools. Crit Care Med 1998;26:1603-1612. [abstract]

1. Was a representative and well-defined sample of patients at a similar point in the course of the disease? Was follow-up sufficiently long and complete?

The sample population was children with clinical MD transported to a tertiary hospital, however the term clinical MD is not explicitly defined in the manuscript. Duration of follow-up is also not explicitly defined but was at least until PICU discharge.

2. Were all potential predictors included?

They included patient's age, Glasgow Meningococcal Septicemia Prognostic Score (GMSPS), Pediatric Index of Mortality (PIM), Malley score, first platelet and neutrophil counts, and the product of the first platelet and neutrophil counts (PN). Malley score was not included in the univariate analysis but included in the logistic regression model. Other potential predictors of poor prognosis like presence of petechiae for less than 12 hours before admission, absence of meningitis, WBC and ESR were not included.

3. Did the investigators test the independent contribution of each predictor variable?

Yes. Eight variables (GMSPS, Malley score, PN product, log₁₀ neutrophil count, log₁₀ platelet count, log10 PN product, log₁₀ age and log₁₀ PIM were analyzed by multiple logistic regression using the developmental sample of patients.

4. Were outcome variables clearly and objectively defined?

Yes. Outcome was death (while in the PICU) and amputation. In the developmental sample, 6/32 patients died and there were no amputations. In the validation sample, 24/195 patients died and 3/195 had an amputation.

III. What are the results?

1. What is(are) the prediction tool(s)?

The product of first platelet and neutrophil counts was the strongest predictor of mortality in the logistic regression model (-2 log likelihood for PN product was lower than the Malley score, GMSPS, PIM score, Age). Although PN product did well as a predictor compared to the other scores, there was no confidence interval for the exponential of estimated coefficient (equivalent of odds ratio) reported. The receiver-operating curve (ROC) for the PN score had a greater area under the curve (AUC) than the ROC for the Malley score or GMSPS. The AUCs for PN score, Malley score, GMSPS are 0.97, 0.92 and 0.81 respectively.

2. How well does the model categorize patients into different levels of risk?

In general, the sensitivity, specificity and positive predictive values (PPV) for the predictor PN product < 40 are higher than for the GMSPS and Malley scores in both the developmental and validation sets. The sensitivity, specificity and PPV for PN product are higher in the developmental set than in the validation set. PN product < 40 has a PPV of 100% in the developmental sample meaning that of those patients that have PN product < 40, all will die. The PPV for PN < 40 drops to 66% in the validation sample.

Score	Metric	Observed DS (%, crude, 95% CI)	Observed VS (%, crude, 95% CI)
PN < 40	PPV	100%; 5/5 (48-100)	66%; 4/6 (22-94)
	Sensitivity	83%; 5/6 ( 35.9-99.6)	17%; 4/24 (4.7-37.4)
	Specificity	100%; 26/26 (86.8-100)	99%; 169/171 (97-100)
Malley3	PPV	83%; 5/6 (35.9-99.6)	39%; 7/18 (17-64)
	Sensitivity	83%; 5/6 (35.9-99.6)	26%; 7/27 (11-46)
	Specificity	96%; 25/26 (80.3-99.9)	93%; 157/168 (89-97)
GMSPS 12-15	PPV	42%; 6/14 (18-71)	30%; 7/21 (14.6-57)
	Sensitivity	100%; 6/6 (54-100)	54%; 7/13 (25.1-80.8)
	Specificity	69%; 18/26 (48.2-85.7)	84%; 75/89 (75-93)

(DS = development set, VS = validation set)

3. How confident are you in the estimates of risk?

As mentioned above, there were no confidence intervals reported for the multiple logistic regression models. Confidence intervals for sensitivity, specificity, PPV are given for the PN product for both the development and validation sets. Assuming they are 95% CI, they are broad. For example, the confidence interval for PPV for PN product < 40 in the developmental sample is 48%-100%, and in the validation sample is 22-94%. This may be explained in part in the developmental set by a small sample size. The wide confidence intervals however mean that we must be cautious in using the PN product to predict mortality.

IV. Will the results help me in caring for my patients?

1. Does the tool maintain its prediction power in a new sample of patients?

The PPV of mortality for PN product <40 is 100% in the development set, but it is 66% in the validation set. Also sensitivity and specificity varied between the developmental and validation samples. The ROC curve for PN product using the validation set had similar AUC as compared to GMSPS and Malley scores.

2. Are your patients similar to those patients used in deriving and validating the tool(s)?

The patients are admitted to a tertiary care center and to a Pediatric intensive care unit but it's not clear that children with this disease are similar across the globe. The demography, socioeconomic status, past medical condition may vary. Indeed, meningococcal disease appears much more prevalent in the UK than the US. It would be difficult to collect 32 cases of this disease from a single center in the US in 2 years!

3. Does the tool improve your clinical decisions?

In the presence of already established scores, PN product per se is not outstanding to use in practice to make individual patient decisions. What may be unique about this score is its objectivity and theoretical potential to identify patients at high risk on presentation, prior to cardiovascular instability.

4. Are the results useful for reassuring or counseling patients?

No. It is too early for me to use this tool from this study. As the author points out that this is an objective measure as compared to other scores, it needs more attention in well-established trials. Unlike PN product, GMSPS and Malley score incorporate historical data and physical examination findings. These can have intra-observer variability and exact timing of recording these variables can be different from patient to patient. The fact that PN product is a readily available test and PN <40 predicts mortality in MD, it can be a valuable tool for identifying at risk patients in clinical trials.

 

Comments

Back to the EB Journal Club Index