Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

Volume-Outcome Relationships in Pediatric Intensive Care Units

Tilford JM, Simpson PM, Green JW, Lensing S, Fiser DH.

Pediatrics. 2000;106(2):289-294. [abstract] [full-text for AAP members/subscribers]

Reviewed by Jeanette White MD, Children's National Medical Center, Washington DC

Review posted April 18, 2001

I. What is being studied?:

The study objective:

This study aimed to examine a risk-adjusted relationship between the volumes of patients admitted to a PICU and patient outcome. The authors specifically query whether an increase in patient volume improves mortality and decreases length of stay. Ultimately, this information would be used to assess the applicability of regionalization tactics to pediatric critical care.

The study design:

Prospective multicenter cohort study. Data from 16 of 21 PICU's that were members of the Pediatric Critical Care Study Group (PCCSG) contributed data about consecutive admissions over a 1-year period. The goal of this study was to use these data to identify whether volume of patient admissions to a PICU was associated with outcome (mortality and length of stay) after adjustment for confounding variables.

The outcomes assessed:

Primary:

• Mortality (death prior to PICU transfer)
• Length of stay in the PICU

Secondary:

• Severe disability

II. Are the recommendations valid?

Note: These questions follow from Randolph AG, Guyatt GH, Carlet J, et al. Understanding articles comparing outcomes among intensive care units to rate quality of care. Crit Care Med 1998; 26: 773-781. [abstract]

1. Are the outcome measures accurate and comprehensive?

The authors are very specific in their choice of primary outcomes: mortality and length of stay. The attractiveness of these outcomes lies in their objective measurability. However, the investigators acknowledge that these measurements may be neither comprehensive, nor an accurate representation of "quality of care". For example, an ICU which has a higher mortality rate may simply care for patients with more severe illness. In addition, lower mortality rates do not insure better long-term patient outcome. As the authors note, "reductions in mortality may come at the expense of increased morbidity". Unfortunately, since it is much more difficult to measure "increased morbidity", authors of outcomes studies often fall back on readily available, easily measurable outcomes such as mortality rate. In addition, there is certainly more to judging the adequacy of a PICU's qualifications than these two measures: patient satisfaction, mild to moderate disability, morbidity affecting quality of life or other parameters, comfort, and efficiency all must be considered. However, there is no single comprehensive outcome measure to study all of these aspects of quality of care in any single PICU.

2. Do the ICU's being compared serve similar patients?

No. Information on each unit and its patient mix was provided by the PCCSG. For each PICU, the following data were collected: teaching status of the PICU, whether the unit had a fellowship training program, whether the hospital in which the unit was located had a separate step-down or monitored care unit, whether the unit was located in a children's hospital, and the number of beds in the unit. The PCCSG reports significant variability between PICU's in age (mean age = 38.7 months, range 26.3 to 60.5 months, SD = 9.9 months), and percent of surgical patients (mean = 35.8%; range 25.7-56.0%, SD = 7.7%) or trauma patients (mean portion = 11.7%, range 1.5 Ð 28.8%, SD = 6.0%). Individual values for each center are listed in table 2 in the paper. No other demographic or patient characteristics are presented for comparison between groups.

3. Was the sample of patients sufficient and unbiased?

Data were collected from 16/21 PICU's surveyed. Each PICU was requested to contribute data on consecutive admissions over a 1-year period beginning in January 1993. A significant number of observations ( 11,106 patients) were recorded. However, there was also significant variability in the number of patient reports contributed by each center (mean = 694 patient records/center; SD = 354 patient-records/center; range = 147 - 1378 patient records/center). It is unclear whether these numbers were a true representation of each PICU's patient load. For example were all data collected for each patient or were some patients missed? A high level of quality control is an important standard in a study such as this one, where patient volume is measured as the main predictor variable.

4. Was appropriate risk adjustment undertaken?

a. Was the model valid? How validated were the severity of illness adjustment models? Were the models well calibrated in all subgroups of patients represented?

The regression models were adjusted for severity of illness using the well-accepted Pediatric Risk of Mortality (PRISM) score and the less well-known Pediatric Overall and Cerebral performance scores.

b. Were the data collected accurately?

The conclusions of this study rely heavily on the accuracy of the collected data. Unfortunately, readers are not given insight into the procedures for data collection. Few details are available about the PCCSG, the consortium that was responsible for the data collection. For example, there is no reference to the methods used by the PCCSG group to collect information. It is unclear whether the "Patient Volume" reported for each PICU in Table one of the paper represents 100% data capture, a random sample, or a non-random (i.e., possibly biased) sample. In addition, the consistency and standardization of data collection (both within and between sites) is unclear. We are given no information quality control standards for data entry, storage, and reporting. In addition, only 16/21 PICUs in the PCCSG reported information: it may have been useful to see baseline data about these excluded PICUs in order to rule out sampling bias. Finally, we do not have enough information to determine whether the PCCSG sample is representative of the entire nation - this is an important point to clarify before a national policy such as regionalization is implemented.

Other deficits with regard to data collection included: lack of information about PICU's that were not included in the analysis, and lack of information about the data collection process. It would be useful to have a reference to the PCCSG's standards for data collection, inter/intra-rater reliability and standardization of format (e.g., definition of "step-down unit" and "teaching status").

5. Do the comparisons really focus on care in the ICU?

Yes. This study did not intend to consider variability in care occurring outside of the PICU, such as prehospital transport, resuscitation methods in the emergency department, surgical methods, or the impact of subspecialty services. Any of these factors could theoretically affect the relationship between mortality and volume of patients admitted to the PICU. However, this evaluation was not within the scope of the present investigation.

III. What are the results?

1. What was the difference in outcome?

Unadjusted results are as expected. This selective population of PICUs included mostly large, university-based, teaching programs. These programs, however, demonstrated considerable variability in terms of unadjusted length of stay (average median length of stay = 1.8 days; median length of stay = 1.8 days; standard deviation = 0.4 days; range = 1 Ð 2.6 days) and mortality rates (average mortality rate = 4.3%; median mortality rate = 3%; standard deviation = 1.7%; range = 1.8% - 9.1%) . Values for mean mortality and length of stay were consistent with those previously reported, lending some credence to the validity of the data collected in this study.

Multivariate regression analysis demonstrated several variables independently associated with both mortality (see table one below) and, in a separate model, with length of stay (see table two below).

Table one: variables significantly associated with PICU mortality

Variable	Adjusted OR (95% CI)	Probability
Volume	.95 (.91-.99)	.045
PRISM	7.61 (6.65-8.71)	.0001
PRISM [2]		.010
Patient age (mo)	1.03 (1.01-1.05)	.007
Trauma patient	1.77 (1.34-2.35)	.0001
Endocrine disease	.35 (.18-.68)	.002
Surgical patient	.45 (.32-.64)	.0001
Oncologic disease	3.24 (1.63-6.47)	.001
Circulatory disease	2.42 (1.58-3.70)	.0001
Congenital anomaly	2.10 (1.25-3.51)	.005
Ill-defined condition	1.70 (1.18-2.45)	.004

 

 

Table two: variables significantly associated with PICU length of stay

Variable	Adjusted IRR (95% CI)	Probability
Volume	.980 (.975-.985)	.030
PRISM	1.58 (1.55-1.62)	<.001
PRISM [2]		<.001
Patient age (mo)	.99 (.98-.99)	<.001
Trauma patient	1.16 (1.02-1.31)	.022
Endocrine disease	.64 (.57-.72)	<.001
Oncologic disease	1.34 (1.04-1.52)	<.001
Circulatory disease	1.29 (1.18-1.42)	<.001
Congenital anomaly	1.15 (1.07-1.23)	<.001
Ill-defined condition	1.26 (1.17-1.37)	<.001
Infection	1.20 (1.05-1.38)	.008
Respiratory condition	1.17 (1.06-1.29)	.002
Nervous system disease	1.09 (1.01-1.18)	.022

The data in table one suggest a significant association between patient volume and mortality. In other words, for every increase in volume of 100 patients, there was an associated relative risk reduction in mortality of ~5% (95% CI 1, 9%) and a ~2% (95% CI 1.5, 2.5%) relative reduction in length of stay. An increase in PRISM score from 10 to 20 was associated with a 7.61 (95% CI 6.65, 8.71) increase in the odds of dying and a 58% increase in the PICU length of stay.

Analysis of other measured "quality-of-care factors", such as presence of a fellowship program, had no impact on PICU mortality risk or patient length of stay. Of note, several other factors were not included in the regression model such as status as a teaching hospital and impact of supportive services (transport, surgical or ancillary staff etc).

2. How confident are you that the outcome differences are important?

With the large number of patients included, the confidence intervals were quite narrow. The 95% confidence intervals of 0.91 and 0.99 for the effect of volume on mortality suggest that if we sampled this population 100 times, 95 times the mean adjusted relative risk of mortality would be between 0.91 and 0.99.

IV. Will the recommendations help me in caring for my patients?

1. Is it possible to conclude which factors might have resulted in these extreme outcomes?

In the introduction to the paper, the authors speculate two ways by which the relationship between increased patient volume and mortality may be explained. They suggest that physicians who care for a specific subtype of patients may have more "practice" caring for these children and thus are more likely to have improved outcomes. Alternatively, referral bias may be at work: challenging patients may be more likely to be referred to institutions where similar patients are more frequently seen. Unfortunately, this "chicken and egg" argument is no closer to resolution with the results of this paper.

2. Can the results be applied to my patient population?

Not directly. These results are another step in the ongoing effort to improve PICU care, in this case on a national level. Ultimately, if these and other data adequately support regionalization of PICU care, practitioners in select PICU's would theoretically see an increased volume of patients. Presumptively, this increased volume of patients would result in improved practice patterns by clinicians. Smaller ICUs may be restricted to seeing a narrow subset of patients, thus limiting their scope of practice and possibly leading to non-accreditation. In that case, the situation may arise where, as the authors comment, patient volume may "become the proxy indicator for quality in health care" in a PICU.

References

1. Randolph AG, Guyatt GH, Carlet J, et al. Understanding articles comparing outcomes among intensive care units to rate quality of care. Crit Care Med 1998; 26: 773-781. [abstract]
2. Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Crit Care Med. 1988;16:1110-1116. [abstract]
3. Fiser DH. Assessing the outcome of pediatric intensive care. J Pediatr. 1992;121:68-74. [abstract]

 

Comments

Back to the EB Journal Club Index