Criteria abstracted from The Users' Guide to Medical Literature, from the Health Information Research Unit and Clinical Epidemiology and Biostatistics, McMaster University

Highlighted lines and questions below provide links to the pertinent description of criteria in The EBM User's Guide, now available at the Canadian Centres for Health Evidence

Article Reviewed:

Please visit the new Evidence Based Journal Club Reviews

Effect of reducing interns' work hours on serious medical errors in intensive care units.

Landrigan CP, Rothschild JM, Cronin JW, et al.

N Engl J Med. 2004 Oct 28;351(18):1838-48 [abstract]

 

Reviewed by Julie Lajoie MD and Lennox Huang MD, Departments of Anaesthesia and Pediatrics, McMaster University, Hamilton, Ontario, Canada

Review posted May 3, 2005

I. What is being studied?:

The study objective: The primary objective of this study was to examine the effects of intern sleep deprivation on serious medical errors in an adult intensive care unit. A secondary objective of the study was to investigate how the ICU system as a whole could be affected by changes in intern schedules.

The study design:

This research represents a prospective randomized cohort study comparing traditional versus intervention work schedules for interns working in a medical intensive care unit (MICU) and coronary care unit (CCU) of a large academic tertiary care center.

During the traditional work schedule, three interns were scheduled to each work 77-81 hours per week, with shifts every third night lasting up to 34 continuous hours. During the intervention work schedule, an additional intern was added to the schedule, each intern was scheduled to work 63 hours per week, with no shift longer than 16 hours. The intervention schedule was previously validated in a pilot study demonstrating successful elimination of shifts longer than 24 hours and reduction in total hours worked by individual interns per week (1,2)

Data collection was performed by continuous observation of interns by physician observers, nurse chart reviewers, voluntary incident reporting and computerized event detection monitors in addition to previously existing medical error monitoring. A medical error was defined in the study as "any error in the delivery of medical care, whether harmful or trivial," while adverse events were defined as "any injury due to medical management." It is important to note that these definitions do not completely overlap as it is possible to have an adverse event as a result of appropriate medical care (nonpreventable adverse event), or a serious medical error that does not result in patient injury (intercepted serious error).

Blinded reviewers not involved in primary data collection determined whether incidents were medical errors or adverse events. Errors were further classified as intercepted or nonintercepted serious errors, or errors with little potential for harm. Adverse events were rated on preventability and then categorized as preventable or nonpreventable.

Rates of intern-associated medical errors per patient-day were compared between the two schedules. The study also compared the total number of serious medical errors completed by all staff, per patient day. The study was powered to detect a 16 percent difference in the rate of serious errors between the groups.

The population investigated:

The primary population investigated were interns in their first year of the internal-medicine residency program at a large tertiary care hospital system. There were no details given about the age, sex, previous experience, nor personal details of the residents.

The patients involved in the study were admitted to the CCU or ICU between June 2002 to June 2003. The study involved 2203 patient-days, which represented 634 admissions. Data was not collected on patients who stayed less than four hours, who were undergoing elective allergy desensitization, or the rare patients who were not cared for by the CCU or ICU teams. Due to the nature of the two schedules, the traditional schedule involved a larger number of patients, larger number of admissions, and larger number of patient days as compared to the intervention schedule. The mean age of patients, the percent of male patients, Charlson comorbidity index, APACHE II score, length of stay, and percentage of deaths during the admissions did not differ significantly between the two schedules.

II. Are the results of the study valid?

Primary questions:

1.Were there clearly identified comparison groups that were similar with respect to important determinants of outcome, other than the one of interest?

No, superficially it appears that the main difference between the traditional and intervention schedules are the number of hours worked per week, the length of shifts worked and potentially the amount of sleep deprivation. While we are provided with patient data showing similar demographics between the two schedules, the comparison groups should utilize caregiver data as the unit of analysis (5).

The study described all of the interns to be in their first year of the internal-medicine residency program. The interns served as their own control, completing three weeks of the traditional schedule in the coronary care unit, then three weeks of the intervention schedule in the medical ICU, or vice versa. There were no additional details of the interns described, thus factors such as previous experience outside the typical medical training program, age, sex or family situation were not compared between the two groups. Since all interns were randomized and rotated through both schedules, demographic confounders should be limited but not fully eliminated. Interns rotating though traditional-CCU, intervention-MICU could differ from those rotating through intervention-CCU, traditional MICU, biasing the results in favour of the intervention schedule. The intervention schedule also had the added advantage of a fourth intern and a one-hour senior supervised signout in the evenings.

2. Were the exposures and outcomes measured in the same way in the groups being compared?

Yes, in both the traditional and intervention schedule, errors were identified in a rigorous and similar manner. Primary data collection showed 82 percent agreement between separate observers regarding the occurrence of a serious medical error during a 10 day period of dual observation. There was no way to blind the physicians who were directly observing the interns as to what schedule they were on. This potentially could have introduced bias, but it was noted that classification of errors and determination of preventability was performed later by investigators blinded to the intern's schedules. For the secondary classification of data, the blinded reviewers showed good interrater reliability with a kappa of 0.80 to 0.90.

3. Was follow-up sufficiently long and complete?

The study dealt with errors during patient's ICU stay. Latent errors that may have been revealed upon discharge from the ICU were not investigated. The goal of the study was not to assess effects of sleep deprivation outside of the intervention schedule, thus there was no long-term follow up of the intern population after their ICU rotation.

Secondary questions:

4. Is the temporal relationship correct?

The study was performed in real-time with two separate and distinct schedules, automatically linking the temporal relationship to the harmful exposure. The effect of sleep deprivation should theoretically affect the number of errors committed immediately.

5. Is there a dose-response gradient?

Not applicable, the interns were placed in one of the two schedules, and there was no demonstration of a gradient of work hours versus the number of errors that occurred.

III. What are the results?

1. How strong is the association between exposure and outcome?

There was a 35.9% increase in the number of serious medical errors, performed by interns, during the traditional schedule than during the intervention schedule, (136.0 vs. 100.1 per 1000 patient-days, P value < 0.001.)

Interns made 20.8 percent more serious medication errors during the traditional schedule than during the intervention schedule (99.7 vs. 82.5 per 1000 patient-days, P = 0.03). Interns also made 5.6 times as many serious diagnostic errors during the traditional schedule (18.6 vs. 3.3 per 1000 patient-days, P < 0.001). This association was very strong.

It was interesting that the number of procedural errors did not significantly differ between the two schedules. When all errors that occurred in the ICU, performed by interns and other staff were compared, there were more errors during the traditional schedule. When the errors committed by interns were removed from the calculation, there was no significant difference in the errors committed by other staff during the two rotations.

There was a 27 percent higher incidence of preventable adverse events during the traditional schedule, as compared to the intervention schedule, but this was not statistically significant (20.9 vs. 16.5 per 1000 patient-days, P = 0.21). Therefore the association between schedules and preventable adverse events is not strong.

2. How precise is the estimate of the risk?

While one would assume that the relative risk of medical errors while on a traditional schedule is higher than the intervention schedule, this is based on the assumption of statistical significance. We do not have enough information about the actual study population (interns) to calculate a true relative risk. Confidence intervals were not provided in the publication, and therefore how precise the risk is cannot be determined.

IV. What are the implications for my practice?

1. Are the results applicable to my practice?

The results of this study are directly applicable to the staffing of adult ICUs and easily generalizable to pediatric ICUs as well. In addition, the issues raised by this study have broad implications for all academic programs. Since sleep deprivation is a large part of most residency programs, the results of this study could be extrapolated to almost any residency rotation.

There are, however, some limitations to this study. Only first year interns, the least experienced of the housestaff, were studied in an adult ICU setting. The greatest subgroup of errors affected involved diagnosis - there was no significant difference in the number of procedural errors between the two schedules. With more experience and training, it could be argued that common diagnosis and management plans become procedural in nature.

There is little demographic information given about the intern population. This could negate the significance of the study's findings by creating artificially low p values, and further limits the generalizability of the study.

Many pediatric residency programs do not allow interns to rotate through the ICU and instead have second, third or in Canada, fourth year residents rotating through the ICU. Level of supervision also varies between ICUs and in more tightly supervised centers, some medical errors may not even be possible depending on the role of the resident within the ICU.

2. What is the magnitude of the risk?

The study reports a statistically significant difference in the number of serious medical errors that occurred during the traditional schedule compared to the intervention schedule. There was a 27 percent higher incidence of intern-associated preventable adverse events during the traditional schedule, but the study was not designed with the power to show that this increase was statistically significant. Therefore the magnitude of risk to ICU or CCU patients cannot confidently be drawn. As was mentioned earlier, the unit of analysis error in omitting data regarding the health care providers may result in artificially inflated p values and thus negate differences detected in this study (5).

3. Should I attempt to stop the exposure?

A 36% increase in the number of serious medical errors resulting from any exposure should warrant an immediate attempt to reduce the exposure. What is unclear from the study is whether the primary exposure relates to sleep, schedule, insufficient number of interns, the presence of interns in the ICU, or all of the above. The Institute of Medicine's 1999 report "To Err Is Human" suggests tremendous mortality as a direct result of medical errors in hospitals (6). While morbidity and mortality were not primary endpoints of this study, the potential association cannot be ignored.

Current dogma and evidence in the medical safety world suggests that systems based issues rather than individual mistakes should be the focus in analyzing and preventing adverse events. This study by Landrigan et al documents what is common sense to most people and illustrates how a systemic change in intern call schedules can have a drastic effect on medical errors.

Previously published research of intern call schedules and medical errors have not shown a significant difference with reduced call (3). However, these studies did not involve a validated intervention schedule, and relied on self-reporting through questionnaires to determine rates of medical errors.

Error reporting in Landrigan et al's study was very thorough. It is amazing to note that the documented errors were committed by interns who knew their actions were directly monitored by physicians. One can wonder how many errors might occur in an average ICU or ward with less rigorous monitoring.

While there is little doubt that the harmful exposure identified by Landrigan et al. must be stopped, the issue of how to stop the exposure identified in this study is a little more difficult. At a minimum, it would be irresponsible for the medical community to continue to allow sleep-deprived interns to care for patients in an ICU setting. The study does not address issues surrounding level of supervision and sleep deprivation in senior trainees and attending physicians. Nor does the study address how to overcome issues of breaks in continuity of care that arise with more frequent handovers and decreased time caring for the same patient. Decreasing the number of work hours per week also has implications on a resident's overall training experience and may affect the quality of physicians graduating from residency programs.

References

1. Landrigan P, Christopher et al. Effect of Reducing Interns' Work Hours on Serious Medical Errors in Intensive Care Units. N Engl J Med. 2004 Oct 28;351(18):1838-1848 [abstract]
2. Lockley SW et al. Effect of Reducing Interns' Weekly Work Hours on Sleep and Attentional Failures. N Engl J Med. 2004 Oct 28;351(18):1829-1837 [abstract]
3. Sawyer RG, Tribble CG, Newberg DS, Pruett TL, Minasi JS. Intern call schedules and their relationship to sleep, operating room participation, stress, and satisfaction. Surgery. 1999 Aug;126(2):337-42 [abstract]
4. Drazen J. Awake and Informed. N Engl J Med. 2004 Oct 28;351(18):1884.[citation]
5. Divine GW, Brown JT, Frazier LM. The unit of analysis error in studies about physician's patient care behavior. J Gen Intern Med. 1992 Nov-Dec;7(6):623-9 [abstract]
6. Kohn LT, Corrigan JM, Donaldson MS, eds. To err is human: building a safer health system. Washington, D.C.: National Academy Press, 1999. [text]

Comments

Back to the EB Journal Club Index