First Year of Mandatory Reporting of Healthcare‐Associated Infections, Pennsylvania: An Infection Control–Chart Abstractor Collaboration
Background. In 2004, the Commonwealth of Pennsylvania mandated hospitals to report healthcare‐associated infections (HAIs). The increased workload led our Infection Control staff to collaborate with Atlas, a group of chart abstractors.
Objective. The objective of this study was to assess our first year of experience with mandatory reporting of HAIs—specifically, to assess Atlas’ contribution to surveillance.
Design. Cases were selected if they had 1 or more of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD‐9‐CM) codes designated by Pennsylvania as a possible HAI. After training by the Infection Control staff, Atlas applied National Nosocomial Infection Surveillance (NNIS) system case definitions for catheter‐associated urinary tract infections (UTIs) and surgical site infections (SSIs), and they applied NNIS chest imaging criteria to eliminate cases that were not ventilator‐associated pneumonia (VAP). To assess Atlas’ performance, Infection Control staff conducted a parallel review.
Results. For discharges from the hospital during the fourth quarter of 2004, a total of 410 UTIs, 59 SSIs, and 56 VAPs were identified on the basis of state‐designated ICD‐9‐CM codes; review by Atlas/Infection Control determined that 15%, 15%, and 16% of cases met case definitions, respectively. Of cases reviewed by both Infection Control and Atlas, 87% of the assessments made by Atlas were correct for UTI, and 96% were correct for SSI. For VAP, Infection Control concluded that 39% of cases could be ruled out on the basis of chest imaging criteria; Atlas correctly dismissed these 12 cases but incorrectly dismissed an additional 6 (error, 19%). Surveillance was not timely: 1‐2 months elapsed between the time of HAI onset and the earliest case review.
Conclusions. With ongoing training by Infection Control, Atlas successfully demonstrated a role in retrospective HAI surveillance. However, despite a major effort to comply with mandates, time lags and other design limitations rendered the data of low utility for Infection Control. States that are planning HAI‐reporting programs should standardize an efficient surveillance methodology that yields data capable of guiding interventions to prevent HAI.
Received September 25, 2005; accepted April 24, 2006; electronically published August 14, 2006.
The staggering magnitude of the mortality, morbidity, and costs of healthcare‐associated infections (HAIs) has been increasingly publicized.1,2 As a strategy to reduce infections, an increasing number of states have initiated mandates requiring hospitals to report HAI rates to the public.3 Theoretically, because informed patients would avoid hospitals with high HAI rates, hospitals would be motivated to improve prevention practices.
In 2003, the Pennsylvania state legislation reauthorized the Pennsylvania Health Care Cost Containment Council (PHC4), enabling implementation of mandatory reporting of HAIs. Originally designed in 1986 to promote control of healthcare costs, PHC4 is an independent agency led by business and labor members that has required hospitals to publicly report data on charges, lengths of stay, and mortality.4 In 2004, PHC4 launched a new initiative that directed hospitals to report, through the billing record format of patient discharges, hospital‐wide catheter‐associated bloodstream infections (BSIs), catheter‐associated urinary tract infections (UTIs), ventilator‐associated pneumonia (VAPs), and cardiovascular, neurosurgical, and orthopedic surgical site infections (SSIs).
Although our 504‐bed tertiary care academic medical center (Penn State Milton S. Hershey Medical Center, Hershey, PA) had previously conducted targeted, problem‐based surveillance for HAI, the PHC4 mandates significantly increased demands on our Infection Control (IC) department. A new collaboration was therefore sought with Atlas, a team of 4 chart abstractors who collect data on severity of illness for PHC4 and quality indicators for other agencies. This report describes the development of this collaboration; a study of the Atlas chart abstractors’ performance in assisting the IC staff to identify patients with catheter‐associated UTI, SSI, and VAP; and an overall assessment of our first year of experience reporting HAIs to the state.
Methods
Training Atlas Chart Abstractors
Using the National Nosocomial Infection Surveillance (NNIS) system case definitions (available at: http://www.cdc.gov/ncidod/dhqp/nnis_pubs.html) as required by PHC4, screening checklist forms were written to guide standardized chart abstraction. Reference materials were written to define terminology, clarify ambiguities in the case definitions, advise where data elements could be located in the medical record, and provide sample cases. Multiple training sessions for Atlas chart abstractors were given by IC staff and a physician from the Division of Infectious Diseases.
Case Finding and Screening by Atlas Chart Abstractors
A list was generated of cases with at least 1 of 142 secondary ICD‐9‐CM discharge diagnosis codes (Table 1) selected by PHC4 to capture UTIs; cardiovascular, neurosurgical, or orthopedic SSIs; and VAPs (“pneumonia” plus “ventilator” codes). Atlas reviewed these available medical records to apply NNIS/PHC4 definitions for healthcare‐associated, catheter‐associated UTIs and for healthcare‐associated SSIs. Atlas chart abstractors also assessed VAP‐coded cases to apply the NNIS imaging criteria for VAP; the goal was for Atlas chart abstractors to perform a first‐level screening to eliminate cases that were obviously not VAP.
Atlas Chart Abstractors’ Performance
To assess Atlas chart abstractors’ application of NNIS/PHC4 definitions for the fourth quarter (October through December) 2004 patient discharges, IC staff reviewed all cases with SSI or VAP codes. Because of the large number of UTI‐coded cases, IC staff reviewed a sample of approximately 10% of cases—cases flagged by Atlas chart abstractors as difficult and a randomly selected subset of cases chosen for quality checks. IC’s interpretation of a case was defined as the final correct conclusion. For UTI‐coded cases not reviewed by IC staff, Atlas chart abstractor’s conclusion was final.
Timeliness of Surveillance
To assess timeliness of identification of cases of HAI, the schedule of coding, ICD‐9‐CM report generation, and Atlas chart abstraction was reviewed.
Resources Used for Surveillance Implementation
Management indicators were used to tabulate the number of hours that Atlas chart abstractors spent in training and during case review. Time required for charts to be gathered for review was estimated on the basis of the mean number of available charts that could be located within 1 h and the mean number that could be refiled within 1 h. Use of IC time was based on the staff’s self‐reported estimates.
Results
As planned, Atlas chart abstractors reviewed 100% of the 410 UTI‐coded cases, and the IC staff reviewed a sample of 47 cases (11%). As a result of logistic problems related to chart availability, Atlas chart abstractors only reviewed 46 (80%) of the 59 cardiovascular, neurosurgical, or orthopedic SSI‐coded cases and 31 (55%) of the 56 VAP‐coded cases; however, IC staff reviewed 100% of each of these latter 2 case types.
Of the UTI‐coded cases, IC/Atlas concluded that 63 (15%) met the NNIS/PHC4 case definition. Similarly, 9 (15%) of the SSI‐coded cases and 9 (16%) of the VAP‐coded cases met their respective case definitions. For UTI‐coded cases, most failed to meet NNIS/PHC4 definitions because either (1) the UTI was diagnosed only at hospital admission, or (2) a healthcare‐associated case of UTI was diagnosed by a physician, but the urine cultures either yielded negative results or were contaminated (ie, multiple organisms were present), and there were not sufficient symptoms to meet the NNIS case definition. For SSI‐coded cases, most infections were present at the time of the patient's admission, before surgery was performed at our hospital. For VAP‐coded cases, pneumonia was frequently diagnosed only at the time of hospital admission, or the physician’s notes commented that a VAP may be present but the cases did not meet the full criteria of the NNIS definition (Table 2).
Atlas Chart Inspectors’ Performance
Of the 45 UTI‐coded cases and 46 SSI‐coded cases reviewed by both Atlas chart abstractors and IC staff, Atlas’ conclusions were correct for 87% and 96% of cases, respectively (Table 3). The 6 errors from the UTI‐coded cases and the 2 errors from the SSI‐coded cases were a mixture of false‐negative and false‐positive results (eg, Atlas chart abstractors overlooked documentation of Foley catheter use, misread the microbiology data because of poor report layout, or reviewed the wrong admission). For VAP‐coded cases, IC staff concluded that 12 (39%) of 31 cases could be ruled out on the basis of chest imaging criteria; Atlas chart abstractors correctly dismissed these 12 cases but incorrectly dismissed an additional 6 cases (error, 19%). For VAPs, the major error was not recognizing patterns in the radiologists’ terminology that indicated suspicion of pneumonia.
Timeliness of Surveillance
The median time from the onset of UTI, SSI, or VAP to the date of discharge from the hospital was 16 days (range, 0‐90 days). For nearly all cases, coding was completed by 2 weeks after the patient was discharged. The Information Technology department generated reports to identify cases with the selected ICD‐9‐CM codes approximately 30 days after the last day of the discharge month (eg, for patients discharged in October 2004, ICD‐9‐CM lists of possible HAI cases were available by December 2, 2004). As a result, at the earliest, Atlas chart abstractors began case reviews approximately 1‐2 months after the onset of the HAI; all case reviews by Atlas/IC were completed before PHC4’s submission deadline of March 31, 2005.
Resources Utilized for Surveillance Implementation
Four Atlas chart abstractors underwent a cumulative 130 h of training during 2004 to conduct chart reviews. For fourth‐quarter discharges coded as possible HAIs, approximately 184 h were required for Atlas chart reviews, and 40 h were required to locate and refile these charts. An estimated 1.25 full‐time equivalents of IC time were spent in 2004 creating training materials, interacting with Atlas, and performing parallel chart reviews.
Discussion
The first year of mandatory reporting of HAI in Pennsylvania provided several lessons at our hospital. First, IC staff and Atlas chart abstractors successfully collaborated to conduct retrospective ICD‐9‐CM code–based surveillance that had been designed to comply with reporting requirements. However, use of ICD‐9‐CM codes for case finding is encumbered by time delays and a lack of specificity. Contrary to statements by PHC4,5‐7 we found that the vast majority of hospitalizations with a secondary infection discharge diagnosis code do not represent HAIs. Finally, the rapidly implemented mandates’ lack of a standardized, focused surveillance methodology led to a major expenditure of effort on collection of data that are of limited internal use and may not be comparable between hospitals.
IC‐Atlas Chart Abstractor Collaboration
With extensive training by IC staff, Atlas chart abstractors demonstrated skills in the application of case definitions for UTI and SSI and of chest imaging criteria to eliminate a subset of non‐VAP cases. The few errors for the UTIs and SSIs were generally not associated with a misunderstanding of case definitions, but were due to misreading of data elements (sometimes in poorly presented formats). VAPs are among the most challenging HAIs to define and detect accurately.8 The inconsistencies of phrases used in chest imaging reports by different radiologists were difficult to interpret, especially in cases that involved a large number of imaging studies. Because of clinical complexities, it was decided that Atlas chart abstractors would only complete the first screening phase (ie, application of chest imaging criteria), to rule out at least some of the non‐VAP cases, and IC staff would review catheter‐associated BSIs without Atlas chart abstractor assistance.
Specificity of ICD‐9‐CM Codes for HAI
To fully comply with PHC4 mandates, our expanded surveillance program was originally designed around evaluation of ICD‐9‐CM infection codes. In statements to hospitals and to the press, PHC4 members have frequently implied that the ICD‐9‐CM infection codes are a proxy for HAIs and have suggested that coding data can be used to estimate hospital compliance with reporting.5‐7 However, we observed that more than 80% of hospital discharges with one of the 142 PHC4‐selected secondary discharge diagnosis ICD‐9‐CM codes for UTI, SSI, or VAP failed to meet PHC4 case definitions. These codes are not designed specifically to identify HAI but to identify a variety of types of infections and complications. Although the accuracy may vary by type of infection and by codes used for case finding, previous studies of discharge diagnosis codes for identification of SSI,9 pneumonia,10 and bacteremia11 have also encountered limitations.
Utility of Mandated Data Collection
Although the use of ICD‐9‐CM codes might offer a relatively standardized case‐finding method that could be adopted across hospitals, the lack of timeliness rendered the data to be of low utility for the purposes of infection control. Current PHC4 case definitions have significant limitations, such as the inclusion of asymptomatic bacteriuria as a UTI and the exclusion of postdischarge SSI and of VAP with onset more than 48 h after discontinuation of mechanical ventilation. Interhospital comparisons may also be invalid because standardized case‐finding methods, use of denominator data, and risk‐adjustment processes were not delineated. PHC4’s reporting of HAI by discharge date also inherently introduces numerator/denominator time‐frame mismatches and distorts data analysis.
The rapidity of the mandates’ implementation precluded careful budgeting and planning, and scientific guidance on efficient means to conduct an internally useful surveillance program was minimal. The mandates were first announced late in November 2003, and the first data submission deadline was June 30, 2004. Because little written information and only one 3‐h training session were provided by PHC4 for our region, our hospital developed its own expanded surveillance strategy, screening tools, and reference training materials and devoted extensive time to training chart abstractors. Not only is this redundant and time‐consuming for each hospital, but it led to a trial‐and‐error approach to different surveillance methodologies. Although IC and Atlas chart abstractors collaborated effectively, an internally useful surveillance program with more timely case‐finding methodology will need to be developed.
Future Directions
Mandatory reporting in Pennsylvania has directed increased attention to HAI surveillance. As professionals in the field of healthcare epidemiology and infection control, we are pleased with this important step. However, resources should not be exclusively expended on collection of an increasingly wide variety of data of limited internal use. Unfortunately, for 2006, PHC4 has scheduled hospitals to also report, without any prioritization, all types of hospital‐wide HAI. Because an increasing number of states are expected to issue mandates on public reporting of HAI, leadership trained in healthcare epidemiology should work with states to create standardized efficient surveillance methodology, training materials, and electronic databases and should evaluate process as well as targeted outcome measures.8,12 If designed to be useful for IC departments, targeted mandatory surveillance may strengthen data collection programs that can guide interventions to prevent HAI.
Acknowledgments
The Penn State Milton S. Hershey Medical Center Atlas staff, central to the work described in this report, are Sirena Garland, Carol Klahr, Sally Plesic, and Darleen Smith; Deborah Davis is a new member of Atlas. We also appreciate the support of Scott Cranston, Tammy Mongold, Patricia Swetland, Linda Ulrich, William Hulme, Jen Weise, and Lynda Martin. Nkuchia M’ikanatha provided excellent guidance for the writing of the manuscript.
References
- 1. Berens MJ. Infection epidemic carves deadly path: poor hygiene, overwhelmed workers contribute to thousands of deaths. Series: Tribune investigation: unhealthy hospitals. First of three parts. Chicago Tribune July 21, 2002:1.
- 2. Connolly C. Data show scourge of hospital infections. Washington Post July 13, 2005:A1.
- 3. Association for Professionals in Infection Control and Epidemiology. Mandatory reporting of infection rates: where does my state stand? Available at: http://www.apic.org/Content/NavigationMenu/GovernmentAdvocacy/MandatoryReporting/state_legislation/state_legislation.htm. Accessed September 23, 2005.
- 4. Pennsylvania Health Care Costs Containment Council. Available at: http://www.phc4.org/. Accessed September 23, 2005.
- 5. Pennsylvania Health Care Costs Containment Council. PHC4 research brief—hospital‐acquired infections in Pennsylvania. July 2005. Available at: http://www.phc4.org/reports/researchbriefs/071205/default.htm. Accessed September 23, 2005.
- 6. Pennsylvania Health Care Costs Containment Council. News release: first state report on hospital‐acquired infections released in PA cost, quality issues raise grave concerns. July 13, 2005. Available at: http://www.phc4.org/reports/researchbriefs/071205/nr071205.htm. Accessed 23 September 2005.
- 7. Glover L. Pa. hospitals underreport infection rates. Pittsburgh Business Times October 3, 2004. Available at: http://www.bizjournals.com/pittsburgh/stories/2004/10/04/story1.html Accessed September 23, 2005.
- 8. McKibben L, Horan T, Tokars JI, et al., Healthcare Infection Control Practices Advisory Committee. Guidance on public reporting of healthcare‐associated infections: recommendations of the Healthcare Infection Control Practices Advisory Committee. Infect Control Hosp Epidemiol 2005; 26:580–587.
- 9. Moro ML, Morsillo F. Can hospital discharge diagnoses be used for surveillance of surgical‐site infections? J Hosp Infect 2004; 56:239‐241.
- 10. Aronsky D, Haug PJ, Lagor C, Dean NC. Accuracy of administrative data for identifying patients with pneumonia. Am J Med Qual 2005; 20:319‐328.
- 11. Madsen KM, Schonheyder HC, Kristensen B, Nielsen GL, Sorensen HT. Can hospital discharge diagnosis be used for surveillance of bacteremia? A data quality study of a Danish hospital discharge registry. Infect Control Hosp Epidemiol 1998; 19:175‐178.
- 12. Wong ES, Rupp ME, Mermel L, et al. Public disclosure of healthcare‐associated infections: the role of the Society for Healthcare Epidemiology of America. Infect Control Hosp Epidemiol 2005; 26:210‐212.