Highly Sensitive and Efficient Computer‐Assisted System for Routine Surveillance for Surgical Site Infection
Objectives. Surveillance of surgical site infections (SSIs) is effective in reducing the rates of these complications, but it is extremely time‐consuming and, consequently, underused. We determined the sensitivity and specificity of a computer‐assisted surveillance system, compared with a conventional method involving review of medical records, and the time saved with the computer‐assisted system.
Method. A prospective study was conducted from January 1 to December 31, 2001. With the computer‐assisted method, screening for SSIs relied on identification in the laboratory database of positive results of microbiological tests of surgical‐site specimens; confirmation was obtained via computer‐generated questionnaires completed by the surgeon in charge of the patient. In the conventional method, SSIs were identified by exhaustive chart review. The time spent on surveillance was recorded for both methods.
Setting. A 25‐bed gastrointestinal surgery unit in a tertiary care hospital.
Patients. A total of 766 consecutive patients who underwent gastrointestinal surgery.
Results. The sensitivity of the computer‐assisted method was 84.3% (95% confidence interval, 0.66‐0.94); the specificity was 99.9%. For the 807 surgical procedures in the study, 197 had an SSI identified by culture of a surgical‐site specimen. After elimination of 63 duplicate cultures with positive results, 134 questionnaires were sent to the surgeons, who confirmed 27 SSIs. The conventional method identified 32 SSIs. The computer‐assisted method required 60% less time than the conventional method (90 hours vs 223 hours).
Conclusion. Surveillance for SSIs using computer‐assisted, laboratory‐based screening and case confirmation by surgeons is as efficient as and far less time‐consuming than the conventional method of chart review. This method permits routine surveillance for SSIs with reliable accuracy.
Received November 15, 2004; accepted June 1, 2005; electronically published July 20, 2006.
Surgical site infection (SSI) remains a major complication of inpatient care. The effectiveness of surveillance in reducing SSIs was demonstrated many years ago,1 and surveillance is recommended in many countries.2,3 SSIs rates are required by accreditation agencies4 and health consumer organizations. In most hospitals, however, routine SSI surveillance is not performed because it is time‐consuming. Conventional data collection includes a chart review, computation of SSI rates, and dissemination of reports. The development of an SSI surveillance method that is highly sensitive yet frugal in terms of time and effort constitutes a challenging task.5 Computer programs save time by cross‐linking data across medical information systems, minimizing data entry tasks and errors, and generating automatic reports.6
Another time‐saving approach involves screening for SSI by means of sensitive indicators. Without using a computer, Glenister et al.7‐9 compared the sensitivity of 8 surveillance methods with that of a reference method requiring ward rounds for reviews of nursing and medical records, temperature charts, drug prescriptions, and laboratory findings.9 Among these methods, laboratory‐based approach involving reviews of microbiological findings and meetings with a ward liaison was the most sensitive.
Subsequent to the work of Glenister and colleagues,7‐9 to save time and resources used for SSIs surveillance, we developed a computer‐assisted surveillance system (CASS) based on the following 3 components: a computer program specifically designed for hospital infection control, a laboratory‐based screening for suspected SSIs, and confirmation of each suspected SSI by the surgeon in charge of the patient. This CASS has been in use since 1999 at the Saint Joseph Hospital, a 450‐bed tertiary care facility in Paris, France. We present the results obtained using the CASS over a 1‐year period in a gastrointestinal and general surgery unit. The sensitivity and specificity of the CASS were assessed comparatively with a reference method. The times required for SSI surveillance by the CASS and the reference method were recorded.
Methods
The study took place between January 1, 2001, and December 31, 2001. For the CASS and the reference method, all the surgical procedures were identified through the operating‐room database, and National Nosocomial Infections Surveillance (NNIS) system definitions were used for surgical procedures and SSI cases.10,11 With the CASS, case finding relied on screening for positive results of microbiological tests and confirmation of these results by the patient's surgeon. Case finding with the conventional method was done by manual review of patients' medical records.
The CASS
Data were handled using NosoCom, a computer program designed for nosocomial infection surveillance.6 NosoCom is compatible with other programs and can import data from nearly all of the databases used in medical information systems. Before the project, data on inpatient identity, surgical procedures performed, and laboratory results were collected routinely in hospital databases. However, some data used for SSI surveillance were not recorded, such as NNIS score components. Before starting the project, we requested that these data be recorded routinely. NosoCom, which is based in the infection control unit, is linked electronically to the databases of the admissions office, operating room, and microbiology laboratory. Every day, selected data are downloaded by NosoCom, which automatically deletes data irrelevant to SSI surveillance (eg, financial and administrative data). Downloaded data are combined and examined to generate SSI rates. This last step involves medical decision making; therefore, it is performed in part by the computer but also requires input from the infection control professionals (ICPs). All databases are stored in a central server for management, maintenance procedures, and data transfer protocol.
Data downloaded automatically to NosoCom from other databases by the CASS. Data on a patient's characteristics, the name of the ward in which the patients were hospitalized, and the admission and discharge dates are automatically transferred to NosoCom. For each surgical procedure, the operating‐room nurse enters data into the operating‐room database. The data to be transferred to NosoCom are those needed to compute SSI incidence rates stratified by surgical procedure and NNIS score and consist of the date of surgery, surgical specialty, surgical procedure category, principal surgeon involved, duration of surgery, wound class, and American Society of Anesthesiologists (ASA) score (Figure 1). From the microbiology laboratory database, NosoCom retrieves the following data on all positive culture results involving specimens obtained from patients hospitalized at any site in the hospital (medical or surgical wards, operating rooms, and outpatient clinic): the date on which the sample was obtained, the ward in which the patient was hospitalized, the anatomical site from which the specimen was taken, and the type(s) of microorganisms that grew on culture (Figure 2). NosoCom flags specimens from surgical sites, on the basis of information provided on forms completed by physicians to request microbiological testing of the specimens. Because of the poor quality of the information entered by ward staff in the open fields of these forms, we created a new form to facilitate identification of surgical‐site specimens that have been sent to the microbiology laboratory for culture. The new form had a question that asked ward staff to specify the anatomical site from which the sample was obtained, because almost all biological specimens can be associated with a surgical site and because words commonly used to designate surgical site sample, such as “pus” or “wound,” are not specific.
Figure 1. Image of a computer screen showing surgical data transferred from the operating room database to NosoCom, a computer program designed for nosocomial infection surveillance. ASA, American Society of Anesthesiologists score; NNIS, National Nosocomial Infection Surveillance system.
Figure 2. Image of a computer screen showing data on positive culture results transferred from the microbiology database to NosoCom. The column “O” indicates whether the patient underwent an operation in the hospital. If a surgical site infection was detected by microbiological analysis, an electronic link is created between data on the positive result and data on the surgical procedures(s) performed (see Figure 3). The column “OS” indicates whether the sample was taken from an operative site. The column “Sample” specifies the anatomic site from which the sample was obtained (O, organ; P, pus; R, respiratory tract; and U, urine). For example, “PWOUND” indicates that pus was taken from a wound.
Automatic linking of data from the microbiology laboratory to data on surgical procedures and input from ICPs. NosoCom not only receives data from various databases, but also automatically links data from various hospital locations (eg, the operating room and the microbiology laboratory) pertaining to a given patient. This is done by generating an identifier from the patient’s family name and social security number. The result is a record for each patient that contains the following data: duration(s) of hospitalization, surgical procedure(s) done in the hospital, and results of microbiological tests. The presence of a link is indicated in the computer by a symbol located in a specific column (Figure 2). This link is generated by the computer and serves to alert the ICP by providing the list of patients with positive results of microbiological tests who have undergone surgical procedure(s) in the hospital. The CASS was introduced in 1999, and records are available from that time onward.
The ICPs interact with NosoCom by selecting surgical procedures and by identifying suspected SSIs. Thus, using the data downloaded from the operating room, the ICPs exclude procedures that are not categorized as surgical procedures by the NNIS,11 such as needle biopsy, procedures without skin incision, or a second surgery for treatment of a complication associated with a previous surgical procedure. For all surgical procedures included in the NNIS definition, the ICPs compute the NNIS score (Figure 1), which incorporates the duration of surgery according to the procedure performed, the ASA score, and the surgical wound class.12
To identify suspected SSIs, the data downloaded to NosoCom from the database of the microbiology laboratory are reviewed by the ICPs 3 times per week. When a patient has a flagged positive result of culture of a surgical‐site specimen after a surgical procedure, the record containing all the surgical procedures that patient underwent in the hospital from 1999 onward can be opened (Figure 3). The ICP creates a second link between the positive result of microbiological culture and the corresponding surgical procedure. If the patient has had more than 1 procedure, the ICP selects the appropriate surgical procedure on the basis of the sampling date, body site sampled, and microorganisms recovered (Figures 3 and 4). If microbiological tests are performed, this protocol ensures identification of suspected SSIs between admission and discharge, at follow‐up outpatient visits and to subsequent readmissions, if any.
Figure 3. Image of a computer screen showing data on all of the surgical procedures performed at Saint Joseph Hospital for a patient with a positive result for a surgical site sample. In this example, the third patient from the top had a positive result of microbiological testing of a specimen taken from a surgical site. The record containing all the surgical procedures done on that patient in the hospital has been opened (bottom), and the infection control professional (ICP) has selected the surgical procedure associated with the surgical site infection. The column “SURG” indicates the surgical specialty (GI, gastrointestinal surgery; V, vascular surgery). The column “SURGPRO” indicates the surgical procedure. In column “D” (denominator), the ICP has entered “N” for nonsurgical procedures, which are excluded from the denominator.
Figure 4. General design and main results of the computer‐assisted surveillance system (CASS) for detection of surgical site infection (SSI).
For each suspected SSI, a questionnaire is generated by the computer and mailed to the surgeon in charge of the patient (Table 1). Four items on the questionnaire can be answered “yes” or “no” to confirm or to refute the presence of an SSI. The SSI is confirmed only if the surgeon answers “yes” to all 4 items. SSI rates are computed automatically, and a standardized electronic report is prepared to provide feedback to the surgical team and to determine confidential surgeon‐specific SSI rates.
Training of the surgical team and ward staff. Before the introduction of NosoCom at our hospital, all of the involved surgeons agreed to the CASS. The infection‐control team designed 2‐hour training sessions, one for surgeons and one for operating‐room nurses. During these sessions, the surgeons and nurses received detailed information on the concepts used for SSI surveillance and the methods used by NosoCom. The surgeons received specific instructions for answering the questionnaire used to confirm the presence or absence of SSI. Throughout the hospital, the microbiology laboratory staff conducted training sessions to familiarize ward staff members with the new form used to facilitate identification of surgical‐site specimens that have been sent to the microbiology laboratory for culture.
The Reference Method
Throughout the study, SSIs were identified by a retrospective chart review. This method was considered to be optimal for identifying SSIs, given the specific characteristics of our hospital (ie, patient medical records are available during hospitalization and the quality of medical records is high). The ICPs using the reference method were blinded to the SSIs identified by the CASS. For each surgical procedure subjected to surveillance, the medical record was reviewed by 2 ICPs. The ICPs identified SSIs by reviewing all medical and nursing records, discharge summaries, surgical reports, body temperature records, antibiotic prescriptions, and laboratory test results (including microbiological findings). The surveillance period included inpatient surveillance and postdischarge surveillance and continued up to 1 month after the day of surgery. Clinical notes written during follow‐up visits always included information on infection status and wound appearance.
The sensitivity and the specificity of SSI rates provided by the CASS and the reference method were compared. The time spent on SSI surveillance was recorded for both methods.
Results
During the study period, there were 6 surgeons on the permanent staff of the gastrointestinal and general surgery unit. Of the 1,092 surgical procedures performed during the study, 807 met criteria for SSI surveillance; they were performed for 766 patients (mean age [±SD], 
years). Data on the 3 components of the NNIS score were available for 760 (94.2%) procedures; there were 45 procedures (5.6%) missing values for the ASA score, 3 for duration of surgery, and 1 for the wound class. The frequency distribution of the surgical procedures is shown in Table 2. Thirty‐day follow‐up data were available for 75% of the surgical procedures.
SSI Screening and Confirmation Using the CASS
The general design and main results of the CASS are presented in Figure 4 and Table 2. Of the specimens obtained for microbiological evaluation after the 807 surgical procedures, 692 were positive for a pathogen; 197 (28.5%) of the 692 specimens were from a surgical site. Of the surgical‐site specimens that tested positive for an SSI, 63 were duplicates, leaving 134 suspected SSIs. Of the 134 questionnaires mailed to the surgeons, 127 (95%) were returned to the infection control unit. An SSI was confirmed in 103 cases, of which 57 were classified as community‐acquired infections, 7 as SSIs acquired in another hospital, 11 as duplicate infections (Table 1), and 27 as SSIs associated with surgical procedures performed in our institution. Of these 27 SSIs, 24 were detected before hospital discharge, and 3 were detected after readmission.
Comparison of the CASS and the Reference Method
With the reference method, 32 SSIs were detected (Table 2). Thus, the sensitivity of the CASS was 84.3% (27 of 32 SSIs; 95% confidence interval, 0.66‐0.94) (Table 3). Of the 5 SSIs that escaped detection by the CASS, 2 were missed because the person who completed the new form for identification of surgical‐site specimens that underwent microbiological culture did not check the box indicating that the specimen was from a surgical site. The other 3 cases were missed because no specimens were taken from the surgical site for microbiological analysis. Failure of surgeons to return confirmation questionnaires did not result in any missed cases with the CASS. Four of the 5 SSIs missed by the CASS were diagnosed after discharge; 3 were diagnosed at readmission (the 3 cases without microbiological specimens), and 1 was diagnosed during an outpatient visit. There were no false‐negative findings via the CASS (ie, no instances in which a surgeon confirmed the absence of an SSI that was detected by the reference method). There was 1 false‐positive finding (ie, a case in which the surgeon confirmed the presence of an SSI that was not detected by the reference method) in a patient who had a specimen that tested positive for an infection that did not meet SSI criteria. For this patient, a consensus was reached between the surgeon and the ICP with regard to SSI status. Finally, the specificity of the CASS was 99.9%.
The time needed for SSI surveillance was 223 hours with the reference method and 90 hours with the CASS. Thus, the CASS reduced the time needed for surveillance by 60.7%.
Discussion
The computer‐assisted and laboratory‐based surveillance system used in our study had a sensitivity of 84.3%. Glenister et al.9 found that a system based on positive results of microbiological tests detected only 71.8% of SSIs. However, they used a ward liaison instead of a computerized system to validate SSIs. With our computerized system, the quality of surveillance was not affected by the geography of the hospital or the availability of records and healthcare staff. Using data from a German national prevalence study, Gastmeier et al.13 estimated that laboratory‐based SSI surveillance would have a sensitivity of only 64.6% because of the fairly small number of surgical‐site specimens that undergo microbiological evaluation in German hospitals. The effectiveness of laboratory‐based surveillance depends on the proportion of suspected SSI cases for which microbiological cultures are performed. Before initiating our study involving laboratory‐based screening, we conducted a 3‐month pilot study.14 SSIs were detected by chart review in 14 of 219 surgical procedures, and 13 SSIs were detected by a positive result of culture of a surgical‐site specimen, indicating that laboratory‐based surveillance would have had a sensitivity of 92.8%, using chart review as the criterion standard. In the present study, microbiological cultures were performed in 29 (90.6%) of the 32 SSI cases, resulting in a high sensitivity for the CASS. Although there is no clear consensus that a suspicion of SSI should lead to routine microbiological evaluation for therapeutic decision‐making, other goals of microbiological analysis for patients with an SSI might include identification of changes in antibiotic resistance15,16 and development of indicators for nosocomial surveillance.
Antibiotic use has been suggested as a useful indicator of SSIs. One study found a sensitivity of 95% for antibiotic prescription–based surveillance of patients who underwent coronary artery bypass graft surgery.17 However, SSI screening based on antibiotic use may lack specificity for patients undergoing gastrointestinal or general surgery, who constitute a heterogeneous population in which antibiotics may be used for many reasons. The sensitivity and specificity of antibiotic prescription–based SSI detection can be improved by simultaneously using another indicator recorded in the medical information system, such as microbiological findings13 or International Classification of Diseases, Ninth Revision, discharge codes.18
The time‐saving effect of automatic methods is crucial to the implementation of routine SSI surveillance. The CASS required 60% less time than the reference method, and similar results have been obtained by Evans et al.18 with a computer program and antibiotic prescription–based SSI screening.
In our study, only 4 of the 32 SSIs identified by the reference method were detected after patient discharge. In some cases, our user‐friendly computer system may ensure early recognition of SSIs in patients who meet criteria for discharge because their clinical status is good, with little evidence of infection. In earlier studies, a large proportion of SSIs became apparent after discharge.19‐21 For the moment, our system is not designed to detect postdischarge SSIs, unless the patient is readmitted to our hospital. We are designing an extension to our computer program that will generate a questionnaire for surgeons to complete at the first outpatient follow‐up visit. Several studies on postdischarge surveillance based on questionnaires mailed to surgeons found encouraging response rates.20,21
In our study, a questionnaire was sent to surgeons for confirmation of suspected SSI. The very high response rate (95%) may be due to the limited number of questionnaires sent to each surgeon, as information was requested only for patients with suspected SSIs. The high specificity of the confirmation questionnaires may be related to the training session attended by the surgeons prior to the study and to the simple design of the questionnaire, which had only 4 items, each requiring an answer of “yes” or “no.” This design facilitated the identification by surgeons of microbiological findings that, on the basis of their clinical judgment, indicated genuine SSI. Validation of SSI cases by surgeons strongly agreed with findings of the ICPs.
The acceptable sensitivity of our CASS (84.3%) and its ability to cut surveillance time by 60% suggests that computer‐assisted SSI surveillance relying on microbiology laboratory–based screening for SSIs can be recommended to infection control units as a means of correcting the current underuse of routine SSI surveillance, which is an invaluable epidemiological tool. Since the interpretation of signs and microbiological findings indicating infection is more ambiguous for SSIs that occur after gastrointestinal surgery, it is most probable that these methods can be generalized to other types of surgery.
Acknowledgments
We thank the following members of the steering group: Prof. ACAR Jacques, Microbiologie, Hôpital Saint Joseph, Paris, France; Barrault Yves, Direction générale, Hôpital Saint Joseph, Paris; Prof. Degos Claude, Neurologie, Hôpital Saint Joseph, Paris; Dr. Guillaumat Michel, Chirurgie orthopédique, Hôpital Saint Joseph, Paris; Prof. Jarlier Vincent, Microbiologie, Hôpital Pitié Salpétrière, Paris; Dr. Morin Jean Paul, Anesthésie, Hôpital Saint Joseph, Paris; and Prof. Valleron Alain Jacques, Unité INSERM 44, Paris.
Financial support: Projet hospitalier de recherche clinique (grant IDF 95002: “Surveillance de l’infection nosocomiale: développement d’un système de routine”).
References
- 1. Haley RW, Culver DH, White JW, et al. The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals. Am J Epidemiol 1985; 121:182‐205.
- 2. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for prevention of surgical site infection, 1999. Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee. Am J Infect Control 1999; 27:97‐134.
- 3. Comité Technique National des Infections Nosocomiales. 100 Recommandations pour la Surveillance et la Prévention des Infections Nosocomiales. Paris, France: Ministère de l’Emploi et de la Solidarité; 1999.
- 4. Agence Nationale d’Accréditation et d’Evaluation en Santé. Manuel d’accréditation des établissements de santé. Agence Nationale d’Accréditation et d’Evaluation en Santé; 1999.
- 5. Tokars JI, Richards C, Andrus M, et al. The changing face of surveillance for health care‐associated infections. Clin Infect Dis 2004; 39:1347‐1352.
- 6. Cauet D, Quenon JL, Desvé G. Surveillance of hospital acquired infections: presentation of a computerized system. Eur J Epidemiol 1999; 15:149‐153.
- 7. Glenister HM, Taylor LJ, Bartlett CL, Cooke EM, Mulhall AB. Introduction of laboratory ward liaison surveillance of hospital infection into six district general hospitals. J Hosp Infect 1993; 25:161‐172.
- 8. Glenister HM, Taylor LJ, Bartlett CL, Cooke EM, Sedgwick JA, Mackintosh CA. An evaluation of surveillance methods for detecting infections in hospital inpatients. J Hosp Infect 1993; 23:229‐242.
- 9. Glenister HM, Taylor LJ, Bartlett CL, Cooke EM, Mackintosh CA, Leigh DA. An 11‐month incidence study of infection in wards of a district general hospital. J Hosp Infect 1992; 21:261‐273.
- 10. Horan TC, Gaynes RP, Martone WJ, Jarvis WR, Emori TG. CDC definitions of nosocomial surgical site infections, 1992: a modification of CDC definitions of surgical wound infections. Infect Control Hosp Epidemiol 1992; 13:606‐608.
- 11. Horan TC, Emori TG. Definition of key terms used in the NNIS system. Am J Infect Control 1997; 25:112‐116.
- 12. Culver DH, Horan TC, Gaynes RP, et al. Surgical wound infection rates by surgical procedure, and patient risk index. National nosocomial infections surveillance system. Am J Med 1991; 16;91:152S‐157S.
- 13. Gastmeier P, Brauer H, Hauer T, Schumacher M, Daschner F, Ruden H. How many nosocomial infections are missed if identification is restricted to patients with either microbiology reports or antibiotic administration. Infect Control Hosp Epidemiol 1999; 20:124‐127.
- 14. Chalfine A, Gonot J, Roche A, et al. Assessment of a laboratory‐based surveillance method for the detection of surgical site infection (SSI) in a general surgery department. In: Proceedings of the 39th Interscience Conference on Antimicrobial Agents and Chemothrapy; September 26‐29, 1999; San Francisco, CA. Abstract 519.
- 15. Goldmann DA, Weinstein RA, Wenzel RP et al. Strategies to prevent and control the emergence and spread of antimicrobial‐resistant microorganisms in hospitals: a challenge to hospital leadership. JAMA 1996; 275:234‐240.
- 16. Campaign to prevent antimicrobial resistance in healthcare settings. 2002. Available at: http://www.cdc.gov/drugresistance/healthcare/tools.htm. Accessed July 18, 2006.
- 17. Yokoe DS, Shapiro M, Simchen E, Platt R. Use of antibiotic exposure to detect postoperative infections. Infect Control Hosp Epidemiol 1998; 19:317‐322.
- 18. Evans RS, Larsen RA, Burke JP, et al. Computer surveillance of hospital‐acquired infections and antibiotic use. JAMA 1986; 256:1007‐1111.
- 19. Gravel‐Tropper D, Oxley C, Memish Z, Garber GE. Underestimation of surgical site infection rates in obstetrics and gynecology. Am J Infect Control 1995; 23:22‐26.
- 20 Sands K, Vineyard G, Platt R. Surgical site infections occurring after hospital discharge. J Infect Dis 1996; 173:963‐970.
- 21. Gravel‐Tropper D, Oxley C, Memish Z, Garber GE. Efficient identification of postdischarge surgical site infections: use of automated pharmacy dispensing information, administrative data, and medical record information. J Infect Dis 1999; 179:434‐441.