Improving Risk-Adjusted Measures of Surgical Site Infection for the National Healthcare Safety Network

Yi Mu PhD, Jonathan R. Edwards MStat, Teresa C. Horan MPH, Sandra I. Berrios-Torres MD and Scott K. Fridkin MD
Infection Control and Hospital Epidemiology
Vol. 32, No. 10 (October 2011), pp. 970-986
DOI: 10.1086/662016
Stable URL: http://www.jstor.org/stable/10.1086/662016
Page Count: 17
  • Download PDF
  • Cite this Item

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

Original Article

Improving Risk-Adjusted Measures of Surgical Site Infection for the National Healthcare Safety Network

Yi Mu, PhD,1
Jonathan R. Edwards, MStat,1
Teresa C. Horan, MPH,1
Sandra I. Berrios-Torres, MD,1 and
Scott K. Fridkin, MD1
1. Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, Georgia
    Address correspondence to Yi Mu, PhD, 1600 Clifton Road NE MS A-24, Atlanta, GA 30329-4018 ().

Background. The National Healthcare Safety Network (NHSN) has provided simple risk adjustment of surgical site infection (SSI) rates to participating hospitals to facilitate quality improvement activities; improved risk models were developed and evaluated.

Methods. Data reported to the NHSN for all operative procedures performed from January 1, 2006, through December 31, 2008, were analyzed. Only SSIs related to the primary incision site were included. A common set of patient- and hospital-specific variables were evaluated as potential SSI risk factors by univariate analysis. Some ific variables were available for inclusion. Stepwise logistic regression was used to develop the specific risk models by procedure category. Bootstrap resampling was used to validate the models, and the c-index was used to compare the predictive power of new procedure-specific risk models with that of the models with the NHSN risk index as the only variable (NHSN risk index model).

Results. From January 1, 2006, through December 31, 2008, 847 hospitals in 43 states reported a total of 849,659 procedures and 16,147 primary incisional SSIs (risk, 1.90%) among 39 operative procedure categories. Overall, the median c-index of the new procedure-specific risk was greater (0.67 [range, 0.59–0.85]) than the median c-index of the NHSN risk index models (0.60 [range, 0.51–0.77]); for 33 of 39 procedures, the new procedure-specific models yielded a higher c-index than did the NHSN risk index models.

Conclusions. A set of new risk models developed using existing data elements collected through the NHSN improves predictive performance, compared with the traditional NHSN risk index stratification.

Surgical site infection (SSI) is one of the most common healthcare-associated infections (HAIs) and is a major cause of increased length of hospital stay and mortality.1-3 SSI surveillance is integral to hospital infection control and quality improvement programs, with feedback of SSI rates being an important component of SSI reduction strategies.4,5 However, hospitals with surgeons who treat patients with multiple nonmodifiable risk factors would expect higher SSI rates. Therefore, risk adjustment that accounts for differences in patient case mix is critical to allow for more meaningful comparisons between surgeons or between hospitals, especially when using SSI summary data as a quality improvement performance metric.6,7

Controversies exist regarding several aspects of such risk adjustment. One is the inclusion of intraoperative or postoperative variables in any risk adjustment strategy, because these variables may reflect surgical technique more than patient case mix, and adjustment for surgical technique may inappropriately allow for adjusting rates down among surgeons with poor technique. Another is the inclusion of SSIs detected through SSI surveillance after discharge from the hospital, which is a setting with great variation in case-finding intensity. In addition, including more procedure-specific variables to generate improved procedure-specific models adds to the data collection burden.

These controversies are relevant to the National Healthcare Safety Network (NHSN), a secure Web-based system used by the Centers for Disease Control and Prevention (CDC) and its healthcare and public health partners for surveillance of HAIs, other adverse events in health care, and adherence to prevention practices in hospitals and other reporting facilities. Traditionally, SSI rates calculated by the CDC and other NHSN data users from data reported to the NHSN have been risk stratified using a risk index of 3 equally weighted factors: the American Society of Anesthesiologists (ASA) score, wound classification, and procedure duration.8,9 However, for some procedures, these variables are not associated with SSI risk, are not equally important in the risk they confer, and are candidates for replacement by other, more important risk factor variables that should be taken into account. Second, beginning in 2012, hospitals participating in the Center for Medicare and Medicaid Services (CMS) Inpatient Prospective Payment System (IPPS) will be required to report SSI data through NHSN, and these data will be included in the Inpatient Quality Reporting data that are publicly reported by CMS at the Hospital Compare Web site.10 Publicly reported SSI data should account for variability in patient case mix, adjust for all possible risk factors, and be based on consistent case detection systems.7,11 Procedure-specific, multivariate risk models that incorporate additional weighted patient factors could calculate more credible, standardized, and reliable risk-adjusted SSI metrics than stratified SSI rates that are limited to the traditional NHSN risk index.12-15

The objectives of our evaluation were to develop procedure-specific risk models for each of the procedure categories reported to the NHSN, incorporating existing NHSN data elements, and to compare their predictive performance with procedure category–specific models composed of only the variable of the traditional NHSN risk index. A secondary objective was to utilize similar methodology to develop models for proposed public reporting metrics (ie, using only deep incisional and organ/space SSIs detected during hospitalization or rehospitalization at the same hospital).

The resulting procedure-specific risk models can be used as a reference of how risk adjustment is currently performed in the NHSN application, and this article will essentially replace the historical annual report containing risk stratification tables.16

Methods

Study Population, Endpoints, and Statistical Approach

As of September 2010, more than 1,900 hospitals reported SSI data to the NHSN. Reporting has been predominately voluntary and confidential; however, during 2008–2009, several states enacted laws mandating SSI reporting to the NHSN for specific procedures at hospitals in their jurisdiction.17 The methodology of SSI surveillance has been described elsewhere.18 In brief, infection preventionists (IPs) choose a procedure category to follow for an entire month and report data on all patients undergoing all procedures within the procedure category for each month of surveillance performed. IPs also are required to identify and report all SSIs detected during the initial hospitalization, through surveillance after hospital discharge, or upon rehospitalization at the same hospital at which the initial procedure was performed. SSIs are classified using standard definitions as superficial incisional, deep incisional (involving the fascia or muscle), or organ/space. SSIs reported to the NHSN are limited to those detected within 30 days after the initial procedure (superficial incisional) or up to 1 year for deep incisional and organ/space if the procedure included an implant (eg, sternal wires or prosthesis).18

SSI data were analyzed for all reported procedures performed from January 1, 2006, through December 31, 2008, including data for all 40 NHSN procedure categories. For this analysis, the NHSN procedure code CBGB (with both sternal and harvest site incisions) and CBGC (with sternal site incision only) were grouped into a single procedure category, CABG, for a total of 39 procedures. In addition, only primary incisional SSIs were analyzed, because no patient- or procedure-specific variables were collected for secondary incision sites; therefore, any SSIs related to secondary incision sites for the NHSN codes CBGB, FUSN, and RFUSN were excluded. All SSIs (superficial incisional, deep incisional, and organ/space) detected through all methods of surveillance (hospitalization, rehospitalization, and surveillance after hospital discharge) for both inpatient and outpatient surgical procedures were included.

Procedures containing outlier values were excluded according to rules described in Appendix A. As a result, a total of 6,432 (0.75%) procedures were excluded from the analysis; the final number of procedures eligible for further analysis was 849,659.

First, patient and hospital characteristic data were evaluated. Second, NHSN risk index models were created for all 39 NHSN procedures. Third, new procedure-specific predictive risk models were created for the same set of procedures through an interactive process that included univariate analysis of all available patient- and hospital-level variables, multivariate modeling, and model validation. SAS, version 9.2 (SAS Institute), was used for data analysis. After completion of the primary analysis, endpoints were altered to include only complex (deep incisional and organ/space) SSIs detected at hospitalization and rehospitalization to develop models appropriate for public reporting, consistent with the 2008 National Quality Forum (NQF) recommendation to exclude superficial SSIs and those detected through surveillance after discharge from the hospital.19

NHSN Risk Index Model

The NHSN risk index comprises 3 dichotomous variables: ASA score (3, 4, or 5), wound classification (contaminated or dirty), and procedure duration in minutes (>75th percentile). Each risk factor represents 1 point; thus, the NHSN SSI risk index ranges from 0 (lowest risk) to 3 (greatest risk).8 Logistic regression of SSIs against the NHSN risk index was used to build the NHSN risk index models by procedure category.

New Procedure-Specific Risk Model

The new model incorporates the 3 NHSN risk index variables and additional data elements currently collected in the NHSN. These are variables of convenience in that they are routinely reported to the NHSN as part of the existing SSI surveillance methodology. Variables were dichotomous (general anesthesia, emergency procedure, gender, trauma association, and medical school affiliation), ordinal (ASA score), categorical (wound classification and number of hospital beds), or continuous (age and procedure duration; Table 1).

Table 1. 
List of Variables Collected and Available for Entry in the Models for All and Selected Procedures
Procedure codeVariable
AllGender, age, emergency, trauma, general anesthesia, ASA score, wound classification, duration, medical school affiliation, no. of hospital beds, endoscope, outpatient
HPROType of surgery (total primary, partial primary, partial revision, total revision)
KPROType of surgery (revision, primary)
CSECLabor, blood loss, body mass index
FUSN/RFUSNApproach, spinal level, diabetes

Procedure-specific supplemental variables include primary versus revision arthroplasty for HPRO and KPRO; total or partial hip arthroplasty for HPRO; body mass index (BMI), history of labor, and estimated blood loss for CSEC; and diagnosis of diabetes, spinal level, and surgical approach for FUSN and RFUSN (Table 1).

Among the variables common to all 849,659 procedures, 7 variables had missing values in 1,304 (0.15%) of the procedures. Variables with missing values were medical school affiliation (931 [0.11%]), trauma (219 [0.03%]), general anesthesia (89 [0.01%]), ASA score (23 [<0.01%]), endoscope (20 [<0.01%]), wound classification (12 [<0.01%]), and emergency (10 [<0.01%]). Among procedure-specific variables, missing values included the following: for CSEC, BMI (242 [0.78%]) and history of labor (5 [0.02%]); for HPRO, type of surgery (16 [0.01%]); for KPRO, type of surgery (15 [0.01%]); and for spine procedures, diabetes (157 [0.37%]), spinal level (3 [0.01%]), and surgical approach (3 [0.01%]; Table 2).

Table 2. 
Patient and Procedure Characteristics for Selected Procedures, National Healthcare Safety Network, 2006–2008
Procedure code, characteristicNo. (%) of procedures
HPRO, type of surgery
 Total primary99,046 (75.09)
 Partial primary19,658 (14.90)
 Total revision10,518 (7.97)
 Partial revision2,661 (2.02)
 Missing16 (0.01)
KPRO, type of surgery
 Revision11,673 (6.78)
 Primary160,382 (92.75)
 Missing15 (0.05)
CSEC, labor
 Y12,519 (40.53)
 N18,365 (59.45)
 Missing5 (0.02)
CSEC, blood loss
 ≤400 mL2,310 (7.48)
 401–800 mL23,854 (77.22)
 >800 mL4,725 (15.30)
CSEC, BMI
 ≤20323 (1.05)
 21–3011,736 (37.99)
 >3018,588 (60.18)
 Missing242 (0.78)
FUSN/RFUSN, approach
 Anterior16,955 (40.08)
 Anterior and posterior1,229 (2.90)
 Lateral transverse1,004 (2.37)
 Posterior16,493 (38.98)
 Not specifieda6,623 (15.65)
 Missing3 (0.01)
FUSN/RFUSN, spinal level
 Atlas-axis284 (0.67)
 Atlas-axis/cervical66 (0.16)
 Cervical16,225 (38.35)
 Cervical/dorsal/dorsolumbar120 (0.28)
 Dorsal/dorsolumbar1,909 (4.51)
 Lumbar/lumbosacral17,923 (42.36)
 Not specifieda5,777 (13.65)
 Missing3 (0.01)
FUSN/RFUSN, diabetes
 Y4,517 (10.68)
 N37,633 (88.95)
 Missing157 (0.37)

Univariate Analysis

The χ2 test was used to test for each individual variable’s association with SSI. Ordinal variables were collapsed into a single group if the χ2 test showed no significant difference between them. For categorical variables, multiple categorizations were used, and only the category most significantly associated with SSI risk was presented as the result of univariate analysis. Continuous variables were divided into quartiles and were compared by means of the χ2 test; continuous variables were coded as binary variables if a significant cutoff point was found. Otherwise, the continuous variable “duration” was coded as “duration10” for every 10-minute increase in duration, and “age” was coded as “age10” for every 10-year increase in age. Variables from the univariate analysis with were considered potential independent variables and entered into the logistic regression model as candidate variables for inclusion.

Multivariate Analysis

Stepwise logistic regression was used to develop the model. For all regression analyses, the referent category was the one that conferred the least risk of SSI. Variables were eligible for inclusion if the likelihood ratio test (LRT) and removed at LRT significance. For variables with multiple categorical, ordinal, or dichotomous cutoff values, the one with the smallest LRT P value was included.

Final Model Variable Selection Procedure

To confirm the appropriateness of the final models, we performed the same stepwise model selection with all variables included regardless of their significance levels in univariate analysis. The interaction terms were tested and kept at LRT significance.

Training and Validation Samples

The models were validated using a bootstrap sample following the steps described in Appendix B.

Model Comparison

The predictive performances of the new and existing NHSN risk index models were assessed by constructing receiver-operating characteristic (ROC) curves and calculating the c-index for the separate logistic regression models. An ROC curve is constructed by plotting the sensitivity (y-axis) versus 1 minus specificity (x-axis) over the range of scores for a given index. The area under the ROC curve (AUC) is the c-index. The c-index is a measure of predictive performance and represents the proportion of instances in which a patient who acquires an SSI is assigned a higher probability of SSI than a patient who does not acquire an SSI. Values for the c-index range from 0.5 (null) to 1.0 (perfect predictive ability).20 The difference in c-index was tested using the method described by Hanley and McNeil.21

Prediction Models for Possible Public Reporting

To be consistent with proposed measures submitted to the NQF regarding public reporting of SSI, we also evaluated the performance characteristics of procedure-specific models for the subset of SSIs classified as deep or organ/space and detected only during the hospitalization during which the surgical procedure was performed or upon rehospitalization at the same facility. To perform this task, we repeated all of the methodologies described for all incisional SSIs for the subset of SSIs classified as complex (deep incisional or organ/space) detected during hospitalization or after rehospitalization at the same hospital. These models are referred to as predictive of complex SSI for public reporting.

Results

Demographic Characteristics

From January 1, 2006, through December 31, 2008, 847 hospitals reported to the NHSN a total of 849,659 procedures and 16,147 SSIs at the primary incision site. The overall risk of SSI was 1.90 per 100 procedures, ranging from 0.26 (THYR) to 13.83 (LTP). The variability in patient and hospital characteristics for some of the main procedure-specific variables is summarized in Table 2.

Univariate Analysis

A list of the significant variables for each of the 39 procedures is summarized in Table 3. As an example, univariate analysis results are shown for hip prostheses (HPRO), for which there were 10 potential independent variables identified for inclusion in the multivariate modeling (Table 4).

Table 3. 
List of Variables That Are Significant on Univariate Analysis for 39 Procedures, National Healthcare Safety Network, 2006–2008
Procedure codeDescriptionList of variables
AAAAbdominal aortic aneurysmEmergency, wound class, ASA score, duration
AMPLimb amputationBed size, duration
APPYAppendectomyEmergency, endoscope, gender, ASA score, wound class
AVSDArteriovenous shunt for dialysisAge, duration
BILIBile duct, liver or pancreatic surgeryEmergency, endoscope, ASA score, wound class, bed size, duration
BRSTBreast surgeryASA score, bed size, duration
CABGCoronary artery bypass graftAnesthesia, gender, medical school affiliation, ASA score, bed size, age, duration
CARDCardiac surgeryASA score, wound class, age, duration
CEACarotid endarterectomy
CHOLCholecystectomyEmergency, endoscope, ASA score, wound class, age, duration
COLOColon surgeryAnesthesia, endoscope, gender, ASA score, wound class, bed size, age, duration
CRANCraniotomyTrauma, bed size, age, duration
CSECCesarean deliveryBody mass index, age, anesthesia, ASA, duration, labor, bed size, wound class, emergency
FUSNSpinal fusionAnesthesia, gender, medical school affiliation, trauma, wound class, diabetes, approach, spinal level, duration, ASA score
FXOpen reduction of long bone fractureASA score, age, duration, outpatient
GASTGastric surgeryEmergency, endoscope, gender, ASA score, wound class, age, duration
HERHerniorrhaphyAnesthesia, emergency, endoscope, gender, medical school affiliation, trauma, ASA score, wound class, duration, outpatient
HPROHip arthroplastyAnesthesia, emergency, gender, trauma, ASA score, wound class, bed size, age, duration, total/primary/partial/revision
HTPHeart transplant
HYSTAbdominal hysterectomyAnesthesia, endoscope, ASA score, wound class, duration
KPROKnee arthroplastyAnesthesia, gender, trauma, ASA score, wound class, age, duration, primary/revision
KTPKidney transplantBed size, age, duration
LAMLaminectomyAnesthesia, endoscope, gender, ASA score, age, duration
LTPLiver transplantAnesthesia, emergency, trauma, age, duration
NECKNeck surgeryWound class, duration
NEPHKidney surgeryDuration
OVRYOvarian surgeryASA score, wound class
PACEPacemaker surgery
PRSTProstate surgeryASA score
PVBYPeripheral vascular bypass surgeryGender, ASA score, age, duration
RECRectal surgeryEndoscope, gender, trauma, wound class, bed size, duration
RFUSNRefusion of spineTrauma, duration, diabetes, spinal level, approach
SBSmall-bowel surgeryBed size, duration
SPLESpleen surgery
THORThoracic surgeryDuration
THYRThyroid and/or parathyroid surgeryAge
VHYSVaginal hysterectomyMedical school affiliation, bed size, age, duration
VSHNVentricular shuntEmergency, wound class, bed size, age
XLAPExploratory abdominal surgeryBed size, duration
Table 4. 
Predictors of Incisional Surgical Site Infection (SSI) by Univariate Analysis among Hip Arthroplasty (HPRO) Procedures Reported to the National Healthcare Safety Network, 2006–2008
Variable, classNo. of proceduresNo. of SSIsRiskP
Age10131,8991,8551.41.0057
Anesthesia<.0001
 N38,2494561.19
 Y93,6461,3991.49
ASAa<.0001
 1/266,9455650.84
 356,8841,0861.91
 4/58,0692042.53
Duration10131,8991,8551.41<.0001
Emergency.0004
 N123,8291,7041.38
 Y8,0701511.87
Endoscope.6686
 N130,9991,8411.41
 Y900141.56
Gender.6022
 F76,6341,0891.42
 M55,2657661.39
Type of surgeryb<.0001
 Total primary99,0461,1341.14
 Partial primary19,6583881.97
 Total revision10,5182512.39
 Partial revision2,661823.08
Medical school affiliation.1784
 N50,7086851.35
 Y81,1381,1701.44
Bed size<.0001
 ≤500100,6541,3421.33
 >50031,2455131.64
Wound class<.0001
 C128,8971,7841.38
 CC/CO/D3,001712.37
Trauma<.0001
 N121,1101,6081.33
 Y10,7892472.29

Procedure-Specific Risk Prediction Model

Table 5 shows the results for models of all SSIs identified at primary incision sites for the 39 procedure categories. Multivariate modeling strategies defined new procedure-specific models for each of the 39 procedure categories. The 3 most common variables included were procedure duration, ASA score, and age (30, 21, and 20 models, respectively). Other common variables were the number of hospital beds (16 models), wound class (8), general anesthesia (6), endoscope (5), medical school affiliation (5), emergency (4), and trauma (4). All procedure-specific supplemental variables, except estimated blood loss, were selected for inclusion into the final model. No variables were selected at the level for 4 procedures (ie, intercept-only models): carotid endarterectomy (CEA), heart transplant (HTP), pacemaker placement (PACE), and splenectomy (SPLE). The observed number of SSIs for these 4 procedures during the study period was small, ranging from 6 to 15 (Table 5).

Table 5. 
Models to Predict All Surgical Site Infections (SSIs) at Primary Incision Site for 39 Procedures, National Healthcare Safety Network (NHSN), 2006–2008
c-index
Procedure codeNo. of proceduresNo. of SSIsEffectEstimateOR (95% CI)PPSMRIMPr>|t|
AAA1,95063.66.64.1749
Intercept−4.20<.0001
Duration100.041.04 (1.02–1.06)<.0001
AMP1,41331.74.62.0007
Intercept−6.74<.0001
Duration, >82 vs ≤821.092.97 (1.43–6.18).0036
Bed size, >200 vs ≤2003.0420.96 (2.83–154.99).0029
APPY6,12285.70.60.0037
Intercept−5.54<.0001
Emergency, Y vs N0.611.84 (1.14–2.99).0135
Gender, M vs F0.531.70 (1.07–2.68).024
Bed size, >500 vs ≤5000.772.15 (1.38–3.34).0007
Wound class, CO vs C/CC0.631.89 (1.07–3.33).0294
Wound class, D vs C/CC1.263.53 (2.04–6.09)<.0001
AVSD86411.77.61.1521
Intercept−1.73.0618
Age10−0.460.63 (0.45–0.89).0082
BILI89489.75.59<.0001
Intercept−4.15<.0001
ASA, ≤3 vs >31.022.76 (1.29–5.89).0087
Duration100.031.03 (1.02–1.05)<.0001
Bed size, 201–500 vs ≤200/>5000.932.54 (1.51–4.29).0005
BRST4,76875.76.71.0147
Intercept−6.22<.0001
ASA, >2 vs ≤20.922.50 (1.57–4.00).0001
Duration100.061.06 (1.05–1.08)<.0001
Bed size, ≤200/>500 vs 201–5000.932.54 (1.29–4.98).0068
CABG133,4882,899.62.54<.0001
Intercept−5.10<.0001
Age10−0.02.4978
Age10∶gender (interaction)−0.24<.0001
ASA (1/2, 3, 4/5)0.281.33 (1.22–1.44)<.0001
Duration100.021.02 (1.02–1.03)<.0001
Gender, F vs M2.16<.0001
Bed size, ≤200/>500 vs 201–5000.151.16 (1.08–1.26).0001
CARD29,758381.60.55.0011
Intercept−4.57<.0001
Age10−0.110.90 (0.86–0.93)<.0001
ASA, >3 vs ≤30.491.63 (1.28–2.07)<.0001
Duration100.021.02 (1.01–1.03)<.0001
CEA4,54815.50.52.5626
Intercept−5.71<.0001
CHOL24,810138.75.71.0001
Intercept−7.16<.0001
Age, >52 vs ≤520.441.55 (1.05–2.29).0272
ASA (1, 2, 3/4/5)0.601.82 (1.30–2.56).0005
Duration100.081.08 (1.06–1.11)<.0001
Endoscope, N vs Y0.431.54 (1.07–2.20).0191
Wound class, CO/D vs C/CC0.681.97 (1.18–3.27).0093
COLO62,7773,647.59.56<.0001
Intercept−3.89<.0001
Age10−0.020.98 (0.96–1.00).0389
Anesthesia, Y vs N0.381.47 (1.02–2.12).0405
ASA, >2 vs ≤20.301.35 (1.26–1.46)<.0001
Duration100.031.03 (1.02–1.03)<.0001
Endoscope, N vs Y0.131.14 (1.04–1.25).0063
Medical school affiliation, N vs Y0.141.15 (1.06–1.25).0008
Bed size, >500 vs ≤5000.261.30 (1.19–1.41)<.0001
Wound class, CO/D vs C/CC0.091.10 (1.01–1.19).0369
CRAN9,918262.65.56<.0001
Intercept−4.05<.0001
Age10−0.140.87 (0.82–0.92)<.0001
ASA, >2 vs ≤20.321.38 (1.04–1.82).0243
Duration100.031.03 (1.02–1.04)<.0001
Bed size, >500 vs ≤5000.451.57 (1.18–2.09).0022
Trauma, Y vs N0.541.72 (1.12–2.65).0141
CSEC30,645574
Intercept−6.56<.0001.66.58<.0001
BMI0.041.04 (1.03–1.05)<.0001
Age, ≤26 vs >260.271.31 (1.11–1.55).0017
Anesthesia, Y vs N0.421.52 (1.15–2.00).0032
ASA (1, 2, 3/4/5)0.281.32 (1.10–1.59).0026
Duration100.131.14 (1.09–1.18)<.0001
Emergency, Y vs N0.211.23 (1.03–1.47).0214
Labor, Y vs N0.411.51 (1.27–1.80)<.0001
Wound class, CO/D vs C/CC0.742.09 (1.39–3.15).0004
FUSN41,160618.75.67<.0001
Intercept−6.40<.0001
Approach, B/L/P vs A0.932.52 (1.96–3.25)<.0001
ASA (1/2, 3, 4/5)0.611.83 (1.60–2.10)<.0001
Diabetes, Y vs N0.421.52 (1.23–1.87).0001
Duration100.031.03 (1.02–1.03)<.0001
Medical school affiliation, Y vs N0.321.38 (1.14–1.68).0011
Spinal level, CD or DL vs AX, AC, or CV0.671.96 (1.41–2.72)<.0001
Spinal level, LL vs AX, AC, or CV0.521.68 (1.32–2.14)<.0001
Wound class, CO/D vs C/CC0.842.31 (1.06–5.03).035
Trauma, Y vs N0.601.83 (1.23–2.71).0026
FX11,361187.65.60.0003
Intercept−6.91<.0001
Age, >25 vs ≤250.722.05 (1.29–3.28).0026
ASA (1, 2, 3/4/5)0.291.34 (1.07–1.68).0119
Duration, >138 vs ≤1380.772.16 (1.58–2.95)<.0001
Bed size, 201–500 vs ≤200/>5000.371.45 (1.07–1.95).0153
Outpatient, N vs Y1.514.52 (1.11–18.36).0349
GAST8,223183.68.62<.0001
Intercept−5.16<.0001
ASA, >2 vs ≤20.471.60 (1.06–2.40).0245
Duration100.061.07 (1.05–1.08)<.0001
Emergency, Y vs N0.641.90 (1.19–3.04).0074
HER18,451227.78.71<.0001
Intercept−7.25<.0001
Age, ≤71 vs >710.742.09 (1.42–3.07).0002
ASA (1, 2, 3/4/5)0.762.15 (1.68–2.74)<.0001
Duration100.051.06 (1.04–1.07)<.0001
Gender, F vs M0.832.30 (1.73–3.04)<.0001
Outpatient, N vs Y0.591.80 (1.28–2.53).0008
HPRO131,8791,855.66.61<.0001
Intercept−5.00<.0001
Age10−0.070.94 (0.90–0.97).0002
Anesthesia, Y vs N0.111.12 (1.01–1.25).0383
ASA, 3 vs 1/20.802.23 (2.01–2.49)<.0001
ASA, 4/5 vs 1/21.072.91 (2.45–3.46)<.0001
Duration100.041.04 (1.03–1.05)<.0001
Type of surgerya0.261.29 (1.22–1.38)<.0001
Bed size, >500 vs ≤5000.191.21 (1.09–1.34).0004
Trauma, Y vs N0.361.43 (1.24–1.65)<.0001
HTP36412.50.54.5898
Intercept−3.38<.0001
HYST54,877975.66.62<.0001
Intercept−6.09<.0001
Age10−0.130.88 (0.83–0.93)<.0001
Anesthesia, Y vs N0.681.97 (1.26–3.07).003
ASA (1, 2, 3/4/5)0.862.37 (2.10–2.67)<.0001
Duration100.041.04 (1.03–1.05)<.0001
Endoscope, N vs Y0.351.43 (1.17–1.74).0005
Bed size, ≤500 vs >5000.221.25 (1.06–1.47).0065
KPRO172,0551,723.64.60<.0001
Intercept−5.77<.0001
Age, ≤58 vs >580.301.34 (1.21–1.49)<.0001
Anesthesia, Y vs N0.111.12 (1.01–1.24).0383
ASA (1/2, 3, 4/5)0.481.62 (1.49–1.76)<.0001
Duration100.051.05 (1.04–1.06)<.0001
Gender, M vs F0.201.22 (1.11–1.34)<.0001
Revision vs primary0.631.89 (1.64–2.17)<.0001
Bed size, >200 vs ≤2000.111.12 (1.01–1.25).039
Trauma, Y vs N0.691.99 (1.31–3.03).0013
KTP1,62575.75.60<.0001
Intercept−5.09<.0001
Age, >59 vs ≤590.772.16 (1.32–3.54).0021
ASA, >3 vs ≤30.511.67 (1.01–2.75).0452
Duration100.051.05 (1.03–1.07)<.0001
Bed size, ≤500 vs >5001.303.65 (2.19–6.09)<.0001
LAM41,414428.62.60.0003
Intercept−6.33<.0001
Anesthesia, Y vs N0.712.03 (1.04–3.94).0371
ASA (1, 2, 3/4/5)0.501.64 (1.43–1.89)<.0001
Duration100.031.03 (1.02–1.04)<.0001
Endoscope, Y vs N1.353.85 (1.57–9.49).0033
LTP824114.71.56<.0001
Intercept−3.30<.0001
Age, ≤43 vs 44–581.072.92 (1.81–4.71)<.0001
Age, >58 vs 44–580.621.86 (1.10–3.15).0215
Duration, >320 vs ≤3201.012.74 (1.75–4.30)<.0001
Emergency, Y vs N0.641.90 (1.22–2.93).0042
NECK60221.81.77.2464
Intercept−4.67<.0001
Duration100.041.04 (1.02–1.06)<.0001
NEPH69110.72.73.9887
Intercept−5.26<.0001
Duration100.051.05 (1.01–1.09).0263
OVRY3,01617.67.68.7069
Intercept−5.84<.0001
ASA, >2 vs ≤21.383.99 (1.47–10.82).0065
PACE3,43813.50.53.5148
Intercept−5.57<.0001
PRST1,03312.67.65.7248
Intercept−5.55<.0001
Duration, >178 vs ≤1781.625.07 (1.11–23.25).0367
PVBY6,210412.60.53<.0001
Intercept−2.70<.0001
Age10−0.160.85 (0.79–0.92)<.0001
ASA, >2 vs ≤20.571.77 (1.16–2.69).0076
Duration100.021.02 (1.01–1.03)<.0001
Gender, F vs M0.321.38 (1.12–1.69).0021
Medical school affiliation, N vs Y0.231.26 (1.02–1.56).0338
REC1,21583.72.62<.0001
Intercept−4.14<.0001
Duration100.041.04 (1.02–1.06)<.0001
Endoscope, Y vs N0.581.78 (1.08–2.95).0242
Gender, M vs F0.481.61 (1.01–2.58).0464
Wound class, CO/D vs C/CC0.822.26 (1.42–3.61).0006
RFUSN98729.73.66.1405
Intercept−6.34<.0001
Approach, B/L/P vs A2.128.35 (1.12–62.16).038
Diabetes, Y vs N1.092.98 (1.16–7.69).024
Duration, >209 vs ≤2091.253.48 (1.39–8.69).008
SB4,200252.65.56<.0001
Intercept−4.07<.0001
Duration, >125 vs ≤1250.902.46 (1.87–3.24)<.0001
Bed size, >200 vs ≤2000.962.60 (1.76–3.84)<.0001
SPLE2576.50.70.0172
Intercept−3.73<.0001
THOR1,97922.72.63.0244
Intercept−5.52<.0001
Duration, >187 vs ≤1871.404.04 (1.72–9.46).0013
Bed size, >500 vs ≤5001.032.79 (1.18–6.60).0198
THYR1,1683.85.63.032
Intercept−3.11.0033
Age10−0.710.49 (0.27–0.91).0244
VHYS19,056185.65.56<.0001
Intercept−5.89<.0001
Age, ≤44 vs >440.661.94 (1.43-2.64)<.0001
ASA, >2 vs ≤20.421.51 (1.03–2.23).0363
Duration, >100 vs ≤1000.501.65 (1.22–2.23).0011
Medical school affiliation, Y vs N0.892.42 (1.76–3.34)<.0001
VSHN5,379288.67.51<.0001
Intercept−6.13<.0001
Age, ≤1 vs >10.772.16 (1.69–2.75)<.0001
Medical school affiliation, Y vs N0.692.00 (1.23–3.23).005
Bed size, ≤200/>500 vs 201–5001.665.24 (2.92–9.40)<.0001
Wound class, C vs CC/CO/D0.822.27 (1.29–4.01).0045
XLAP5,115100.63.60.3044
Intercept−3.95<.0001
Age10−0.090.91 (0.84–1.00).0434
Duration, >197 vs ≤1970.661.93 (1.28–2.92).0017
Bed size, >500 vs ≤5000.531.71 (1.13–2.57).0104

Model Performance

For the NHSN risk index models, the c-index ranged from 0.51 (VSHN) to 0.77 (NECK), compared with 0.59 (COLO) to 0.85 (THYR) for the new procedure-specific risk models (resultant increase in the c-index from 0 to 0.2). For 33 procedures, the new models yielded a higher c-index than did the NHSN index models, and for 28 of these, the improvement was statistically significant (; Table 5).

The subset analysis of only complex (deep incisional and organ/space) SSIs that occurred during hospitalization or rehospitalization at the same hospital resulted in prediction models that, overall, had a c-index similar to or higher than that for all SSIs, but 9 procedures had intercept-only models, which was more than what was observed in all SSIs models (Table 6).

Table 6. 
Multivariate Models Predicting Deep Incisional and Organ/space Surgical Site Infections (SSIs) Detected During Initial Hospitalization or Rehospitalization at the Same Hospital for 39 Procedures Reported to the National Healthcare Safety Network, 2006–2008
Procedure codeNo. of proceduresNo. of SSIsEffectEstimateOR (95% CI)Pc-index
AAA1,95030.70
Intercept−5.15<.0001
Duration100.041.04 (1.02–1.07).0004
Wound class, CO/D vs C/CC2.3710.72 (3.19–36.07).0001
AMP1,4139.50
Intercept−5.05<.0001
APPY5,88950.74
Intercept−6.62<.0001
Emergency, Y vs N0.872.38 (1.21–4.67).0116
Gender, M vs F0.842.31 (1.22–4.38).0099
Bed size, >500 vs ≤5000.942.56 (1.44–4.54).0013
Wound class, CO/D vs C/CC1.072.90 (1.64–5.15).0003
AVSD8648.77
Intercept−1.89.0761
Age10−0.500.61 (0.41–0.91).0152
BILI89463.76
Intercept−4.88<.0001
ASA, ≤3 vs >31.343.83 (1.36–10.82).0113
Duration100.031.03 (1.02–1.05)<.0001
Bed size, 201–500 vs ≤200/>5001.253.49 (1.97–6.20)<.0001
BRST3,16725.81
Intercept−7.91<.0001
ASA, >2 vs ≤21.454.25 (1.84–9.79).0007
Duration100.061.06 (1.04–1.08)<.0001
Bed size, ≤200/>500 vs 201–5001.514.51 (1.05–19.32).0422
CABG133,4881,644.62
Intercept−6.55<.0001
Age100.07.0187
Age10∶gender (interaction)−0.26<.0001
ASA (1/2, 3, 4/5)0.381.47 (1.31–1.65)<.0001
Duration100.031.03 (1.02–1.03)<.0001
Gender, F vs M2.29<.0001
Medical school affiliation, Y vs N0.191.21 (1.08–1.36).0009
CARD29,757229.59
Intercept−5.23<.0001
Age, ≤56 vs >56 years0.351.42 (1.09–1.85).0093
Duration, >306 vs ≤3060.611.83 (1.40–2.40)<.0001
Emergency, Y vs N0.481.61 (1.07–2.41).0215
CEA4,5485.50
Intercept−6.81<.0001
CHOL14,72663.77
Intercept−7.65<.0001
Age, >52 vs ≤520.792.21 (1.18–4.13).0131
ASA, >2 vs ≤20.621.86 (1.03–3.35).0382
Duration100.071.08 (1.04–1.11)<.0001
Bed size, >200 vs ≤2000.962.61 (1.41–4.82).0022
COLO62,7821,825.61
Intercept−4.72<.0001
Age, ≤75 vs >750.151.16 (1.03–1.30).0137
ASA, >2 vs ≤20.331.39 (1.26–1.54)<.0001
Duration100.031.03 (1.03–1.04)<.0001
Endoscope, N vs Y0.181.19 (1.05–1.36).0088
Medical school affiliation, N vs Y0.161.18 (1.06–1.31).0028
Bed size, >200 vs ≤2000.211.23 (1.10–1.37).0004
Wound class, CO/D vs C/CC0.191.21 (1.08–1.36).0013
CRAN9,918198.65
Intercept−4.02<.0001
Age10−0.150.86 (0.81–0.92)<.0001
Duration100.021.02 (1.01–1.03)<.0001
Bed size, >500 vs ≤5000.561.75 (1.24–2.46).0013
CSEC30,645160.75
Intercept−7.63<.0001
BMI0.031.03 (1.01–1.05).0078
Age10−0.480.62 (0.47–0.81).0004
Anesthesia, Y vs N0.551.74 (1.09–2.78).0209
ASA (1, 2, 3/4/5)0.531.69 (1.21–2.37).0023
Duration100.221.25 (1.17–1.33)<.0001
Labor, Y vs N0.832.29 (1.65–3.18)<.0001
Bed size, >200 vs ≤2000.842.32 (1.53–3.52)<.0001
Wound class, CO/D vs C/CC1.072.91 (1.50–5.65).0015
FUSN41,161383.75
Intercept−6.90<.0001
Approach, B/L/P vs A0.942.56 (1.85–3.55)<.0001
ASA (1/2, 3, 4/5)0.601.82 (1.54–2.16)<.0001
Diabetes, Y vs N0.391.48 (1.13–1.93).0045
Duration100.031.03 (1.02–1.04)<.0001
Medical school affiliation, Y vs N0.341.41 (1.10–1.80).0067
Spinal level, CD/DL vs AX/AC/CV0.822.26 (1.51–3.40)<.0001
Spinal level, LL vs AX/AC/CV0.491.63 (1.20–2.22).002
FX10,646117.64
Intercept−5.80<.0001
Age, >25 vs ≤250.832.29 (1.36–3.86).0018
Duration, >138 vs ≤1380.922.52 (1.72–3.69)<.0001
Bed size, >200 vs ≤2000.551.73 (1.17–2.56).0064
GAST8,223104.66
Intercept−6.18<.0001
Age100.211.24 (1.08–1.41).0017
Duration100.061.06 (1.04–1.08)<.0001
HER7,48792.77
Intercept−8.11<.0001
Age, ≤71 vs >710.932.53 (1.36–4.71).0035
ASA (1, 2, 3/4/5)0.752.12 (1.39–3.22).0005
Duration100.061.06 (1.04–1.08)<.0001
Gender, F vs M0.852.35 (1.50–3.69).0002
Bed size, >200 vs ≤2000.832.30 (1.31–4.01).0035
HPRO131,8261,183.67
Intercept−5.69<.0001
Age10−0.090.92 (0.88–0.96)<.0001
Anesthesia, Y vs N0.171.19 (1.03–1.36).016
ASA, 3 vs 1/20.822.27 (1.98–2.59)<.0001
ASA, 4/5 vs 1/21.072.91 (2.34–3.61)<.0001
Duration100.041.04 (1.03–1.05)<.0001
Type of surgerya0.351.43 (1.32–1.54)<.0001
Medical school affiliation, Y vs N0.191.21 (1.07–1.37).003
Bed size, >200 vs ≤2000.311.37 (1.20–1.56)<.0001
Trauma, Y vs N0.241.27 (1.05–1.53).0126
HTP36411.50
Intercept−3.47<.0001
HYST54,877389.64
Intercept−5.82<.0001
Age10−0.170.85 (0.77–0.93).0003
ASA (1, 2, 3/4/5)0.732.08 (1.73–2.50)<.0001
Duration100.041.04 (1.03–1.06)<.0001
Bed size, ≤500 vs >5000.331.39 (1.07–1.80).0137
KPRO172,0391,108.65
Intercept−6.39<.0001
Age, ≤58 vs >580.341.41 (1.24–1.61)<.0001
ASA (1/2, 3, 4/5)0.491.64 (1.47–1.82)<.0001
Duration100.051.05 (1.04–1.06)<.0001
Gender, M vs F0.351.42 (1.26–1.60)<.0001
Revision vs primary0.782.18 (1.85–2.58)<.0001
Medical school affiliation, Y vs N0.161.18 (1.04–1.33).0096
Bed size, >200 vs ≤2000.181.20 (1.04–1.38).01
Trauma, Y vs N0.681.97 (1.18–3.31).0099
KTP1,62533.67
Intercept−5.38<.0001
ASA, >3 vs ≤30.872.39 (1.09–5.22).0292
Duration100.041.05 (1.02–1.07).0012
LAM40,513218.64
Intercept−6.89<.0001
ASA (1, 2, 3, 4/5)0.521.68 (1.38–2.03)<.0001
Duration100.031.03 (1.02–1.05)<.0001
Medical school affiliation, N vs Y0.661.94 (1.37–2.76).0002
Bed size, >500 vs ≤5000.611.84 (1.32–2.56).0003
LTP82496.72
Intercept−3.34<.0001
Age, ≤43 vs 44–581.303.66 (2.18–6.16)<.0001
Age, >58 vs 44–580.782.18 (1.23–3.86).0074
Duration, >320 vs ≤3201.143.12 (1.92–5.06)<.0001
NECK60212.85
Intercept−5.43<.0001
Duration100.041.04 (1.02–1.06)<.0001
NEPH6919.50
Intercept−4.33<.0001
OVRY3,0162.50
Intercept−7.32<.0001
PACE3,4387.50
Intercept−6.20<.0001
PRST1,0335.50
Intercept−5.33<.0001
PVBY6,210176.63
Intercept−4.50<.0001
Age, ≤58 vs >580.561.75 (1.27–2.39).0005
ASA, >3 vs ≤30.391.47 (1.07–2.02).0173
Duration100.021.02 (1.01–1.04).0013
Medical school affiliation, N vs Y0.621.86 (1.36–2.55).0001
REC1,21538.77
Intercept−5.90<.0001
Duration100.041.04 (1.02–1.06)<.0001
Gender, M vs F1.062.87 (1.39–5.92).0043
Bed size, >500 vs ≤5001.243.46 (1.43–8.40).006
RFUSN99124.65
Intercept−4.59<.0001
Duration, >209 vs ≤2091.373.94 (1.46–10.63).0069
SB4,200141.65
Intercept−4.79<.0001
Duration, >125 vs ≤1250.922.51 (1.73–3.64)<.0001
Bed size, 201–500 vs ≤2000.992.8 (1.40–5.12).0028
Bed size, >500 vs ≤2001.082.96 (1.71–5.12).0001
SPLE2574.50
Intercept−4.15<.0001
THOR1,97913.72
Intercept−5.91<.0001
Duration, >187 vs ≤1871.936.85 (2.10–22.35).0014
THYR1,1681.50
Intercept−7.06<.0001
VHYS19,009122.67
Intercept−3.96<.0001
Age10−0.460.63 (0.53–0.76)<.0001
Duration100.031.03 (1.00–1.07).0366
Medical school affiliation, Y vs N0.872.38 (1.61–3.53)<.0001
VSHN5,379270.66
Intercept−6.17<.0001
Age, ≤1 vs >10.762.14 (1.67–2.75)<.0001
Medical school affiliation, Y vs N0.621.86 (1.15–3.02).012
Bed size, ≤200/>500 vs 201–5001.775.87 (3.11–11.11)<.0001
Wound class, C vs CC/CO/D0.752.12 (1.20–3.74).0094
XLAP5,11539.59
Intercept−5.48<.0001
Duration100.041.04 (1.01–1.06).001

Discussion

Risk models based on the NHSN risk index, although simple in design, showed poor predictive performance for many procedures. New procedure-specific predictive models developed with currently collected NHSN data elements significantly improved the predictive performance for most procedures, including all of the most common procedures reported to the NHSN.

This study represents a large and robust data set of almost 850,000 surgical procedures among 39 procedure categories reported since 2006 by 847 hospitals in 43 states. Most of the potential predictive factors included have been previously identified as risk factors in other studies.6,22-32 The c-indices also approximate what has been reported in other studies,6,25,32 which suggests some reproducibility in these findings.

We found that the procedure duration was the most common of the 3 traditional NHSN risk index parameters selected by 30 of the 39 models; ASA score was the next most common (21 models). Age, which is not a component of the traditional NHSN risk index, was the third most commonly selected factor (included in 18 models). Because patient-specific variables available for analysis were limited, we also included hospital-level variables. These likely serve as proxy indicators for patient case mix or possibly for surgical programs. We incorporated hospital-specific information, including the number of hospital beds (16 models) and medical school affiliation (5). Including these latter variables as well as procedure duration could introduce some risk adjustment for surgical performance (ie, surgical residents performing at teaching facilities) and/or for patient case mix (higher risk patients cared for at teaching facilities). Until further patient- (eg, BMI and diabetes) and procedure-specific data are available to allow comparable risk adjustment without including such proxy indicators or intermediate outcomes (like duration), we decided to maximize risk adjustment using all of the information available.

Although SSI prediction improved considerably with our models, the resulting c-indices still remained relatively low. This may result from the characteristics of the NHSN surveillance data in which, for most procedures, there are no procedure-specific risk factors. For the 5 procedures for which procedure-specific data elements were collected, improvement was noted. For example, in addition to those factors collected across all procedures, our CSEC model included BMI and whether the patient was in labor. This resulted in a model with significant improvement in predictive performance, compared with that reported by Brandt et al6 (0.66 vs 0.55), which included only ASA score, procedure duration, age, and wound class. Further improvement can be expected with additional patient- and procedure-specific factors, such as diabetes, duration of preoperative hospital stay, indication for surgery, and the number of discharge diagnoses.33 Beginning in January 2013, the NHSN will require submission of BMI and diabetes information for all procedures.

Our findings indicate that, with use of the currently available NHSN data, the new procedure-specific risk models significantly improved SSI prediction. This justifies their use in facility-specific performance comparisons with an external benchmark, which serve as guides for internal quality improvement efforts. To enable NHSN users to take advantage of the new procedure-specific risk models, the CDC has incorporated them into the NHSN application. The new models supersede the NHSN risk index for procedures in which the traditional NHSN risk index has little discriminatory power. Improved risk adjustment may provide SSI data that are more convincing to clinicians and thus more effective in guiding changes in infection-prevention practices. In addition, separate models for predicting the subset of SSIs classified as complex (deep incisional or organ/space infections) detected during initial hospitalization or upon rehospitalization at the same hospital were developed (Table 6). These models may be more acceptable for public reporting, because there may be less variability to detect this subset between facilities when excluding those infections detected by surveillance after hospital discharge and superficial infections. In addition, the model fit, as measured by the c-index, was improved for a number of the procedures, which indicates that perhaps the identified risk factors are better at predicting this subset of SSIs. However, any models developed for public reporting will need frequent reevaluation as more information becomes available and the quality measure environment changes.19 Likewise, even for the overall SSI models, caution should also be exercised when evaluating some of these models. Specifically, 9 procedure categories (AVSD, CEA, HTP, NEPH, OVRY, PACE, PRST, SPLE, and THYR) had fewer than 20 SSI events, and for 4 of these (CEA, HTP, PACE, and SPLE), we were able to construct intercept-only models. The intercept-only models produce essentially unadjusted infection rates for comparison, and for the other models, the risk estimates might not be stable because of an insufficient number of SSI cases. These models can and should be modified as additional information on methods to improve risk adjustment (eg, the addition of specific patient-level variables) for specific procedures and the ability to reliably and effortlessly acquire these variables from surgical or facility information systems as part of routine SSI surveillance improve. This is an ongoing, deliberate, and iterative process. The NHSN is committed to pursue additional efforts to present the best available risk-adjusted SSI data to reporting facilities and to make accurate overall assessments of the status of SSI prevention efforts in the United States.

Acknowledgments

We thank the NHSN participants for their ongoing efforts to monitor healthcare-associated infections and improve patient safety and our colleagues in the Division of Healthcare Quality Promotion for their tireless support of this unique public health network.

Potential conflicts of interest. All authors report no conflicts of interest relevant to this article.

Appendix A

Outlier exclusion rules:

(1) Exclude all procedures where duration in minutes = 0 ();

(2) Exclude all procedures where patient was less than 1 day old or greater than 109 years old ();

(3) Exclude all procedures where wound class was undefined ();

(4) Exclude all procedures with duration between 0 and 5 minutes or more than 5 times the interquartile range ().

Appendix B

Bootstrap resampling steps:

(1) For each procedure category, 100 independent samples of the same size as the original sample were obtained, each of which was a simple random sample with replacement;

(2) Logistic regression was applied to each sample using selected risk factors;

(3) The 95% confidence intervals based on 100 independent samples for the estimated effects (of the risk factors) were calculated;

(4) If the effects at the 2.5th percentile and the 97.5th percentile were both positive (being risk factors) or negative (being protective factors), the effects were deemed to be significant; if the lower and the upper bound of the effects pointed to different directions (one being positive and the other being negative), the effect was deemed to be nonsignificant;

(5) Nonsignificant effect was removed from the models, and the stepwise model selection was run to see whether other new effects could enter the models with this effect absent. The above bootstrapping process was repeated to validate the new models.

(6) If several effects were found to be nonsignificant through bootstrapping, we removed the least significant effect in step 5.

References

  1. 1. Klevens RM, Edwards JR, Richards CL Jr, et al. Estimating health care–associated infections and deaths in U.S. hospitals. Public Health Reports 2002;122:160–167.
  2. 2. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for prevention of surgical site infection. Infect Control Hosp Epidemiol 1999;20:250–280.
  3. 3. Scott II RD. The direct medical costs of healthcare-associated infections in U.S. hospitals and the benefits of prevention. 2009. http://www.cdc.gov/ncidod/dhqp/pdf/Scott_CostPaper.pdf. Accessed August 12, 2011.
  4. 4. Haley RW, Culver DH, White JW, et al. The efficacy of infection surveillance and control programs in preventing nosocomial infections in United States hospitals. Am J Epidemiol 1985;121:182–205.
  5. 5. Gaynes R, Richards C, Edwards JR, et al. Feeding back surveillance data to prevent hospital-acquired infections. Emerg Infect Dis 2001;7:295–298.
  6. 6. Brandt C, Hansen S, Sohr D, Daschner F, Ruden H, Gastmeier P. Finding a method for optimizing risk adjustment when comparing surgical-site infecions rates. Infect Control Hosp Epidemiol 2004;25:13–18.
  7. 7. Anderson DJ, Kaye KS, Classen D, et al. Strategies to prevent surgical site infections in acute care hospitals. Infect Control Hosp Epidemiol 2008;(suppl 1):S51–S61.
  8. 8. Culver DH, Horan TC, Gaynes RP, et al. Surgical wound-infection rates by wound class, operative procedure, and patient risk index. Am J Med 1991;91:S152–S157.
  9. 9. American Society of Anesthesiologists (ASA). ASA physical status classification system. http://www.asahq.org/clinical/physicalstatus.htm. Accessed March 2, 2011.
  10. 10. US Department of Health and Human Services (HHS). HHS action plan to prevent healthcare associated infections: appendices. Appendix G. http://www.hhs.gov/ophs/initiatives/hai/appendices.html Accessed July 28, 2010.
  11. 11. Anderson DJ, Chen LF, Sexton DJ, Kaye KS. Complex surgical site infections and the devilish details of risk adjustment: important implications for public reporting. Infect Control Hosp Epidemiol 2008;29:941–946.
  12. 12. Shahian DM, O’Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB, et al. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1—coronary artery bypass grafting surgery. Ann Thorac Surg 2009;88:S2–S22.
  13. 13. Friedman ND, Bull AL, Russo PL, Gurrin L, Richards M. Performance of the National Nosocomial Infections Surveillance risk index in predicting surgical site infection in Australia. Infect Control Hosp Epidemiol 2007;28:55–59.
  14. 14. Paul M, Raz A, Leibovici L, Madar H, Holinger R, Rubinovitch B. Sternal wound infection after coronary artery bypass graft surgery: validation of existing risk scores. J Thorac Cardiovasc Surg 2007;133:397–403.
  15. 15. Roy MC, Herwaldt LA, Embrey R, Kuhns K, Wenzel RP, Eerl TM. Does the Centers for Disease Control’s NNIs system risk index stratify patients undergoing cardiothoracic operations by their risk of surgical-site infection. Infect Control Hosp Epidemiol 2000;21:186–190.
  16. 16. Edwards JR, Peterson KD, Mu Y, et al. National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009. Am J Infect Control 2009;37:783–805.
  17. 17. Association for Professionals in Infection Control. HAI reporting laws and regulation. http://www.apic.org/downloads/legislation/HAI_map.gif. Accessed August 12, 2011.
  18. 18. Centers for Disease Control and Prevention. National Healthcare Safety Network: surgical site infection (SSI) event. http://www.cdc.gov/nhsn/PDFs/pscManual/9pscSSIcurrent.pdf. Accessed June 26, 2010.
  19. 19. National Quality Forum. National voluntary consensus standards for the reporting of healthcare-associated infections data. National Quality Forum, 2008. http://www.qualityforum.org/Publications/2008/03/National_Voluntary_Consensus_Standards_for_the_Reporting_of_Healthcare-Associated_Infection_Data.aspx. Accessed August 9, 2010.
  20. 20. Hanley JA, McNeil BJ. The meaning and use of the area under a receive operating characteristic (ROC) curve. Radiology 1982;143:29–36.
  21. 21. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839–843.
  22. 22. Abboud CS, Wey SB, Baltar VT. Risk factors for mediastinitis after cardiac surgery. Ann Thorac Surg 2004;77:676–683.
  23. 23. de Boer AS, Mintjes-de Groot AJ, Severijnen AJ, van den Berg JM, van Pelt W. Risk assessment for surgical-site infections in orthopedic patients. Infect Control Hosp Epidemiol 1999;20:402–407.
  24. 24. Gaynes RP. Surgical-site infections (SSI) and the NNIS SSI risk index, part II: room for improvement. Infect Control Hosp Epidemiol 2001;22:268–272.
  25. 25. The Parisian Mediastinitis Study Group. Risk factors for deep sternal wound infections after sternotomy: a prospective, multicenter study. J Thorac Cardiovasc Surg 2006;111:1200–1207.
  26. 26. Harrignton G, Russo P, Spelman D, et al. Surgical-site infection rates and risk factor analysis in coronary artery bypass graft surgery. Infect Control Hosp Epidemiol 2004;25:472–476.
  27. 27. Killian CA, Graffunder EM, Vinciguerra TJ, Venezia RA. Risk factors for surgical-site infections following cesarean section. Infect Control Hosp Epidemiol 2001;22:613–617.
  28. 28. Kivi M, Mannien J, Wille JC, van den Hof S. Surgical site infection surveillance and the predictive power of the National Nosocomial Infection Surveillance index as compared with alternative determinants in the Netherlands. Am J Infect Control 2008;36:S27–S31.
  29. 29. Neumayer L, Hosokawa P, Itani K, El-Tamer M, Henderson WG, Khuri SF. Multivariable predictors of postoperative surgical site infection after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg 2007;204:1178–1187.
  30. 30. Olsen MA, Lock-Buckley P, Hopkins D, Polish LB, Sundt TM, Fraser VJ. The risk factors for deep and superficial chest surgical-site infections after coronary artery bypass graft surgery are different. J Thorac Cardiovasc Surg 2002;124:136–145.
  31. 31. Russo PL, Spelman DW. A new surgical-site infection risk index using risk factors identified by multivariate analysis for patients undergoing coronary artery bypass graft surgery. Infect Control Hosp Epidemiol 2002;23:372–376.
  32. 32. Tran TS, Jamulitrat S, Chongsurvivatwong V, Geater A. Risk factors for postcesarean surgical site infection. Obstet Gynecol 2000;95:367–371.
  33. 33. Geubbels ELPE, Grobbee DE, Vandenbroucke-Grauls CMJE, Wille JC, de Boer AS. Improved risk adjustment for comparison of surgical site infection rates. Infect Control Hosp Epidemiol 2006;27:1330–1339.

Acknowledgments

We thank the NHSN participants for their ongoing efforts to monitor healthcare-associated infections and improve patient safety and our colleagues in the Division of Healthcare Quality Promotion for their tireless support of this unique public health network.

Potential conflicts of interest. All authors report no conflicts of interest relevant to this article.

Appendix A

Outlier exclusion rules:

(1) Exclude all procedures where duration in minutes = 0 ();

(2) Exclude all procedures where patient was less than 1 day old or greater than 109 years old ();

(3) Exclude all procedures where wound class was undefined ();

(4) Exclude all procedures with duration between 0 and 5 minutes or more than 5 times the interquartile range ().

Appendix B

Bootstrap resampling steps:

(1) For each procedure category, 100 independent samples of the same size as the original sample were obtained, each of which was a simple random sample with replacement;

(2) Logistic regression was applied to each sample using selected risk factors;

(3) The 95% confidence intervals based on 100 independent samples for the estimated effects (of the risk factors) were calculated;

(4) If the effects at the 2.5th percentile and the 97.5th percentile were both positive (being risk factors) or negative (being protective factors), the effects were deemed to be significant; if the lower and the upper bound of the effects pointed to different directions (one being positive and the other being negative), the effect was deemed to be nonsignificant;

(5) Nonsignificant effect was removed from the models, and the stepwise model selection was run to see whether other new effects could enter the models with this effect absent. The above bootstrapping process was repeated to validate the new models.

(6) If several effects were found to be nonsignificant through bootstrapping, we removed the least significant effect in step 5.

References

  1. 1. Klevens RM, Edwards JR, Richards CL Jr, et al. Estimating health care–associated infections and deaths in U.S. hospitals. Public Health Reports 2002;122:160–167.
  2. 2. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for prevention of surgical site infection. Infect Control Hosp Epidemiol 1999;20:250–280.
  3. 3. Scott II RD. The direct medical costs of healthcare-associated infections in U.S. hospitals and the benefits of prevention. 2009. http://www.cdc.gov/ncidod/dhqp/pdf/Scott_CostPaper.pdf. Accessed August 12, 2011.
  4. 4. Haley RW, Culver DH, White JW, et al. The efficacy of infection surveillance and control programs in preventing nosocomial infections in United States hospitals. Am J Epidemiol 1985;121:182–205.
  5. 5. Gaynes R, Richards C, Edwards JR, et al. Feeding back surveillance data to prevent hospital-acquired infections. Emerg Infect Dis 2001;7:295–298.
  6. 6. Brandt C, Hansen S, Sohr D, Daschner F, Ruden H, Gastmeier P. Finding a method for optimizing risk adjustment when comparing surgical-site infecions rates. Infect Control Hosp Epidemiol 2004;25:13–18.
  7. 7. Anderson DJ, Kaye KS, Classen D, et al. Strategies to prevent surgical site infections in acute care hospitals. Infect Control Hosp Epidemiol 2008;(suppl 1):S51–S61.
  8. 8. Culver DH, Horan TC, Gaynes RP, et al. Surgical wound-infection rates by wound class, operative procedure, and patient risk index. Am J Med 1991;91:S152–S157.
  9. 9. American Society of Anesthesiologists (ASA). ASA physical status classification system. http://www.asahq.org/clinical/physicalstatus.htm. Accessed March 2, 2011.
  10. 10. US Department of Health and Human Services (HHS). HHS action plan to prevent healthcare associated infections: appendices. Appendix G. http://www.hhs.gov/ophs/initiatives/hai/appendices.html Accessed July 28, 2010.
  11. 11. Anderson DJ, Chen LF, Sexton DJ, Kaye KS. Complex surgical site infections and the devilish details of risk adjustment: important implications for public reporting. Infect Control Hosp Epidemiol 2008;29:941–946.
  12. 12. Shahian DM, O’Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB, et al. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1—coronary artery bypass grafting surgery. Ann Thorac Surg 2009;88:S2–S22.
  13. 13. Friedman ND, Bull AL, Russo PL, Gurrin L, Richards M. Performance of the National Nosocomial Infections Surveillance risk index in predicting surgical site infection in Australia. Infect Control Hosp Epidemiol 2007;28:55–59.
  14. 14. Paul M, Raz A, Leibovici L, Madar H, Holinger R, Rubinovitch B. Sternal wound infection after coronary artery bypass graft surgery: validation of existing risk scores. J Thorac Cardiovasc Surg 2007;133:397–403.
  15. 15. Roy MC, Herwaldt LA, Embrey R, Kuhns K, Wenzel RP, Eerl TM. Does the Centers for Disease Control’s NNIs system risk index stratify patients undergoing cardiothoracic operations by their risk of surgical-site infection. Infect Control Hosp Epidemiol 2000;21:186–190.
  16. 16. Edwards JR, Peterson KD, Mu Y, et al. National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009. Am J Infect Control 2009;37:783–805.
  17. 17. Association for Professionals in Infection Control. HAI reporting laws and regulation. http://www.apic.org/downloads/legislation/HAI_map.gif. Accessed August 12, 2011.
  18. 18. Centers for Disease Control and Prevention. National Healthcare Safety Network: surgical site infection (SSI) event. http://www.cdc.gov/nhsn/PDFs/pscManual/9pscSSIcurrent.pdf. Accessed June 26, 2010.
  19. 19. National Quality Forum. National voluntary consensus standards for the reporting of healthcare-associated infections data. National Quality Forum, 2008. http://www.qualityforum.org/Publications/2008/03/National_Voluntary_Consensus_Standards_for_the_Reporting_of_Healthcare-Associated_Infection_Data.aspx. Accessed August 9, 2010.
  20. 20. Hanley JA, McNeil BJ. The meaning and use of the area under a receive operating characteristic (ROC) curve. Radiology 1982;143:29–36.
  21. 21. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839–843.
  22. 22. Abboud CS, Wey SB, Baltar VT. Risk factors for mediastinitis after cardiac surgery. Ann Thorac Surg 2004;77:676–683.
  23. 23. de Boer AS, Mintjes-de Groot AJ, Severijnen AJ, van den Berg JM, van Pelt W. Risk assessment for surgical-site infections in orthopedic patients. Infect Control Hosp Epidemiol 1999;20:402–407.
  24. 24. Gaynes RP. Surgical-site infections (SSI) and the NNIS SSI risk index, part II: room for improvement. Infect Control Hosp Epidemiol 2001;22:268–272.
  25. 25. The Parisian Mediastinitis Study Group. Risk factors for deep sternal wound infections after sternotomy: a prospective, multicenter study. J Thorac Cardiovasc Surg 2006;111:1200–1207.
  26. 26. Harrignton G, Russo P, Spelman D, et al. Surgical-site infection rates and risk factor analysis in coronary artery bypass graft surgery. Infect Control Hosp Epidemiol 2004;25:472–476.
  27. 27. Killian CA, Graffunder EM, Vinciguerra TJ, Venezia RA. Risk factors for surgical-site infections following cesarean section. Infect Control Hosp Epidemiol 2001;22:613–617.
  28. 28. Kivi M, Mannien J, Wille JC, van den Hof S. Surgical site infection surveillance and the predictive power of the National Nosocomial Infection Surveillance index as compared with alternative determinants in the Netherlands. Am J Infect Control 2008;36:S27–S31.
  29. 29. Neumayer L, Hosokawa P, Itani K, El-Tamer M, Henderson WG, Khuri SF. Multivariable predictors of postoperative surgical site infection after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg 2007;204:1178–1187.
  30. 30. Olsen MA, Lock-Buckley P, Hopkins D, Polish LB, Sundt TM, Fraser VJ. The risk factors for deep and superficial chest surgical-site infections after coronary artery bypass graft surgery are different. J Thorac Cardiovasc Surg 2002;124:136–145.
  31. 31. Russo PL, Spelman DW. A new surgical-site infection risk index using risk factors identified by multivariate analysis for patients undergoing coronary artery bypass graft surgery. Infect Control Hosp Epidemiol 2002;23:372–376.
  32. 32. Tran TS, Jamulitrat S, Chongsurvivatwong V, Geater A. Risk factors for postcesarean surgical site infection. Obstet Gynecol 2000;95:367–371.
  33. 33. Geubbels ELPE, Grobbee DE, Vandenbroucke-Grauls CMJE, Wille JC, de Boer AS. Improved risk adjustment for comparison of surgical site infection rates. Infect Control Hosp Epidemiol 2006;27:1330–1339.