PRINOI avril 2012 Page 1

AMELIORATION DES PRATIQUES

IMPACT DES INDICATEURS DU TABLEAU DE BORD DES INFECTIONS NOSOCOMIALES

Journeacutee Preacutevention du Risque Infectieux Nosocomial de lrsquoOceacutean Indien

Vendredi 27 avril 2012

Didier LEPELLETIER

PRINOI avril 2012 Page 2

Les indicateurs

Existence drsquoindicateurs de moyens drsquoorganisation et de reacutesultats

Quels objectifs pour un indicateur laquo Benchmarking raquo

Professionnels ameacutelioration des pratiques Deacutecideurs paiement agrave la performance

Diffusion publique Droit des usagers Outil de seacutelection des ES () Equilibre entre pression politique validiteacute scientifique et outil de

communication (compreacutehensibiliteacute)

PRINOI avril 2012 Page 3

PRINOI avril 2012 Page 4

Critegraveres drsquoeacutevaluation drsquoun indicateur (1)

Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration

Faisabiliteacute

Charge de travail suppleacutementaire acceptable pour les professionnels des ES

Accessibiliteacute des donneacutees (par exemple dans le SIH)

PRINOI avril 2012 Page 5

Critegraveres drsquoeacutevaluation drsquoun indicateur (2)

Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute

reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace

Utiliteacute pour les professionnels

en vue drsquoune ameacutelioration des pratiques ou de benchmarking

Utiliteacute pour les usagers

transparence arguments pour un choix raisonneacute des ES par les usagers

PRINOI avril 2012 Page 6

Recommandations HICPAC 2005 (USA)

Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)

Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement

ISO agrave la suite de certains types drsquointervention chirurgicale

Non recommandeacutes

Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)

Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)

Guidance on public reporting of healthcare-associated infections

recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ

Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26

PRINOI avril 2012 Page 2

Les indicateurs

Existence drsquoindicateurs de moyens drsquoorganisation et de reacutesultats

Quels objectifs pour un indicateur laquo Benchmarking raquo

Professionnels ameacutelioration des pratiques Deacutecideurs paiement agrave la performance

Diffusion publique Droit des usagers Outil de seacutelection des ES () Equilibre entre pression politique validiteacute scientifique et outil de

communication (compreacutehensibiliteacute)

PRINOI avril 2012 Page 3

PRINOI avril 2012 Page 4

Critegraveres drsquoeacutevaluation drsquoun indicateur (1)

Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration

Faisabiliteacute

Charge de travail suppleacutementaire acceptable pour les professionnels des ES

Accessibiliteacute des donneacutees (par exemple dans le SIH)

PRINOI avril 2012 Page 5

Critegraveres drsquoeacutevaluation drsquoun indicateur (2)

Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute

reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace

Utiliteacute pour les professionnels

en vue drsquoune ameacutelioration des pratiques ou de benchmarking

Utiliteacute pour les usagers

transparence arguments pour un choix raisonneacute des ES par les usagers

PRINOI avril 2012 Page 6

Recommandations HICPAC 2005 (USA)

Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)

Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement

ISO agrave la suite de certains types drsquointervention chirurgicale

Non recommandeacutes

Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)

Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)

Guidance on public reporting of healthcare-associated infections

recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ

Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26

PRINOI avril 2012 Page 3

PRINOI avril 2012 Page 4

Critegraveres drsquoeacutevaluation drsquoun indicateur (1)

Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration

Faisabiliteacute

Charge de travail suppleacutementaire acceptable pour les professionnels des ES

Accessibiliteacute des donneacutees (par exemple dans le SIH)

PRINOI avril 2012 Page 5

Critegraveres drsquoeacutevaluation drsquoun indicateur (2)

Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute

reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace

Utiliteacute pour les professionnels

en vue drsquoune ameacutelioration des pratiques ou de benchmarking

Utiliteacute pour les usagers

transparence arguments pour un choix raisonneacute des ES par les usagers

PRINOI avril 2012 Page 6

Recommandations HICPAC 2005 (USA)

Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)

Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement

ISO agrave la suite de certains types drsquointervention chirurgicale

Non recommandeacutes

Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)

Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)

Guidance on public reporting of healthcare-associated infections

recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ

Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26

PRINOI avril 2012 Page 4

Critegraveres drsquoeacutevaluation drsquoun indicateur (1)

Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration

Faisabiliteacute

Charge de travail suppleacutementaire acceptable pour les professionnels des ES

Accessibiliteacute des donneacutees (par exemple dans le SIH)

PRINOI avril 2012 Page 5

Critegraveres drsquoeacutevaluation drsquoun indicateur (2)

Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute

reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace

Utiliteacute pour les professionnels

en vue drsquoune ameacutelioration des pratiques ou de benchmarking

Utiliteacute pour les usagers

transparence arguments pour un choix raisonneacute des ES par les usagers

PRINOI avril 2012 Page 6

Recommandations HICPAC 2005 (USA)

Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)

Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement

ISO agrave la suite de certains types drsquointervention chirurgicale

Non recommandeacutes

Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)

Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)

Guidance on public reporting of healthcare-associated infections

recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ

Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26

PRINOI avril 2012 Page 5

Critegraveres drsquoeacutevaluation drsquoun indicateur (2)

Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute

reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace

Utiliteacute pour les professionnels

en vue drsquoune ameacutelioration des pratiques ou de benchmarking

Utiliteacute pour les usagers

transparence arguments pour un choix raisonneacute des ES par les usagers

PRINOI avril 2012 Page 6

Recommandations HICPAC 2005 (USA)

Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)

Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement

ISO agrave la suite de certains types drsquointervention chirurgicale

Non recommandeacutes

Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)

Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)

Guidance on public reporting of healthcare-associated infections

recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ

Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26

PRINOI avril 2012 Page 6

Recommandations HICPAC 2005 (USA)

Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)

Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement

ISO agrave la suite de certains types drsquointervention chirurgicale

Non recommandeacutes

Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)

Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)

Guidance on public reporting of healthcare-associated infections

recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ

Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26

PRINOI avril 2012 Page 7

Technical Advisory Committee Maryland Health-Care Commission

laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)

ISO apregraves pontage coronarien (2230)

ISO apregraves prothegravese totale de hanche ou de genou (2130)

Autres laquo Outcomes raquo MRSAVRE en reacuteanimation

pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie

Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74

PRINOI avril 2012 Page 8

En Europe

Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies

associeacutees aux CVC Taux drsquoincidence des pneumopathies

associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs

invasifs Volontariat (reacuteseau KISS)

Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave

Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies

associeacutees aux CVC Taux drsquoincidence des infections

urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies

associeacutees agrave la ventilation meacutecanique

Rapporteacutes publiquement Taux drsquoincidence des

bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)

Taux drsquoincidence des bacteacuterieacutemies agrave SARM

Taux drsquoincidence des infections agrave Clostridium difficile

Nb MRSA et ICD dans les 12 derniegraveres semaines

Nb semaines sans MRSA

Variable Taux drsquoincidence des ISO Taux de preacutevalence de la

reacutesistance microbienne

ALLEMAGNE ANGLETERRE

Haustein et al Lancet ID 2011

PRINOI avril 2012 Page 9

Calcul et expression de lrsquoindicateur

QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou

densiteacute) Taux de preacutelegravevements

cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS

Nb attendu IAS

QUALITATIF Preacutesence drsquoau moins 1 type

drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)

Intervalle (nb semaines) sans IAS

Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)

RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle

deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)

PRINOI avril 2012 Page 10

Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts

(France 2003)

Pertinence Faisabiliteacute Qualiteacute

meacutetrologique

Utilisation

agrave lrsquoexteacuterieur de

lrsquoES

Passaretti et coll

(Maryland USA ICHE 2011)

Impact

Improvability

Feasibility

Frequency

Feasibility

(lsquoSystem for collecting

data already in placersquo)

Inclusiveness

Functionality

Mc Kibben et coll

(HICPAC USA AJIC 2006)

Severity

Preventability

Detectability

Accuracy

HELICS-IPSE

(Europe)

Clinical relevance Simplicity

Acceptability

Timeliness

Completeness

Validity

Sensitivity

Readibility

Pas de distingo par les usagers vs par les professionnels

PRINOI avril 2012 Page 11

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 12

DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central

59

98

53

32

09 09 12 14

04 02

13 11

0

5

10

15

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

NNIS Cardiothoracic ICU

REA-RAISIN

Reacuteanimation CTCV (HGRL CHU Nantes)

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition

3 Audit drsquoobservation

4 V2 mode opeacuteratoire pose CVC

et feuille de suivi hors USI

2 Surveillance prospective et publication de lrsquoindicateur

5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes

PRINOI avril 2012 Page 13

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 9

Calcul et expression de lrsquoindicateur

QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou

densiteacute) Taux de preacutelegravevements

cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS

Nb attendu IAS

QUALITATIF Preacutesence drsquoau moins 1 type

drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)

Intervalle (nb semaines) sans IAS

Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)

RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle

deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)

PRINOI avril 2012 Page 10

Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts

(France 2003)

Pertinence Faisabiliteacute Qualiteacute

meacutetrologique

Utilisation

agrave lrsquoexteacuterieur de

lrsquoES

Passaretti et coll

(Maryland USA ICHE 2011)

Impact

Improvability

Feasibility

Frequency

Feasibility

(lsquoSystem for collecting

data already in placersquo)

Inclusiveness

Functionality

Mc Kibben et coll

(HICPAC USA AJIC 2006)

Severity

Preventability

Detectability

Accuracy

HELICS-IPSE

(Europe)

Clinical relevance Simplicity

Acceptability

Timeliness

Completeness

Validity

Sensitivity

Readibility

Pas de distingo par les usagers vs par les professionnels

PRINOI avril 2012 Page 11

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 12

DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central

59

98

53

32

09 09 12 14

04 02

13 11

0

5

10

15

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

NNIS Cardiothoracic ICU

REA-RAISIN

Reacuteanimation CTCV (HGRL CHU Nantes)

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition

3 Audit drsquoobservation

4 V2 mode opeacuteratoire pose CVC

et feuille de suivi hors USI

2 Surveillance prospective et publication de lrsquoindicateur

5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes

PRINOI avril 2012 Page 13

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 10

Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts

(France 2003)

Pertinence Faisabiliteacute Qualiteacute

meacutetrologique

Utilisation

agrave lrsquoexteacuterieur de

lrsquoES

Passaretti et coll

(Maryland USA ICHE 2011)

Impact

Improvability

Feasibility

Frequency

Feasibility

(lsquoSystem for collecting

data already in placersquo)

Inclusiveness

Functionality

Mc Kibben et coll

(HICPAC USA AJIC 2006)

Severity

Preventability

Detectability

Accuracy

HELICS-IPSE

(Europe)

Clinical relevance Simplicity

Acceptability

Timeliness

Completeness

Validity

Sensitivity

Readibility

Pas de distingo par les usagers vs par les professionnels

PRINOI avril 2012 Page 11

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 12

DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central

59

98

53

32

09 09 12 14

04 02

13 11

0

5

10

15

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

NNIS Cardiothoracic ICU

REA-RAISIN

Reacuteanimation CTCV (HGRL CHU Nantes)

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition

3 Audit drsquoobservation

4 V2 mode opeacuteratoire pose CVC

et feuille de suivi hors USI

2 Surveillance prospective et publication de lrsquoindicateur

5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes

PRINOI avril 2012 Page 13

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 11

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 12

DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central

59

98

53

32

09 09 12 14

04 02

13 11

0

5

10

15

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

NNIS Cardiothoracic ICU

REA-RAISIN

Reacuteanimation CTCV (HGRL CHU Nantes)

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition

3 Audit drsquoobservation

4 V2 mode opeacuteratoire pose CVC

et feuille de suivi hors USI

2 Surveillance prospective et publication de lrsquoindicateur

5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes

PRINOI avril 2012 Page 13

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 12

DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central

59

98

53

32

09 09 12 14

04 02

13 11

0

5

10

15

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

NNIS Cardiothoracic ICU

REA-RAISIN

Reacuteanimation CTCV (HGRL CHU Nantes)

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition

3 Audit drsquoobservation

4 V2 mode opeacuteratoire pose CVC

et feuille de suivi hors USI

2 Surveillance prospective et publication de lrsquoindicateur

5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes

PRINOI avril 2012 Page 13

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 13

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 14

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 15

Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees

aux actes invasifs

Source Circulaire du 26 aoucirct 2009

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 16

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 17

DI infections urinaires 1000 jours de sondage urinaire

National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009

Proceacutedure sondage urinaire Mateacuteriel de

diuregravese horaire preacute-connecteacute

Formation des internes agrave chaque semestre au BO

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 18

Evolution des indicateurs du tableau de bord

Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires

Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques

Nouvelle organisation des ES et ARS (loi HPST)

Deacutefinition drsquoobjectifs quantifieacutes pour 2012

Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 19

Tableau de bord des IN

Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN

Mise en place en 2006

ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute

Indicateurs de 2e geacuteneacuteration

Nouvelle eacutetape drsquoameacutelioration des pratiques

Mise en place en 2012

ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute

Am

eacutelio

ratio

n de

s ES

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 20

Nouveau rocircle des instances dans la politique de lutte contre les IN

Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins

ICALIN 2

ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 21

Nouveauteacutes ICALIN 2 Protection du personnel

proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)

Suivi de la consommation des PHA et restitution au moins 1 fois par an

Check-list SU CVP CCI CVC CSC

Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique

Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau

Reacutesultats des surveillances preacutesenteacutes en CME

Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave

Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS

PCC PCAG sonde urinaire CVPCVCCCICSC

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 22

Indicateur ICABMR

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 23

Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 24

Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 25

Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline

au sein de lrsquoespegravece SA

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 26

Source httpwwwrivmnlearssdatabase

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 27

Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM

ne le sont pas sur les EBLSE

Que nous disent les indicateurs de surveillance

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 28

DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)

02018018

012007003

016

019024

026033

036039

05048043

0

01

02

03

04

05

06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

DI EBLSE 1000 JH DI SARM 1000 JH

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 29

Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)

Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 30

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 31

au moins 100 actes

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 32

Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)

Surveillance continue des infections du site opeacuteratoire (ISO)

Innovation introduction des DM impreacutegneacutes

2001

Lepelletier et al Infect Control Hosp Epidemiol 2005

Lepelletier et al Antibiotiques 2005

Lepelletier et al Arch Cardiovasc Dis 2009

Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire

Audit drsquoobservation des pratiques

de preacuteparation

cutaneacutee

+

Enquecircte reacutetrospective

Taux drsquoISO

Mise en œuvre drsquoune surveillance prospective continue des

ISO

2002

Reacutedaction drsquoun mode

opeacuteratoire sur la preacuteparation

cutaneacutee du site opeacuteratoire

Application du mode

opeacuteratoire par lrsquoensemble de

lrsquoeacutequipe chirurgicale

Introduction de fils de suture impreacutegneacutes

drsquoantiseptique

2e eacutevaluation des pratiques

de la preacuteparation

cutaneacutee (audit POP)

2003 2005 2007

Introduction

drsquoune eacuteponge impreacutegneacutee

drsquoantibiotique

2009 2008

Arrecirct utilisation

eacuteponge impreacutegneacutee

2011

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 33

copy Photos DL GH Bichat-CB AP-HP 2009

Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren

1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France

Abst ract

Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance

Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale

Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition

Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties

Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131

Editor Daniel Morgan University of Maryland United States of America

Received November 10 2011 Accept ed March 13 2012 Published April 17 2012

Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited

Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Compet ing Interests The authors have declared that no competing interests exist

E-mail jean-christophelucetbchaphpfr

curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France

Int roduct ion

Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]

Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be

obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]

We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates

PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 34

and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]

Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the

single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]

Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )

Number of vignettesscored

Scoring without the SSIdefinit ion (95CI)

Number of vignet tesscored

Scoring with the SSIdefinit ion (95CI)

Intraspecialty correlat ion

Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)

Surgeon 32 045 (000ndash081) 28 042 (009ndash080)

Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)

Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)

Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)

Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)

Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)

Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002

Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals

SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)

Scoring without the SSI definit ion (95CI)

Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)

Intraspecialty

Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)

Surgeon 32 688 (413ndash890) 038 (2 005ndash080)

Public health specialist 40 750 (509ndash913) 052 (020ndash084)

Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)

Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)

Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)

Microbiologist 40 800 (563ndash943) 060 (026ndash094)

Interspecialty 238 - 028 (021ndash036)

Scoring with the SSI definit ion (95CI)

Intraspecialty

Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)

Surgeon 28 714 (419ndash916) 028 (2 005ndash063)

Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)

Infection control physician 40 550 (315ndash769) 2 003 (2 045041)

Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)

Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)

Microbiologist 40 700 (457ndash881) 041 (002ndash080)

Interspecialty 238 - 041 (034ndash048)

Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003

Agreement in Diagnosing SSI

PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 35

Discussion (1)

Les indicateurs

Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN

Importance des critegraveres drsquoeacutevaluation

Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs

Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 36

Discussion (2)

Les indicateurs

Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les

indicateurs de reacutesultats Changement des classes de performance

Importance de deacutevelopper des indicateurs au sein des ES

par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 37

Quel regard des usagers sur les indicateurs (en routine en cas de crise)

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients

PRINOI avril 2012 Page 38

Choisir des indicateurs utiles aux patients