Statistical Methods for Healthcare Performance Monitoring

Description

Healthcare is important to everyone, yet large variations in its quality have been well documented both between and within many countries. With demand and expenditure rising, it’s more crucial than ever to know how well the healthcare system and all its components – from staff member to regional network – are performing. This requires data, which inevitably differ in form and quality. It also requires statistical methods, the output of which needs to be presented so that it can be understood by whoever needs it to make decisions. Statistical Methods for Healthcare Performance Monitoring covers measuring quality, types of data, risk adjustment, defining good and bad performance, statistical monitoring, presenting the results to different audiences and evaluating the monitoring system itself. Using examples from around the world, it brings all the issues and perspectives together in a largely non-technical way for clinicians, managers and methodologists. Statistical Methods for Healthcare Performance Monitoring is aimed at statisticians and researchers who need to know how to measure and compare performance, health service regulators, health service managers with responsibilities for monitoring performance, and quality improvement scientists, including those involved in clinical audits. Cover Half Title Title Page Copyright page Table of Contents List of Illustrations List of Tables Preface Authors 1: Introduction 1.1 The Need for Performance Monitoring 1.2 Measuring and Monitoring Quality 1.3 The Need for This Book 1.4 Who Is This Book For and How Should It Be Used? Common Abbreviations Used in the Book Acknowledgment 2: Origins and Examples of Monitoring Systems Aims of This Chapter 2.1 Origins 2.2 Healthcare Scandals 2.2.1 Responses to the Scandals 2.3 Examples of Monitoring Schemes 2.4 Goals of Monitoring 2.4.1 Accountability 2.4.2 Regulation and Accreditation 2.4.3 Patient Choice 2.4.4 Openness and Transparency 2.4.5 Quality Improvement 2.4.6 Prevent Harm and Unsafe Care 2.4.7 Professionalism 2.4.8 Informed Consent 3: Choosing the Unit of Analysis and Reporting Aims of This Chapter 3.1 Issues Principally Concerning the Analysis 3.1.1 Clustering (*) 3.1.2 Episode Treatment Groups 3.2 Issues More Relevant to Reporting: Attributing Performance to a Given Unit in a System 4: What to Measure: Choosing and Defining Indicators Aims of This Chapter 4.1 How Can We Define Quality? 4.2 Common Indicator Taxonomies 4.3 Particular Challenges of Measuring Patient Safety 4.4 Particular Challenges of Multimorbidity 4.5 Measuring the Health of the Population and Quality of the Whole Healthcare System 4.5.1 The WHO Annual World Health Statistics Report 4.6 Efficiency and Value 4.6.1 Data Envelopment Analysis and Stochastic Frontier Analysis (*) 4.7 Features of an Ideal Indicator 4.8 Steps in Construction and Common Issues in Definition 4.9 Validation of Indicators 4.10 Some Strategies for Choosing among Candidates 4.11 Time to Go: When to Withdraw Indicators 4.12 Conclusion 5: Sources of Data Aims of This Chapter 5.1 How to Assess Data Quality 5.2 Administrative Data 5.2.1 Coding Systems for Administrative Data 5.2.2 Use of Administrative Databases to Flag Patient Safety Events 5.3 Clinical Registry Data 5.4 Accuracy of Administrative and Clinical Databases Compared 5.5 Incident Reports and Other Ways to Capture Safety Events 5.6 Surveys 5.7 Other Sources 5.8 Other Issues Concerning Data Sources 5.9 Conclusion 6: Risk-Adjustment Principles and Methods Aims of This Chapter 6.1 Risk Adjustment and Risk Prediction 6.2 When and Why Should We Adjust for Risk? 6.3 Alternatives to Risk Adjustment 6.4 What Factors Should We Adjust For? 6.4.1 Factors Not under the Control of the Provider 6.4.2 Proxies Such as Age and Socioeconomic Status 6.4.3 Comorbidity 6.4.4 Disease Severity 6.5 Selecting an Initial Set of Candidate Variables 6.6 Dealing with Missing and Extreme Values 6.7 Timing of the Risk Factor Measurement 6.8 Building the Model 6.8.1 Choosing the Final Set of Variables from the Initial Set of Candidates 6.8.2 Decide How Each Variable Should Be Entered into the Model 6.8.3 Decide on the Statistical Method for Modelling (*) 6.8.4 Assess the Fit of the Model (*) 6.8.4.1 Adjusted R2 6.8.4.2 Area under the Receiver Operating Characteristic Curve: c Statistic 6.8.4.3 The Hosmer–Lemeshow Statistic for Calibration 6.8.5 Which Is More Important, Discrimination or Calibration? 6.8.6 What Can Be Done If the Model Fit or Performance Is Unacceptable? 6.8.7 Convert Regression Coefficients into a Risk Score If Desired 7: Output the Observed and Model-Predicted Outcomes (*) Aims of This Chapter 7.1 Ratios versus Differences 7.2 Deriving SMRs from Standardisation and Logistic Regression 7.3 Other Fixed Effects Approaches to Generate an SMR 7.4 Random Effects–Based SMRs (*) 7.5 Marginal versus Multilevel Models (*) 7.6 Which Is the “Best” Modelling Approach Overall? (*) 7.7 Further Reading on Producing Risk-Adjusted Outcomes by Unit 8: Composite Measures Aims of This Chapter 8.1 Some Examples 8.2 Steps in the Construction 8.2.1 Specify the Scope and Purpose 8.2.2 Choose the Unit 8.2.3 Select the Data and Deal with Missing Values 8.2.4 Choose the Indicators and Run Descriptive Analyses 8.2.5 Normalise the Metrics 8.2.6 Assign Weights and Aggregate the Component Indicators 8.2.7 Run Sensitivity Analyses 8.2.8 Present the Results 8.3 Some Real Examples 8.3.1 AHRQ’s Patient Safety Indicator Composite 8.3.2 Leapfrog Group Patient Safety Composite 8.4 Pros and Cons of Composites 8.5 Alternatives to the Use of Composites 9: Setting Performance Thresholds and Defining Outliers Aims of This Chapter 9.1 Defining Acceptable Performance 9.1.1 Targets 9.1.2 Historical Benchmarks 9.1.3 Referring to Inter-Unit Variation 9.2 Bayesian Methods for Comparing Providers 9.3 Statistical Process Control and Funnel Plots 9.4 Multiple Testing (*) 9.4.1 Multivariate Statistical Process Control Methods (*) 9.4.2 Further Reading on SPC 9.5 Ways of Assessing Variation between Units 9.6 How Much Variation Is “Acceptable”? 9.7 Impact on Outlier Status of Using Fixed versus Random Effects to Derive SMRs 9.8 How Reliably Can We Detect Poor Performance? 9.9 Some Resources for Quality Improvement Methods 10: Making Comparisons across National Borders Aims of This Chapter 10.1 Examples of Multinational Patient-Level Databases 10.2 Challenges 10.2.1 Worked Example of Combining Administrative Databases from Multiple Countries: Stroke Mortality 10.2.2 Clustering within Countries 10.2.3 Countries with Unusual Data or Apparent Performance 10.3 Interpreting Apparent Differences in Performance between Countries 10.4 Conclusion 11: Presenting the Results to Stakeholders Aims of This Chapter 11.1 The Main Ways of Presenting Comparative Performance Data 11.2 Effect on Behaviour of the Choice of Format When Providing Performance Data 11.3 The Importance of the Method of Presentation 11.3.1 Presenting Performance Data to Managers and Clinicians 11.3.2 Presenting Results to the Public 11.4 Examples of Giving Performance Information to Units 11.5 Examples of Giving Performance Information to the Public 11.6 Metadata 12: Evaluating the Monitoring System Aims of This Chapter 12.1 Study Design and Statistical Approaches to Evaluating a Monitoring System 12.1.1 Interrupted Time-Series Design and Analysis (*) 12.1.2 Adjusting for Confounding (*) 12.1.3 Difference-in-Differences 12.1.4 Instrumental Variable Analysis 12.1.5 Regression Discontinuity Designs 12.2 Economic Evaluation Methods 13: Concluding Thoughts 13.1 Simple versus Complex 13.2 Specific versus General 13.3 The Future Appendix A: Glossary of Main Statistical Terms Used References Index