Principles for Evaluation of One Health Surveillance: The EVA Book

Description

This book outlines essential elements of the evaluation of health surveillance within the One Health concept. It provides an introduction to basic theoretical notions of evaluation and vividly discusses related challenges. Expert authors cover the entire spectrum of available, innovative methods, from those for system process evaluations to methods for the economic evaluation of the surveillance strategies. Each chapter provides a detailed description of the methodology required and the tools available as illustrated by practical examples of animal health or One Health surveillance evaluations in both developed and developing countries. Targeting not only scientists, including epidemiologists, but also technical advisers of decision-makers, the present work is suitable for the evaluation of any type of health surveillance system - animal, human or combined - regardless of the socio-economic context. The volume is richly equipped with practical tools and examples, which enables the reader to apply the methods described. Increasing importance of health surveillance, and threats from disease outbreaks such as the coronavirus pandemic, underline the practical relevance of this work, which will fill an important gap in the literature. Foreword EVA Book Sections Summary and Highlights Needs for Evaluation of Health Surveillance Evaluation Frameworks and Tools Economic Evaluation Qualitative Evaluation Methods Quantitative Evaluation Methods Evaluation in the Policy Cycle About the Book Target Users Overview of the Book Content Content Not Covered Contents About the Editors About the Lead Authors Part I: General Introduction on Health Surveillance Evaluation 1: Why Do We Need to Evaluate Health Surveillance Systems? 1.1 Introduction 1.2 Part I: Surveillance 1.2.1 What Is Animal Health Surveillance? 1.2.1.1 Surveillance Networks 1.2.1.2 Surveillance Systems and Components 1.2.2 Surveillance Is Different from Epidemiological Surveys 1.2.3 Why Do We Need Animal Health Surveillance? 1.2.4 Challenges in Animal Health Surveillance 1.2.4.1 Influence of the Animal Health Management Cycle and the Socioeconomic Contexts 1.2.4.2 Which Priorities for Surveillance, Which Point of View? 1.2.4.3 A Complex System of Information Sharing 1.2.4.4 Seeing the Unseen: Dealing with Underreporting Issues 1.3 Part II: Evaluation 1.3.1 What Is Evaluation? 1.3.2 Historical Overview of Animal Health Surveillance Evaluation 1.3.3 What Is Evaluation of Animal Health Surveillance? 1.3.4 What Is Economic Evaluation? 1.3.5 Why Do we Need to Evaluate AHS? 1.3.6 Brief Overview of the Evaluation Process 1.3.6.1 Limits of the Existing Evaluation Frameworks 1.3.6.2 The Need for an Effectiveness Measure and the Benefit Issue 1.3.6.3 The Decision-Makers’ Point of View (See Part VI) 1.4 Conclusion on the Need for Animal Health Surveillance Evaluation References 2: An Overview of the Different Types and Level of Evaluation: From Theory to Application in Health Surveillance 2.1 Introduction 2.1.1 Evaluation Ideology and Classification Attempts 2.1.2 Methodological Specificities 2.1.3 Decision-Making in Evaluation Models 2.1.4 Revisiting the Main Evaluation Models to Animal Health Evaluation 2.2 The Different Types of Evaluation 2.2.1 The Different Purposes of Animal Health Surveillance Evaluation 2.2.2 When to Evaluate 2.3 Best Evaluation Practices 2.3.1 Evaluation of Best Practices Initiatives 2.4 Conclusion References Part II: What to Evaluate and How 3: Frameworks and Tools for Evaluating Health Surveillance Systems 3.1 Introduction 3.2 Evaluation Vs. Assessment 3.3 Overview of Current Animal and Human Health Surveillance Evaluation Guides: Advantages and Limits 3.4 The Evaluation Process 3.4.1 Planning Phase 3.4.1.1 Engaging Stakeholders 3.4.1.2 Needs and Required Resources for the Evaluation 3.4.2 Framing the Evaluation 3.4.2.1 Define the Purpose of the Evaluation 3.4.2.2 How to Define the Evaluation Question(S)? 3.4.3 Which Evaluation Attributes Should Address the Question(s)? 3.4.4 Implementation Phase 3.4.5 Reporting Phase 3.5 Conclusion References 4: The EVA Survtool: An Integrated Framework to Plan Health Surveillance Evaluation 4.1 Overview of the EVA Survtool 4.1.1 Step 1: Describing the Surveillance System 4.1.2 Step 2: Describing the Surveillance Evaluation Context 4.1.3 Step 3: Selecting the Evaluation Question 4.1.4 Step 4: Selecting the Relevant Evaluation Attributes 4.1.5 Step 5: Selecting the Evaluation Attribute Assessment Methods Including Economic Analysis 4.1.6 Steps 6 and 7: Implementing the Evaluation and Reporting on the Evaluation Outputs 4.2 Practical Application of the EVA Survtool 4.2.1 Case Study 1. Evaluation of the Swine Disease Surveillance System in Vietnam 4.2.1.1 Evaluation Plan—Outputs of EVA Tool Steps 1–5 Selected Evaluation Question(S) Selected Evaluation Attributes Assessment Methods 4.2.1.2 Implementation of the Evaluation: Step 6 Descriptive Analysis (Qualitative Assessment) Acceptability of Surveillance and Control Policy by Pig Farmers Timelines of Swine Disease Reporting Sensitivity of Swine Disease Surveillance System 4.2.1.3 Cost Analysis 4.2.1.4 Cost-Effectiveness Analysis 4.2.1.5 Cost–Benefit Analysis 4.2.1.6 Recommendations: Step 6 (Addressing the Evaluation Questions) and Step 7 (Reporting on the Evaluation Outputs) Quality of the Evaluation Performed Recommendations to Improve Surveillance Strategies 4.2.2 Case Study 2: Evaluation of Classical Swine Fever Surveillance System in Germany 4.2.2.1 Evaluation Plan—Outputs of EVA Tool Steps 1–5 Selected Evaluation Question(S) Selected Evaluation Attributes Assessment Methods 4.2.2.2 Implementation of the Evaluation: Step 6 Descriptive Analysis (Qualitative Assessment) SWOT Analysis Report Analysis of Changes in the System Process Incurred by the Change in Surveillance Design 4.2.2.3 Cost Analysis 4.2.2.4 Effectiveness Assessment 4.2.2.5 Cost-Effectiveness Assessment 4.2.2.6 Recommendations: Step 6 (Addressing the Evaluation Questions) and Step 7 (Reporting on the Evaluation Outputs) Quality of the Evaluation Performed Recommendations for Surveillance Design Changes 4.3 Conclusion References Part III: Methods for Economic Evaluation 5: The Economics of Surveillance 5.1 Introduction 5.2 Surveillance Information Is an Economic Product 5.3 Surveillance Is a Production Process 5.4 Choice of Surveillance Approach 5.5 The Value of Surveillance 5.5.1 Cost-Effectiveness Criteria 5.5.2 Willingness-to-Pay Approaches 5.6 Surveillance and Intervention Are Interdependent 5.7 Surveillance, Intervention, and the Benefits 5.8 Relationship Between Surveillance, Intervention, and Disease Mitigation 5.8.1 Iso-Mitigation Maps and Technical Relationships Between Surveillance and Intervention 5.8.2 Budget Constraints and Monetary Costs of Loss Avoidance 5.8.2.1 The Overall Economic Optimum for Combining Surveillance and Intervention 5.9 Further Considerations in Economic Evaluation 5.10 New Directions for Surveillance Evaluation References 6: Applications of Principles to Case Studies Focusing on Non-Monetary Surveillance Values 6.1 Introduction 6.1.1 Overview 6.1.2 Examples of Economic Efficiency Studies 6.1.3 The Challenge of Capturing Non-Market Values 6.2 Case Study 1: Evaluation of Costs and Benefits of Passive Surveillance Through Qualitative Surveys: Results of Surveys Conducted in Vietnam and Thailand 6.2.1 Introduction 6.2.2 Materials and Methods 6.2.2.1 Participatory Approaches and Snowball Sampling Method 6.2.2.2 Analysis of Qualitative Data 6.2.3 Results 6.2.3.1 Choices Operated by the Different Stakeholders Facing HPAI Suspicion 6.2.3.2 Perceived Cost and Benefits of Sharing Health Information 6.2.4 Discussion 6.2.5 Implication in Terms of Disease Management Policy 6.3 Case Study 2: The Economic Value of One Health Surveillance—An Assessment of the One Health Approach to Campylobacter Surveillance in Switzerland 6.3.1 Introduction 6.3.2 Methods 6.3.3 Results 6.3.4 Discussion 6.3.5 Implication in Terms of Disease Management Policy 6.4 Case Study 3: Quantifying the Perceived Benefits and Costs of Surveillance Through Discrete Choice Experiments: A Pilot Study in North Vietnam 6.4.1 Introduction 6.4.2 Materials and Methods 6.4.2.1 Contingent Valuation to Estimate the Benefits of Health Information Sharing 6.4.2.2 Discrete Choice Experiment to Value the Non-Monetary Costs and Benefits 6.4.3 Results 6.4.4 Discussion 6.4.5 Implication in Terms of Disease Management Policy 6.5 Case Study 4: The Willingness to Forego Compensation: Application of the Discrete Choice Experiment to Evaluate Swine Disease Surveillance in Vietnam 6.5.1 Introduction 6.5.2 Materials and Methods 6.5.2.1 Discrete Choice Experiment Applied to Passive Surveillance 6.5.2.2 Definition of Policy Options To Be Tested 6.5.2.3 Study Location and Participants 6.5.3 Results 6.5.4 The Burden of Distrust, the Unbearable Cost of Massive Culling Outweighs Compensation Levels 6.5.5 Implication in Terms of Disease Management Policy 6.6 General Discussion References Part IV: Qualitative and Semi-Quantitative Methods for Process Evaluation 7: Qualitative Methods to Evaluate Health Surveillance Systems 7.1 Introduction 7.2 OASIS: A Tool Adapted to the Evaluation of the Surveillance Process 7.2.1 Tool Principles 7.2.2 Practical Use 7.3 The Use of Social Network Analysis to Characterize Health Information-Sharing Systems 7.3.1 Importance of Private Actors in the Sharing of Health Information in Vietnam: Impact on the Performance of the Official Surveillance System 7.4 Socioeconomic Factors that Impact the Performance of Health Surveillance 7.4.1 Quick Sale of Animals as an Alternative to Reporting 7.4.2 The Importance of Economic Impact, Trust, and Cultural Issues 7.4.3 The Impact on the Market of a Declaration of Cases of Diseases and Risks of Stigmatization of the Actors 7.5 Conclusion References 8: The Use of Participatory Methods in the Evaluation of Health Surveillance Systems 8.1 Introduction 8.2 Importance of Sociological and Economic Factors in the Surveillance System Performances 8.3 Advantages of Using Participatory Approaches in the Evaluation Process 8.4 Application of Participatory Methods to Surveillance System Evaluation 8.4.1 The AccePT Method to Assess Surveillance System Acceptability 8.4.1.1 Definition of the Acceptability of a Surveillance System 8.4.1.2 General Method Flow 8.4.1.3 Presentation of the Tools Relational Diagrams and Rating Smileys Flow Diagrams Associated with Proportional Piling Impact Diagrams Associated with Proportional Piling Analysis and Presentation of Results 8.4.1.4 Benefits, Limits, and Outlook 8.5 Case Study 1: Application of Participatory Approaches to Evaluate Avian Influenza Surveillance in Vietnam 8.6 Case Study 2: Evaluation of the Acceptability of Classical Swine Fever Surveillance System in Germany 8.7 Case Study 3: Evaluation of the Acceptability of a Pilot Multistakeholder Wildlife Health Surveillance Network in Cambodia References 9: Evaluation of Collaboration in a Multisectoral Surveillance System: The ECoSur Tool 9.1 The Conceptual Approach Behind the Development of the Tool 9.2 The Evaluation Process Used in ECoSur 9.3 The Structure of ECoSur 9.4 The Application of ECoSur 9.4.1 First Step: Defining the Evaluation Question and the Evaluation Boundaries 9.4.2 Second Step: Collecting Data 9.4.3 Third Step: Scoring the Criteria of the Organizational and Function Attributes 9.4.4 Fourth Step: Interpreting Evaluation Results 9.4.5 Fifth Step: Organizing a Workshop to Validate the Evaluation Results 9.4.6 Sixth Step: Drafting the Report References Part V: Statistical and Modeling Approaches to Evaluate and Inform Surveillance Design 10: Quantitative Methods to Evaluate Health Surveillance Effectiveness 10.1 Introduction 10.2 Surveillance Effectiveness 10.3 Methods to Quantitatively Assess Surveillance Effectiveness 10.3.1 Methods Relying on the Analysis/Statistical Modelling of the Data Generated by the Surveillance Systems (Case Study 2) 10.3.1.1 General Principles 10.3.1.2 Type of Inference Obtained 10.3.1.3 Strengths 10.3.1.4 Weaknesses 10.3.2 Methods Relying on the Modelling of the Surveillance Processes Only (Case Study 3) 10.3.2.1 General Principles 10.3.2.2 Type of Inference Obtained 10.3.2.3 Strengths 10.3.2.4 Weaknesses 10.3.3 Methods Relying on the Simultaneous Modelling of the Surveillance Processes and of Disease Transmission in the Population under Surveillance (Case Study 4) 10.3.3.1 General Principles 10.3.3.2 Type of Inference Obtained 10.3.3.3 Strengths 10.3.3.4 Weaknesses 10.4 Synthesis (Table 10.1) 10.5 Case Study 1: A Rationale to Unify Surveillance Effectiveness Attributes [6] 10.5.1 Information Required for the Evaluation of Surveillance Effectiveness (Table 10.2) 10.6 Case Study 2—Results of the Sensitivity Evaluation of the Avian Influenza Surveillance System in Vietnam: Application of the Unlisted Capture–Recapture Method [21] 10.7 Case Study 3—The Use of Models of the Surveillance Processes Only: Quantitative Assessment of the Sensitivity of Avian Influenza Surveillance Systems Using the Scenario Tree Approach [24] 10.7.1 HPAI Surveillance System Organization in Thailand 10.7.2 Quantitative Evaluation of HPAI H5N1 Surveillance in Backyard Poultry in Thailand Using Scenario Tree Modelling 10.8 Case Study 4—The Use of Dynamic Models [16] References 11: Network Analysis for Surveillance Design and Evaluation 11.1 Introduction 11.2 Network Data 11.2.1 Data Structures and Network Construction 11.2.2 Study Design and Data Collection 11.2.3 Network Analysis Software 11.3 Network Structural Properties 11.3.1 Node-Level Centrality 11.3.2 Large-Scale Structure of Networks 11.3.2.1 Cohesion 11.3.2.2 Degree Distribution 11.3.2.3 Partition 11.4 Models of Disease Transmission 11.5 Conclusions References Part VI: Evaluation in the Policy Cycle 12: Health Surveillance Evaluation in the Policy Cycle 12.1 Introduction 12.1.1 Evaluation at the Strategic Level 12.1.2 The Value of Surveillance and Its Evaluation 12.1.3 The Delivery of Value Through Implementation of Evaluation Recommendations 12.1.4 Barriers to Implementation of Recommendations from Evaluation 12.2 The Rest of This Section References 13: Animal Health Surveillance: From Compliance to Real Governance 13.1 Introduction 13.2 Governance and Compliance from a Social Sciences Perspective 13.3 Assessing the Governance of Animal Health Surveillance System: Available Tools 13.4 Compliance of Surveillance Systems Relying on Farmers’ Reporting 13.5 Real Governance, “Practical” Information Systems and Their Limits 13.6 Conclusion References 14: Integrating Health Surveillance Evaluation Outputs in the Policy Cycle 14.1 Introduction 14.2 Limited Application of the Evaluation of Surveillance System to Inform Policy 14.3 Loose Links Between Surveillance Cycle and Policy Cycle: The Need for Evaluation 14.4 Practical Implementation of Animal Health Evaluation Versus Standard Requirements 14.5 Evaluation to Inform Dynamic Changes 14.6 Conclusion and Way Forward References 15: Added Value and Impact of the Evaluation Process in Health Surveillance 15.1 Introduction 15.1.1 What is a Theory of Change? 15.1.2 The Process of Creating a Theory of Change 15.1.2.1 Data Collection and Analysis 15.1.3 The Value of a Theory of Change 15.2 IMPRESS Impact Assessment Method 15.3 Application of Principles to Case Studies 15.3.1 Case Study 1: Evaluation of the Impact of the Development of a Cost-Effective Swine Influenza Surveillance Protocol in Vietnam 15.3.1.1 Introduction 15.3.1.2 Story of the innovation 15.3.1.3 Added value of the impact evaluation 15.3.2 Case Study 2: Assessing the Impact of the Evaluation Process of Avian Influenza Surveillance Strategy in Vietnam: a Way to Promote Changes 15.3.2.1 Introduction 15.3.2.2 Story of the Innovation Improving Animal Health Surveillance in Vietnam Using Evaluation Methods and Tools Scenarios and Activities to Implement to Ensure the Impacts Reach Out 15.3.2.3 Added Value of the Impact Evaluation References 16: Synthesis—Evaluate to Better Inform: A Way to Strengthening Health Surveillance Systems 16.1 Introduction 16.1.1 Accounting for Major Socioeconomic Constraints that Impact the Performance of Health Surveillance Systems 16.2 Acceptability: A Critical Issue to Ensure Stakeholder Engagement 16.3 Private Surveillance Networks: How to Integrate Them? 16.4 The Importance of an Integrated Evaluation Approach to Inform Decision 16.5 Remaining Challenges and Future Perspectives to Further Improve Animal Health Surveillance Systems 16.5.1 Participatory Modeling to Build User-Centered Surveillance Systems—Adapted to the Socioeconomic Constraints of Its Actors 16.5.2 The Importance of Understanding the Utility of Health Programs (Surveillance and Control) 16.6 From Policy to Field Implementation: The Need to Combine Top-Down and Bottom-Up Approaches 16.7 Public–Private Partnerships (PPP): At the Heart of Surveillance in Animal Health 16.8 Conclusion References Glossary