Meta-Research: Methods and Protocols

Description

This volume presents state-of-the art design, analysis and integration approaches for biomedical data including novel statistical models for a comprehensive and powerful synthesis and assessment of scientific evidence. Chapters detail principles of systematic reviews, semi-automated tools for systematic searches, fixed- and random-effects meta-analytical models, living systematic reviews, meta-analysis of genetic studies, meta-analysis of pragmatic and explanatory trials, network meta-analysis, and other modern approaches for data synthesis. Written in the highly successful Methods in Molecular Biology series format, chapters include introductions to their respective topics, lists of the necessary materials, and tips on troubleshooting and avoiding known pitfalls.   Authoritative and cutting-edge, the Meta- Research: Methods and Protocols book, written by global experts, will introduce the reader in a step-by-step process to the methods of the vital and highly promising field of evidence synthesis. Preface Contents Contributors Chapter 1: Principles of Systematic Reviews and Meta-analyses 1 Overview 2 The Team 3 The Question 4 The Search 5 Selection Process 6 Data Extraction 7 Risk of Bias 8 The Synthesis 9 Certainty of the Evidence 10 Tools to Assess the Quality and the Report of a Systematic Review 11 Conclusion References Chapter 2: Semi-automated Tools for Systematic Searches 1 Introduction 1.1 How Literature Identification Has Been Done to Date 1.1.1 Step 1: Create Search Queries 1.1.2 Step 2: Screen Citations 1.2 Limitations of the Current Process 2 Semi-automating Parts of the Traditional Two-Step Approach 3 Tools that Support the Development of Search Queries to Fix a Corpus 3.1 Tools to Identify Search Terms 3.2 Tools to Cluster Terms and Visualize Term Relationships 3.3 Development of Tools to Semi-automatically Generate Queries 3.4 To Trace Citation Linkages 4 Methodological Filters 5 Tools that Translate Search Queries Across Databases 6 Tools for Deduplication 7 Tools that Support Screening 8 Semi-automating a Unified Approach 9 Evaluation 10 Conclusion References Chapter 3: Fixed- and Random-Effects Models 1 Introduction 1.1 Example Data and Terminology 1.2 An Overview of Fixed-Effect Modeling vs. Random-Effects Modeling 2 Fixed-Effect Modeling 2.1 Statistical Concepts of Fixed-Effect Modeling 2.2 A Worked Example 2.3 Effect Sizes for Different Data Types 3 Basic Random-Effects Modeling 3.1 Statistical Concepts of Random-Effects Modeling 3.2 Conducting Random-Effects Meta-analysis 3.3 Another Example 4 Modeling with Arm-Based Data 4.1 Mantel-Haenszel Meta-analysis 4.2 Additional Random-Effects Models 5 Fixed and Random-Effects Using the Bayesian Framework 5.1 Use of Empirical Priors 6 Identifying the Source of Heterogeneity References Chapter 4: Assessing Heterogeneity in Random-Effects Meta-analysis 1 Introduction 1.1 Types of Heterogeneity 1.1.1 Methodological Heterogeneity 1.1.2 Clinical Heterogeneity 1.2 Approaches to Addressing Heterogeneity 2 Q-Statistic 3 Methods to Estimate the Heterogeneity Variance 3.1 DerSimonian-Laird 3.2 Paul-Mandel and the Method of Moments Approach 3.3 Restricted Maximum Likelihood (REML) Estimator 4 The I2 Statistic 5 Problems with Heterogeneity Estimation 6 Exploring Heterogeneity 6.1 Sub-group Analysis 6.2 Meta-Regression 6.2.1 Single-Variable Meta-Regression 6.2.2 Multivariable Meta-Regression 7 Publication Bias 7.1 Identifying Publication Bias 7.2 The Funnel Plot 7.3 Tests for Funnel Plot Asymmetry 7.4 A Final Note on Publication Bias 8 Final Notes on Assessing Heterogeneity References Chapter 5: Performing Meta-analyses with Very Few Studies 1 Introduction 2 Materials 2.1 Overview of Basic Estimation Methods 2.2 Qualitative Evidence Synthesis (QES) 3 Methods 3.1 Step 1: Initial Choice of Model for a Meta-analysis 3.2 Step 2: Assessment of Heterogeneity 3.3 Step 3: Making the Final Model Choice for Meta-analysis 4 Discussion and Outlook References Chapter 6: Systematic Versus Rapid Versus Scoping Reviews 1 Overview 2 Systematic Reviews 2.1 Steps in a Systematic Review 2.1.1 Formulating a Clear and Concise Review Question 2.1.2 Develop and Register the Review Protocol 2.1.3 Performing a Comprehensive Search to Identify Studies 2.1.4 Screening and Study Selection, Data Extraction, and Risk of Bias Assessment 2.1.5 Data Synthesis 2.1.6 Presentation and Interpretation of the Results 2.2 Limitations 3 Rapid Reviews 3.1 Steps in a Rapid Review 3.1.1 Formulating the Review Question, Eligibility Criteria, and Search Strategy 3.1.2 Develop and Register the Review Protocol 3.1.3 Screening, Data Extraction, and Risk of Bias Assessment 3.1.4 Data Synthesis 3.1.5 Presentation and Interpretation of Results 3.2 Limitations 4 Scoping Reviews 4.1 Steps in a Scoping Review 4.1.1 Formulating the Review Question, Eligibility Criteria, and Search Strategy 4.1.2 Develop the Review Protocol 4.1.3 Performing a Comprehensive Search to Identify Studies 4.1.4 Screening, Selection, and Data Extraction 4.1.5 Data Synthesis 4.1.6 Presenting and Interpretation of Results 4.2 Limitations 4.3 Rapid Scoping Reviews 5 Final Remarks References Chapter 7: Living Systematic Reviews 1 What Is a Living Systematic Review? 2 Performing a Living Systematic Review 3 Meta-Analysis in a Living Systematic Review 4 Avoiding Errors When Updating Meta-Analyses 4.1 Trial Sequential Analysis 4.2 Sequential Meta-Analysis 5 Example: Meta-Analysis of Peptic Ulcer Trials 6 Methods for Network Meta-Analysis 7 Maintaining and Updating a Living Systematic Review 8 Publishing and Disseminating a Living Systematic Review 9 Conclusions and Recommendations References Chapter 8: Umbrella Reviews: What They Are and Why We Need Them 1 Introduction 1.1 Hierarchy of Evidence 1.2 Need for Broader Assessments 2 Umbrella Reviews 2.1 Basic Terminology and Definitions 2.2 Methods 2.2.1 Designing the Review 2.2.2 Registration 2.2.3 Inclusion and Exclusion Criteria 2.2.4 Search Strategy and Databases 2.2.5 Basic Data Extraction 2.2.6 Quality Considerations 2.2.7 Quantitative Synthesis Measures of Association and Exposure-Outcome Categories 2.2.8 Inconsistency and Other Biases 2.2.9 Presentation of Findings and Grading Criteria 2.3 Conclusions and Limitations 3 Conclusions References Chapter 9: Meta-analysis of Pragmatic and Explanatory Trials 1 Introduction 2 Explanatory and Pragmatic Trials 3 Measurement of Pragmatism: The Efficacy-Effectiveness Continuum 4 Trial Differences and Level of Pragmatism 5 Pooling Findings from Pragmatic and Explanatory Trials 5.1 Plan to Assess Pragmatism at the Protocol Stage 5.2 Collect Data on Pragmatism 5.3 Extract Data on Treatment Effect 5.4 Incorporate Pragmatism in Meta-analysis 5.5 Reporting and Interpreting Results of Meta-analyses of Pragmatic and Explanatory Trials 6 Conclusion References Chapter 10: Meta-Analysis of Proportions 1 Introduction 2 Illustrative Example: Prevalence of KRAS Mutations by Tumor Sidedness in Colorectal Cancer 3 Meta-Analysis of Single Proportions 3.1 Transformations of Proportions 3.1.1 Logit Transformation 3.1.2 Arcsine Transformation 3.1.3 Freeman-Tukey Double Arcsine Transformation 3.1.4 Back-Transformations 3.2 Confidence Intervals for Individual Studies 3.3 Classic Inverse Variance Meta-Analysis 3.4 Meta-Analysis Based on Generalized Linear Mixed Model 3.5 Back-Transformation of Meta-Analysis Results 3.6 Application to KRAS Meta-Analysis 4 Meta-Analysis Comparing Two Proportions 4.1 Classic Inverse Variance Meta-Analysis 4.2 Mantel-Haenszel and Peto Method 4.3 Meta-Analysis Based on Generalized Linear Mixed Model 4.4 Application to KRAS Meta-Analysis 5 Discussion and Conclusion References Chapter 11: Meta-Analysis Methods of Diagnostic Test Accuracy Studies 1 Introduction 2 Analysis of Diagnostic Test Accuracy Studies 3 Univariate Methods for Meta-Analysis of a Single Diagnostic Test in Diagnostic Test Accuracy Studies 4 Multivariate Methods for Meta-Analysis of a Single Diagnostic Test in Diagnostic Test Accuracy Studies 5 Methods for Meta-Analysis of Multiple Diagnostic Tests in Diagnostic Test Accuracy Studies 6 Meta-Analysis of Diagnostic Accuracy Tests in Practice 7 Discussion References Chapter 12: Network Meta-Analysis 1 Introduction 2 What Are the Similarities and Differences Between Pairwise and Network Meta-Analysis? 3 What Are Direct, Indirect, and Mixed Treatment Comparisons? 4 How Are Network Meta-Analysis Treatment Effects Derived? 5 What Are Key Criteria Predicating Completion of a Network Meta-Analysis (NMA)? 5.1 Network Connectivity 5.2 Homogeneity 5.3 Transitivity 5.4 Consistency 6 How Do I Interpret Network Meta-Analysis (NMA) Outputs? 7 What Are Areas of Ongoing Methodological Research in Network Meta-Analysis (NMA)? 8 How Do I Conduct a Systematic Review and Network Meta-Analysis (NMA)? 9 What Are Common Problems in Conducting a Network Meta-Analysis (NMA)? 10 How Do I Critically Appraise a Systematic Review and Network Meta-Analysis (NMA)? 11 Conclusion References Chapter 13: Contrast-Based and Arm-Based Models for Network Meta-Analysis 1 Introduction 2 Models 2.1 Statistical Notation 2.2 Contrast-Based Model 2.3 Arm-Based Model 3 Theoretical Differences Between Contrast-Based and Arm-Based Models 3.1 Estimands 3.2 Model Specification and Symmetry 3.3 Between-Study Information 3.4 Treatment Effect Modified by Baseline Risk 4 Evidence from Simulation Studies and Empirical Investigations 5 Worked Example 6 Conclusion References Chapter 14: Software to Conduct a Meta-Analysis and Network Meta-Analysis 1 Introduction 2 Key Considerations When Selecting Software 3 Criteria for Describing Software 3.1 Data Preparation and Input 3.2 Statistically Assessing the Validity of the Models 3.3 Summarizing the Results 4 Software for Meta-Analysis 5 Software for Network Meta-Analysis References Chapter 15: Modeling Multicomponent Interventions in Network Meta-Analysis 1 Introduction 2 Meta-Analytical Approaches to Evaluate Complex Interventions 2.1 Single-Effect Model 2.2 Standard NMA Model 2.3 Component NMA (CNMA) Model 2.3.1 The Additive Main Effects Model 2.3.2 Interaction CNMA Model 3 Illustrative Example: Comparative Effectiveness of Self-Management Interventions for Type 2 Diabetes 3.1 The Dataset 3.2 Results 4 Discussion References Chapter 16: Practical Considerations and Challenges When Conducting an Individual Participant Data (IPD) Meta-Analysis 1 Types of Data for Meta-Analysis 1.1 Aggregate Data Meta-Analysis 1.2 Individual Participant Data Meta-Analysis 2 Practical Considerations for IPD Meta-Analysis 2.1 Planning an IPD Meta-Analysis 2.2 Collecting Data for IPD Meta-Analysis 2.2.1 Where to Collect IPD from? 2.2.2 How to Collect IPD? Requesting Data Directly from a Study Investigator Requesting Data from a Data-Sharing Platform or an Archive 2.3 Preparing Data for IPD Meta-Analysis 3 Challenges When Conducting IPD Meta-Analysis 3.1 Obtaining IPD 3.2 Availability Bias 3.2.1 Characteristics of Studies Providing IPD and Not Providing IPD? 3.2.2 Results of Studies Providing IPD and Not Providing IPD? 3.2.3 Reasons Why IPD Are Not Available? 4 Summary of the Chapter References Chapter 17: Individual Patient Data Meta-Analysis and Network Meta-Analysis 1 Introduction 2 Two-Stage Models Versus One-Stage Models 3 One-Stage IPD MA Models 4 Two-Stage IPD MA Models 5 IPD NMA Models 5.1 Generalization to Networks of Any Size 6 Including Patient-Level Covariates 7 Treatment-Covariate Interactions 8 Missing Covariate Data 9 Assessing Consistency 10 Combining IPD and Aggregate Data 11 Time-to-Event Outcomes 11.1 Reconstructing IPD for Time-to-Event Outcomes 12 Conclusion References Chapter 18: Challenges in Comparative Meta-Analysis of the Accuracy of Multiple Diagnostic Tests 1 Introduction 2 Approaches Commonly Used to Compare Multiple Tests 3 Illustrative Example: Comparison of HPV DNA, Cytology, and mRNA Tests for Cervical Cancer 3.1 The Dataset 3.2 Results 4 Extending Meta-Analysis to NMA of Diagnostic Tests 4.1 NMA Models of Multiple Diagnostic Tests 4.2 Basic Assumptions in NMA of DTA Studies 4.3 Test Ranking 5 Discussion References Index