Modern Statistical Methods for HCI (Human–Computer Interaction Series)

Description

This book critically reflects on current statistical methods used in Human-Computer Interaction (HCI) and introduces a number of novel methods to the reader.  Covering many techniques and approaches for exploratory data analysis including effect and power calculations, experimental design, event history analysis, non-parametric testing and Bayesian inference; the research contained in this book discusses how to communicate statistical results fairly, as well as presenting a general set of recommendations for authors and reviewers to improve the quality of statistical analysis in HCI. Each chapter presents [R] code for running analyses on HCI examples and explains how the results can be interpreted. Modern Statistical Methods for HCI is aimed at researchers and graduate students who have some knowledge of “traditional” null hypothesis significance testing, but who wish to improve their practice by using techniques which have recently emerged from statistics and related fields. This book critically evaluates current practices within the field and supports a less rigid, procedural view of statistics in favour of fair statistical communication. Foreword Preface About the Authors Maurits Judy Who Is This Book For? The Structure of the Book The Sample Dataset: The Mango Watch Our Approach Acknowledgements Contents Contributors 1 An Introduction to Modern Statistical Methods in HCI 1.1 Introduction 1.2 Earlier Commentaries on Statistical Methods in HCI and Beyond 1.3 Some of the Very Basic Ideas of Our Common ``NHST'' Method 1.4 The Common, and Well-Acknowledged, Problems with Our Current NHST Methods 1.4.1 Misinterpretations of the P-Value 1.4.2 The Fallacy of the Transposed Conditional 1.4.3 A Lack of Power 1.4.4 Confusion of P-Values and Effect Size Estimates 1.4.5 Multiple Comparisons 1.4.6 Researcher Degrees of Freedom 1.4.7 What Is the Aim of Statistical Analysis? 1.5 Broadening the Scope of Statistical Analysis in HCI References Part I Getting Started With Data Analysis 2 Getting Started with [R]; a Brief Introduction 2.1 Introduction 2.1.1 Data Types 2.1.2 Storage 2.1.3 Storage Descriptives 2.2 Working with [R] 2.2.1 Writing Functions 2.2.2 Data: Input and Output 2.2.3 For Loop 2.2.4 Apply Function 2.2.5 Common Used Functions 2.2.6 Packages 2.3 Mastering [R] 2.3.1 Further Reading References 3 Descriptive Statistics, Graphs, and Visualisation 3.1 Introduction 3.1.1 Why Do We Visualise Data? 3.2 Descriptive Statistics and Exploratory Data Analysis 3.3 Principles of Visualisation 3.4 ggplot2---A Grammar of Graphics 3.4.1 ggplot2 3.5 Case Study 3.5.1 Team Size 3.5.2 SUS Scores 3.5.3 Response Times 3.5.4 Model Predictions 3.6 Summary References 4 Handling Missing Data 4.1 Introduction 4.2 Missing Data: Problems and Pitfalls 4.2.1 Mechanisms of Misssingness 4.2.2 Comparing Three Missing Data Scenarios 4.3 Missing Data: Potential Solutions 4.3.1 Popular but Inadequate `Solutions' 4.3.2 Multiple Imputation 4.3.3 Pooling Imputations Using Rubin's Equations 4.3.4 Multiple Imputation in Practice 4.4 Maximum Likelihood and Bayesian in Approaches to Missing Data 4.4.1 Maximum Likelihood Estimation in Ignorable Models Using Expectation-Maximization 4.4.2 Bayesian Inference with Missing Data 4.5 Conclusion References Part II Classical Null Hypothesis Significance Testing Done Properly 5 Effect Sizes and Power Analysis in HCI 5.1 Introduction 5.2 Myths of the p Value 5.2.1 Myth 1: Threshold of the P Value 5.2.2 Myth 2: Magnitude of the Effect 5.2.3 Myth 3: ``Significant'' is ``Important'' 5.2.4 Summary 5.3 Effect Size 5.3.1 Interpretation of Effect Sizes 5.3.2 Paired t-Test 5.3.3 Unpaired t-Test 5.3.4 ANOVA 5.4 One-Way ANOVA 5.5 One-Way Repeated-Measure ANOVA 5.6 Two-Way ANOVA 5.7 Non-parametric Tests 5.8 Pearson's Correlation 5.9 Power Analysis 5.9.1 Type I Error and Type II Error 5.9.2 Conducting Power Analysis 5.9.3 Sample Size Estimation 5.9.4 Retrospective Power Analysis (Caution: Not Recommended) 5.10 Conclusion References 6 Using R for Repeated and Time-Series Observations 6.1 Repeated Measures and Time Series in HCI 6.2 An Introduction to Within-Subjects Analysis of Variance (ANOVA) 6.2.1 Assumptions of Within-Subjects ANOVA 6.2.2 Sphericity 6.2.3 Correcting for Sphericity 6.3 Conceptual Model for Within-Subjects ANOVA 6.4 Statistical Inference for Within-Subjects ANOVA 6.5 Calculating Within-Subjects ANOVA Using R 6.6 Interpreting and Presenting the Within-Subjects ANOVA Findings 6.7 An Introduction to Event History Analysis 6.7.1 Censoring 6.8 Statistical Model for Event History Analysis 6.9 Calculating Event History Analysis Using R 6.10 Interpreting and Presenting the Event History Analysis Findings References 7 Nonparametric Statistics in Human--Computer Interaction 7.1 Introduction 7.2 When to Use Nonparametric Analyses 7.2.1 Assumptions of Analysis of Variance (ANOVA) 7.2.2 Table of Analogous Parametric and Nonparametric Tests 7.3 Tests of Proportions 7.3.1 One-Sample Tests of Proportions 7.3.2 N-Sample Tests of Proportions 7.4 Single Factor Tests 7.4.1 Testing ANOVA Assumptions 7.4.2 Single-Factor Between-Subjects Tests 7.4.3 Single-Factor Within-Subjects Tests 7.4.4 Multifactor Tests 7.4.5 N-Way Analysis of Variance 7.4.6 Aligned Rank Transform (ART) 7.4.7 Generalized Linear Models (GLM) 7.4.8 Generalized Linear Mixed Models (GLMM) 7.4.9 Generalized Estimating Equations (GEE) 7.5 Summary References Part III Bayesian Inference 8 Bayesian Inference 8.1 Introduction 8.2 Introduction to Bayes' Theorem 8.3 Computing Bayesian Statistics 8.4 Bayesian Two Group Comparison for Binary Variables 8.5 Bayesian Two Group Comparison for Numeric Variables 8.6 Bayesian Regression with Numeric Predicted Variable 8.7 Do Not Reinvent the Wheel References 9 Bayesian Testing of Constrained Hypotheses 9.1 Introduction 9.1.1 Multiple Hypothesis Testing in HCI 9.1.2 Limitations of Classical Methods 9.1.3 Bayes Factors for Multiple Hypothesis Testing 9.1.4 Outline 9.2 Bayesian Estimation in the HCI Application 9.2.1 The Multivariate Normal Model 9.2.2 The Prior: A Formalization of Prior Beliefs 9.2.3 The Posterior: Our Belief After Observing the Data 9.2.4 HCI Data Example 9.3 Bayes Factors and Posterior Model Probabilities 9.3.1 Definition of the Bayes Factor 9.3.2 Interpreting Bayes Factors 9.3.3 Posterior Model Probabilities 9.3.4 Prior Specification When Computing Bayes Factors 9.4 A Bayesian T Test Using BIEMS 9.4.1 Hypotheses 9.4.2 Prior Specification 9.4.3 Bayes Factors and Posterior Model Probabilities 9.5 Evaluation of the HCI Application 9.6 Discussion References Part IV Advanced Modeling in HCI 10 Latent Variable Models 10.1 Introduction 10.1.1 Data 10.2 Path Models 10.2.1 Variables Types in a Path Model 10.2.2 Variables Relations in a Path Model 10.2.3 Tracing Rules 10.3 Latent Variable Models 10.3.1 Identification of Latent Variable Models 10.4 Latent Variable Models in R 10.4.1 Entering Data 10.4.2 Specifying Models in Lavaan 10.4.3 More Complex Models 10.5 Cautions and Considerations in Using Latent Variable Models 10.6 Conclusion References 11 Using Generalized Linear (Mixed) Models in HCI 11.1 Introduction 11.2 Linear Models 11.2.1 Fitting a Line 11.2.2 [R] Model Formulas 11.3 Regularization 11.4 Generalized Linear Models 11.5 Linear Mixed Models 11.5.1 Random Intercept Models 11.5.2 More Random Effect Specifications 11.5.3 Generalized Mixed Models 11.6 Conclusions and Recommendations References 12 Mixture Models: Latent Profile and Latent Class Analysis 12.1 Introduction 12.2 Mixtures of Continuous Variables: Latent Profile Analysis 12.3 Mixtures of Categorical Variables: Latent Class Analysis 12.4 Other Uses of Latent Class/Profile Analysis 12.5 Further References References Part V Improving Statistical Practice in HCI 13 Fair Statistical Communication in HCI 13.1 Introduction 13.2 p-Values, Effect Sizes and Confidence Intervals 13.2.1 A Minimalistic Example and Quick Reminders 13.2.2 Choosing a Pill 13.2.3 How Useful Is the Information Conveyed by P? 13.2.4 Usability Problems with p-Values 13.2.5 Conclusion 13.3 Null Hypothesis Significance Testing Versus Estimation 13.3.1 A Few More Reminders 13.3.2 How Useful Is the α Cut-Off? 13.3.3 More Usability Problems Brought by the α Cut-Off 13.3.4 Conclusion 13.4 Fair Statistical Communication Through Estimation 13.4.1 General Principles 13.4.2 Before Analyzing Data 13.4.3 Calculating Confidence Intervals 13.4.4 Plotting Confidence Intervals 13.4.5 Interpreting Confidence Intervals 13.5 Conclusion References 14 Improving Statistical Practice in HCI 14.1 Introduction 14.2 Case Studies from the HCI Literature 14.2.1 Misinterpretations of the p-Value 14.2.2 The Fallacy of the Transposed Conditional 14.2.3 A Lack of Power (Type II Errors) 14.2.4 Confusion of p-Values and Effect Size Estimates 14.2.5 Multiple Comparisons (Type I Errors) 14.2.6 Researcher Degrees of Freedom 14.3 What Do We Know Now? 14.4 Recommendations for Improving Statistical Methodology 14.5 Closing Remarks References