Unobtrusive Observations of Learning in Digital Environments: Examining Behavior, Cognition, Emotion, Metacognition and Social Processes Using Learning Analytics

Description

This book integrates foundational ideas from psychology, immersive digital learning environments supported by theories and methods of the learning sciences, particularly in pursuit of questions of cognition, behavior and emotion factors in digital learning experiences. New and emerging foundations of theory and analysis based on observation of digital traces are enhanced by data science, particularly machine learning, with extensions to deep learning, natural language processing and artificial intelligence brought into service to better understand higher-order thinking capacities such as self-regulation, collaborative problem-solving and social construction of knowledge. As a result, this edited volume presents a collection of indicators or measurements focusing on learning processes and related behavior, (meta-)cognition, emotion and motivation, as well as social processes. In addition, each section of the book includes an invited commentary from a related field, such as educational psychology, cognitive science, learning science, etc. Preface Contents About the Editors Part I: Learning Processes Chapter 1: Unobtrusive Observations of Learning Processes 1 Section Overview Chapter 2: A Review of Measurements and Techniques to Study Emotion Dynamics in Learning 1 Introduction 2 The Features of Emotion Dynamics 2.1 Emotional Variability 2.2 Emotional Instability 2.3 Emotional Inertia 2.4 Emotional Cross-lags 2.5 Emotional Patterns 3 The Measurements of Emotion Dynamics 3.1 Experience Sampling Method 3.2 Emote-Aloud 3.3 Facial Expressions 3.4 Vocal Expressions 3.5 Language and Discourse 3.6 Physiological Sensors 4 The Techniques for Analyzing Emotion Dynamics 4.1 Conventional Statistical Methods 4.2 Entropy Analysis 4.3 Growth Curve Modeling 4.4 Time Series Analysis 4.5 Network Analysis 4.6 Recurrence Quantification Analysis 4.7 Sequential Pattern Mining 5 The Challenges of Studying Emotion Dynamics in Learning 5.1 Deciding What to Measure About Emotion Dynamics 5.2 Deciding How to Analyze Emotion Dynamics 5.3 Addressing Individual and Developmental Differences 5.4 Differentiating Between Short-Term and Long-Term Emotion Dynamics 6 Concluding Remarks and Directions for Future Research References Chapter 3: Applying Log Data Analytics to Measure Problem Solving in Simulation-Based Learning Environments 1 Introduction 2 Background 3 Methods 3.1 Experiment 1 3.2 Experiment 2 3.3 Log Data Processing 4 Results 4.1 Problem-Solving Outcomes as Measured by Solution Quality 4.2 Problem-Solving Processes as Captured by Features Extracted from Log Data 4.3 Pause as a Generalizable Indicator of Deliberate Problem Solving 4.4 How Log Data-Based Features Were Associated with Specific Problem-Solving Practices 5 Discussion 6 Limitations 7 Conclusion References Chapter 4: Challenges in Assessments of Soft Skills: Towards Unobtrusive Approaches to Measuring Student Success 1 Introduction 2 Background 2.1 Developing Soft Skills 2.2 Leadership Skills 2.3 Challenges of Assessing Soft Skills 3 Case Study 3.1 Study Context 3.2 Extracting Unobtrusive Measures 3.3 Assessing Leadership Mastery 3.4 Assessing Systematic Progression 4 Conclusion References Chapter 5: Reconfiguring Measures of Motivational Constructs Using State-Revealing Trace Data 1 Introduction: Self-Regulated Learning 2 Dynamic Nature of Motivation 2.1 How to Capture Motivation 2.2 A Role for Trace Data in Motivational Studies 3 Critiques of Recent Studies 3.1 Hershkovitz and Nachmias (2008) 3.1.1 Theoretical Framework 3.1.2 Contexts 3.1.3 Data and Indicators 3.1.4 Data Analysis and Results 3.2 Cocea and Weibelzahl (2011) 3.2.1 Theoretical Framework 3.2.2 Contexts 3.2.3 Data and Indicators 3.2.4 Data Analysis and Results 3.3 Zhou and Winne (2012) 3.3.1 Theoretical Framework 3.3.2 Contexts 3.3.3 Data and Indicators 3.3.4 Data Analysis and Results 3.4 Critiques of the Select Studies 3.4.1 Importance of Design Processes 3.4.2 Weak Evaluation Process of Indicators 3.4.3 Lack of Discussion on How Trace Measures Were Introduced to Users 4 Proposals 4.1 Implementing Design Framework 4.2 Evaluating Indicator Designs for Future Studies 4.3 Introducing Interventions Less Obtrusively 5 Conclusion References Chapter 6: Measuring Collaboration Quality Through Audio Data and Learning Analytics 1 Introduction 2 Defining Collaboration Quality 3 Background 4 Automated Collaboration Analytics 5 Toward Collaboration Quality Detection: From Analytics to Visualizations 6 From Visualizations to Meaningful Feedback 7 Challenges 8 Discussion and Conclusion References Chapter 7: Unobtrusively Measuring Learning Processes: Where Are We Now? 1 Introduction 2 Critical Overview of the Chapters 3 What Is Currently Missing in the Modelling of Learning Processes? References Part II: Learning Data Chapter 8: Data for Unobtrusive Observations of Learning: From Trace Data to Multimodal Data 1 Section Overview Chapter 9: Measuring and Validating Assumptions About Self-Regulated Learning with Multimodal Data 1 Introduction 2 The Theory of Self-Regulated Learning 3 Self-Reported SRL Measurement 4 Observational SRL Measurement 4.1 Multimodal Observation of SRL 4.2 Establishing the Validity of Inferences from Observational Data in Multimodal Designs 4.3 Implications of Multimodal Designs for Research on SRL 4.4 Limitations 5 Conclusions References Chapter 10: Measuring Multidimensional Facets of SRL Engagement with Multimodal Data 1 Introduction 2 What Is Engagement? 3 Extension of the Integrative Model of Self-Regulated Learning (SRL) Engagement 4 Unimodal Methods for Studying Engagement 4.1 Clickstream Data/Log Files 4.2 Eye Tracking and Gaze Patterns 4.3 Audio/Video (Think and Emote-Alouds, Observations, and Interviews) 4.4 Electrodermal Activity and Heart Rate Variability 4.5 Self-Reports and Experience Sampling 4.6 Facial Expressions 4.7 EEG 4.8 Convergence Approaches 5 Theoretically Grounded Approach for Measuring Engagement with Multimodal Data 6 Limitations and Future Directions 7 Concluding Thoughts References Chapter 11: Roles for Information in Trace Data Used to Model Self-Regulated Learning 1 Introduction 2 Learning Events 2.1 Modeling One Learning Event: If-Then-Else 3 Information Is the Subject of Operations 3.1 Motivation 3.2 Cognition 3.3 Metacognition 4 Integrating Information with Trace Data 4.1 Examining Effects of One Operation 4.2 How Information Enriches Trace Data About Operations 4.2.1 Operations Mark Conditions Learners Monitor 4.2.2 Standards Can Be Supplied Explicitly in Sources 4.2.3 Information in Sources 4.2.4 Selections, Notes, and Tags 5 Analyzing Information-Enriched Trace Data 6 Conclusion 6.1 Next Steps References Chapter 12: Multimodal Measures Characterizing Collaborative Groups’ Interaction and Engagement in Learning 1 Introduction 1.1 Engagement in Collaborative Learning 1.2 Cognitive and Socio-Emotional Interaction Reflecting Students’ Engagement in Collaborative Learning 2 Studying Cognitive and Socio-Emotional Interactions as Part of Collaborative Engagement with Multimodal Data 2.1 Socio-Emotional Interaction Facilitates the Emergence of Group-Level Regulation 2.2 Cognitive Interaction Supports the Function of Group-Level Regulation 2.3 Case Example – Analysis of Interactions in Engaged Collaboration 2.3.1 Data Collection 2.3.2 Analysis Protocol 3 Building Collaborative Engagement in Group Interaction – A Multimodal Data Case Example 4 Practical Implications and Future Potential of the Research Reviewed References Chapter 13: Electrodermal Activity Wearables and Wearable Cameras as Unobtrusive Observation Devices in Makerspaces 1 Introduction 2 Related Literature 2.1 Engagement 2.2 Wearable Electrodermal Activity Sensing 3 Empirical Person-in-Context Research with EDA 4 Study 1: EDA and Wearable Still Image Cameras in a Maker Project 5 Study 2: EDA Referenced Engagement in Two Maker Camps 6 Study 3: EDA Referenced Engagement in an Extended Museum-Based Afterschool Maker Program 7 Summary References Chapter 14: Collecting Unobtrusive Data: What Are the Current Challenges? 1 A Critical Overview of the Chapters 2 Using Multimodal Data for Unobtrusive Measurement of Learning: Where Are We Now? References Index