Explainable Human-ai Interaction: A Planning Perspective

Description

From its inception, artificial intelligence (AI) has had a rather ambivalent relationship with humans—swinging between their augmentation and replacement. Now, as AI technologies enter our everyday lives at an ever-increasing pace, there is a greater need for AI systems to work synergistically with humans. One critical requirement for such synergistic human?AI interaction is that the AI systems' behavior be explainable to the humans in the loop. To do this effectively, AI agents need to go beyond planning with their own models of the world, and take into account the mental model of the human in the loop. At a minimum, AI agents need approximations of the human's task and goal models, as well as the human's model of the AI agent's task and goal models. The former will guide the agent to anticipate and manage the needs, desires and attention of the humans in the loop, and the latter allow it to act in ways that are interpretable to humans (by conforming to their mental models of it), and be ready to provide customized explanations when needed. The authors draw from several years of research in their lab to discuss how an AI agent can use these mental models to either conform to human expectations or change those expectations through explanatory communication. While the focus of the book is on cooperative scenarios, it also covers how the same mental models can be used for obfuscation and deception. The book also describes several real-world application systems for collaborative decision-making that are based on the framework and techniques developed here. Although primarily driven by the authors' own research in these areas, every chapter will provide ample connections to relevant research from the wider literature. The technical topics covered in the book are self-contained and are accessible to readers with a basic background in AI. Preface Acknowledgments Introduction Humans and AI Agents: An Ambivalent Relationship Explanations in Humans When and Why Do Humans Expect Explanations from Each Other? How Do Humans Exchange Explanations? (Why) Should AI Systems Be Explainable? Dimensions of Explainable AI systems Use Cases for Explanations in Human–AI Interaction Requirements on Explanations Explanations as Studied in the AI Literature Explainable AI: The Landscape and The Tribes Our Perspective on Human-Aware and Explainable AI Agents How Do We Make AI Agents Human-Aware? Mental Models in Explainable AI Systems Overview of This Book Measures of Interpretability Planning Models Modes of Interpretable Behavior Explicability Legibility Predictability Communication to Improve Interpretability Communicating Model Information Other Considerations in Interpretable Planning Generalizing Interpretability Measures Bibliographic Remarks Explicable Behavior Generation Explicable Planning Problem Model-Based Explicable Planning Plan Generation Through Reconciliation Search Possible Distance Functions Model-Free Explicable Planning Problem Formulation Learning Approach Plan Generation Environment Design for Explicability Problem Setting Framework for Design for Explicability Search for Optimal Design Demonstration of Environment Design for Explicability Bibliographic Remarks Legible Behavior Controlled Observability Planning Problem Human's Belief Space Computing Solutions to COPP Variants Variants of COPP Goal Legibility Computing Goal Legible Plans Plan Legibility Computing Plan Legible Plans Bibliographic Remarks Explanation as Model Reconciliation Model-Reconciliation as Explanation The Fetch Domain Explanation Generation Explanation Types Desiderata for Explanations as Discussed in Social Sciences Model Space Search for Minimal Explanations Approximate Explanations Explicit Contrastive Explanations Approximate MCE User Studies Other Explanatory Methods Bibliographic Remarks Acquiring Mental Models for Explanations The Urban Search and Reconnaissance Domain Model Uncertainty Model-Free Explanations Assuming Prototypical Models Bibliographic Remarks Balancing Communication and Behavior Modified USAR Domain Balancing Explanation and Explicable Behavior Generation Generating Balanced Plans Stage of Interaction and Epistemic Side Effects Optimizing for Explicability of the Plan Balancing Communication and Behavior For Other Measures Bibliographic Remarks Explaining in the Presence of Vocabulary Mismatch Representation of Robot Model Setting Local Approximation of Planning Model Montezuma's Revenge Acquiring Interpretable Models Query-Specific Model Acquisition Explanation Generation Identifying Explanations Through Sample-Based Trials Explanation Confidence Handling Uncertainty in Concept Mapping Acquiring New Vocabulary Bibliographic Remarks Obfuscatory Behavior and Deceptive Communication Obfuscation Goal Obfuscation Secure Goal Obfuscation Plan Obfuscation Deception Multi-Observer Simultaneous Obfuscation and Legibility Mixed-Observer Controlled Observability Planning Problem Plan Computation Lies When Should an Agent Lie? How Can an Agent Lie? Implications of Lies Bibliographical Remarks Applications Collaborative Decision-Making Humans as Actors RADAR MA-RADAR RADAR-X Model Transcription Assistants D3WA+ Bibliographic Remarks Conclusion Bibliography Authors' Biographies Index