The Sonification Handbook

Description

This book is a comprehensive introductory presentation of the key research areas in the interdisciplinary fields of sonification and auditory display. Chapters are written by leading experts, providing a wide-ranging coverage of the central issues, and can be read from start to finish, or dipped into as required. Sonification conveys information by using non-speech sounds. To listen to data as sound and noise can be a surprising new experience with diverse applications ranging from novel interfaces for visually impaired people to data analysis problems in many scientific fields. This book gives a solid introduction to the field of auditory display, the techniques for sonification, suitable technologies for developing sonification algorithms, and the most promising application areas. The book is accompanied by an online repository of sound examples. Introduction Auditory Display and Sonification The Potential of Sonification and Auditory Display Structure of the book How to Read Fundamentals of Sonification, Sound and Perception Theory of Sonification Chapter Overview Sonification and Auditory Displays Towards a Taxonomy of Auditory Display & Sonification Data Properties and Task Dependency Representation and Mappings Limiting Factors for Sonification: Aesthetics, Individual Differences, and Training Conclusions: Toward a Cohesive Theoretical Account of Sonification Psychoacoustics Introduction The transduction of mechanical sound energy into biological signals in the auditory nervous system The perception of loudness The perception of pitch The perception of temporal variation Grouping spectral components into auditory objects and streams The perception of space Summary Further reading Perception, Cognition and Action in Auditory Displays Introduction Perceiving Auditory Dimensions Auditory-Visual Interaction Auditory Space and Virtual Environments Space as a Dimension for Data Representation Rhythm and Time as Dimensions for Auditory Display Auditory Scene Analysis Auditory Cognition Summary Sonic Interaction Design Introduction A psychological perspective on sonic interaction Product sound design Interactive art and music Sonification and Sonic Interaction Design Open challenges in SID Evaluation of Auditory Display Chapter Overview General Experimental Procedures Data Collection Methods for Evaluating Perceptual Qualities and Relationships among Auditory Stimuli Analysis of Data Obtained from Identification, Attribute Rating, Discrimination, and Dissimilarity Rating Tasks Using ``Distance'' Data Obtained by Dissimilarity Ratings, Sorting, and Other Tasks Usability Testing Issues and Active Use Experimental Procedures Conclusion Sonification Design and Aesthetics Background Design Aesthetics: sensuous perception Towards an aesthetic of sonification Where do we go from here? Sonification Technology Statistical Sonification for Exploratory Data Analysis Introduction Datasets and Data Analysis Methods Sonifications of Iris Dataset Discussion Conclusion and Caveat Sound Synthesis for Auditory Display Introduction and Chapter Overview Parametric vs. Non-Parametric Models Digital Audio: The Basics of PCM Fourier (Sinusoidal) ``Synthesis'' Modal (Damped Sinusoidal) Synthesis Subtractive (Source-Filter) synthesis Time Domain Formant Synthesis Waveshaping and FM Synthesis Granular and PhISEM Synthesis Physical Modeling Synthesis Non-Linear Physical Models Synthesis for Auditory Display, Conclusion Laboratory Methods for Experimental Sonification Programming as an interface between theory and laboratory practice Overview of languages and systems SuperCollider: Building blocks for a sonification laboratory Example laboratory workflows and guidelines for working on sonification designs Coda: back to the drawing board Interactive Sonification Chapter Overview What is Interactive Sonification? Principles of Human Interaction Musical instruments -- a 100,000 year case study A brief History of Human Computer Interaction Interacting with Sonification Guidelines & Research Agenda for Interactive Sonification Conclusions Sonification Techniques Audification Introduction Brief Historical Overview (before ICAD, 1800-1991) Methods of Audification Audification now (1992-today) Conclusion: What audification should be used for Towards Better Audification Tools Auditory Icons Auditory icons and the ecological approach Auditory icons and events Applications using auditory icons Designing auditory icons Conclusion Earcons Introduction Initial Earcon Research Creating Earcons Earcons and Auditory Icons Using Earcons Future Directions Conclusions Parameter Mapping Sonification Introduction Data Features Connecting Data and Sound Mapping Topology Signal and Sound Listening, Thinking, Tuning Integrating Perception in PMSon Auditory graphs Vowel / Formant based PMSon Features of PMSon Design Challenges of PMSon Synthesis and signal processing methods used in PMSon Artistic applications of PMSon Conclusion Model-Based Sonification Introduction Definition of Model-Based Sonification Sonification Models MBS Use and Design Guidelines Interaction in Model-Based Sonification Applications Discussion Conclusion Applications Auditory Display in Assistive Technology Introduction The Power of Sound Visually Disabled People Computer Access Electronic Travel Aids Other Systems Discussion Conclusion Sonification for Process Monitoring Types of monitoring --- basic categories Modes of Listening Environmental awareness (workspaces and living spaces) Monitoring program execution Monitoring interface tasks Potential pitfalls The road ahead Intelligent auditory alarms Introduction The concept of auditory alarms Problems linked to non-speech auditory alarm design Acoustic properties of non-speech sound alarms A cognitive approach to the problem Spatialization of alarms Contribution of learning Ergonomic approach to the problem Intelligent alarm systems Conclusion Navigation of Data Navigation Control Loop Wayfinding Methods For Navigating Through Data Using Auditory Displays For Navigation Of Data Considerations for the Design of Auditory Displays for the Navigation of Data Aiding Movement with Sonification in ``Exercise, Play and Sport'' Multidisciplinary Applications of Sonification in the Field of ``Exercise, Play and Sport'' Use of Sound for Physiotherapy Analysis and Feedback Interaction with Sound in auditory computer games Sonification-based Sport games and Performance Tests in Adapted Physical Activity Enhancing Motor Control and Learning by Additional Movement Sonification Concluding Remarks Index