Near-sensor and In-sensor Computing

Description

This book provides a detailed introduction to near-sensor and in-sensor computing paradigms, their working mechanisms, development trends and future directions.  The authors also provide a comprehensive review of current progress in this area, analyze existing challenges in the field, and offer possible solutions.  Readers will benefit from the discussion of computing approaches that intervene in the vicinity of or inside sensory networks to help process data more efficiently, decreasing power consumption and reducing the transfer of redundant data between sensing and processing units. Provides readers with a detailed introduction to the near-sensor and in-sensor computing paradigms;Includes in-depth and comprehensive summaries of the state-of-the-art development in this field;Discusses and compares various neuromorphic sensors and neural networks:Describes integration technology for near-/in-sensor computing;Reveals the relationship between near-/in-sensor computing and other computing paradigms, such as neuromorphic computing, edge computing, intuitive computing, and in-memory computing. Contents Chapter 1: Neuromorphic Computing Based on Memristor Dynamics 1.1 Introduction 1.2 Artificial Synapses 1.2.1 Long-Term Plasticity 1.2.2 Short-Term Plasticity 1.3 Artificial Neuron 1.3.1 H-H Neuron 1.3.2 LIF Neurons 1.3.3 Oscillation Neuron 1.3.4 Artificial Dendrites 1.4 Memristor-Based Neuromorphic Computing Systems 1.4.1 Memristive Reservoir Computing Systems 1.4.2 Memristor-Based Coupled Oscillator Network 1.4.3 Memristor-Based Continuous Attractor Neural Network 1.4.4 Memristive Spiking Neural Network 1.4.5 Memristor-Based Chaotic Computing 1.5 Conclusions and Outlook References Chapter 2: Short-Term Plasticity in 2D Materials for Neuromorphic Computing 2.1 Introduction 2.2 Sound Localization via 2D Synaptic Devices 2.2.1 Short-Term Plasticity 2.2.2 Joule Heating for STP 2.2.3 Tunable STP in 2D Material-Based Synaptic Devices 2.2.4 Synaptic Computation for Sound Localization 2.2.5 2D Materials Engineering for Optimized STP 2.3 In-Sensor Reservoir Computing for Language Learning via 2D Memristors 2.3.1 Recurrent Neural Networks and Reservoir Computing 2.3.2 SnS Memristor for Optoelectronic RC 2.3.3 Spatiotemporal Signal Processing in a Circuit Based on SnS Memristors 2.3.4 Optoelectronic RC for the Learning of the Korean Language 2.3.5 Inference of Korean Sentences via Optoelectronic RC 2.4 Conclusion and Outlook References Chapter 3: Bioinspired In-Sensor Computing Devices for Visual Adaptation 3.1 Introduction 3.2 The Visual Adaptation of the Retina 3.3 In-Sensor Visual Adaptation Based on Emerging Devices 3.3.1 Two-Terminal Optoelectronic Devices 3.3.2 Three-Terminal Phototransistors 3.3.3 Optoelectronic Circuits 3.4 Conclusion and Future Prospects References Chapter 4: Neuromorphic Vision Based on van der Waals Heterostructure Materials 4.1 Introduction 4.2 Retinomorphic Sensor 4.2.1 Mimicking Retinal Cells with Vertical Heterostructure Devices 4.2.2 Reconfigurable Retinomorphic Vision Sensor 4.3 Neuromorphic Vision System 4.3.1 The Architecture of Neuromorphic Vision System 4.3.2 The Function of Neuromorphic Vision System 4.4 Conclusion References Chapter 5: Neuromorphic Vision Chip 5.1 Introduction 5.2 Vision Chip Architectures 5.2.1 Early Frame-Driven Vision Chips 5.2.2 Dynamically Reconfigurable Vision Chips 5.2.2.1 Architecture Design of Dynamically Reconfigurable Vision Chips 5.2.2.2 Architecture Characteristics of Dynamically Reconfigurable Vision Chips 5.2.3 Convolutional Neural Network–Oriented Vision Chip Architecture 5.2.3.1 Convolutional Neural Network–Oriented Vision Chip Architecture Design 5.2.3.2 Optimization Strategy 5.2.3.3 Development Status of the Convolutional Neural Network Accelerator 5.2.4 Programmable Parallel Vision Chip 5.2.4.1 Design of Programmable Parallel Architecture 5.2.4.2 Features of Programmable Parallel Architecture 5.3 Visual Tasks and Software on Vision Chips 5.3.1 Categories of Various Visual Tasks and Model Selection 5.3.2 Network Architecture Determination 5.3.2.1 On-Chip Memory 5.3.2.2 On-Chip Computing Resources 5.3.2.3 Specialized Hardware Units 5.3.2.4 Other Constraints 5.3.3 Model Compression on Vision Chips 5.3.3.1 Lightweight Model Pruning 5.3.3.1.1 Fine-Grained Pruning and Structured Pruning 5.3.3.1.2 Regularization-Based Structured Pruning 5.3.3.2 Network Quantization 5.3.3.2.1 Post-training Quantization and Quantization-Aware Training 5.3.3.3 Precision Alignment After Quantization 5.3.3.3.1 Model Accuracy Verification on a Vision Chip Simulator 5.3.3.3.2 Solutions for Accuracy Mismatch 5.3.4 Model Mapping on Vision Chips 5.3.4.1 Dataflow on Vision Chips 5.3.4.1.1 Integer Inference Dataflow 5.3.4.1.2 Overflow Handling and Bit-Shifting 5.3.4.2 Manually Designed Operator Library 5.3.4.2.1 Interface Design of the Library 5.3.4.2.2 Performance Optimization of the Library 5.3.4.2.3 Mapping of the Model to the Library 5.4 Conclusion References Chapter 6: Collision Avoidance Systems and Emerging Bio-inspired Sensors for Autonomous Vehicles 6.1 Introduction 6.2 Sensors for CAS 6.2.1 Light Detection and Ranging (LiDAR) 6.2.2 Radio Detection and Ranging (Radar) 6.2.3 Ultrasonic Sensors 6.2.4 Image Sensors 6.3 Bio-inspired Sensors 6.4 Sensor Fusion Technology 6.5 Summary References Chapter 7: Emerging Devices for Sensing-Memory-Computing Applications 7.1 Introduction 7.1.1 History and Emerging Technology 7.1.2 The Challenge for Sensing-Memory-Computing 7.1.2.1 Material System and Device Structure 7.1.2.2 Array Integration 7.2 Metal Oxide-Based Sensing-Memory-Computing Device 7.2.1 Research Background and Status 7.2.2 Several Device Types Based on Metal Oxide Film 7.2.2.1 Single Two-Terminal Memristor 7.2.2.2 Memristor Array 7.2.3 Issues and Challenges 7.2.4 Conclusion and Outlook 7.3 Two-Dimensional Sensing-Memory-Computing Device 7.3.1 Research Background and Status 7.3.1.1 Two-Dimensional Materials 7.3.1.2 Library of 2D Materials and Their Heterostructures 7.3.1.3 Preparation of 2D Materials and Their Heterostructures 7.3.2 Several Device Types Based on Two-Dimensional Film 7.3.2.1 Charge-Based Device 7.3.2.2 Resistive Switching Device 7.3.3 Issues and Challenges 7.3.4 Conclusion and Outlook 7.4 Organic Sensing-Memory-Computing Device 7.4.1 Research Background and Status 7.4.2 Several Device Types Based on Organic Film 7.4.2.1 Tactile Sensing-Memory-Computing Device 7.4.2.2 Visual Sensing-Memory-Computing Device 7.4.2.3 Olfaction Sensing-Memory-Computing Device 7.4.2.4 Auditory Sensing-Memory-Computing Device 7.4.3 Issues and Challenges 7.4.4 Conclusion and Outlook 7.5 Phase Change Sensing-Memory-Computing Device 7.5.1 Research Background and Status 7.5.2 Several Device Types Based on Phase Change Material 7.5.3 Issues and Challenges 7.5.4 Conclusion and Outlook 7.6 Ferroelectric Sensing-Memory-Computing Device 7.6.1 Research Background and Status 7.6.2 Several Device Types Based on Ferroelectric Material 7.6.2.1 Piezoelectric Device 7.6.2.2 Logic Device 7.6.2.3 Storage Device 7.6.2.4 Optical Device 7.6.3 Issues and Challenges 7.6.4 Conclusion and Outlook 7.7 Conclusion References Chapter 8: Neural Computing with Photonic Media 8.1 Introduction 8.2 Nanophotonic Medium for Neural Computing 8.2.1 Implementation 8.2.2 Training Process 8.3 Fabrication Constraints 8.3.1 B-Splines 8.4 Neuromorphic Metasurfaces 8.4.1 System Setup 8.4.2 Training Process 8.4.3 Results 8.5 Conclusion References Chapter 9: Multimodal Sensory Computing 9.1 Introduction 9.2 Multisensory Integration Modeling 9.2.1 Optimal Cue Integration 9.2.2 Normalization Model 9.2.3 Dynamic Adjustment 9.3 Hardware Implementation 9.4 Outlook References Index