Imaging and Sensing for Unmanned Aircraft Systems: Control and Performance (Control, Robotics and Sensors)

Description

This two-volume book set explores how sensors and computer vision technologies are used for the navigation, control, stability, reliability, guidance, fault detection, self-maintenance, strategic re-planning and reconfiguration of unmanned aircraft systems (UAS). Volume 1 concentrates on UAS control and performance methodologies including Computer Vision and Data Storage, Integrated Optical Flow for Detection and Avoidance Systems, Navigation and Intelligence, Modeling and Simulation, Multisensor Data Fusion, Vision in Micro-Aerial Vehicles (MAVs), Computer Vision in UAV using ROS, Security Aspects of UAV and Robot Operating System, Vision in Indoor and Outdoor Drones, Sensors and Computer Vision, and Small UAV for Persistent Surveillance. Volume 2 focuses on UAS deployment and applications including UAV-CPSs as a Testbed for New Technologies and a Primer to Industry 5.0, Human-Machine Interface Design, Open Source Software (OSS) and Hardware (OSH), Image Transmission in MIMO-OSTBC System, Image Database, Communications Requirements, Video Streaming, and Communications Links, Multispectral vs Hyperspectral Imaging, Aerial Imaging and Reconstruction of Infrastructures, Deep Learning as an Alternative to Super Resolution Imaging, and Quality of Experience (QoE) and Quality of Service (QoS). Cover Contents About the editors Preface 1 Introduction to advances in UAV avionics for imaging and sensing 1.1 Basic concepts 1.2 Navigation and intelligence 1.3 Communications 1.4 Sensors 1.5 Computational aspects: image/video processing, computer graphics, modelling, and visualisation 1.6 Security, health, and standards 1.7 Applications 1.8 Book organization References 2 Computer vision and data storage in UAVs 2.1 Introduction 2.1.1 Requirements 2.1.2 Root file system 2.1.3 Data logging 2.1.4 Cloud support and virtualisation 2.2 The architecture of the cloud-based UAV cyber-physical system 2.3 UAV needs versus memory use 2.3.1 Limitations of OVP 2.3.2 General solutions and their viability analysis 2.4 UAV data logging 2.5 Types of data logging 2.5.1 Requirements and recommended solutions 2.5.2 Internal RAM with SD 2.5.3 External RAM with SD 2.5.4 External flash memory 2.6 Discussion and future trends 2.6.1 UAV-based data storage 2.6.2 UAV-based data processing 2.6.3 Distributed versus centralised control 2.6.4 Impact of big data in UAV-CPSs 2.6.4.1 Infrastructure readiness 2.6.4.2 Complexity 2.6.4.3 Privacy 2.6.4.4 Barriers to BD processing in UAV-CPSs 2.6.5 Challenges related to privacy and the protection of personal information 2.6.6 Organisational and cultural barriers 2.7 Conclusions References 3 Integrated optical flow for situation awareness, detection and avoidance systems in UAV systems 3.1 Introduction 3.2 Computer vision 3.2.1 Optical Flow 3.2.1.1 Methods based on the brightness gradient 3.2.1.2 Feature extractor algorithm 3.3 Optical flow and remote sensing 3.3.1 Aerial Triangulation 3.4 Optical flow and situational awareness 3.4.1 Detect and avoidance system 3.4.1.1 Perception 3.4.1.2 Comprehension 3.4.1.3 Projection 3.5 Optical flow and navigation by images 3.5.1 Egomotion 3.6 Case study: INS using FPGA 3.6.1 Architectural proposals 3.6.1.1 Control unit (CU) 3.6.1.2 Generation of time 3.6.1.3 Feature points detector 3.6.1.4 OF calculation 3.6.1.5 Input and output component 3.6.2 Integration INS/GPS/OF using a Kalman filter 3.7 Future trends and discussion 3.7.1 3D optical flow 3.7.2 Multispectral and hyperspectral images 3.8 Conclusion References 4 Introduction to navigation and intelligence for UAVs relying on computer vision 4.1 Introduction 4.2 Basic terminology 4.2.1 Visual servoing 4.2.2 Visual odometry 4.2.3 Terrain-referenced visual navigation 4.3 Future trends and discussion 4.4 Conclusions References 5 Modelling and simulation of UAV systems 5.1 Need for modelling and simulation 5.1.1 Control systems design 5.1.2 Operator training 5.1.3 Sub-system development and testing 5.2 History and adoption 5.2.1 Early aviation 5.2.2 First computerised simulations 5.2.3 Entry of UAVs into service 5.2.4 Commercial and consumer drones 5.3 Modelling of UAV dynamics 5.3.1 Model representation methods 5.3.1.1 Differential equations 5.3.1.2 State-space representation 5.3.2 Common reference frames 5.3.2.1 Inertial frame of reference 5.3.2.2 Earth-centre frames of reference 5.3.2.3 Navigation frame of reference 5.3.2.4 Body frames of reference 5.3.3 Representation of state variables 5.3.3.1 Euler angles 5.3.3.2 Rotation matrices 5.3.3.3 Quaternions 5.3.4 Deriving the system equations of motion 5.3.4.1 Conservation of momentum 5.3.4.2 Euler–Lagrange method 5.3.4.3 Newton–Euler recursive method 5.3.5 Flight physics models 5.3.5.1 Fixed-wing flight 5.3.5.2 Multi-rotors and VTOL 5.4 Flight dynamics simulation 5.4.1 Integration of the equations of motion 5.4.1.1 Euler method 5.4.1.2 Runga–Kutta methods 5.5 Conclusion References 6 Multisensor data fusion for vision-based UAV navigation and guidance 6.1 Introduction 6.2 Data-fusion algorithms 6.2.1 Extended Kalman filter 6.2.2 Unscented Kalman filter 6.2.3 Integration architectures 6.3 Fusion of visual sensors References 7 Vision-based UAV pose estimation 7.1 Introduction 7.2 INS–GNSS drawbacks 7.2.1 Inertial navigation systems 7.2.2 Global navigation satellites systems 7.3 Visual navigation: A viable alternative 7.4 Visual navigation strategies 7.4.1 Photogrammetry: Extracting pose information from images 7.4.2 Template matching 7.4.3 Landmark recognition 7.4.3.1 Knowing the exact landmark 7.4.3.2 Identifying the landmarks' classes 7.4.4 Visual odometry 7.4.5 Combination of methods 7.5 Future developments on visual navigation systems 7.6 Conclusion References 8 Vision in micro-aerial vehicles 8.1 Introduction 8.1.1 Fixed-wing MAVs 8.1.1.1 Longitudinal dynamics 8.1.1.2 Lateral dynamic 8.1.2 Rotary-wing MAVs 8.1.3 Flapping-wing or biomimetic MAVs 8.1.4 Hybrid MAVs 8.2 Computer vision as a biological inspiration 8.3 The role of sensing in MAVs 8.3.1 Pose-estimation sensors 8.3.2 Environmental awareness sensors 8.3.3 Sonar ranging sensor 8.3.4 Infrared-range sensors 8.3.5 Thermal imaging 8.3.6 LIDAR 8.3.7 Cameras 8.4 Illumination 8.5 Navigation, pathfinding, and orientation 8.6 Communication and polarisation-inspired machine vision applications 8.6.1 Robot orientation and navigation 8.6.2 Polarisation-opponent sensors 8.7 CCD cameras and applications in machine vision 8.8 Error modelling of environments with uncertainties 8.9 Further work and future trends 8.9.1 MAV challenges 8.9.2 Proposed solutions for MAV design challenges 8.9.3 New frontiers in sensors 8.10 Conclusion References 9 Computer vision in UAV using ROS 9.1 Introduction 9.2 Computer vision on ROS 9.3 Applications 9.3.1 OpenCV in ROS 9.3.1.1 Object detection 9.3.2 Visual navigation 9.3.2.1 Parallel tracking and mapping (PTAM) 9.3.2.2 ROS package –autonomous flight 9.3.2.3 tum ardrone GUI 9.3.2.4 PTAM UAV camera feed and navigation 9.3.3 Setting the drone state estimation node 9.3.3.1 Simple navigation 9.4 Future developments and trends in ROS 9.5 Conclusion References 10 Security aspects of UAV and robot operating system 10.1 Introduction 10.2 Unmanned aerial vehicles 10.3 ROS basic concepts 10.4 Security UAV review 10.5 Security ROS review 10.6 UAV security scenarios 10.7 Security assessment on consumer UAV operation with ROS 10.8 Future trends 10.9 Conclusion References 11 Vision in indoor and outdoor drones 11.1 Computer vision in unmanned aerial vehicles 11.1.1 Indoor environments 11.1.2 Outdoor environments 11.2 Other approaches handling both indoor and outdoor environments 11.3 Conclusion References 12 Sensors and computer vision as a means to monitor and maintain a UAV structural health 12.1 Introduction 12.1.1 Case study: aeroelastic instability flutter phenomenon 12.2 Related work 12.2.1 Structural health monitoring 12.2.2 Computer vision for structural health 12.2.3 Flutter certification 12.2.4 Computer vision and in in-flight measurements: future trends 12.3 Signal processing on flutter certification 12.4 Experiments and results 12.4.1 Synthetic data 12.4.1.1 Model of the typical wing section 12.4.1.2 Pre-processing 12.4.1.3 Extraction of dynamic characteristics 12.4.1.4 Results for synthetic data 12.4.2 Wind tunnel experiment 12.4.2.1 Experiment description 12.4.2.2 Results for experimental data 12.5 Discussion 12.5.1 Computer vision 12.6 Final remarks References 13 Small UAV: persistent surveillance made possible 13.1 Introduction 13.2 System view 13.2.1 System description 13.2.2 Hardware components 13.2.3 Components recommendation 13.3 Software components 13.3.1 Camera calibration 13.3.2 Image stitching 13.3.3 Stabilization 13.3.4 Background subtraction 13.3.5 Object tracking 13.3.6 Geo-location pointing 13.4 Future trends 13.5 Conclusion References 14 Conclusions Index Back Cover