Autonomous Real-Time Testing: Testing Artificial Intelligence and Other Complex Systems

Description

Software testing is becoming increasingly important because more and more products are software-intensive. Cars, for example, contain more and more control software (ECUs) that are networked with each other. With new rail vehicles, software problems delay commissioning by months, even years, because the different components are not coordinated with each other. A timely system test would help, but there is a lack of time and resources. The functionality of the software is simply too great. So, you must automate. Automation is not only necessary for the execution of tests, but above all for the generation of suitable test cases. This is possible with Combinatory Logic, the Analytic Hierarchy Process (AHP), and Quality Function Deployment (QFD). When today's cars use map services from the cloud, or their own sensors, for an Advanced Driving Assistance System (ADAS) to perform driving decisions; or when in the future an autonomous car meets another; or with truck platooning; or when adding a new, previously unknown device to an IoT orchestra, the original base system expands its functionality. Therefore, such an expanding system needs being retested before it can do decisions with the potential of affecting harm to humans or things, after each update, after each learning. This is Continuous Testing during operation; it supplements Continuous Delivery and Continuous Integration. Disruptive innovations in automotive require an equally disruptive new approach to testing of software-intense systems. This requires moving from once-upon-a-time testing before release to autonomous real-time software & systems testing during operations, with indications to users and suppliers about the actual state and testing results. This book explains the theory and the implementation approach for a framework for Autonomous Real-time Testing (ART) of a software-intense system while in operation. Chapter 1: Why Autonomous Real-time Testing? 1-1 Introduction 1-2 What is Software Testing? 1-3 Representing Unlimited Knowledge 1-4 Autonomous Real-time Testing 1-5 Outlook Chapter 2: Test Metrics 2-1 Introduction 2-2 Modeling Software 2-3 A Short Primer on Six Sigma Transfer Functions 2-4 Measuring Tests 2-5 Test Metrics for the Navigator Application 2-6 Conclusion Chapter 3: Testing the Internet of Things 3-1 Introduction 3-2 Testing the Internet of Things (IoT) 3-3 Conclusions and Next Steps Chapter 4: Testing Privacy Protection and Safety Risks 4-1 Introduction 4-2 Consumer Metrics 4-3 ART for ADAS 4-4 Conclusion Chapter 5: Artificial Intelligence for Testing 5-1 What is the Goal of Testing? 5-2 Generating New Test Cases 5-3 The Test Case Generator 5-4 Three Standard Tests 5-5 The DevOps Paradigm and Software Testing 5-6 Three Innovations needed Chapter 6: Testing Highly Complex Technical Systems 6-1 Testing Digital Twins 6-2 The Fundamentals of Testing Complex Systems 6-3 AHP for Testing 6-4 Open Questions 6-5 Conclusion Chapter 7: Testing Artificial Intelligence 7-1 Introduction 7-2 How to Test Artificial Intelligence 7-3 A Deep Learning Application as a Sample 7-4 Next Steps, and a Preliminary Conclusion 7-5 A Side Note Chapter 8: Agile Testing with the Buglione-Trudel Matrix 8-1 Introduction 8-2 Story Cards with Test Stories 8-3 Selecting Test Stories for Story Cards 8-4 Creating Test Stories by the Development Team 8-5 Test Management 8-6 Conclusions Bibliography Reference Index