Spatially Explicit Hyperparameter Optimization for Neural Networks

Description

Neural networks as the commonly used machine learning algorithms, such as artificial neural networks (ANNs) and convolutional neural networks (CNNs), have been extensively used in the GIScience domain to explore the nonlinear and complex geographic phenomena. However, there are a few studies that investigate the parameter settings of neural networks in GIScience. Moreover, the model performance of neural networks often depends on the parameter setting for a given dataset. Meanwhile, adjusting the parameter configuration of neural networks will increase the overall running time. Therefore, an automated approach is necessary for addressing these limitations in current studies. This book proposes an automated spatially explicit hyperparameter optimization approach to identify optimal or near-optimal parameter settings for neural networks in the GIScience field. Also, the approach improves the computing performance at both model and computing levels. This book is written for researchers of the GIScience field as well as social science subjects. Preface Acknowledgements Contents List of Figures List of Tables 1 Introduction 1.1 Background 1.2 Research Objectives 1.2.1 Objective 1 1.2.2 Objective 2 1.2.3 Objective 3 References 2 Literature Review 2.1 Artificial Neural Network 2.2 Hyperparameter Optimization 2.3 Cyberinfrastructure and High-Performance and Parallel Computing 2.4 Evolutionary Algorithms References 3 Methodology 3.1 Overview 3.2 Component 1—Automatic Search of Hyperparameters 3.3 Component 2—Spatial Prediction of Hyperparameter Space 3.4 Component 3—Acceleration of Hyperparameter Search References 4 Study I. Hyperparameter Optimization of Neural Network-Driven Spatial Models Accelerated Using Cyber-Enabled High-Performance Computing 4.1 Introduction 4.2 Literature Review 4.2.1 Artificial Neural Networks 4.2.2 Hyperparameter Optimization 4.3 Study Area and Data 4.4 Methodology 4.4.1 Land Price Evaluation Model 4.4.2 Hyperparameter Optimization 4.4.3 Determining Optimal Sample Size 4.4.4 Parallel Computing and Implementation 4.5 Results 4.5.1 Results of Grid Search and Random Search 4.5.2 Prediction Performance of Hyperparameters 4.5.3 Parallel Computing Performance 4.6 Discussions 4.6.1 Necessity of the Framework 4.6.2 Feasibility of the Framework 4.6.3 Computing Performance 4.7 Conclusion References 5 Study II. Spatially Explicit Hyperparameter Optimization of Neural Networks Accelerated Using High-Performance Computing 5.1 Introduction 5.2 Study Area and Data 5.3 Methodology 5.4 Implementation 5.5 Results 5.5.1 Model Performance 5.5.2 Prediction Performance of Hyperparameters 5.5.3 Parallel Computing Performance 5.6 Discussions 5.6.1 The Prediction of Generalization Performance 5.6.2 Computing Performance 5.7 Conclusion References 6 Study III. An Integration of Spatially Explicit Hyperparameter Optimization with Convolutional Neural Networks-Based Spatial Models 6.1 Introduction 6.2 Hyperparameters of Convolutional Neural Networks 6.3 Study Area and Data 6.4 Experimental Design 6.4.1 Setting of CNN Model 6.4.2 CNN-Based Cellular Automata 6.4.3 Implementation 6.5 Results 6.5.1 Accuracy Assessment 6.5.2 Model Performance 6.5.3 Generalization Performance of Hyperparameters 6.5.4 Prediction Performance 6.5.5 Parallel Computing Performance 6.6 Discussions 6.6.1 The Simulation Performance of CNN-CA Model 6.6.2 Computing Performance 6.7 Conclusion References 7 Conclusion