Nested algorithms for optimal reservoir operation and their embedding in a decision support platform

Description

Reservoir operation is a multi-objective optimization problem, and is traditionally solved with dynamic programming (DP) and stochastic dynamic programming (SDP) algorithms. The thesis presents novel algorithms for optimal reservoir operation, named nested DP (nDP), nested SDP (nSDP), nested reinforcement learning (nRL) and their multi-objective (MO) variants, correspondingly MOnDP, MOnSDP and MOnRL. The idea is to include a nested optimization algorithm into each state transition, which reduces the initial problem dimension and alleviates the curse of dimensionality. These algorithms can solve multi-objective optimization problems, without significantly increasing the algorithm complexity or the computational expenses. It can additionally handle dense and irregular variable discretization. All algorithms are coded in Java and were tested on the case study of the Knezevo reservoir in the Republic of Macedonia. Nested optimization algorithms are embedded in a cloud application platform for water resources modeling and optimization. The platform is available 24/7, accessible from everywhere, scalable, distributed, interoperable, and it creates a real-time multiuser collaboration platform. This thesis contributes with new and more powerful algorithms for an optimal reservoir operation and cloud application platform. All source codes are available for public use and can be used by researchers and practitioners to further advance the mentioned areas. Content: 1 INTRODUCTION 1.1 Motivation 1.2 Problem description 1.2.1 Optimal reservoir operation 1.2.2 Development of a cloud decision support platform 1.3 Research objectives 1.4 Outline of the thesis 2 OPTIMAL RESERVOIR OPERATION: THE MAIN APPROACHES RELEVANT FOR THIS STUDY 2.1 Mathematical formulation of reservoir optimization problem 2.2 Dynamic programming 2.3 Stochastic dynamic programming 2.4 Reinforcement learning 2.5 Approaches to multi-objective optimization 2.5.1 Multi-objective optimization by a sequence of single-objective optimization searches 2.5.2 Multi-objective and multi-agent reinforcement learning 2.6 Conclusions 3 NESTED OPTIMIZATION ALGORITHMS 3.1 Nested dynamic programming (nDP) algorithm 3.2 Nested optimization algorithms 3.2.1 Linear formulation 3.2.2 Non-linear formulation 3.3 Nested stochastic dynamic programming (nSDP) algorithm 3.4 Nested reinforcement learning (nRL) algorithm 3.5 Multi-objective nested algorithms 3.6 Synthesis: methodology and experimental workflow 3.7 Conclusions 4 CASE STUDY: ZLETOVICA HYDRO SYSTEM OPTIMIZATION PROBLEM 4.1 General description 4.2 Zletovica river basin 4.3 Zletovica hydro system 4.4 Optimization problem formulation 4.4.1 Decision variables 4.4.2 Constraints 4.4.3 Aggregated objective function 4.4.4 Objectives weights magnitudes 4.5 Conclusions 5 ALGORITHMS IMPLEMENTATION ISSUES 5.1 nDP implementation 5.2 nSDP implementation 5.2.1 Implementation issues 5.2.2 Transition matrices 5.2.1 Optimal number of clusters 5.3 nRL implementation 5.3.1 nRL design and memory implications 5.3.2 nRL parameters 5.3.3 Agent starting state, action list and convergence criteria 5.4 Conclusions 6 EXPERIMENTS, RESULTS AND DISCUSSION 6.1 Experiments with nDP using monthly data 6.2 Comparison of nDP with other DP algorithms 6.2.1 nDP compared with a classical DP algorithm 6.2.2 nDP compared with an aggregated water demand DP algorithm 6.3 Experiments with nDP using weekly data 6.4 Experiments with nSDP and nRL using weekly data and their comparison to nDP 6.5 Identification of optimal solutions in multi-objective setting using MOnDP, MOnSDP and MOnRL 6.6 Conclusions 7 CLOUD DECISION SUPPORT PLATFORM 7.1 Background 7.2 Architecture and implementation 7.2.1 Data infrastructure web service 7.2.2 Web service for support of Water Resources Modelling 7.2.3 Web service for water resources optimization 7.2.4 Web service for user management 7.3 Results and tests 7.4 Discussion 7.5 Conclusion 8 CONCLUSIONS AND RECOMMENDATIONS 8.1 Summary 8.2 Conclusions 8.2.1 Conclusions concerning the algorithms 8.2.2 Conclusions concerning the decision support platform 8.3 Recommendations