Adaptive Learning of Polynomial Networks. Genetic Programming, Backpropagation and Bayesian Methods

Description

Contents......Page 3 Preface......Page 8 1 Introductin......Page 11 Inductive Learning......Page 13 Evolutionary Search......Page 26 STROGANOFF and its Variants......Page 27 Organization of the Book......Page 33 2 Inductive Genetic Programming......Page 35 PNN Approaches......Page 37 Tree-structured PNN......Page 39 Biological Interpretation......Page 46 Context-preserving Mutation......Page 48 Size-biasing of the Genetic Operators......Page 51 Random Tree Generation......Page 56 Schema Theorem of IGP......Page 60 Chapter Summary......Page 64 3 Tree-like PNN Representations......Page 65 Discrete Volterra Series......Page 66 Errors of Approximation Approximation Error Bounds......Page 69 Kernel PNN Models......Page 76 Orthogonal PNN Models......Page 79 Trigonometric PNN Models......Page 81 Chapter Summary......Page 90 4 Fitness Functions & Landscapes......Page 91 Fitness Functions......Page 93 Static Fitness Functions......Page 94 Fitness Magnitude......Page 104 Statistical Correlation Measures......Page 106 Information Measures......Page 114 Quantitative Measures......Page 117 5 Search Navigation......Page 121 The Reproduction Operator......Page 122 Selection Strategies......Page 123 Replacement Strategies......Page 127 Implementing Reproduction......Page 128 Performance Examination......Page 138 Chapter Summary......Page 156 6 Backpropagation Techniques......Page 157 First-Order Backpropagation......Page 159 Batch Backpropagation......Page 167 Incremental Backpropagation......Page 168 Control of the Learning Step......Page 169 Second-Order Backpropagation......Page 173 Second-Order Error Derivatives......Page 174 Newton's Method......Page 179 Conjugate Gradients......Page 180 Levenberg-Marquardt Method......Page 181 First-Order Network Pruning......Page 186 Second-Order Network Pruning......Page 187 Chapter Summary......Page 189 7 Temporal Backpropagation......Page 191 Recurrent PNN as State-Space Models......Page 192 Backpropagation Through Time......Page 194 Real-Time BPTT Algorithm......Page 199 Epochwise BPTT Algorithm......Page 200 Real-Time Recurrent Learning......Page 201 Subgrouping......Page 209 Second-Order Temporal BP......Page 210 Recurrent Network Optimization......Page 216 Recurrent Network Pruning......Page 217 Chapter Summary......Page 218 8 Bayesian Inference Techniques......Page 219 Bayesian Error Function......Page 221 Deriving Hyperparameters......Page 225 Evidence Procedure for PNN Models......Page 228 Choosing a Weight Prior......Page 231 Sequential Dynamic Hessian Estimation......Page 240 Sequential Hyperparameter Estimation......Page 242 Hybrid Sampling Resampling......Page 247 Chapter Summary......Page 249 9 Statistical Model Diagnostics......Page 251 Residual Bootstrap Sampling......Page 253 Bias/Variance Dilemma......Page 254 Interval Estimation by the Delta Method......Page 258 Bootstrapping Confidence Intervals......Page 262 Prediction Intervals......Page 264 Analytical Prediction Intervals......Page 265 Empirical Learning of Prediction Bars......Page 266 Chapter Summary......Page 281 10 Time Series Modelling......Page 283 Data Preprocessing......Page 286 PNN vs. Genetically Programmed Functions......Page 289 PNN vs. Statistical Learning Networks......Page 291 PNN vs. Kernel Models......Page 295 Recurrent PNN vs. Recurrent Neural Networks......Page 298 Chapter Summary......Page 300 11 Conclusions......Page 301 Refs......Page 305 Index......Page 322