Better Deep Learning: Train Faster, Reduce Overfitting, and Make Better Predictions

Description

Copyright Contents Preface Introduction Welcome Framework for Better Deep Learning Diagnostic Learning Curves I Better Learning Improve Learning by Understanding Optimization Neural Nets Learn a Mapping Function Learning Network Weights Is Hard Key Features of the Error Surface Navigating the Non-Convex Error Surface Implications for Training Components of the Learning Algorithm Further Reading Summary Configure Capacity with Nodes and Layers Neural Network Model Capacity Nodes and Layers Keras API Model Capacity Case Study Extensions Further Reading Summary Configure Gradient Precision with Batch Size Batch Size and Gradient Descent Gradient Descent Keras API Batch Size Case Study Extensions Further Reading Summary Configure What to Optimize with Loss Functions Loss Functions Regression Loss Functions Case Study Binary Classification Loss Functions Case Study Multiclass Classification Loss Functions Case Study Extensions Further Reading Summary Configure Speed of Learning with Learning Rate Learning Rate Learning Rate Keras API Learning Rate Case Study Extensions Further Reading Summary Stabilize Learning with Data Scaling Data Scaling Data Scaling scikit-learn API Data Scaling Case Study Extensions Further Reading Summary Fix Vanishing Gradients with ReLU Vanishing Gradients and ReLU ReLU Keras API ReLU Case Study Extensions Further Reading Summary Fix Exploding Gradients with Gradient Clipping Exploding Gradients and Clipping Gradient Clipping Keras API Gradient Clipping Case Study Extensions Further Reading Summary Accelerate Learning with Batch Normalization Batch Normalization Batch Normalization Keras API Batch Normalization Case Study Extensions Further Reading Summary Deeper Models with Greedy Layer-Wise Pretraining Greedy Layer-Wise Pretraining Greedy Layer-Wise Pretraining Case Study Extensions Further Reading Summary Jump-Start Training with Transfer Learning Transfer Learning Transfer Learning Case Study Extensions Further Reading Summary II Better Generalization Fix Overfitting with Regularization Problem of Model Generalization and Overfitting Reduce Overfitting by Constraining Complexity Regularization Methods for Neural Networks Regularization Recommendations Further Reading Summary Penalize Large Weights with Weight Regularization Weight Regularization Weight Regularization Keras API Weight Regularization Case Study Extensions Further Reading Summary Sparse Representations with Activity Regularization Activity Regularization Activity Regularization Keras API Activity Regularization Case Study Extensions Further Reading Summary Force Small Weights with Weight Constraints Weight Constraints Weight Constraints Keras API Weight Constraints Case Study Extensions Further Reading Summary Decouple Layers with Dropout Dropout Dropout Keras API Dropout Case Study Extensions Further Reading Summary Promote Robustness with Noise Noise Regularization Noise Regularization Keras API Noise Regularization Case Study Extensions Further Reading Summary Halt Training at the Right Time with Early Stopping Early Stopping Early Stopping Keras API Early Stopping Case Study Extensions Further Reading Summary III Better Predictions Reduce Model Variance with Ensemble Learning High Variance of Neural Network Models Reduce Variance Using an Ensemble of Models How to Ensemble Neural Network Models Summary of Ensemble Techniques Further Reading Summary Combine Models From Multiple Runs with Model Averaging Ensemble Model Averaging Ensemble Ensembles in Keras Model Averaging Ensemble Case Study Extensions Further Reading Summary Contribute Proportional to Trust with Weighted Average Ensemble Weighted Average Ensemble Weighted Average Ensemble Case Study Extensions Further Reading Summary Fit Models on Different Samples with Resampling Ensembles Resampling Ensembles Resampling Ensembles Case Study Extensions Further Reading Summary Models from Contiguous Epochs with Horizontal Voting Ensembles Horizontal Voting Ensemble Horizontal Voting Ensembles Case Study Extensions Further Reading Summary Cyclic Learning Rate and Snapshot Ensembles Snapshot Ensembles Snapshot Ensembles Case Study Extensions Further Reading Summary Learn to Combine Predictions with Stacked Generalization Ensemble Stacked Generalization Ensemble Stacked Generalization Ensemble Case Study Extensions Further Reading Summary Combine Model Parameters with Average Model Weights Ensemble Average Model Weight Ensemble Average Model Weight Ensemble Case Study Extensions Further Reading Summary IV Appendix Getting Help Applied Neural Networks Official Keras Destinations Where to Get Help with Keras How to Ask Questions Contact the Author How to Setup Your Workstation Overview Download Anaconda Install Anaconda Start and Update Anaconda Install Deep Learning Libraries Further Reading Summary V Conclusions How Far You Have Come