Feature Engineering & Selection for Explainable Models

Description

I found the root cause of many challenges faced by my students who recently transitioned into data science and machine learning. I have tried to address these issues in my book and would like to dedicate this book to all my students for all the love and respect I have received. Contents Foreword Before We Start Section I: Introduction Chapter 1: Introduction 1.1 Terminology 1.1.1 Dataset, Variable, and Observation 1.1.2 Feature Engineering 1.1.3 Feature Extraction 1.1.4 Feature Selection 1.1.5 Cost Function 1.2 Process of Training a Machine Learning Model 1.3 Preventing Overfitting 1.4 Code Conventions 1.5 Datasets Used 1.5.1 Hotel Booking Demand Datasets 1.5.2 Car Sales 1.5.3 Coupon Recommendation 1.5.4 Raman Spectroscopy of Skimmed Milk Samples 1.5.5 Beaver Body Temperatures 1.6 References Section II: Feature Engineering Chapter 2: Domain Specific Feature Engineering 2.1 Introduction 2.2 Domain-Specific Feature Engineering 2.2.1 Ask Probing Questions 2.2.2 Literature Review 2.3 References Contents Foreword Before We Start Section I: Introduction Chapter 1: Introduction 1.1 Terminology 1.1.1 Dataset, Variable, and Observation 1.1.2 Feature Engineering 1.1.3 Feature Extraction 1.1.4 Feature Selection 1.1.5 Cost Function 1.2 Process of Training a Machine Learning Model 1.3 Preventing Overfitting 1.4 Code Conventions 1.5 Datasets Used 1.5.1 Hotel Booking Demand Datasets 1.5.2 Car Sales 1.5.3 Coupon Recommendation 1.5.4 Raman Spectroscopy of Skimmed Milk Samples 1.5.5 Beaver Body Temperatures 1.6 References Section II: Feature Engineering Chapter 2: Domain Specific Feature Engineering 2.1 Introduction 2.2 Domain-Specific Feature Engineering 2.2.1 Ask Probing Questions 2.2.2 Literature Review 2.3 References Chapter 3: EDA Feature Engineering 3.1 Introduction 3.2 Car Sales 3.3 Coupon Recommendation 3.4 Conclusion Chapter 4: Higher Order Feature Engineering 4.1 Engineering Categorical Features 4.1.1 Dummy Encoding or One-Hot Encoding 4.1.2 Label Encoding 4.1.3 Count, and Percentage Encoding 4.1.4 Encoding by Rank of Counts 4.1.5 Target Encoding 4.2 Engineering Ordinal Features 4.2.1 Rank Encoding 4.2.2 Polynomial Encoding 4.2.3 Backward Difference Encoding 4.3 Engineering Numerical Features 4.3.1 Binning 4.3.2 Square and Cube 4.3.3 Regression Splines 4.3.4 Square Root and Cube Root 4.3.5 Log Transformation 4.3.6 Standardization and Normalization 4.3.7 Box-cox Transformation 4.3.8 Yeo-Johnson Transformation 4.4 Conclusion Chapter 5: Interaction Effect Feature Engineering 5.1 Interaction Plot 5.2 SHAP 5.2.1 Car Sales 5.2.2 Coupon Recommendation 5.3 Putting Everything Together 5.3.1 Hotel Total Room Booking 5.3.2 Hotel Booking Cancellation 5.3.3 Car Sales 5.3.4 Coupon Recommendation 5.4 Conclusion 5.5 References Section III: Feature Selection Chapter 6: Fundamentals of Feature Selection 6.1 Introduction 6.2 Different Feature Selection Methods 6.3 Filter Method 6.4 Wrapper Method 6.4.1 Forward Selection 6.4.2 Backward Selection 6.4.3 Stepwise Selection 6.4.4 Recursive Feature Elimination 6.5 Putting Everything Together 6.5.1 Hotel Total Room Booking 6.5.2 Hotel Booking Cancellation 6.5.3 Car Sales 6.5.4 Coupon Recommendation 6.6 Conclusion Chapter 7: Feature Selection Concerning Modeling Techniques 7.1 Lasso, Ridge, and ElasticNet 7.2 Feature Importance of Tree Models 7.3 Boruta 7.4 Using Tree-Based Feature Importance for Linear Model 7.5 Using Linear Model Feature Importance for Tree Models 7.6 Linear Regression 7.7 SVM 7.8 PCA 7.9 Putting Everything Together 7.9.1 Hotel Total Room Booking 7.9.2 Hotel Booking Cancellation 7.9.3 Car Sales 7.9.4 Coupon Recommendation 7.10 Conclusion Chapter 8: Feature Selection Using Metaheuristic Algorithms 8.1 Exhaustive Feature Selection 8.2 Genetic Algorithm 8.3 Simulated Annealing 8.4 Ant Colony Optimization 8.5 Particle Swarm Optimization 8.6 Putting Everything Together 8.6.1 Hotel Total Room Booking 8.6.2 Hotel Booking Cancellation 8.6.3 Car Sales 8.6.4 Coupon Recommendation 8.7 Conclusion 8.8 References Section IV: Model Explanation Chapter 9: Explaining Model and Model Predictions to Layman 9.1 Introduction 9.2 Explainable models 9.2.1 Linear Regression 9.2.2 Logistic Regression 9.2.3 Decision Tree 9.3 Explanation Techniques 9.3.1 Explaining Overall Model 9.3.1.1 Partial Dependence Plot 9.3.1.2 Accumulated Local Effects Plot 9.3.1.3 Permutation Feature Importance 9.3.1.4 Surrogate Model 9.3.2 Explaining Individual Predictions 9.3.2.1 Individual Conditional Expectation Plots 9.3.2.2 Local interpretable model-agnostic explanations 9.3.2.3 Counterfactual Model Explanations 9.3.2.4 SHAP 9.4 Putting Everything Together 9.4.1 Hotel Total Room Booking 9.4.2 Hotel Booking Cancellation 9.5 Conclusion 9.6 References Section V: Special Chapters Chapter 10: Feature Engineering & Selection for Text Classification 10.1 Introduction 10.2 Feature Construction 10.2.1 N-gram 10.2.2 Syntactic N-gram 10.2.3 Domain-Specific Taxonomy Features 10.2.4 Meta Features 10.3 Feature Selection 10.3.1 Filter Method 10.3.1.1 Document Frequency 10.3.1.2 Chi-Square 10.3.1.3 Mutual Information 10.3.1.4 Proportional Difference 10.3.1.5 Information Gain 10.3.2 Metaheuristics Algorithms 10.3.3 Ensemble Feature Selection 10.4 Feature Extraction 10.4.1 Bag of Words 10.4.2 Term Frequency Inverse Document Frequency 10.4.3 Word2vec 10.5 Feature Reduction 10.5.1 Singular Value Decomposition 10.5.2 Non-Negative Matrix Factorization 10.6 Conclusion 10.7 References Chapter 11: Things That Can Give Additional Improvement 11.1 Introduction 11.2 Hyperparameter Tuning 11.3 Ensemble Learning 11.4 Signal Processing 11.4.1 Filtering 11.4.2 Baseline Removal 11.5 Conclusion 11.6 References