Transfer Learning

Description

Transfer learning deals with how systems can quickly adapt themselves to new situations, tasks and environments. It gives machine learning systems the ability to leverage auxiliary data and models to help solve target problems when there is only a small amount of data available. This makes such systems more reliable and robust, keeping the machine learning model faced with unforeseeable changes from deviating too much from expected performance. At an enterprise level, transfer learning allows knowledge to be reused so experience gained once can be repeatedly applied to the real world. For example, a pre-trained model that takes account of user privacy can be downloaded and adapted at the edge of a computer network. This self-contained, comprehensive reference text describes the standard algorithms and demonstrates how these are used in different transfer learning paradigms. It offers a solid grounding for newcomers as well as new insights for seasoned researchers and developers. Contents......Page 6 Preface......Page 9 PART I FOUNDATIONS OF TRANSFER LEARNING......Page 14 1.1 AI, Machine Learning and Transfer Learning......Page 16 1.2 Transfer Learning: A Definition......Page 20 1.3 Relationship to Existing Machine Learning Paradigms......Page 24 1.4 Fundamental Research Issues in Transfer Learning......Page 26 1.5 Applications of Transfer Learning......Page 27 1.6 Historical Notes......Page 30 1.7 About This Book......Page 31 2.1 Introduction......Page 36 2.2 Instance-Based Noninductive Transfer Learning......Page 38 2.3 Instance-Based Inductive Transfer Learning......Page 41 3.1 Introduction......Page 47 3.2 Minimizing the Domain Discrepancy......Page 48 3.3 Learning Universal Features......Page 54 3.4 Feature Augmentation......Page 56 4.1 Introduction......Page 58 4.2 Transfer through Shared Model Components......Page 60 4.3 Transfer through Regularization......Page 63 5.1 Introduction......Page 71 5.3 Relation-Based Transfer Learning Based on MLNs......Page 74 6.1 Introduction......Page 81 6.2 The Heterogeneous Transfer Learning Problem......Page 83 6.3 Methodologies......Page 84 6.4 Applications......Page 103 7.1 Introduction......Page 106 7.2 Generative Adversarial Networks......Page 107 7.3 Transfer Learning with Adversarial Models......Page 110 7.4 Discussion......Page 117 8.1 Introduction......Page 118 8.2 Background......Page 120 8.3 Inter-task Transfer Learning......Page 126 8.4 Inter-domain Transfer Learning......Page 135 9.1 Introduction......Page 139 9.3 Multi-task Supervised Learning......Page 141 9.4 Multi-task Unsupervised Learning......Page 150 9.6 Multi-task Active Learning......Page 151 9.8 Multi-task Online Learning......Page 152 9.10 Parallel and Distributed Multi-task Learning......Page 153 10.1 Introduction......Page 154 10.2 Generalization Bounds for Multi-task Learning......Page 155 10.3 Generalization Bounds for Supervised Transfer Learning......Page 158 10.4 Generalization Bounds for Unsupervised Transfer Learning......Page 161 11.1 Introduction......Page 164 11.2 TTL over Mixed Graphs......Page 166 11.3 TTL with Hidden Feature Representations......Page 171 11.4 TTL with Deep Neural Networks......Page 175 12.1 Introduction......Page 181 12.2 The L2T Framework......Page 182 12.3 Parameterizing What to Transfer......Page 183 12.4 Learning from Experiences......Page 184 12.6 Connections to Other Learning Paradigms......Page 187 13.1 Introduction......Page 190 13.2 Zero-Shot Learning......Page 191 13.3 One-Shot Learning......Page 197 13.4 Bayesian Program Learning......Page 200 13.5 Poor Resource Learning......Page 203 13.6 Domain Generalization......Page 206 14.1 Introduction......Page 209 14.2 Lifelong Machine Learning: A Definition......Page 210 14.3 Lifelong Machine Learning through Invariant Knowledge......Page 211 14.4 Lifelong Machine Learning in Sentiment Classification......Page 212 14.5 Shared Model Components as Multi-task Learning......Page 216 14.6 Never-Ending Language Learning......Page 217 PART II APPLICATIONS OF TRANSFER LEARNING......Page 222 15.1 Introduction......Page 224 15.2 Differential Privacy......Page 225 15.3 Privacy-Preserving Transfer Learning......Page 228 16.1 Introduction......Page 234 16.2 Overview......Page 235 16.3 Transfer Learning for Medical Image Analysis......Page 242 17.2 Transfer Learning in NLP......Page 247 17.3 Transfer Learning in Sentiment Analysis......Page 254 18.1 Introduction......Page 270 18.3 Transfer Learning in Spoken Language Understanding......Page 272 18.4 Transfer Learning in Dialogue State Tracker......Page 275 18.5 Transfer Learning in DPL......Page 276 18.6 Transfer Learning in Natural Language Generation......Page 281 18.7 Transfer Learning in End-to-End Dialogue Systems......Page 282 19.1 Introduction......Page 292 19.2 What to Transfer in Recommendation......Page 293 19.3 News Recommendation......Page 297 19.4 VIP Recommendation in Social Networks......Page 301 20.1 Introduction......Page 306 20.2 Machine Learning Problems in Bioinformatics......Page 307 20.3 Biological Sequence Analysis......Page 308 20.5 Systems Biology......Page 312 20.6 Biomedical Text and Image Mining......Page 314 20.7 Deep Learning for Bioinformatics......Page 315 21.2 Transfer Learning for Wireless Localization......Page 320 21.3 Transfer Learning for Activity Recognition......Page 329 22.1 Introduction......Page 337 22.2 “What to Transfer” in Urban Computing......Page 338 22.3 Key Issues of Transfer Learning in Urban Computing......Page 339 22.4 Chain Store Recommendation......Page 340 22.5 Air-Quality Prediction......Page 343 23 Concluding Remarks......Page 347 References......Page 349 Index......Page 390