Fast Python for Data Science

Description

Master these effective techniques to reduce costs and run times, handle huge datasets, and implement complex machine learning applications efficiently in Python. Fast Python for Data Science is a hands-on guide to writing Python code that can process more data, faster, and with less resources. It takes a holistic approach to Python performance, showing you how your code, libraries, and computing architecture interact and can be optimized together. Written for experienced practitioners, Fast Python for Data Science dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in lower-level Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. Fast Python for Data Science MEAP V03 Copyright welcome brief contents Chapter 1: The need for efficient computing and data storage 1.1 The overwhelming need for efficient computing in Python 1.2 The impact of modern computing architectures on high performance computing 1.2.1 Changes inside the computer 1.2.2 Changes in the network 1.2.3 The cloud 1.3 Working with Python’s limitations 1.3.1 The Global Interpreter Lock (GIL) 1.4 What will you learn from this book 1.5 The reader for this book 1.6 Summary Chapter 2: Extracting maximum performance from built-in features 2.1 Introducing the project dataset 2.1.1 An architecture for big data processing 2.1.2 Preparing the data 2.2 Profiling code to detect performance bottlenecks 2.2.1 Using Python’s built-in profiling module 2.2.2 Visualizing profiling information 2.2.3 Line profiling 2.3 Optimizing basic data structures for speed: lists, sets, dictionaries 2.3.1 Performance of list searches 2.3.2 Using the bisect module 2.3.3 Content aware search approaches 2.3.4 Searching using sets or dictionaries 2.3.5 List complexity in Python 2.4 Finding excessive memory allocation 2.4.1 Navigating the minefield of Python memory estimation 2.4.2 Using more compact representations 2.4.3 Packing many observations in a number 2.4.4 Use the array module 2.4.5 Systematizing what we have learned: Estimating the memory usage of Python objects 2.5 Using laziness and generators for big-data pipelining 2.5.1 Using generators instead of standard functions 2.5.2 Enabling code pipelining with generators 2.6 Summary Chapter 3: Concurrency, parallelism and asynchronous processing in Python 3.1 Writing the scaffold of an asynchronous server 3.1.1 The implementation of the scaffold for communicating with clients 3.1.2 Programming with coroutines 3.1.3 Sending complex data from simple synchronous client 3.1.4 Alternative approaches 3.2 Implementing the first MapReduce engine 3.2.1 Understanding MapReduce frameworks 3.2.2 Developing a very simple test scenario 3.2.3 Implementing a too-simple MapReduce framework 3.3 Implementing a concurrent version of a MapReduce engine 3.3.1 Using concurrent.futures to implement a threaded server 3.3.2 Asynchronous execution with Futures 3.3.3 The GIL and multi-threading 3.4 Using multi-processing to implement MapReduce 3.4.1 A solution based on concurrent.futures 3.4.2 A solution based on the multiprocessing module 3.4.3 Monitoring the progress of the multiprocessing solution 3.4.4 Transferring data in chunks 3.5 Tying it all together: an asynchronous multi-threaded and multi-processing MapReduce server 3.5.1 Architecting a complete high-performance solution 3.5.2 Creating a robust version of the server 3.6 Summary Chapter 4: Using NumPy more efficiently 4.1 Understanding NumPy from a performance perspective 4.1.1 Copies and views 4.1.2 Understanding NumPy’s view machinery 4.1.3 Making use of views for efficiency 4.2 Using array programming 4.2.1 Broadcasting in NumPy 4.2.2 Applying array programming to image manipulation 4.2.3 Developing a "vectorized mentality" 4.3 Tuning NumPy’s internal architecture for performance 4.3.1 An overview of NumPy dependencies 4.3.2 How to tune NumPy in your Python distribution 4.3.3 Threads in NumPy 4.4 Summary Appendix: Setting up the environment A.1 Code style and organization Notes