Mastering Python

Description

Cover Copyright Credits About the Author About the Reviewers www.PacktPub.com Table of Contents Preface Chapter 1: Getting Started – One Environment per Project Creating a virtual Python environment using venv Creating your first venv venv arguments Differences between virtualenv and venv Bootstrapping pip using ensurepip ensurepip usage Manual pip install Installing C/C++ packages Debian and Ubuntu Red Hat, CentOS, and Fedora OS X Windows Summary Chapter 2: Pythonic Syntax, Common Pitfalls, and Style Guide Code style – or what is Pythonic code? Formatting strings – printf-style or str.format? PEP20, the Zen of Python Beautiful is better than ugly Explicit is better than implicit Simple is better than complex Flat is better than nested Sparse is better than dense Readability counts Practicality beats purity Errors should never pass silently In the face of ambiguity, refuse the temptation to guess One obvious way to do it Now is better than never Hard to explain, easy to explain Namespaces are one honking great idea Conclusion Explaining PEP8 Duck typing Differences between value and identity comparisons Loops Maximum line length Verifying code quality, pep8, pyflakes, and more flake8 Pylint Common pitfalls Scope matters! Function arguments Class properties Modifying variables in the global scope Overwriting and/or creating extra built-ins Modifying while iterating Catching exceptions – differences between Python 2 and 3 Late binding – be careful with closures Circular imports Import collisions Summary Chapter 3: Containers and Collections – Storing Data the Right Way Time complexity – the big O notation Core collections list – a mutable list of items dict – unsorted but a fast map of items set – like a dict without values tuple – the immutable list Advanced collections ChainMap – the list of dictionaries counter – keeping track of the most occurring elements deque – the double ended queue defaultdict – dictionary with a default value namedtuple – tuples with field names enum – a group of constants OrderedDict – a dictionary where the insertion order matters heapq – the ordered list bisect – the sorted list Summary Chapter 4: Functional Programming – Readability versus Brevity Functional programming list comprehensions dict comprehensions set comprehensions lambda functions The Y combinator functools partial – no need to repeat all arguments every time reduce – combining pairs into a single result Implementing a factorial function Processing trees itertools accumulate – reduce with intermediate results chain – combining multiple results combinations – combinatorics in Python permutations – combinations where the order matters compress – selecting items using a list of Booleans dropwhile/takewhile – selecting items using a function count – infinite range with decimal steps groupby – grouping your sorted iterable islice – slicing any iterable Summary Chapter 5: Decorators – Enabling Code Reuse by Decorating Decorating functions Why functools.wraps is important How are decorators useful? Memoization using decorators Decorators with (optional) arguments Creating decorators using classes Decorating class functions Skipping the instance – classmethod and staticmethod Properties – smart descriptor usage Decorating classes Singletons – classes with a single instance Total ordering – sortable classes the easy way Useful decorators Single dispatch – polymorphism in Python Contextmanager, with statements made easy Validation, type checks, and conversions Useless warnings – how to ignore them Summary Chapter 6: Generators and Coroutines – Infinity, One Step at a Time What are generators? Advantages and disadvantages of generators Pipelines – an effective use of generators tee – using an output multiple times Generating from generators Context managers Coroutines A basic example Priming Closing and throwing exceptions Bidirectional pipelines Using the state Summary Chapter 7: Async IO – Multithreading without Threads Introducing the asyncio library The async and await statements Python 3.4 Python 3.5 Choosing between the 3.4 and 3.5 syntax A simple example of single-threaded parallel processing Concepts of asyncio Futures and tasks Event loops Processes Asynchronous servers and clients Basic echo server Summary Chapter 8: Metaclasses – Making Classes (Not Instances) Smarter Dynamically creating classes A basic metaclass Arguments to metaclasses Accessing metaclass attributes through classes Abstract classes using collections.abc Internal workings of the abstract classes Custom type checks Using abc.ABC before Python 3.4 Automatically registering a plugin system Importing plugins on-demand Importing plugins through configuration Importing plugins through the file system Order of operations when instantiating classes Finding the metaclass Preparing the namespace Executing the class body Creating the class object (not instance) Executing the class decorators Creating the class instance Example Storing class attributes in definition order The classic solution without metaclasses Using metaclasses to get a sorted namespace Summary Chapter 9: Documentation – How to Use Sphinx and reStructuredText The reStructuredText syntax Getting started with reStructuredText Inline markup Headers Lists Enumerated list Bulleted list Option list Definition list Nested lists Links, references, and labels Images Substitutions Blocks, code, math, comments, and quotes Conclusion The Sphinx documentation generator Getting started with Sphinx Using sphinx-quickstart Using sphinx-apidoc Sphinx directives The table of contents tree directive (toctree) Autodoc, documenting Python modules, classes, and functions Sphinx roles Documenting code Documenting a class with the Sphinx style Documenting a class with the Google style Documenting a class with the NumPy style Which style to choose Summary Chapter 10: Testing and Logging – Preparing for Bugs Using examples as tests with doctest A simple doctest example Writing doctests Testing with pure documentation The doctest flags True and False versus 1 and 0 Normalizing whitespace Ellipsis Doctest quirks Testing dictionaries Testing floating-point numbers Times and durations Testing with py.test The difference between the unittest and py.test output The difference between unittest and py.test tests Simplifying assertions Parameterizing tests Automatic arguments using fixtures Print statements and logging Plugins Mock objects Using unittest.mock Using py.test monkeypatch Logging Configuration Basic logging configuration Dictionary configuration JSON configuration Ini file configuration The network configuration Logger Usage Summary Chapter 11: Debugging – Solving the Bugs Non-interactive debugging Inspecting your script using trace Debugging using logging Showing call stack without exceptions Debugging asyncio Handling crashes using faulthandler Interactive debugging Console on demand Debugging using pdb Breakpoints Catching exceptions Commands Debugging using ipdb Other debuggers Debugging services Summary Chapter 12: Performance – Tracking and Reducing your Memory and CPU Usage What is performance? Timeit – comparing code snippet performance cProfile – finding the slowest components First profiling run Calibrating your profiler Selective profiling using decorators Using profile statistics Line profiler Improving performance Using the right algorithm Global interpreter lock Try versus if Lists versus generators String concatenation Addition versus generators Map versus generators and list comprehensions Caching Lazy imports Using optimized libraries Just-in-time compiling Converting parts of your code to C Memory usage Tracemalloc Memory profiler Memory leaks Reducing memory usage Generators versus lists Recreating collections versus removing items Using slots Performance monitoring Summary Chapter 13: Multiprocessing – When a Single CPU Core Is not Enough Multithreading versus multiprocessing Hyper-threading versus physical CPU cores Creating a pool of workers Sharing data between processes Remote processes Distributed processing using multiprocessing Distributed processing using IPyparallel ipython_config.py ipython_kernel_config.py ipcontroller_config.py ipengine_config.py ipcluster_config.py Summary Chapter 14: Extensions in C/C++, System Calls, and C/C++ Libraries Introduction Do you need C/C++ modules? Windows OS X Linux/Unix Calling C/C++ with ctypes Platform-specific libraries Windows Linux/Unix OS X Making it easy Calling functions and native types Complex data structures Arrays Gotchas with memory management CFFI Complex data structures Arrays ABI or API? CFFI or ctypes? Native C/C++ extensions A basic example C is not Python – size matters The example explained static PyObject* Parsing arguments C is not Python – errors are silent or lethal Calling Python from C – handling complex types Summary Chapter 15: Packaging – Creating Your Own Libraries or Applications Installing packages Setup parameters Packages Entry points Creating global commands Custom setup.py commands Package data Testing packages Unittest py.test Nosetests C/C++ extensions Regular extensions Cython extensions Wheels – the new eggs Distributing to the Python Package Index Summary Index