Expert Python Programming

Description

Become an ace Python programmer by learning best coding practices and advance-level concepts with Python 3.5 About This Book Based on the latest stable version of Python (version 3.5) Creating well manageable code that will run in various environments with different sets of dependencies Packed with advanced concepts and best practices to write efficient Python code Who This Book Is For The book would appeal to web developers and Python programmers who want to start using version 3.5 and write code efficiently. Basic knowledge of Python programming is expected. What You Will Learn Conventions and best practices that are widely adopted in the python community Package python code effectively for community and production use Easy and lightweight ways to automate code deployment on remote systems Improve your code's quality, reliability, and performance Write concurrent code in python Extend python with code written in different languages In Detail Python is a dynamic programming language, used in a wide range of domains by programmers who find it simple, yet powerful. Even if you find writing Python code easy, writing code that is efficient and easy to maintain and reuse is a challenge. The focus of the book is to familiarize you with common conventions, best practices, useful tools and standards used by python professionals on a daily basis when working with code. You will begin with knowing new features in Python 3.5 and quick tricks for improving productivity. Next, you will learn advanced and useful python syntax elements brought to this new version. Using advanced object-oriented concepts and mechanisms available in python, you will learn different approaches to implement metaprogramming. You will learn to choose good names, write packages, and create standalone executables easily. You will also be using some powerful tools such as buildout and vitualenv to release and deploy the code on remote servers for production use. Moving on, you will learn to effectively create Python extensions with C, C++, cython, and pyrex. The important factors while writing code such as code management tools, writing clear documentation, and test-driven development are also covered. You will now dive deeper to make your code efficient with general rules of optimization, strategies for finding bottlenecks, and selected tools for application optimization. By the end of the book, you will be an expert in writing efficient and maintainable code. Style and approach An easy-to-follow guide that covers industry followed best practices in Python programming" Cover Copyright Credits About the Authors About the Reviewer www.PacktPub.com Table of Contents Preface Chapter 1: Current Status of Python Where are we now and where we are going? Why and how does Python change? Getting up to date with changes – PEP documents Python 3 adoption at the time of writing this book The main differences between Python 3 and Python 2 Why should I care? The main syntax differences and common pitfalls Syntax changes Changes in the standard library Changes in datatypes and collections The popular tools and techniques used for maintaining cross-version compatibility Not only CPython Why should I care? Stackless Python Jython IronPython PyPy Modern approaches to Python development Application-level isolation of Python environments Why isolation? Popular solutions virtualenv venv buildout Which one to choose? System-level environment isolation Virtual development environments using Vagrant Containerization versus virtualization Popular productivity tools Custom Python shells – IPython, bpython, ptpython, and so on Setting up the PYTHONSTARTUP environment variable IPython bpython ptpython Interactive debuggers Useful resources Summary Chapter 2: Syntax Best Practices – below the Class Level Python's built-in types Strings and bytes Implementation details String concatenation Collections Lists and tuples Dictionaries Sets Beyond basic collections – the collections module Advanced syntax Iterators The yield statement Decorators General syntax and possible implementations Usage and useful examples Context managers – the with statement General syntax and possible implementations Other syntax elements you may not know yet The for … else … statement Function annotations The general syntax The possible uses Summary Chapter 3: Syntax Best Practices – above the Class Level Subclassing built-in types Accessing methods from superclasses Old-style classes and super in Python 2 Understanding Python's Method Resolution Order super pitfalls Mixing super and explicit class calls Heterogeneous arguments Best practices Advanced attribute access patterns Descriptors Real-life example – lazily evaluated attributes Properties Slots Metaprogramming Decorators – a method of metaprogramming Class decorators Using the __new__() method to override instance creation process Metaclasses The general syntax New Python 3 syntax for metaclasses Metaclass usage Metaclass pitfalls Some tips on code generation exec, eval, and compile Abstract Syntax Tree Projects using code generation patterns Summary Chapter 4: Choosing Good Names PEP 8 and naming best practices Why and when to follow PEP 8? Beyond PEP 8 – team-specific style guidelines Naming styles Variables Constants Naming and usage Public and private variables Functions and methods The private controversy Special methods Arguments Properties Classes Modules and packages The naming guide Using the has or is prefix for Boolean elements Using plurals for variables that are collections Using explicit names for dictionaries Avoiding generic names Avoiding existing names Best practices for arguments Building arguments by iterative design Trust the arguments and your tests Using *args and **kwargs magic arguments carefully Class names Module and package names Useful tools Pylint pep8 and flake8 Summary Chapter 5: Writing a Package Creating a package The confusing state of Python packaging tools The current landscape of Python packaging thanks to PyPA Tool recommendations Project configuration setup.py setup.cfg MANIFEST.in Most important metadata Trove classifiers Common patterns The custom setup command Working with packages during development setup.py install Uninstalling packages setup.py develop or pip -e Namespace packages Why is it useful? PEP 420 – implicit namespace packages Namespace packages in previous Python versions Uploading a package PyPI – Python Package Index Uploading to PyPI – or other package index .pypirc Source packages versus built packages sdist bdist and wheels Standalone executables When are standalone executables useful? Popular tools PyInstaller cx_Freeze py2exe and py2app Security of Python code in executable packages Making decompilation harder Summary Chapter 6: Deploying Code The Twelve-Factor App Deployment automation using Fabric Your own package index or index mirror PyPI mirroring Deployment using a package Common conventions and practices The filesystem hierarchy Isolation Using process supervision tools Application code should be run in user space Using reverse HTTP proxies Reloading processes gracefully Code instrumentation and monitoring Logging errors – sentry/raven Monitoring system and application metrics Dealing with application logs Basic low-level log practices Tools for log processing Summary Chapter 7: Python Extensions in Other Languages Different language means – C or C++ How do extensions in C or C++ work Why you might want to use extensions Improving performance in critical code sections Integrating existing code written in different languages Integrating third-party dynamic libraries Creating custom datatypes Writing extensions Pure C extensions A closer look at Python/C API Calling and binding conventions Exception handling Releasing GIL Reference counting Cython Cython as a source to source compiler Cython as a language Challenges Additional complexity Debugging Interfacing with dynamic libraries without extensions ctypes Loading libraries Calling C functions using ctypes Passing Python functions as C callbacks CFFI Summary Chapter 8: Managing Code Version control systems Centralized systems Distributed systems Distributed strategies Centralized or distributed? Use Git if you can Git flow and GitHub flow Continuous development processes Continuous integration Testing every commit Merge testing through CI Matrix testing Continuous delivery Continuous deployment Popular tools for continuous integration Jenkins Buildbot Travis CI GitLab CI Choosing the right tool and common pitfalls Problem 1 – too complex build strategies Problem 2 – too long building time Problem 3 – external job definitions Problem 4 – lack of isolation Summary Chapter 9: Documenting Your Project The seven rules of technical writing Write in two steps Target the readership Use a simple style Limit the scope of information Use realistic code examples Use a light but sufficient approach Use templates A reStructuredText primer Section structure Lists Inline markup Literal block Links Building the documentation Building the portfolio Design Usage Operations Making your own portfolio Building the landscape Producer's layout Consumer's layout Documentation building and continuous integration Summary Chapter 10: Test-Driven Development I don't test Test-driven development principles Preventing software regression Improving code quality Providing the best developer documentation Producing robust code faster What kind of tests? Acceptance tests Unit tests Functional tests Integration tests Load and performance testing Code quality testing Python standard test tools unittest doctest I do test unittest pitfalls unittest alternatives nose py.test Testing coverage Fakes and mocks Building a fake Using mocks Testing environment and dependency compatibility Dependency matrix testing Document-driven development Writing a story Summary Chapter 11: Optimization – General Principles and Profiling Techniques The three rules of optimization Make it work first Work from the user's point of view Keep the code readable and maintainable Optimization strategy Find another culprit Scale the hardware Writing a speed test Finding bottlenecks Profiling CPU usage Macro-profiling Micro-profiling Measuring Pystones Profiling memory usage How Python deals with memory Profiling memory C code memory leaks Profiling network usage Summary Chapter 12: Optimization – Some Powerful Techniques Reducing the complexity Cyclomatic complexity The big O notation Simplifying Searching in a list Using a set instead of a list Using collections deque defaultdict namedtuple Using architectural trade-offs Using heuristics and approximation algorithms Using task queues and delayed processing Using probabilistic data structures Caching Deterministic caching Nondeterministic caching Cache services Memcached Summary Chapter 13: Concurrency Why concurrency? Multithreading What is multithreading? How Python deals with threads When should threading be used? Building responsive interfaces Delegating work Multiuser applications An example of a threaded application Multiprocessing The built-in multiprocessing module Using process pools Using multiprocessing.dummy as a multithreading interface Asynchronous programming Cooperative multitasking and asynchronous I/O Python async and await keywords asyncio in older versions of Python A practical example of asynchronous programming Integrating nonasynchronous code with async using futures Executors and futures Using executors in an event loop Summary Chapter 14: Useful Design Patterns Creational patterns Singleton Structural patterns Adapter Interfaces Proxy Facade Behavioral patterns Observer Visitor Template Summary Index