Mastering Python for data science : explore the world of data science through Python and learn how to make sense of data

Description

About This BookMaster data science methods using Python and its librariesCreate data visualizations and mine for patternsAdvanced techniques for the four fundamentals of Data Science with Python - data mining, data analysis, data visualization, and machine learningWho This Book Is For If you are a Python developer who wants to master the world of data science then this book is for you. Some knowledge of data science is assumed. What You Will LearnManage data and perform linear algebra in PythonDerive inferences from the analysis by performing inferential statisticsSolve data science problems in PythonCreate high-end visualizations using PythonEvaluate and apply the linear regression technique to estimate the relationships among variables.Build recommendation engines with the various collaborative filtering algorithmsApply the ensemble methods to improve your predictionsWork with big data technologies to handle data at scaleIn Detail Data science is a relatively new knowledge domain which is used by various organizations to make data driven decisions. Data scientists have to wear various hats to work with data and to derive value from it. The Python programming language, beyond having conquered the scientific community in the last decade, is now an indispensable tool for the data science practitioner and a must-know tool for every aspiring data scientist. Using Python will offer you a fast, reliable, cross-platform, and mature environment for data analysis, machine learning, and algorithmic problem solving. This comprehensive guide helps you move beyond the hype and transcend the theory by providing you with a hands-on, advanced study of data science. Beginning with the essentials of Python in data science, you will learn to manage data and perform linear algebra in Python. You will move on to deriving inferences from the analysis by performing inferential statistics, and mining data to reveal hidden patterns and trends. You will use the matplot library to create high-end visualizations in Python and uncover the fundamentals of machine learning. Next, you will apply the linear regression technique and also learn to apply the logistic regression technique to your applications, before creating recommendation engines with various collaborative filtering algorithms and improving your predictions by applying the ensemble methods. Cover Copyright Credits About the Author About the Reviewers www.PacktPub.com Table of Contents Preface Chapter 1: Getting Started with Raw Data The world of arrays with NumPy Creating an array Mathematical operations Array subtraction Squaring an array A trigonometric function performed on the array Conditional operations Matrix multiplication Indexing and slicing Shape manipulation Empowering data analysis with pandas The data structure of pandas Series DataFrame Panel Inserting and exporting data CSV XLS JSON Database Data cleansing Checking the missing data Filling the missing data String operations Merging data Data operations Aggregation operations Joins The inner join The left outer join The full outer join The groupby function Summary Chapter 2: Inferential Statistics Various forms of distribution A normal distribution A normal distribution from a binomial distribution A Poisson distribution A Bernoulli distribution A z-score A p-value One-tailed and two-tailed tests Type 1 and Type 2 errors A confidence interval Correlation Z-test vs T-test The F distribution The chi-square distribution Chi-square for the goodness of fit The chi-square test of independence ANOVA Summary Chapter 3: Finding a Needle in a Haystack What is data mining? Presenting an analysis Studying the Titanic Which passenger class has the maximum number of survivors? What is the distribution of survivors based on gender among the various classes? What is the distribution of nonsurvivors among the various classes who have family aboard the ship? What was the survival percentage among different age groups? Summary Chapter 4: Making Sense of Data through Advanced Visualization Controlling the line properties of a chart Using keyword arguments Using the setter methods Using the setp() command Creating multiple plots Playing with text Styling your plots Box plots Heatmaps Scatter plots with histograms A scatter plot matrix Area plots Bubble charts Hexagon bin plots Trellis plots A 3D plot of a surface Summary Chapter 5: Uncovering Machine Learning Different types of machine learning Supervised learning Unsupervised learning Reinforcement learning Decision trees Linear regression Logistic regression The naive Bayes classifier The k-means clustering Hierarchical clustering Summary Chapter 6: Performing Predictions with a Linear Regression Simple linear regression Multiple regression Training and testing a model Summary Chapter 7: Estimating the Likelihood of Events Logistic regression Data preparation Creating training and testing sets Building a model Model evaluation Evaluating a model based on test data Model building and evaluation with SciKit Summary Chapter 8: Generating Recommendations with Collaborative Filtering Recommendation data User-based collaborative filtering Finding similar users The Euclidean distance score The Pearson correlation score Ranking the users Recommending items Item-based collaborative filtering Summary Chapter 9: Pushing Boundaries with Ensemble Models The census income dataset Exploring the census data Hypothesis 1: People who are older earn more Hypothesis 2: Income bias based on working class Hypothesis 3: People with more education earn more Hypothesis 4: Married people tend to earn more Hypothesis 5: There is a bias in income based on race Hypothesis 6: There is a bias in the income based on occupation Hypothesis 7: Men earn more Hypothesis 8: People who clock in more hours earn more Hypothesis 9: There is a bias in income based on the country of origin Decision trees Random forests Summary Chapter 10: Applying Segmentation with k-means Clustering The k-means algorithm and its working A simple example The k-means clustering with countries Determining the number of clusters Clustering the countries Summary Chapter 11: Analyzing Unstructured Data with Text Mining Preprocessing data Creating a wordcloud Word and sentence tokenization Parts of speech tagging Stemming and lemmatization Stemming Lemmatization The Stanford Named Entity Recognizer Performing sentiment analysis on world leaders using Twitter Summary Chapter 12: Leveraging Python in the World of Big Data What is Hadoop? The programming model The MapReduce architecture The Hadoop DFS Hadoop's DFS architecture Python MapReduce The basic word count A sentiment score for each review The overall sentiment score Deploying the MapReduce code on Hadoop File handling with Hadoopy Pig Python with Apache Spark Scoring the sentiment The overall sentiment Summary Index