Malware Data Science: Attack Detection and Attribution

Description

Malware Data Science explains how to identify, analyze, and classify large-scale malware using machine learning and data visualization. Security has become a "big data" problem. The growth rate of malware has accelerated to tens of millions of new files per year while our networks generate an ever-larger flood of security-relevant data each day. In order to defend against these advanced attacks, you'll need to know how to think like a data scientist. In Malware Data Science, security data scientist Joshua Saxe introduces machine learning, statistics, social network analysis, and data visualization, and shows you how to apply these methods to malware detection and analysis. You'll learn how to: - Analyze malware using static analysis - Observe malware behavior using dynamic analysis - Identify adversary groups through shared code analysis - Catch 0-day vulnerabilities by building your own machine learning detector - Measure malware detector accuracy - Identify malware campaigns, trends, and relationships through data visualization Whether you're a malware analyst looking to add skills to your existing arsenal, or a data scientist interested in attack detection and threat intelligence, Malware Data Science will help you stay ahead of the curve. Title Page Copyright Page Dedication About the Authors About the Technical Reviewer BRIEF CONTENTS CONTENTS IN DETAIL FOREWORD by Anup Ghosh ACKNOWLEDGMENTS INTRODUCTION What Is Data Science? Why Data Science Matters for Security Applying Data Science to Malware Who Should Read This Book? About This Book How to Use the Sample Code and Data 1 BASIC STATIC MALWARE ANALYSIS The Microsoft Windows Portable Executable Format Dissecting the PE Format Using pefile Examining Malware Images Examining Malware Strings Summary 2 BEYOND BASIC STATIC ANALYSIS: X86 DISASSEMBLY Disassembly Methods Basics of x86 Assembly Language Disassembling ircbot.exe Using pefile and capstone Factors That Limit Static Analysis Summary 3 A BRIEF INTRODUCTION TO DYNAMIC ANALYSIS Why Use Dynamic Analysis? Dynamic Analysis for Malware Data Science Basic Tools for Dynamic Analysis Limitations of Basic Dynamic Analysis Summary 4 IDENTIFYING ATTACK CAMPAIGNS USING MALWARE NETWORKS Nodes and Edges Bipartite Networks Visualizing Malware Networks Building Networks with NetworkX Adding Nodes and Edges Network Visualization with GraphViz Building Malware Networks Building a Shared Image Relationship Network Summary 5 SHARED CODE ANALYSIS Preparing Samples for Comparison by Extracting Features Using the Jaccard Index to Quantify Similarity Using Similarity Matrices to Evaluate Malware Shared Code Estimation Methods Building a Similarity Graph Scaling Similarity Comparisons Building a Persistent Malware Similarity Search System Running the Similarity Search System Summary 6 UNDERSTANDING MACHINE LEARNING–BASED MALWARE DETECTORS Steps for Building a Machine Learning–Based Detector Understanding Feature Spaces and Decision Boundaries What Makes Models Good or Bad: Overfitting and Underfitting Major Types of Machine Learning Algorithms Summary 7 EVALUATING MALWARE DETECTION SYSTEMS Four Possible Detection Outcomes Considering Base Rates in Your Evaluation Summary 8 BUILDING MACHINE LEARNING DETECTORS Terminology and Concepts Building a Toy Decision Tree–Based Detector Building Real-World Machine Learning Detectors with sklearn Building an Industrial-Strength Detector Evaluating Your Detector’s Performance Next Steps Summary 9 VISUALIZING MALWARE TRENDS Why Visualizing Malware Data Is Important Understanding Our Malware Dataset Using matplotlib to Visualize Data Using seaborn to Visualize Data Summary 10 DEEP LEARNING BASICS What Is Deep Learning? How Neural Networks Work Training Neural Networks Types of Neural Networks Summary 11 BUILDING A NEURAL NETWORK MALWARE DETECTOR WITH KERAS Defining a Model’s Architecture Compiling the Model Training the Model Evaluating the Model Enhancing the Model Training Process with Callbacks Summary 12 BECOMING A DATA SCIENTIST Paths to Becoming a Security Data Scientist A Day in the Life of a Security Data Scientist Traits of an Effective Security Data Scientist Where to Go from Here APPENDIX AN OVERVIEW OF DATASETS AND TOOLS Overview of Datasets Tool Implementation Guide Index