Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software

Description

Malware analysis is big business, and attacks can cost a company dearly. When malware breaches your defenses, you need to act quickly to cure current infections and prevent future ones from occurring. For those who want to stay ahead of the latest malware, Practical Malware Analysis will teach you the tools and techniques used by professional analysts. With this book as your guide, you'll be able to safely analyze, debug, and disassemble any malicious software that comes your way. You'll learn how to: Set up a safe virtual environment to analyze malware Quickly extract network signatures and host-based indicators Use key analysis tools like IDA Pro, OllyDbg, and WinDbg Overcome malware tricks like obfuscation, anti-disassembly, anti-debugging, and anti-virtual machine techniques Use your newfound knowledge of Windows internals for malware analysis Develop a methodology for unpacking malware and get practical experience with five of the most popular packers Analyze special cases of malware with shellcode, C++, and 64-bit code Hands-on labs throughout the book challenge you to practice and synthesize your skills as you dissect real malware samples, and pages of detailed dissections offer an over-the-shoulder look at how the pros do it. You'll learn how to crack open malware to see how it really works, determine what damage it has done, thoroughly clean your network, and ensure that the malware never comes back. Malware analysis is a cat-and-mouse game with rules that are constantly changing, so make sure you have the fundamentals. Whether you're tasked with securing one network or a thousand networks, or you're making a living as a malware analyst, you'll find what you need to succeed in Practical Malware Analysis. Warning About the Authors About the Technical Reviewer About the Contributing Authors Foreword Acknowledgments Individual Thanks Introduction What Is Malware Analysis? Prerequisites Practical, Hands-On Learning What’s in the Book? 0: Malware Analysis Primer The Goals of Malware Analysis Malware Analysis Techniques Basic Static Analysis Basic Dynamic Analysis Advanced Static Analysis Advanced Dynamic Analysis Types of Malware General Rules for Malware Analysis Part 1: Basic Analysis 1: Basic Static Techniques Antivirus Scanning: A Useful First Step Hashing: A Fingerprint for Malware Finding Strings Packed and Obfuscated Malware Packing Files Detecting Packers with PEiD Portable Executable File Format Linked Libraries and Functions Static, Runtime, and Dynamic Linking Exploring Dynamically Linked Functions with Dependency Walker Imported Functions Exported Functions Static Analysis in Practice PotentialKeylogger.exe: An Unpacked Executable PackedProgram.exe: A Dead End The PE File Headers and Sections Examining PE Files with PEview Viewing the Resource Section with Resource Hacker Using Other PE File Tools PE Header Summary Conclusion Lab 1-1 Questions Lab 1-2 Questions Lab 1-3 Questions Lab 1-4 Questions 2: Malware Analysis in Virtual Machines The Structure of a Virtual Machine Creating Your Malware Analysis Machine Configuring VMware Using Your Malware Analysis Machine Connecting Malware to the Internet Connecting and Disconnecting Peripheral Devices Taking Snapshots Transferring Files from a Virtual Machine The Risks of Using VMware for Malware Analysis Record/Replay: Running Your Computer in Reverse Conclusion 3: Basic Dynamic Analysis Sandboxes: The Quick-and-Dirty Approach Using a Malware Sandbox Sandbox Drawbacks Running Malware Monitoring with Process Monitor The Procmon Display Filtering in Procmon Viewing Processes with Process Explorer The Process Explorer Display Using the Verify Option Comparing Strings Using Dependency Walker Analyzing Malicious Documents Comparing Registry Snapshots with Regshot Faking a Network Using ApateDNS Monitoring with Netcat Packet Sniffing with Wireshark Using INetSim Basic Dynamic Tools in Practice Conclusion Lab 3-1 Questions Lab 3-2 Questions Lab 3-3 Questions Lab 3-4 Questions Part 2: Advanced Static Analysis 4: A Crash Course in x86 Disassembly Levels of Abstraction Reverse-Engineering The x86 Architecture Main Memory Instructions Opcodes and Endianness Operands Registers Simple Instructions The Stack Conditionals Branching Rep Instructions C Main Method and Offsets More Information: Intel x86 Architecture Manuals Conclusion 5: IDA Pro Loading an Executable The IDA Pro Interface Disassembly Window Modes Useful Windows for Analysis Returning to the Default View Navigating IDA Pro Searching Using Cross-References Code Cross-References Data Cross-References Analyzing Functions Using Graphing Options Enhancing Disassembly Renaming Locations Comments Formatting Operands Using Named Constants Redefining Code and Data Extending IDA with Plug-ins Using IDC Scripts Using IDAPython Using Commercial Plug-ins Conclusion Lab 5-1 Questions 6: Recognizing C Code Constructs in Assembly Global vs. Local Variables Disassembling Arithmetic Operations Recognizing if Statements Analyzing Functions Graphically with IDA Pro Recognizing Nested if Statements Recognizing Loops Finding for Loops Finding while Loops Understanding Function Call Conventions cdecl stdcall fastcall Push vs. Move Analyzing switch Statements If Style Jump Table Disassembling Arrays Identifying Structs Analyzing Linked List Traversal Conclusion Lab 6-1 Questions Lab 6-2 Questions Lab 6-3 Questions Lab 6-4 Questions 7: Analyzing Malicious Windows Programs The Windows API Types and Hungarian Notation Handles File System Functions Special Files The Windows Registry Registry Root Keys Regedit Programs that Run Automatically Common Registry Functions Analyzing Registry Code in Practice Registry Scripting with .reg Files Networking APIs Berkeley Compatible Sockets The Server and Client Sides of Networking The WinINet API Following Running Malware DLLs Processes Threads Interprocess Coordination with Mutexes Services The Component Object Model Exceptions: When Things Go Wrong Kernel vs. User Mode The Native API Conclusion Lab 7-1 Questions Lab 7-2 Questions Lab 7-3 Questions Part 3: Advanced Dynamic Analysis 8: Debugging Source-Level vs. Assembly-Level Debuggers Kernel vs. User-Mode Debugging Using a Debugger Single-Stepping Stepping-Over vs. Stepping-Into Pausing Execution with Breakpoints Exceptions First- and Second-Chance Exceptions Common Exceptions Modifying Execution with a Debugger Modifying Program Execution in Practice Conclusion 9: OllyDbg Loading Malware Opening an Executable Attaching to a Running Process The OllyDbg Interface Memory Map Rebasing Viewing Threads and Stacks Executing Code Breakpoints Software Breakpoints Conditional Breakpoints Hardware Breakpoints Memory Breakpoints Loading DLLs Tracing Standard Back Trace Call Stack Run Trace Tracing Poison Ivy Exception Handling Patching Analyzing Shellcode Assistance Features Plug-ins OllyDump Hide Debugger Command Line Bookmarks Scriptable Debugging Conclusion Lab 9-1 Questions Lab 9-2 Questions Lab 9-3 Questions 10: Kernel Debugging with WinDbg Drivers and Kernel Code Setting Up Kernel Debugging Using WinDbg Reading from Memory Using Arithmetic Operators Setting Breakpoints Listing Modules Microsoft Symbols Searching for Symbols Viewing Structure Information Configuring Windows Symbols Kernel Debugging in Practice Looking at the User-Space Code Looking at the Kernel-Mode Code Finding Driver Objects Rootkits Rootkit Analysis in Practice Interrupts Loading Drivers Kernel Issues for Windows Vista, Windows 7, and x64 Versions Conclusion Lab 10-1 Questions Lab 10-2 Questions Lab 10-3 Questions Part 4: Malware Functionality 11: Malware Behavior Downloaders and Launchers Backdoors Reverse Shell RATs Botnets RATs and Botnets Compared Credential Stealers GINA Interception Hash Dumping Keystroke Logging Persistence Mechanisms The Windows Registry Trojanized System Binaries DLL Load-Order Hijacking Privilege Escalation Using SeDebugPrivilege Covering Its Tracks—User-Mode Rootkits IAT Hooking Inline Hooking Conclusion Lab 11-1 Questions Lab 11-2 Questions Lab 11-3 Questions 12: Covert Malware Launching Launchers Process Injection DLL Injection Direct Injection Process Replacement Hook Injection Local and Remote Hooks Keyloggers Using Hooks Using SetWindowsHookEx Thread Targeting Detours APC Injection APC Injection from User Space APC Injection from Kernel Space Conclusion Lab 12-1 Questions Lab 12-2 Questions Lab 12-3 Questions Lab 12-4 Questions 13: Data Encoding The Goal of Analyzing Encoding Algorithms Simple Ciphers Caesar Cipher XOR Other Simple Encoding Schemes Base64 Common Cryptographic Algorithms Recognizing Strings and Imports Searching for Cryptographic Constants Searching for High-Entropy Content Custom Encoding Identifying Custom Encoding Advantages of Custom Encoding to the Attacker Decoding Self-Decoding Manual Programming of Decoding Functions Using Instrumentation for Generic Decryption Conclusion Lab 13-1 Questions Lab 13-2 Questions Lab 13-3 Questions 14: Malware-Focused Network Signatures Network Countermeasures Observing the Malware in Its Natural Habitat Indications of Malicious Activity OPSEC = Operations Security Safely Investigate an Attacker Online Indirection Tactics Getting IP Address and Domain Information Content-Based Network Countermeasures Intrusion Detection with Snort Taking a Deeper Look Combining Dynamic and Static Analysis Techniques The Danger of Overanalysis Hiding in Plain Sight Understanding Surrounding Code Finding the Networking Code Knowing the Sources of Network Content Hard-Coded Data vs. Ephemeral Data Identifying and Leveraging the Encoding Steps Creating a Signature Analyze the Parsing Routines Targeting Multiple Elements Understanding the Attacker’s Perspective Conclusion Lab 14-1 Questions Lab 14-2 Questions Lab 14-3 Questions Part 5: Anti-Reverse-Engineering 15: Anti-Disassembly Understanding Anti-Disassembly Defeating Disassembly Algorithms Linear Disassembly Flow-Oriented Disassembly Anti-Disassembly Techniques Jump Instructions with the Same Target A Jump Instruction with a Constant Condition Impossible Disassembly NOP-ing Out Instructions with IDA Pro Obscuring Flow Control The Function Pointer Problem Adding Missing Code Cross-References in IDA Pro Return Pointer Abuse Misusing Structured Exception Handlers Thwarting Stack-Frame Analysis Conclusion Lab 15-1 Questions Lab 15-2 Questions Lab 15-3 Questions 16: Anti-Debugging Windows Debugger Detection Using the Windows API Manually Checking Structures Checking for System Residue Identifying Debugger Behavior INT Scanning Performing Code Checksums Timing Checks Interfering with Debugger Functionality Using TLS Callbacks Using Exceptions Inserting Interrupts Debugger Vulnerabilities PE Header Vulnerabilities The OutputDebugString Vulnerability Conclusion Lab 16-1 Questions Lab 16-2 Questions Lab 16-3 Questions 17: Anti-Virtual Machine Techniques VMware Artifacts Bypassing VMware Artifact Searching Checking for Memory Artifacts Vulnerable Instructions Using the Red Pill Anti-VM Technique Using the No Pill Technique Querying the I/O Communication Port Using the str Instruction Anti-VM x86 Instructions Highlighting Anti-VM in IDA Pro Using ScoopyNG Tweaking Settings Escaping the Virtual Machine Conclusion Lab 17-1 Questions Lab 17-2 Questions Lab 17-3 Questions 18: Packers and Unpacking Packer Anatomy The Unpacking Stub Loading the Executable Resolving Imports The Tail Jump Unpacking Illustrated Identifying Packed Programs Indicators of a Packed Program Entropy Calculation Unpacking Options Automated Unpacking Manual Unpacking Rebuilding the Import Table with Import Reconstructor Finding the OEP Repairing the Import Table Manually Tips and Tricks for Common Packers UPX PECompact ASPack Petite WinUpack Themida Analyzing Without Fully Unpacking Packed DLLs Conclusion Labs Part 6: Special Topics 19: Shellcode Analysis Loading Shellcode for Analysis Position-Independent Code Identifying Execution Location Using call/pop Using fnstenv Manual Symbol Resolution Finding kernel32.dll in Memory Parsing PE Export Data Using Hashed Exported Names A Full Hello World Example Shellcode Encodings NOP Sleds Finding Shellcode Conclusion Lab 19-1 Questions Lab 19-2 Questions Lab 19-3 Questions 20: C++ Analysis Object-Oriented Programming The this Pointer Overloading and Mangling Inheritance and Function Overriding Virtual vs. Nonvirtual Functions Use of Vtables Recognizing a Vtable Creating and Destroying Objects Conclusion Lab 20-1 Questions Lab 20-2 Questions Lab 20-3 Questions 21: 64-Bit Malware Why 64-Bit Malware? Differences in x64 Architecture Differences in the x64 Calling Convention and Stack Usage 64-Bit Exception Handling Windows 32-Bit on Windows 64-Bit 64-Bit Hints at Malware Functionality Conclusion Lab 21-1 Questions Lab 21-2 Questions A: Important Windows Functions B: Tools for Malware Analysis C: Solutions to Labs Chapter 1 Lab 1-1 Solutions Short Answers Detailed Analysis Lab 1-2 Solutions Short Answers Detailed Analysis Lab 1-3 Solutions Short Answers Detailed Analysis Lab 1-4 Solutions Short Answers Detailed Analysis Chapter 3 Lab 3-1 Solutions Short Answers Detailed Analysis Lab 3-2 Solutions Short Answers Detailed Analysis Lab 3-3 Solutions Short Answers Detailed Analysis Lab 3-4 Solutions Short Answers Detailed Analysis Chapter 5 Lab 5-1 Solutions Short Answers Detailed Analysis Chapter 6 Lab 6-1 Solutions Short Answers Detailed Analysis Lab 6-2 Solutions Short Answers Detailed Analysis Lab 6-3 Solutions Short Answers Detailed Analysis Lab 6-4 Solutions Short Answers Detailed Analysis Chapter 7 Lab 7-1 Solutions Short Answers Detailed Analysis Lab 7-2 Solutions Short Answers Detailed Analysis Lab 7-3 Solutions Short Answers Detailed Analysis Chapter 9 Lab 9-1 Solutions Short Answers Detailed Analysis Lab 9-2 Solutions Short Answers Detailed Analysis Lab 9-3 Solutions Short Answers Detailed Analysis Chapter 10 Lab 10-1 Solutions Short Answers Detailed Analysis Lab 10-2 Solutions Short Answers Detailed Analysis Lab 10-3 Solutions Short Answers Detailed Analysis Chapter 11 Lab 11-1 Solutions Short Answers Detailed Analysis Lab 11-2 Solutions Short Answers Detailed Analysis Lab 11-3 Solutions Short Answers Detailed Analysis Chapter 12 Lab 12-1 Solutions Short Answers Detailed Analysis Lab 12-2 Solutions Short Answers Detailed Analysis Lab 12-3 Solutions Short Answers Detailed Analysis Lab 12-4 Solutions Short Answers Detailed Analysis Chapter 13 Lab 13-1 Solutions Short Answers Detailed Analysis Lab 13-2 Solutions Short Answers Detailed Analysis Lab 13-3 Solutions Short Answers Detailed Analysis Chapter 14 Lab 14-1 Solutions Short Answers Detailed Analysis Lab 14-2 Solutions Short Answers Detailed Analysis Lab 14-3 Solutions Short Answers Detailed Analysis Chapter 15 Lab 15-1 Solutions Short Answers Detailed Analysis Lab 15-2 Solutions Short Answers Detailed Analysis Lab 15-3 Solutions Short Answers Detailed Analysis Chapter 16 Lab 16-1 Solutions Short Answers Detailed Analysis Lab 16-2 Solutions Short Answers Detailed Analysis Lab 16-3 Solutions Short Answers Detailed Analysis Chapter 17 Lab 17-1 Solutions Short Answers Detailed Analysis Lab 17-2 Solutions Short Answers Detailed Analysis Lab 17-3 Solutions Short Answers Detailed Analysis Chapter 18 Lab 18-1 Solutions Lab 18-2 Solutions Lab 18-3 Solutions Lab 18-4 Solutions Lab 18-5 Solutions Chapter 19 Lab 19-1 Solutions Short Answers Detailed Analysis Lab 19-2 Solutions Short Answers Detailed Analysis Lab 19-3 Solutions Short Answers Detailed Analysis Chapter 20 Lab 20-1 Solutions Short Answers Detailed Analysis Lab 20-2 Solutions Short Answers Detailed Analysis Lab 20-3 Solutions Short Answers Detailed Analysis Chapter 21 Lab 21-1 Solutions Short Answers Detailed Analysis Lab 21-2 Solutions Short Answers Detailed Analysis Index Updates