Mastering Malware Analysis

Description

Title Page......Page 2 Copyright and Credits......Page 3 Mastering Malware Analysis......Page 4 About Packt......Page 5 Why subscribe?......Page 6 Contributors......Page 7 About the authors......Page 8 About the reviewers......Page 9 Packt is searching for authors like you......Page 11 Preface......Page 26 Who this book is for......Page 27 What this book covers......Page 28 To get the most out of this book......Page 31 Download the example code files......Page 32 Download the color images......Page 33 Conventions used......Page 34 Get in touch......Page 35 Reviews......Page 36 Section 1: Fundamental Theory......Page 37 A Crash Course in CISC/RISC and Programming Basics......Page 38 Basic concepts......Page 40 Registers......Page 41 Memory......Page 42 Virtual memory......Page 43 Stack......Page 45 Branches, loops, and conditions......Page 46 Exceptions, interrupts, and communicating with other devices......Page 47 Assembly languages......Page 48 CISC versus RISC......Page 49 Types of instructions......Page 50 Becoming familiar with x86 (IA-32 and x64)......Page 51 Registers......Page 52 Special registers......Page 54 The instruction structure......Page 55 opcode......Page 56 dest......Page 57 src......Page 58 The instruction set......Page 59 Data manipulation instructions......Page 60 Data transfer instructions......Page 62 Flow control instructions......Page 64 Arguments, local variables, and calling conventions (in x86 and x64)......Page 65 stdcall......Page 66 Arguments......Page 67 Local variables......Page 69 cdecl......Page 71 fastcall......Page 72 thiscall......Page 73 The x64 calling convention......Page 74 Exploring ARM assembly......Page 75 Basics......Page 78 Instruction sets......Page 82 Basics of MIPS......Page 85 Basics......Page 87 The instruction set......Page 89 Diving deep into PowerPC......Page 92 Basics......Page 93 The instruction set......Page 95 Covering the SuperH assembly......Page 97 Basics......Page 98 The instruction set......Page 99 Working with SPARC......Page 101 Basics......Page 102 The instruction set......Page 104 From assembly to high-level programming languages......Page 105 Arithmetic statements......Page 106 If conditions......Page 108 While loop conditions......Page 111 Summary......Page 112 Section 2: Diving Deep into Windows Malware......Page 113 Basic Static and Dynamic Analysis for x86/x64......Page 114 Working with the PE header structure......Page 115 Why PE?......Page 116 Exploring PE structure......Page 117 MZ header......Page 118 PE header......Page 119 File header......Page 120 Optional header......Page 121 Data directory......Page 123 Section table......Page 124 PE+ (x64 PE)......Page 125 PE analysis tools......Page 126 Static and dynamic linking......Page 129 Static linking......Page 130 Dynamic linking......Page 131 Dynamic link libraries......Page 132 Application programming interface......Page 133 Dynamic API loading......Page 134 Using PE header information for static analysis......Page 135 How to use PE header for incident handling......Page 136 How to use a PE header for threat intelligence......Page 138 PE loading and process creation......Page 141 Basic terminology......Page 142 What's process?......Page 143 Virtual memory to physical memory mapping......Page 145 Threads......Page 147 Important data structures: TIB, TEB, and PEB......Page 149 Process loading step by step......Page 150 PE file loading step by step......Page 151 WOW64 processes......Page 154 Dynamic analysis with OllyDbg/immunity debugger......Page 156 Debugging tools......Page 157 How to analyze a sample with OllyDbg......Page 161 Types of breakpoints......Page 167 Step into/step over breakpoint......Page 168 INT3 breakpoint......Page 169 Memory breakpoints......Page 170 Hardware breakpoints......Page 171 Modifying the program execution......Page 172 Patching—modifying the program's assembly instructions......Page 173 Change EFlags......Page 174 Modifying the instruction pointer value......Page 175 Changing the program data......Page 176 Debugging malicious services......Page 177 What is service?......Page 178 Attaching to the service......Page 180 Summary......Page 182 Unpacking, Decryption, and Deobfuscation......Page 183 Exploring packers......Page 184 Exploring packing and encrypting tools......Page 186 Identifying a packed sample......Page 188 Technique 1 – checking PE tool static signatures......Page 189 Technique 2 – evaluating PE section names......Page 190 Technique 3 – using stub execution signs......Page 191 Technique 4 – detecting a small import table......Page 192 Automatically unpacking packed samples......Page 193 Technique 1 – the official unpacking process......Page 194 Technique 2 – using OllyScript with OllyDbg......Page 195 Technique 3 – using generic unpackers......Page 196 Technique 4 – emulation......Page 199 Technique 5 – memory dumps......Page 200 Manual unpacking using OllyDbg......Page 201 Technique 6 – memory breakpoint on execution......Page 202 Step 1 – setting the breakpoints......Page 203 Step 2 – turning on Data Execution Prevention......Page 204 Step 3 – preventing any further attempts to change memory permissions......Page 207 Step 4 – executing and getting the OEP......Page 208 Technique 7 – call stack backtracing......Page 210 Step 1 – setting the breakpoints......Page 213 Step 2 – following the call stack......Page 214 Step 3 – reaching the OEP......Page 215 Technique 8 – monitoring memory allocated spaces for unpacked code......Page 216 Technique 9 – in-place unpacking......Page 219 Technique 10 – stack restoration based......Page 220 Dumping the unpacked sample and fixing the import table......Page 221 Dumping the process......Page 222 Fixing the import table......Page 224 Identifying different encryption algorithms and functions......Page 228 Types of encryption algorithms......Page 229 Basic encryption algorithms......Page 231 How to identify encryption functions......Page 232 String search detection techniques for simple algorithms......Page 234 The basics of X-RAYING......Page 235 Simple static encryption......Page 236 Other encryption algorithms......Page 237 X-RAYING tools for malware analysis and detection......Page 238 Identifying the RC4 encryption algorithm......Page 240 The RC4 encryption algorithm......Page 241 Key-scheduling algorithm......Page 242 Pseudo-random generation algorithm......Page 243 Identifying RC4 algorithms in a malware sample......Page 244 Standard symmetric and asymmetric encryption algorithms......Page 246 Extracting information from Windows cryptography APIs......Page 247 Step 1 – initializing and connecting to the cryptographic service provider (CSP)......Page 248 Step 2 – preparing the key......Page 249 Step 3 – encrypting or decrypting the data......Page 251 Step 4 – freeing the memory......Page 252 Cryptography API next generation (CNG)......Page 253 Applications of encryption in modern malware – Vawtrak banking Trojan......Page 254 String and API name encryption......Page 255 Network communication encryption......Page 261 Using IDA for decryption and unpacking......Page 264 IDA tips and tricks......Page 265 Static analysis......Page 266 Dynamic analysis......Page 271 Classic and new syntax of IDA scripts......Page 274 Dynamic string decryption......Page 278 Dynamic WinAPIs resolution......Page 279 Summary......Page 280 Inspecting Process Injection and API Hooking......Page 281 Understanding process injection......Page 282 What's process injection?......Page 283 Why process injection?......Page 284 DLL injection......Page 285 Windows-supported DLL injection......Page 286 A simple DLL injection technique......Page 290 Working with process injection......Page 292 Getting the list of running processes......Page 293 Code injection......Page 295 Advanced code injection-reflective DLL injection......Page 297 Stuxnet secret technique-process hollowing......Page 299 Dynamic analysis of code injection......Page 302 Technique 1—debug it where it is......Page 303 Technique 2—attach to the targeted process......Page 305 Technique 3—dealing with process hollowing......Page 306 Memory forensics techniques for process injection......Page 308 Technique 1—detecting code injection and reflective DLL injection ......Page 309 Technique 2—detecting process hollowing......Page 312 Technique 3—detecting process hollowing using the HollowFind plugin......Page 314 Understanding API hooking......Page 316 Why API hooking?......Page 317 Working with API hooking......Page 318 Inline API hooking......Page 319 Inline API hooking with trampoline......Page 320 Inline API hooking with a length disassembler......Page 322 Detecting API hooking using memory forensics......Page 324 Exploring IAT hooking......Page 325 Summary......Page 327 Bypassing Anti-Reverse Engineering Techniques......Page 328 Exploring debugger detection......Page 329 Direct check for debugger presence......Page 330 Detecting a debugger through an environment change......Page 331 Detecting a debugger using parent processes......Page 332 Handling debugger breakpoints evasion......Page 334 Detecting software breakpoints (INT3)......Page 335 Detecting single-stepping breakpoints (trap flag)......Page 338 Detecting a trap flag using the SS register......Page 339 Detecting single-stepping using timing techniques......Page 341 Evading hardware breakpoints......Page 343 What is structured exception handling?......Page 344 Detecting and removing hardware breakpoints......Page 347 Memory breakpoints......Page 348 Escaping the debugger......Page 349 Process injection......Page 350 TLS callbacks......Page 351 Windows events callbacks......Page 353 Obfuscation and anti-disassemblers......Page 354 Encryption......Page 355 Junk code insertion......Page 356 Code transportation......Page 358 Dynamic API calling with checksum......Page 360 Proxy functions and proxy argument stacking......Page 361 Detecting and evading behavioral analysis tools......Page 363 Finding the tool process......Page 364 Searching for the tool window......Page 367 Detecting sandboxes and virtual machines......Page 369 Different output between virtual machines and real machines......Page 370 Detecting virtualization processes and services......Page 371 Detecting virtualization through registry keys......Page 372 Detecting virtual machines using PowerShell......Page 373 Detecting sandboxes by using default settings......Page 374 Other techniques......Page 375 Summary......Page 376 Understanding Kernel-Mode Rootkits......Page 377 Kernel mode versus user mode......Page 378 Protection rings......Page 379 Windows internals......Page 381 The infrastructure of Windows......Page 382 The execution path from user mode to kernel mode......Page 386 Rootkits and device drivers......Page 389 What is a rootkit?......Page 390 Types of rootkits......Page 391 What is a device driver?......Page 392 Hooking mechanisms......Page 393 SSDT hooking......Page 396 Hooking the SYSENTER entry function......Page 397 Modifying SSDT in an x86 environment......Page 399 Modifying SSDT in an x64 environment......Page 402 Hooking SSDT functions......Page 404 IRP hooking......Page 405 Devices and major functions......Page 406 Attaching to a device......Page 408 Modifying the IRP response and setting a completion routine......Page 409 DKOM......Page 410 The kernel objects—EPROCESS and ETHREAD......Page 411 How do rootkits perform an object manipulation attack?......Page 413 Process injection in kernel mode......Page 416 Executing the inject code using APC queuing......Page 421 KPP in x64 systems (PatchGuard)......Page 424 Bypassing driver signature enforcement......Page 425 Bypassing PatchGuard—the Turla example......Page 426 Bypassing PatchGuard—GhostHook......Page 427 Disabling PatchGuard using the Command Prompt......Page 428 Static and dynamic analysis in kernel mode......Page 429 Static analysis......Page 430 Tools......Page 431 Tips and tricks......Page 432 Dynamic and behavioral analysis......Page 433 Tools......Page 434 Monitors......Page 436 Rootkit detectors......Page 437 Setting up a testing environment......Page 438 Setting up the debugger......Page 441 Stopping at the driver's entrypoint......Page 445 Loading the driver......Page 449 Restoring the debugging state......Page 450 Summary......Page 451 Section 3: Examining Cross-Platform Malware......Page 452 Handling Exploits and Shellcode......Page 453 Getting familiar with vulnerabilities and exploits......Page 454 Types of vulnerabilities......Page 455 Stack overflow vulnerability......Page 456 Heap overflow vulnerabilities......Page 458 The use-after-free vulnerability......Page 459 Logical vulnerabilities......Page 460 Types of exploits......Page 461 Cracking the shellcode......Page 463 What's shellcode?......Page 464 Linux shellcode in x86-64......Page 465 Getting the absolute address......Page 466 Null-free shellcode......Page 467 Local shell shellcode......Page 468 Reverse shell shellcode......Page 470 Linux shellcode for ARM......Page 473 Null-free shellcode......Page 474 Windows shellcode......Page 475 Getting the Kernel32.dll ImageBase......Page 476 Getting the required APIs from Kernel32.dll......Page 477 The download and execute shellcode......Page 480 Static and dynamic analysis of exploits......Page 481 Analysis workflow......Page 482 Shellcode analysis......Page 484 Exploring bypasses for exploit mitigation technologies......Page 485 Data execution prevention (DEP/NX)......Page 486 Return-oriented programming......Page 487 Address space layout randomization......Page 489 DEP and partial ASLR......Page 490 DEP and full ASLR – partial ROP and chaining multiple vulnerabilities......Page 491 DEP and full ASLR – heap spray technique......Page 493 Other mitigation technologies......Page 494 Analyzing Microsoft Office exploits......Page 495 File structures......Page 496 Compound file binary format......Page 497 Rich text format......Page 499 Office open XML format......Page 500 Static and dynamic analysis of MS Office exploits......Page 501 Static analysis......Page 502 Dynamic analysis......Page 504 Studying malicious PDFs......Page 505 File structure......Page 506 Static and dynamic analysis of PDF files......Page 509 Static analysis......Page 510 Dynamic analysis......Page 513 Summary......Page 514 Reversing Bytecode Languages: .NET, Java, and More......Page 515 Exploring the theory of bytecode languages......Page 516 Object-oriented programming......Page 517 Inheritance......Page 518 Polymorphism......Page 519 .NET explained......Page 520 .NET file structure......Page 521 .NET COR20 header......Page 522 Metadata streams......Page 524 How to identify a .NET application from PE characteristics......Page 525 The CIL language instruction set......Page 527 Pushing into stack instructions......Page 528 Pulling out a value from the stack......Page 530 Mathematical and logical operations......Page 531 Branching instructions......Page 532 CIL language to higher-level languages......Page 533 Local variable assignments......Page 534 Local variable assignment with a method return value......Page 535 Basic branching statements......Page 536 Loops statements......Page 537 .NET malware analysis......Page 538 .NET analysis tools......Page 539 Static and dynamic analysis (with Dnspy)......Page 540 .NET static analysis......Page 541 .NET dynamic analysis......Page 543 Patching a .NET sample......Page 544 Dealing with obfuscation......Page 545 Obfuscated names for classes, methods, and others......Page 546 Encrypted strings inside the binary......Page 548 The sample is obfuscated using an obfuscator......Page 550 The essentials of Visual Basic......Page 552 File structure......Page 553 P-code versus native code......Page 557 Common p-code instructions......Page 560 Dissecting Visual Basic samples......Page 562 Static analysis......Page 563 P-code......Page 564 Native code......Page 566 Dynamic analysis......Page 569 P-code......Page 570 Native code......Page 571 The internals of Java samples......Page 572 File structure......Page 573 JVM instructions......Page 575 Static analysis......Page 576 Dynamic analysis......Page 579 Dealing with anti-reverse engineering solutions......Page 580 Python—script language internals......Page 581 File structure......Page 582 Bytecode instructions......Page 584 Analyzing compiled Python......Page 586 Static analysis......Page 587 Dynamic analysis......Page 589 Summary......Page 590 Scripts and Macros: Reversing, Deobfuscation, and Debugging......Page 591 Classic shell script languages......Page 592 Windows batch scripting......Page 593 Bash......Page 596 VBScript explained......Page 598 Basic syntax......Page 599 Static and dynamic analysis......Page 603 Deobfuscation......Page 606 Those evil macros inside documents......Page 607 Basic syntax......Page 608 Static and dynamic analysis......Page 610 Besides macros......Page 611 The power of PowerShell......Page 613 Basic syntax......Page 614 Static and dynamic analysis......Page 618 Handling JavaScript......Page 621 Basic syntax......Page 622 Static and dynamic analysis......Page 624 Anti-reverse engineering tricks......Page 630 Behind C&C—even malware has its own backend......Page 631 Things to focus on......Page 632 Static and dynamic analysis......Page 633 Other script languages......Page 634 Where to start from......Page 635 Questions to answer......Page 636 Summary......Page 637 Section 4: Looking into IoT and Other Platforms......Page 638 Dissecting Linux and IoT Malware......Page 639 Explaining ELF files ......Page 640 ELF structure......Page 641 System calls......Page 643 Filesystem......Page 644 Network......Page 645 Process management......Page 646 Other......Page 647 Syscalls in assembly......Page 648 Common anti-reverse engineering tricks......Page 652 Exploring common behavioral patterns......Page 653 Initial delivery and lateral movement......Page 654 Persistence......Page 656 Privilege escalation......Page 658 Interaction with the command and control server......Page 659 Attacking stage......Page 661 Static and dynamic analysis of x86 (32- and 64-bit) samples......Page 662 Static analysis......Page 663 File type detectors......Page 664 Data carving......Page 665 Disassemblers......Page 666 Actual tools......Page 667 Engines......Page 673 How to choose......Page 674 Dynamic analysis......Page 675 Tracers......Page 676 Network monitors......Page 677 Debuggers......Page 678 Binary emulators......Page 679 Radare2 cheat sheet......Page 680 Anti-reverse engineering techniques......Page 684 Learning Mirai, its clones, and more......Page 685 High-level functionality......Page 686 Propagation......Page 687 Weaponry......Page 688 Self-defense......Page 689 Later derivatives......Page 690 Other widespread families......Page 692 Static and dynamic analysis of RISC samples......Page 694 ARM......Page 697 MIPS......Page 699 PowerPC......Page 700 SuperH......Page 702 SPARC......Page 703 Handling other architectures......Page 704 What to start from......Page 705 Summary......Page 706 Introduction to macOS and iOS Threats......Page 707 Understanding the role of the security model......Page 708 macOS......Page 709 Security policies......Page 710 Filesystem hierarchy and encryption......Page 711 Directory structure......Page 712 Encryption......Page 713 Apps protection......Page 714 Gatekeeper......Page 715 App sandbox......Page 716 Other technologies......Page 718 iOS......Page 719 System security......Page 720 Data encryption and password management......Page 722 Apps' security......Page 725 File formats and APIs......Page 727 Mach-O......Page 728 Thin......Page 729 Fat......Page 732 Application bundles (.app)......Page 734 Info.plist......Page 735 macOS......Page 736 iOS......Page 737 Installer packages (.pkg)......Page 738 Apple disk images (.dmg)......Page 739 iOS app store packages (.ipa)......Page 740 APIs......Page 741 Static and dynamic analyses of macOS and iOS samples......Page 744 Static analysis......Page 745 Retrieving samples......Page 746 Disassemblers and decompilers......Page 747 Auxiliary tools and libraries......Page 749 Dynamic and behavioral analysis......Page 750 macOS......Page 751 Debuggers......Page 752 Monitoring and dynamic instrumentation......Page 754 Network analysis......Page 756 iOS......Page 757 Installers and loaders......Page 758 Debuggers......Page 760 Dumping and decryption......Page 761 Monitors and in-memory patching......Page 762 Network analysis......Page 763 Attack stages......Page 764 Jailbreaks on demand......Page 765 Penetration......Page 767 Deployment and persistence......Page 769 macOS......Page 770 iOS......Page 772 Action phase......Page 773 macOS......Page 774 iOS......Page 777 Other attack techniques......Page 779 macOS......Page 780 iOS......Page 781 Advanced techniques......Page 783 Anti-reverse-engineering (RE) tricks......Page 784 Misusing dynamic data exchange (DDE)......Page 785 User hiding......Page 786 Use of AppleScript......Page 787 API hijacking......Page 788 Rootkits for Mac—do they exist?......Page 789 Analysis workflow......Page 790 Summary......Page 792 Analyzing Android Malware Samples......Page 793 (Ab)using Android internals ......Page 794 File hierarchy......Page 795 Android security model......Page 798 Process management......Page 799 Filesystem......Page 800 App permissions......Page 801 Security services......Page 803 Console......Page 804 To root or not to root?......Page 807 Understanding Dalvik and ART ......Page 810 Dalvik VM (DVM)......Page 811 Android runtime (ART)......Page 812 APIs......Page 817 File formats......Page 819 DEX......Page 820 ODEX......Page 823 OAT......Page 824 VDEX......Page 825 ART......Page 826 ELF......Page 827 APK......Page 828 Bytecode set......Page 831 Malware behavior patterns......Page 837 Attack stages......Page 838 Penetration......Page 839 Deployment......Page 840 Action phase......Page 841 Advanced techniques—investment pays off......Page 844 Patching system libraries......Page 845 Keylogging......Page 846 Self-defense......Page 847 Rootkits—get it covered......Page 848 Static and dynamic analysis of threats......Page 849 Static analysis......Page 850 Disassembling and data extraction......Page 851 Decompiling......Page 854 Dynamic analysis......Page 857 Android debug bridge......Page 858 Emulators......Page 860 Behavioral analysis and tracing......Page 861 Debuggers......Page 862 Analysis workflow......Page 863 Summary......Page 865 Other Books You May Enjoy......Page 866 Leave a review - let other readers know what you think......Page 868