Writing A Compiler In Go

Description

This is the sequel to Writing An Interpreter In Go. We're picking up right where we left off and write a compiler and a virtual machine for Monkey. Runnable and tested code front and center, built from the ground up, step by step — just like before. But this time, we're going to define bytecode, compile Monkey and execute it in our very own virtual machine. It's the next step in Monkey's evolution. It's the sequel to … a programming language Writing A Compiler In Go is the sequel to Writing An Interpreter In Go. It starts right where the first one stopped, with a fully-working, fully-tested Monkey interpreter in hand, connecting both books seamlessly, ready to build a compiler and a virtual machine for Monkey. In this book, we use the codebase (included in the book!) from the first part and extend it. We take the lexer, the parser, the AST, the REPL and the object system and use them to build a new, faster implementation of Monkey, right next to the tree-walking evaluator we built in the first book. The approach is unchanged, too. Working, tested code is the focus, we build everything from scratch, do baby steps, write tests firsts, use no 3rd-party-libraries and see and understand how all the pieces fit together. It's a continuation in prose and in code. Do you need to read the first part before this one? If you're okay with treating the code from the first book as black box, then no. But that's not what these books are about; they're about opening up black boxes, looking inside and shining a light. You'll have the best understanding of where we're going in this book, if you know where we started. Learn how to write a compiler and a virtual machine Our main goal in in this book is to evolve Monkey. We change its architecture and turn it into a bytecode compiler and virtual machine. We'll take the lexer, the parser, the AST and the object system we wrote in the first book and use them to build our own Monkey compiler and virtual machine … from scratch! We'll build them side-by-side so that we'll always have a running system we can steadily evolve. What we end up with is not only much closer to the programming languages we use every day, giving us a better understanding of how they work, but also 3x faster. And that's without explicitly aiming for performance. Here's what we'll do: - We define our own bytecode instructions, specifying their operands and their encoding. Along the way, we also build a mini-disassembler for them. - We write a compiler that takes in a Monkey AST and turns it into bytecode by emitting instructions - At the same time we build a stack-based virtual machine that executes the bytecode in its main loop We'll learn a lot about computers, how they work, what machine code and opcodes are, what the stack is and how to work with stack pointers and frame pointers, what it means to define a calling convention, and much more. We also - build a symbol table and a constant pool - do stack arithmetic - generate jump instructions - build frames into our VM to execute functions with local bindings and arguments! - add built-in functions to the VM - get real closures working in the virtual machine and learn why closure-compilation is so tricky Acknowledgments Introduction Evolving Monkey The Past and Present The Future Use This Book Compilers & Virtual Machines Compilers Virtual and Real Machines Real Machines What Is a Virtual Machine? Why Build One? Bytecode What We're Going to Do, or: the Duality of VM and Compiler Hello Bytecode! First Instructions Starting With Bytes The Smallest Compiler Bytecode, Disassemble! Back to the Task at Hand Powering On the Machine Adding on the Stack Hooking up the REPL Compiling Expressions Cleaning Up the Stack Infix Expressions Booleans Comparison Operators Prefix Expressions Conditionals Jumps Compiling Conditionals Executing Jumps Welcome Back, Null! Keeping Track of Names The Plan Compiling Bindings Introducing: the Symbol Table Using Symbols in the Compiler Adding Globals to the VM String, Array and Hash String Array Hash Adding the index operator Functions Dipping Our Toes: a Simple Function Representing Functions Opcodes to Execute Functions Compiling Function Literals Compiling Function Calls Functions in the VM A Little Bonus Local Bindings Opcodes for Local Bindings Compiling Locals Implementing Local Bindings in the VM Arguments Compiling Calls With Arguments Resolving References to Arguments Arguments in the VM Built-in Functions Making the Change Easy Making the Change: the Plan A New Scope for Built-in Functions Executing built-in functions Closures The Problem The Plan Everything's a closure Compiling and resolving free variables Creating real closures at run time Recursive Closures Taking Time Resources Feedback Changelog