Developing your own 32-bit operating system

Description

This helpful guide is designed to build upon an intermediate programmer's knowledge and explain how to design and develop a feature rich, full operating system. Among the topics discussed are tasking models, memory management, hardware interfaces, systems programming, and much more. Chapter 1, Introduction Overview What This Book Is About Who Should Use This Book My Basic Design Goals (yours may vary) A Brief Description of MMURTL Uses for MMURTL Similarities to Other Operating Systems Hardware Requirements Chapter 2, General Discussion and Background Where Does One Begin? You Are Where You Were When The Development Platform and the Tools The Chicken and the Egg Early Hardware Investigation The Real Task at Hand (Pun intended) A Starting Point The Critical Path Working Attitude Chapter 3, The Tasking Model Terms We Should Agree On TASK KERNEL PREEMPTIVE MULTITASKING Resource Management Task Management Single vs. Multi User The Hardware State The Software State CPU Time Single Tasking Multitasking Cooperative Multitasking Preemptive Multitasking Task Scheduling Who, When, and How Long? Scheduling Techniques Time Slicing Simple Cooperative Queuing Prioritized Cooperative Scheduling Variable Time Slicing Other Scheduling Factors Mixed Scheduling Techniques Fully Time Sliced, Some Cooperation Time Sliced, More Cooperative Primarily Cooperative, Limited Time Slicing Primarily Cooperative, More Time Slicing The Tasking Model I Chose Interrupt Handling Chapter 4, Interprocess Communications Introduction Messaging and Tasks Synchronization Semaphores Pipes Messages Send and Wait Request() Respond() Link Blocks Reentrancy Issues Interrupt Latency Memory Management Issues Chapter 5, Memory Management Introduction Basic Terms Memory Model Simple Flat Memory Figure 5.1, Simple Flat Memory Paged Flat Memory Figure 5.2 - Paged Flat Memory Demand-Paged Flat Memory Figure 5.3 - Demand Paged Flat Memory Virtual Paged Memory Figure 5.4 - Virtual Paged Memory Demand Paged Virtual Memory Demand-Paged Memory Management Segmented Memory Memory Management Tracking Linear Memory Allocation Basic Memory Allocation Unit Linked List Management Memory Management With Tables Tracking Physical Memory Initialization of Memory Management Memory Protection An Intel Based Memory Management Implementation A Few More Words On Segmentation How MMURTL Uses Paging Page Tables (PTs) Page Directories (PDs) The Memory Map Table 5.1 - MMURTL Memory Map Page Directory Entries (PDEs) Table 5.2 Page Directory Example Allocation of Linear Memory Deallocation of Linear Memory Allocation of Physical Memory Loading Things Into Memory Messaging and Address Aliases Memory and Pointer Management for Messaging Summary of MMURTL Memory Management Chapter 6, Hardware Interface Hardware Isolation The CPU The Bus Structure Serial I/O Parallel I/O Block-Oriented Devices Keyboard Video Direct Memory Access (DMA) Timers Priority Interrupt Controller Unit (PICU) Chapter 7, OS Initialization Getting Booted Boot ROM The Boot Sector Operating System Boot Image Other Boot Options Basic Hardware Initialization Static Tables and Structures Initialization of Task Management Initialization of Memory Dynamic Tables and Structures Chapter 8, Programming Interfaces Application Programming Interface Documenting Your API Procedural Interfaces Calling Conventions Mechanical Details Portability Considerations Error Handling and Reporting Systems Programming Interface Internal Cooperation Device Driver Interfaces A Device-Driver Interface Example Announcing the Driver to the OS A Call to the driver Chapter 9, Application Programming Introduction Terminology Understanding 32-BIT Software Table 9.1 - Memory Alignment Operating System Calling Conventions Stack Usage Memory Management Table 9.2 - Basic Memory Map Operating System Protection Application Messaging Starting a New Thread Job Control Block Basic Keyboard Basic Video Table 9.3 - Foreground Colors (Low nibble) Table 9.4 - Background Colors (High nibble) Terminating Your Application Replacing Your Application Chapter 10, Systems Programming Introduction Writing Message-Based System Services Initializing Your System Service A Simple System Service Example Listing 10.1 - A System Service. The Request Block Items In the Request Block The Service Code Caller Information in a Request Asynchronous Services System Service Error Handling Writing Device Drivers Device Driver Theory Building Device Drivers How Callers Reach Your Driver Device Driver Setup and Installation System Resources For Device Drivers Interrupts Direct Memory Access Device (DMA) Timer Facilities Message Facilities for Device Drivers Detailed Device Interface Specification Device Control Block Setup and Use Table 10.1-. Device control block definition Standard Device Call Definitions DeviceOp Function Implementation DeviceStat Function Implementation DeviceInit Function Implementation Initializing Your Driver OS Functions for Device Drivers Standard Device Error Codes Chapter 11, The Monitor Program Introduction Active Job (Video & keyboard) Initial Jobs Listing 11.1. Sample Initial.Job file. Monitor Function Keys Table 11.1. Monitor Function Keys Monitor Program Theory Performance Monitoring Monitor Source Listing Listing 11.2.Monitor program source listing. Chapter 12, Debugger Introduction Using the Debugger Entering the Debugger Exiting the Debugger Debugger Display Debugger Function Keys Debugging Your Application Debugger Theory Table 12.1.Processor exceptions. Chapter 13, Keyboard Service Introduction Available Services Table 13.1.Available Keyboard Services Read Keyboard Notify On Global Keys Cancel Notify On Global Keys Assign Keyboard Key codes and Status Alpha-Numeric Key Values Table 13.2 - .Shift State Bits Lock-State Byte (third byte) Table 13.3 - Lock State Bits Numeric Pad Indicator Key Codes Table 13.4 - .Keys Values Function Strip Key Codes Table 13.5 - Function Key Strip Codes Numeric Pad Key Codes Table 13.6 - Numeric Pad Key Codes Cursor, Edit, and Special Pad Key Codes 13.7.Additional Key Codes Your Keyboard Implementation Chapter 14, The File System Service Introduction File Specifications Network File Request Routing File Handles File Open Modes File Access Type Block Mode Stream Mode Logical File Address File System Requests Listing 14.1 - Openfile request in C Table 14.1 - File System Service Codes Procedural Interfaces Device Access Through the File System Table 14.2 - Device Stream Access File System Functions in Detail OpenFile CloseFile ReadBlock WriteBlock ReadBytes WriteBytes GetFileLFA SetFileLFA GetFileSize SetFileSize CreateFile RenameFile DeleteFile CreateDirectory DeleteDirectory GetDirSector File System Theory Internal File System Structures File Control Blocks File User Blocks FAT Buffers File Operations Read Write Close Chapter 15, API Specification Introduction Public Calls Parameters to Calls (Args) Table 15.1 - Size and Type Prefixes Table 15.2 - Additional Prefixes Table1 5.3 - Compound Prefixes Stack Conventions Calls Grouped by Functionality Alphabetical Call Listing AddCallGate AddIDTGate Alarm AliasMem AllocDMAPage AllocExch AllocOSPage AllocPage Beep Chain CheckMsg ClrScr Compare CompareNCS CopyData CopyDataR DeAliasMem DeAllocExch DeAllocPage DeviceInit DeviceOp DeviceStat DMASetUp EditLine EndOfIRQ ExitJob FillData GetCmdLine GetCMOSTime GetCMOSDate GetDMACount GetExitJob GetIRQVector GetJobNum GetNormVid GetPath GetPhyAdd GetpJCB GetSysIn GetSysOut GetTimerTick GetTSSExch GetUserName GetVidChar GetVidOwner GetXY InByte InDWord InWord InWords InitDevDr ISendMsg KillAlarm LoadNewJob MaskIRQ MicroDelay MoveRequest NewTask OutByte OutDWord OutWord OutWords PutVidAttrs PutVidChars QueryPages ReadCMOS ReadKbd RegisterSvc Request Respond ScrollVid SendMsg SetCmdLine SetExitJob SetIRQVector Table 15.4 - Hardware IRQs SetJobName SetNormVid SetPath SetPriority SetSysIn SetSysOut SetUserName SetVidOwner SetXY Sleep SpawnTask Tone TTYOut UnMaskIRQ UnRegisterSvc WaitMsg Chapter 16, MMURTL Sample Software Introduction Command Line Interpreter (CLI) Internal Commands Cls (Clear Screen) Copy (Copy a file) Dir (Directory listing) Debug (Enter Debugger) Del (Delete a File) Dump (Hex dump of a file) Exit (Exit the CLI) Help (List internal commands) Monitor (Return to Monitor) MD (Make Directory) Path (Set file access path) RD (Remove Directory) Rename (Rename a file) Run (Execute a .RUN file) Type (View a text file on the screen) External Commands Commands.CLI File Global "Hot Keys" CLI Source Listing Listing 16.1.CLI Source Code Listing Ver. 1.0. A Simple Editor Editor Screen Editor Commands Table 16.1 - File Management and General Commands Table 16.2 - Cursor and Screen Management Commands Listing 16.2.Editor Source Code. DumbTerm Listing 16.3 - DumbTerm source code. Print Listing 16.4.Print source code. System Service Example Service.C Listing Listing 16.5 - Simple Service Source Code. TestSvc.C Listing Listing 16.6 - TestSvc.C. Chapter 17, Introduction to the Source Code Comments In the Code Calling Conventions Organization Building MMURTL Listing 17.1 – Assembler Template File Using Pieces of MMURTL in Your OS Using Pieces of MMURTL in Other Projects Chapter 18, The Kernel Introduction Naming Conventions Kernel Data Listing 18.1.Kernel data segment source. Local Kernel Helper Functions enQueueMsg Listing 18.2.Queueing for messages at an exchange. deQueueMsg Listing 18.3.De-queueing a message from an exchange. deQueueTSS Listing 18.4.De-queueing a task from an exchange. enQueueRdy Listing 18.5.Adding a task to the ready queue. deQueueRdy Listing 18.6.De-queueing the highest priority task. ChkRdyQ Listing 18.7.Finding the highest priority task Internal Public Helper Functions RemoveRdyJob Listing 18.8.Removing a terminated task from the ready queue. GetExchOwner Listing 18.9.Finding the owner of an exchange. SetExchOwner Listing 18.10.Changing the owner of an exchange. SendAbort Listing 18.11.Notifying services of a job’s demise. Public Kernel Functions Request() Listing 18.12.Request kernel primitive code. Respond() Listing 18.13.Respond kernel primitive code. MoveRequest() Listing 18.14, MoveRequest kernel primitive code. SendMsg() Listing 18.15.SendMsg kernel primitive code. ISendMsg() Listing 18.16.IsendMsg kernel primitive code. WaitMsg() Listing 18.17.WaitMsg kernel primitive code. CheckMsg() Listing 18.18 -CheckMsg kernel primitive code. NewTask() Listing 18.19.NewTaskkernel primitive code. SpawnTask() Listing 18.20.SpawnTask kernel primitive code AllocExch() Listing 18.21.Allocate exchange code DeAllocExch() Listing 18.22.Deallocate exchange code GetTSSExch() Listing 18.23.GetTSS exchange code. SetPriority() Listing 18.24.Set priority code. GetPriority() Listing 18.25.Get priority code. Chapter 19, Memory Management Code Introduction Memory Management Data Listing 19.1.Memory management constants and data Internal Code InitMemMgmt Listing 19.2.Memory management initialization code Listing 19.3.Continuation of memory management init. Listing 19.4.Continuation of memory management init. Listing 19.5.Turning on paged memory management Listing 19.6.Allocation of memory management exchange. Listing 19.7.Finishing memory management initi. FindHiPage Listing 19.8.Find highest physical page code. FindLoPage Listing 19.9.Find lowest physical page code. MarkPage Listing 19.10.Code to mark a physical page in use. UnMarkPage Listing 19.11.Code to free a physical page ofr reuse. LinToPhy Listing 19.12.Code to convert linear to physical addresses. FindRun Listing 19.13.Code to find a free run of linear memory AddRun Listing 19.14.Adding a run of linear memory. AddAliasRun Listing 19.15.Adding an aliased run of linear memory. AddUserPT Listing 19.16.Adding a page table for user memory AddOSPT Listing 19.17.Adding an operating system page table. Public Memory Management Calls AddGDTCallGate Listing 19.18.Adding a call gate to the GDT. AddIDTGate Listing 19.19.Adding entries to the IDT. AllocOSPage Listing 19.20.Allocating a page of linear memory. AliasMem Listing 19.21.Code to alias memory. DeAliasMem Listing 19.22.Code to remove alias PTEs. DeAllocPage Listing 19.23.Deallocating linear memory. QueryMemPages Listing 19.24.Code to find number of free pages GetPhyAdd Listing 19.25.Public for linear to physical conversion. Chapter 20, Timer Management Source Code Introduction Choosing a Standard Interval Listing 20.1.Loop in timer ISR Timer Data Listing 20.2 - Data and constants for timer code Timer Code Listing 20.3 - External functions for timer code The Timer Interrupt Listing 20.4 - The timer interrupt service routine Sleep() Listing 20.5 - Code for the Sleep function Alarm() Listing 20.6 - Code for the Alarm function KillAlarm() Listing 20.7 - Code for the KillAlarm function MicroDelay() Listing 20.8 - Code for the MicrDelay function GetCMOSTime() Listing 20.9.Code to read the CMOS time GetCMOSDate() Listing 20.10 - Code to read the CMOS date GetTimerTick() Listing 20.11.Code to return the timer tick Beep_Work() Listing 20.12.Helper function for Beep() and Tone() Beep() Listing 20.13.Code to produce a beep. Tone() Listing 20.14 - Code to produce a tone Chapter 21, Initialization Code Introduction OS Global Data Listing 21.1 - Main Operating System Data Operating System Entry Point Listing 21.2 - OS Initialization Code and Entry Point Listing 21.3.Continuation of OS initialization. Listing 21.4.Continuation of OS initialization Listing 21.5 - End of Initialization Code Initialization Helpers Listing 21.6 - Initialization Subroutines Listing 21.7 - Initialize call gates amd the IDT Chapter 22, Job Management Code Introduction Reclamation of Resources Job Management Helpers Listing 22.1 - Job Management Subroutines Job Management Listing Listing 22.2 – Jobc.c Source code Chapter 23, Debugger Code Introduction Debugger Interrupt Procedure Listing 23.1 - Exception Handlers for Entering Debugger Debugger Source Listing Listing 23.2 – Debugger Data and Code Chapter 24, Selected Device Driver Code Introduction IDE Disk Device Driver Listing 24.1 - IDE Device Driver Data (Defines and Externs) Listing 24.2 - Continuation of IDE Driver Data (protos and defines) Listing 24.3 - Continuation of IDE Driver Data (data structures) Listing 24.4 - IDE Device Driver Code (Initialization) Listing 24.5 - Continuation of IDE Driver Code (code) Listing 24.6 - Continuation of IDE Driver Code (Device Interface) The RS-232 Asynchronous Device Driver Listing 24.7 - RS-232.h Header File Listing 24.8 - RS-232 Device Driver Code (defines and externs) Listing 24.9 - RS-232 Device Driver Code Continued (Register bits) Listing 24.10 - RS-232 Device Driver Code Continued (status record) Listing 24.11 - RS-232 Device Driver Code Continued (DCB) Listing 24.12 - RS-232 Device Driver Code Continued (DCB Init) Listing 24.13 - RS-232 Device Driver Code Continued (ISR and code) Listing 24.14 - RS-232 Device Driver Code Continued (Interface) Chapter 25, Keyboard Source Code Introduction Keyboard Data Listing 25.1 - Keyboard Data and Tables Keyboard Code Keyboard ISR Listing 25.2 - Keyboard ISR Code Keyboard Translation Listing 25.3 - Keyboard Translation Code Reading the Final Buffer Listing 25.4.Read Keyboard Buffer Code The Keyboard Service Listing 25.5.Keyboard system service code. Keyboard Procedural Interface Listing 25.6 - Code for Blocking Read Keyboard Call Debugger Keyboard Listing 25.7 - Debugger Read Keyboard Code Keyboard Hardware Set Up Listing 25.8 - Code to Initialize the Keyboard Hardware Keyboard Service Initialization Listing 25.9 - Keyboard Service Initialization Hardware Helpers Listing 25.10 - Low-level Keyboard Control Code Chapter 26, Video Code Introduction Virtual Video Concept VGA Text Video The Video Code Listing 26.1.Video constants and data Listing 26.2 - Code to Initialize Video Listing 26.3 - Code to Set the Video Owner. Listing 26.4 - Code to set Normal Video Attribute. Listing 26.5 - Code to get the Normal Video Attribute Listing 26.6 - Code to Get the Current Video Owner Listing 26.7 - Code to Clear the Screen Listing 26.8 - Code for TTY Stream Output to Screen. Listing 26.9.Code to Screen Video Attributes Listing 26.10 - Code to Place Video chars Anywhere on the Screen Listing 26.11.Code to Get a Character and Attribute. Listing 26.12.Code to Scroll an Area of the Screen Listing 26.13 - Support Code to Set Hardware Cursor Position. Listing 26.14.Code to Set Cursor Position Listing 26.15.Code to Return Current Cursor Position Listing 26.16 - Code to Edit a Line on the Screen Relating MMURTL’s Video Code to Your Operating System Chapter 27, File System Code Introduction How MMURTL Handles FAT Physical (Absolute) Sector Numbers Hidden Sectors Logical Block Address (LBA) MS-DOS Boot Sector File System Initialization File System Listing Listing 27.1 - MS-DOS FAT-Compatible File System Code Chapter 28, DASM; A 32-Bit Intel-Based Assembler Introduction DASM Concepts Using DASM Application Template File Listing 28.1 - Simple Application Template File Listing 28.2 - A More Complicated Template File Command Line Options List File Option Errors Only Option Dynamic Link Libraries Device Drivers DASM Source Files Local (Non-Public) Declarations Memory Model DASM Program Structure Listing 28.3 - Alternating data and code in a template file DOT Commands DASM versus DOS Assemblers DASM Addressing Modes Table 28.1 - .Specifying Memory Operands Storage Declarations DB (Define Byte) DW (Define Word) DD (Define Double Word) DF (Define Far Word) DASM Peculiar Syntax PUBLIC Declarations EXTRN Declarations Labels and Address Segment and Instruction Prefixes CALL and JUMP Addresses 386 Instruction Set Table 28.2 - Supported Instructions Executable File Format Table 28.3 - Tag Usage Tag Descriptions Error Codes from DASM Your Own Assembler? Chapter 29, CM32; A 32-Bit C Compiler Introduction Memory Usage and Addressing Important Internal Differences Variable Length Argument Lists CM32 Language Specifics FAR Calls Interrupt Functions Supported C Language Features Library Header File Access Limitations (compared to ANSI) Structure Limitations Far External Functions Type Conversion and Pointers Initialization of Static Variables Other Implementation Dependent Information Using CM32 Command Line Options Library Functions Listing 29.1 - Supported Library Functions Library Code Examples Listing 29.2 - Standard file I/O for MMURTL Listing 29.3 - Standard String Functions for MMURTL