9 min readAug 12, 2019

–

Recently I was lucky enough to take a one-month sabbatical away from work. While I spent much of that time traveling and staying away from a computer, it’s hard for an engineer to turn off completely. If I was going to create something for amusement, it needed to be distinctly different from my day job. I ended up building a DOS-compatible OS straight out of the 80s.

Press enter or click to view image in full size

Uh, what? Why?

I’ve always been a huge retro-computing nerd. After I began playing with writing emulators last year, I resurrected some of that fanaticism by building a series of MOS 65XX emulators — C64, Atari 2600, NES… Thinking about the internals of these systems led to reading about how early IBM PCs operated. Rather than taking on the significant task of building a PC emulator (I’ll admit, implementing processor instructions gets pretty sleepy after a while), I decided to write a DOS-compatible OS.

Here’s a quick primer/refresher on what DOS-compatible means. Back in the 80s, Microsoft had the most popular PC operating system: MS-DOS. While companies like Compaq built PC clones that were hardware-compatible with IBM’s computers, other companies built operating systems that were code-compatible with MS-DOS. Some added new competitive features, others were released with specific hardware in mind. As long as you implemented the same APIs as MS-DOS, software written for Microsoft’s OS would run on yours as well. This was important when folks in your office needed to run Lotus 1–2–3 on your off-brand system.

My first computer was a 486 PC that eventually ran DOS 6-point-something. While I wrote QBasic programs on that thing, I certainly didn’t write any assembly on it, let alone know what a syscall was. Implementing the DOS API was a deep learning experience through documentation often found in the older corners of the internet.

So What’d I Build?

After a month of off-and-on coding, here’s what I have at the time of writing: a kernel that implements about half of the extended DOS API; basic driver support for disk drives, the console, and the system clock; a FAT-12 filesystem implementation; a COMMAND.COM command prompt that leverages the DOS APIs to run basic internal commands, list directories, and execute other COM programs. Much of it is hacky and half-finished, but it achieves the initial goal of being able to run some made-for-DOS programs.

There’s still a LOT of work I hope to do: better directory support, handling multiple drives at once, piping and redirection, support for FAT-16 and maybe other filesystems, and a text editor. The good news is that after my initial experiments, I went back and restructured the core of the kernel to make it easier to switch between disks, drivers, and filesystems, so many of these things should be possible.

Getting Real

Pretty much every operating system tutorial today starts by getting your CPU out of Real Mode. Before processors implemented modern OS features like virtual memory and hardware protection, running code was a mad free-for-all where any program could overwrite anything in memory — this is known as Real Mode, and every processor in the x86 family starts off this way. You need to tell the processor to jump into Protected Mode before you can access more than 1 MB of memory or leverage any sort of memory safety.

The first versions of DOS were written for the Intel 8086, when Real Mode was the only mode. Through the entire life of MS-DOS, every version was implemented in Real Mode in order to maintain compatibility. As a result, my DOS was also going to need to be a Real Mode program, which flies in the face of all OS guides written in the last 30 years. There’s no paging, GDT, or rings where we’re headed.

One of the weirdest quirks of Real Mode is memory segmentation. Real Mode only uses 16-bit registers for memory addresses, but lets you access a lot more than 64 KB of RAM. This was achieved with a second set of registers called segments: one for code, one for data, and one for the stack. A full 1 MB of memory was addressable using a combination of a **segment **and an offset. The segment was shifted four bits and added to the offset to create a 20-bit address.

Press enter or click to view image in full size

Segmented addresses are represented using four hex digits for the segment, and four hex digits for the offset, separated by a colon: 0x2020:4300. Since each segment is a 16-byte offset, there are many different ways to represent the same location. I’ll talk more about this when talking about executing programs within DOS, but for now I’ll say that dealing with segments is a huge pain when switching contexts. Unfortunately, the DOS APIs rely heavily on segments, so they’re unavoidable at certain points.

Once 32-bit processors were introduced, a full 4 GB was addressable without segmentation, and segments have pretty much been ignored since then.

Booting Up

Booting a PC started with a program called BIOS, which set up the processor and provided basic access to hardware. Nowadays most Intel-based computers use UEFI, but it’s chronologically appropriate for our DOS to rely on BIOS. Before I picked up this project, I underestimated just how much functionality is provided by BIOS. When you’re in Real Mode, you can access a wide range of disk and video functionality through software interrupts.

When BIOS finds a bootable disk, it copies a small chunk of code from the first section of the disk to memory at the address 0x7c00 and begins running it. Ideally this code — the bootloader — will copy your operating system from disk into memory and jump to it.

My bootloader tries to copy much of what the original DOS systems did. They would load the first few chunks of the hardware drivers, IO.SYS, into memory. That first bit of IO.SYS would then load the rest of itself from disk, configure the system, load the OS kernel from another file, and finally start the command prompt. For the sake of simplicity, I’ve combined the drivers and kernel into a single executable, and just load the whole thing at once from the bootloader.

The Kernel

The main function of the kernel is to set up a system call API, which programs can call into and trigger some effect. There’s a lot of complexity under the hood to implement these system calls, but on the surface that’s basically all the kernel does. These system calls are triggered by calling software interrupt 0x21, or as it was often written in code, int 21h. Depending on the values of other CPU registers at the time of the system call, DOS will determine which method to run.

Most of these system calls allow managed access to hardware, like displaying text, reading/writing from the filesystem, or allocating memory. A filesystem call may touch global kernel state, a filesystem implementation, and a physical device driver before it can do something like reading a file.

My kernel is implemented as a simple setup method that initializes all drivers and launches the command line, and a huge interrupt handler that forks depending on which method is called. It saves the state of all registers, runs the syscall, and restores the register state before returning control to the caller. Each interrupt call runs in isolation, so any shared state needs to be in a predictable memory location — no stack is shared between syscalls.

Doing it with Rust

My emulators have all been written in Rust, and it seemed like a good systems programming language for this project. For modern operating systems, Rust is a fantastic choice given its modern feature set and memory safety, but targeting the behavior of DOS is a different story. I would quickly discover that a lot of DOS functionality relies on unsafe behavior, and some standard functions can’t be made to work in a Real-Mode, static executable. Still, it ended up being a fun challenge to minimize the unsafe surface of the kernel and leverage as much idiomatic Rust as possible during this project.

My kernel started like many of the fantastic Rust OS tutorials out there: an executable built with no stdlib, no main, and an externally-located entry point:

#![no_std]#![no_main]#[no_mangle]pub extern "C" fn _start() {  // kernel code here}// Also, we need to override panic behavior#[no_mangle]#[panic_handler]fn panic(_info: &PanicInfo) -> ! {  loop {}}

Much of the scaffolding for building a static target came from the first section of Philipp Oppermann’s thorough guide, with some other pieces pulled in to make the executable run in Real Mode. I wrote a brief entry point in assembly that sets up the segment registers and stack before calling into the _start function.

The kernel and all sub-programs are built as static programs with no relocation. When a program is compiled down to assembly, the addresses of static data or functions need to be known at runtime. Modern executable programs can usually be run in any memory location because the host operating system performs extra setup or rewrites these addresses, but my kernel running with zero scaffolding needs to be static. This did leave to some unexpected challenges, chief among these being that I can’t use the built-in string formatting methods. I actually don’t want to use them — I’m trying to keep my kernel slim since there’s only 1 MB of total memory — but it means that code that might panic will not compile. This usually boils down to two different cases: indexing arrays/slices by a variable that might be out of bounds, or dividing by a variable that might be zero. This is addressed by adding bounds checks around such code, though, which isn’t such a bad thing. In the end, it’s like having a compile-time check for unsafe code, and who can complain about that?

During the development process, I also encountered a weird issue where code past a certain size was having trouble being statically linked — the linker refused to compute certain offsets. This ultimately derived from how the kernel was being built as an executable. Because everything is static, and I don’t rely on anything specific to executable formats, I actually don’t need the kernel to be an “executable.” I rejiggered the build scripts, compile the kernel as a static library, and everything works fine. I honestly don’t know enough about the LLVM linker to debug this further, but my current solution unblocked me for the moment.

Stacks, Heaps, and Statics

Much of Rust’s memory safety focuses on variables allocated on the stack, where they can be cleaned up when they move out of scope, and their ownership can be tracked. My kernel doesn’t use heap space or an allocator, because all objects I deal with are either “stack”-based objects that live for the entire lifetime of the kernel, or static objects with locations known at compile time.

Around the time DOS was written, assembly programs would outline areas of static memory that they would use to store variables. These could be initialized or uninitialized, but they needed to be known at compile time. This meant they couldn’t be variable-sized, but you could still do a lot with static objects and arrays. Between system calls the kernel needs to maintain some state, such as the current directory. In order to address them at predictable locations, I use a few static mut variables, the equivalent of the static memory areas from those original DOS versions. These are considered unsafe in Rust because they can’t be shared between threads, but in a single-threaded kernel they can be used without fear.

Building the OS

I’ve already mentioned a bit about how the kernel gets compiled from the Rust source, but I’ll explain it in more detail. The Rust code is compiled down to a static program, and combined with a simple assembly entry point using a linker script that throws out all the unneeded extras. The helper programs like the command line are also compiled and combined with their own assembly headers. A FAT-12 floppy disk image is initialized, and the bootloader is copied to part of the first sector of the disk. Finally, each of the system files are copied to the disk image using GNU mtools.

The floppy image is loaded up in QEMU, where the bootloader kicks off loading the kernel, drivers, and command prompt:

Press enter or click to view image in full size

Up Next

UPDATE: I’ve continued this post with an exploration of the internals of the kernel. Another post will discuss COM files, how they can be written in Rust, and how they’re able to call back into the DOS API.

Once I return to work and get approval to open up the source code, I’ll make the repo public as well.