Design decisions
Basics
| Topic | Decision | Why |
|---|---|---|
| Language | Rust nightly, #![no_std], some inline assembly | memory safety; nightly is needed for custom_test_frameworks and abi_x86_interrupt |
| Target | x86_64-unknown-none (built-in) | bare metal, soft-float (no SSE in the kernel) → no custom target JSON needed |
| Linker | GNU ld (ld.bfd, binutils 2.46), flavor ld | my preference for GNU tools; linker script kernel/linker.ld |
| Architecture | x86_64 | matches the host PC, best documentation (OSDev, Intel SDM) |
| Bootloader | Limine v11, boot protocol base revision 6, crate limine 0.6.5 | starts directly in long mode with paging, provides memory map, HHDM and framebuffer |
| Boot medium | hybrid ISO (BIOS + UEFI) built with xorriso | boots in QEMU with -cdrom, both firmware paths testable |
| External crates | only limine, spin | everything else is written from scratch |
| Tools | GNU make, gdb, binutils (objdump/nm/readelf/addr2line), gcc for host helpers | my preference: Rust kernel, GNU tooling around it |
Memory layout
- The kernel lives in the higher half at
0xffffffff80000000(top 2 GiB),code-model=kernel,relocation-model=static. - Limine additionally maps all physical memory at the HHDM offset (
boot::HHDM): physical addresspis reachable atp + hhdm_offset. - Linker sections:
.limine_requests(between start and end marker),.text,.rodata,.data/.bss, each page-aligned with its own PT_LOAD segment.
Physical memory
mem::pmmis a bitmap allocator: one bit per 4 KiB frame (1 = used), covering physical memory up to the end of the highest usable region.- The bitmap lives in the first usable region big enough for it and is reserved in itself. Frame 0 is never handed out.
- The bitmap covers
USABLEandBOOTLOADER_RECLAIMABLEmemory; the latter starts as used (it holds Limine's stack, page tables and responses) and is handed over bymem::reclaim_bootloader_memory(M33) at the start ofkernel_main: page tables living there are copied (paging::copy_kernel_tables– tables elsewhere are kept and only repointed), every thread's saved CR3 and every CPU's CR3 move to the new PML4, thenpmm::release_regionfrees the frames. Limine responses must not be read afterwards –pmm::initkeeps a copy of the memory map (pmm::boot_memmap). Test kernels other thantests/reclaim.rsdo not reclaim (they still run on Limine's stack). - All access to physical memory goes through the HHDM (
PhysAddr::to_virt). Allocator calls run with interrupts disabled. free_framepanics on double free / invalid addresses – that is always a kernel bug.
Virtual address space
| Range | PML4 slot | Use |
|---|---|---|
0x0000_0000_0000_0000 – 0x0000_7FFF_FFFF_FFFF | 0–255 | user space: private per process (see "User space") |
0xFFFF_8000_0000_0000 + phys | 256 | HHDM (Limine): all RAM + framebuffer, no MMIO |
0xFFFF_FE00_0000_0000 | 508 | kernel heap (grows upward, max 1 GiB) |
0xFFFF_FE80_0000_0000 | 509 | scratch area for paging unit tests |
0xFFFF_FF00_0000_0000 | 510 | kernel stacks: [guard page][stack pages] per stack, never reused |
0xFFFF_FFFF_8000_0000 | 511 | kernel image (linker script) |
paging::ensure_hhdm_mapped(phys, size, flags)maps pages Limine left out of the HHDM (ACPI tables, LAPIC0xFEE0_0000, I/O APIC0xFEC0_0000) at their usual HHDM address, so every physical address is still reached asphys + hhdm_offset.- Page tables: we modify Limine's tables through the HHDM; intermediate tables come from the frame allocator and are not freed.
paging::initremembers Limine's PML4 askernel_root()and creates all 256 upper-half PML4 entries up front (empty PDPTs). Every process PML4 copies these entries, so later kernel mappings (heap growth, new stacks, MMIO) land in shared lower-level tables and are visible in every address space. The PML4 entries themselves never change after boot.- Limine sets the USER bit on its upper-half intermediate tables (leaves are supervisor-only).
paging::initclears USER in all upper-half PML4 entries, so ring 3 cannot reach the kernel half, whatever the lower levels say. - Limine enables EFER.NXE (so
NO_EXECUTEis usable) and CR0.WP (the kernel cannot write read-only pages either). kmainswitches from Limine's stack to a 64 KiB guarded stack (mem::stack) immediately afterkernel::init().
Threads and scheduling
- SMP (since M26):
current[cpu]/idle[cpu]per CPU, one shared ready queue, every CPU's LAPIC timer preempts. Each thread hason_cpu(set while a CPU executes it or is still switching away from it).schedule()takes the scheduler lock, picks the next thread, forgets the guard and switches; the next thread callsfinish_switch(), which clears the previous thread'son_cpuand releases the lock (force_unlock). New threads do that first thing inthread_start. - Blocking:
wait_for(reason, ready)marks the current threadBlocked(reason), then evaluatesready()(interrupts off, scheduler lock not held –readymay take other locks), then schedules. Wakers (notify,wake,make_ready): a blocked thread that is stillon_cpuis set back toRunning(it will see its condition), otherwise it is queued and an idle CPU gets aWAKE_VECTORIPI. Lock order:PROCESSES→SCHEDULER; pipe/keyboard/serial locks are leaves. - TLB: kernel virtual ranges are never reused after unmapping (stacks) or never unmapped (heap), so the kernel half needs
no shootdowns. User pages of a process with threads may be cached on several CPUs:
sys_brkunmaps without freeing (unmap_keeping_frames), callspaging::shoot_down(root)(arun_onjob per CPU that reloads CR3 if it runs inroot) and only then frees the frames. A CPU always reloads CR3 when it switches address spaces. - Sleeping lock (
sync::Mutex, M36): CAS on a flag; a contendedlockblocks onWait::Lock(address)(spins only before the scheduler runs or with interrupts off); unlock notifies when the waiter count is non-zero. Used fordrivers::DISKandfs::ROOT, whose holders block on the AHCI interrupt during I/O. Not for interrupt handlers. task::Scheduler(behind a spin lock, only touched with interrupts disabled):BTreeMap<Tid, Box<Thread>>(boxed so the saved RSP has a stable address), a FIFO ready queue, the current TID and the idle TID. Thread 0 = the boot code ("main"), runs the shell.- Context switch:
kernel_switch_context(old_rsp_ptr, new_rsp)pushes rbp/rbx/r12–r15, swaps RSP, pops,ret. A new thread's stack is prepared so thatretlands inkernel_thread_trampoline, which callsthread_start(entry)with the boxed closure from r12. schedule()must run with interrupts disabled. A resumed thread continues with interrupts off and restores them itself (viawithout_interrupts,iretqof the interrupted handler, orstiinthread_start).- Preemption: every timer tick, if another thread is ready. The timer handler sends EOI first, then calls
task::on_tick. - Wake-ups: timer (
Until), keyboard/COM1 IRQ on the BSP (Input),exit(Join), pipes (Pipe(id)),kill(wake). - Dead threads are reaped (stack unmapped and frames freed) by a later
schedule()call – never while running on their stack.
Storage
- PCI: legacy configuration mechanism (ports 0xCF8/0xCFC), brute-force scan of all bus/device/function numbers.
- AHCI (q35's ICH9 controller, BAR5): one driver instance per port, probed once at boot into
drivers::DISK. Per port: one frame for command list + received FIS, one for the command table (FIS + 1 PRD), one 4 KiB DMA bounce frame. Commands are polled (no interrupts yet) and limited to 8 sectors; larger requests are split. drivers::DISKis a spin lock held with interrupts enabled during I/O – never use it from interrupt handlers.- File systems only see the
BlockDevicetrait (512-byte sectors); tests useRamDisk. - Disk images: FAT32 without partition table ("superfloppy"), 64 MiB, 512-byte clusters (FAT32 needs ≥ 65525 clusters),
created by
scripts/mkdisk.shfromrootfs/. QEMU:-device ide-hd,bus=ide.0= AHCI port 0 (the boot CD is ATAPI on port 2). Run mode uses the persistentbuild/disk.img; every test binary and the E2E test get fresh images.
Disk caching and interrupts (M32)
- The root file system runs on
BlockCache<SystemDisk>: write-through (the disk is always current, a power-off loses nothing), 1024 sectors, LRU eviction of the oldest eighth when full. Runs of missing sectors are fetched with one request. FatFsadditionally keeps the last cluster chain and the last path lookup; any write to the volume drops both.- AHCI: MSI to the boot CPU's LAPIC on vector 0x50; the handler acknowledges all ports, remembers task-file errors per port
and notifies
Wait::Disk. Commands wait withwait_forwhen interrupts are on and the scheduler runs, else they poll. TheDISKlock stays held while waiting (other threads wanting the disk spin; fine for now).
File system
- One FAT32 volume on the system disk, mounted at boot as
fs::ROOT(a spin lock likeDISK: never from interrupt handlers).FatFs<D: BlockDevice>is generic, so the same code runs onSystemDisk(adapter fordrivers::DISK) and onRamDiskin tests. - Mount: FAT32 boot sector at LBA 0 (superfloppy), otherwise the first MBR partition of type 0x0B/0x0C. Only 512-byte sectors.
- Caching: every sector – FAT and data – goes through the write-through
BlockCachebelow the file system (see "Disk caching and interrupts");FatFsitself keeps the FAT sector it used last, the last cluster chain and the last path lookup. FAT updates are written through to every FAT copy immediately. Every mutating operation ends withsync(FSInfo hints + disk flush), so apoweroffor reset right after a command never leaves the volume inconsistent. A power loss or disk error during a command is a different case. What holds then is measured (M53,fs::fat32::fault_tests: every I/O of create, replace, append,write_at, mkdir, rename and remove made to fail once, as a transient error and as a power loss with a torn write, then the volume re-mounted and checked):- the command returns
FsError::Io, never panics, and the volume mounts again; - files the command did not touch are intact; its target is in the old or the new state (
write_atmay have written part of its bytes); - the damage is what
fsck.fatrepairs without losing data that was there before: lost clusters (new clusters not yet in a directory entry, or old ones not yet freed), the two FAT copies differing in one sector, after append/write_ata chain longer than the file size, after rename both names pointing to the same clusters (the new entry is written before the old one is freed – the order Linux's vfat uses too); - after a transient error the same mount keeps working (later writes do not damage the volume).
There is no journal: FAT32 has none, and the kernel does not run
fsckat boot.
- the command returns
- Names: a name that fits 8.3 in one case per part (
hello.txt,README) is stored as a plain short entry with the NT lower-case flags, like Windows/Linux do. Everything else gets LFN entries plus a unique 8.3 alias (MYNOTE~1). Lookup is case-insensitive on long and short names. - Paths at the FS API are absolute and normalized (
/a/b); the shell owns the current directory and callsfs::normalize. rmrefuses non-empty directories;mvonly rewrites directory entries (and the..entry of moved directories).- Timestamps come from the CMOS RTC (
drivers/rtc.rs, treated as UTC). - Validation outside the kernel: after
make test,scripts/check-test-disks.shrunsfsck.fat -non every test disk and checks kernel-written files with mtools; the E2E test does the same for its disk after two boots.
User space
- Address space (
mem::address_space::AddressSpace): own PML4, user half empty, kernel half shared. Frames are filled via the HHDM, so loading never switches CR3.Dropfrees every user frame and page table (paging::free_user_half); all user mappings own their frames. It must not be the active address space when dropped. - User layout: ELF segments between
USER_IMAGE_START(64 KiB, null page area stays unmapped) andUSER_IMAGE_END; our programs link at0x40_0000. The heap starts at the first page after the image and is moved withbrk(max 64 MiB). Stack: up to 1 MiB (USER_STACK_MAX) belowUSER_STACK_TOP = 0x7FFF_FFFF_0000, the top 4 pages mapped at start, the rest on demand (M35); nothing is mapped below it (overflow → page fault → process killed). Below that gap: 64 thread stack slots (M52), each[64 KiB guard][256 KiB stack]; a slot belongs to a thread started withthread_spawn(entry, 0, arg), its stack pages are demand-paged, the guard never. A fault in a guard kills the process with "stack overflow in a thread". Slots of ended threads are unmapped (TLB shoot-down first) at the nextthread_spawnorthread_join.USER_IMAGE_ENDleaves room for the largest heap plus a guard below the slots (bug #21). The argument block is copied to the stack top;_startgets pointer and length in RDI/RSI. Arguments inside the block are separated by NUL bytes (ulib::Args); the shell joins its words that way. - ELF loader (
process::elf): staticET_EXECx86_64 only,PT_LOADsegments with R/W/X → USER, WRITABLE, NO_EXECUTE. Every header field is bounds-checked; segments outside the user image area or an entry point outside an X segment are refused. - Processes (
process::PROCESSES, interrupts off while locked): one kernel thread per process.task::spawn_usercreates a thread with its own CR3 and aUserEntry;thread_startdrops the closure and theniretqs to ring 3 (enter_user). The thread's kernel stack is abandoned at that point and reused from its top by every interrupt/syscall from ring 3. - Scheduler: each thread has
cr3; on a switch the scheduler loads it (if different) and sets TSS.RSP0 + the syscall stack to the next thread's kernel stack top (syscall::set_kernel_stack). Switching CR3 on the old thread's stack is fine: kernel stacks are in the shared half. - End of a process:
exitsyscall, CPU exception with CS.RPL = 3 (handlers callprocess::kill_currentinstead of panicking), or a fatal signal (process::deliver_signals, run when the timer/wake IPI interrupts user code and on every syscall return).exit_currentrecords the status, moves the thread to the kernel CR3, frees the address space and exits the thread. The status stays in the table untilprocess::wait(zombie); the shell always waits. - System calls (
syscall/sysret): STAR = kernel CS 0x08 / sysret base 0x10, LSTAR =syscall_entry, FMASK clears IF/TF/DF/AC. The stub doesswapgs(IA32_KERNEL_GS_BASE = this CPU'sCpuLocal), parks the user RSP ings:[0], loads the kernel stack fromgs:[8], pushes the user RSP and swaps GS back before enabling interrupts (a preempted syscall may resume on another CPU, so GS must never stay swapped), saves aSyscallFrame, enables interrupts and callsprocess::syscall::dispatch. Convention like Linux: number in RAX, args in RDI/RSI/RDX/R10/R8/R9, result in RAX, negative =-errno; only RCX/R11 are clobbered. Syscalls may block (read, sleep).# Name 0 exit(code)never returns 1 write(fd, buf, len)fd 1/2 → console (or a capture buffer in tests) 2 read(fd, buf, len)fd 0 → keyboard/serial keys as terminal bytes ( \n,0x08, UTF-8), blocks for ≥ 1 byte3 sleep_ms(ms)-EINTRwhen a signal arrives4 getpid()5 uptime_ms()6 yield()7 open(path, len, flags)flags READ=1, WRITE=2, CREATE=4, TRUNCATE=8, APPEND=16; a directory opens for readdir8 close(fd)9 readdir(fd, *DirEntryInfo)1 = entry written, 0 = end; 296-byte repr(C)struct shared withulib10 stat(path, len, *Stat)32-byte repr(C): size, kind (1 file, 2 dir), modified time11 getcwd(buf, len)-ERANGEif too small12 unlink(path, len)files and empty directories 13 mkdir(path, len)14 brk(new_end)0 = query; growing only moves the end (pages come on first touch), shrinking frees used pages; -ENOMEMoutside the range15 spawn(path, len, args, args_len)4th argument in R10; child inherits cwd and output; -ENOENT,-ENOEXEC,-E2BIG16 wait(pid, *WaitStatus)children only ( -ECHILD); 128-byte status: kind (1 exited, 2 killed), code (exit code / signal number), kill reason17 kill(pid, sig)any user process ( -ESRCH); sig 0 = existence check18 chdir(path, len)-ENOTDIRfor files19 kshell(line, len)transitional: runs a kernel shell built-in (not cd/pwd/history/run) in the caller's cwd and output 20 kcommands(buf, len)NUL-separated names usable with kshell21 pipe(*[u32; 2])read end, write end 22 dup(fd)lowest free fd for the same object 23 dup2(fd, new_fd)closes what was at new_fd24 isatty(fd)1 for the console (stdin, stdout to the console), else 0 25 sysinfo(*SysInfo)176 bytes: uptime, ticks, memory, kernel heap, FS size/free/label, CPUs, CPU vendor/brand, date 26 proclist(*[ProcInfo], max)one 64-byte entry per thread (tid, cpu ticks, parent, state, user?, name) 27 rename(from, len, to, len)28 seek(fd, offset, whence)SET/CUR/END for files, -ESPIPEotherwise; past the end is allowed (gap = zeros on write)29 termsize()console rows << 32 | columns (the serial side has no size; programs draw for the console) 30 ignore_interrupt(on)Ctrl+C arrives as 0x03 instead of killing the caller (the editor) 31 net_info(*NetInfo)40 bytes: MAC, link, IPv4, netmask, gateway, frame counters; -ENODEVwithout a card32 ping(ip, seq, timeout_ms)round trip in µs; -ETIMEDOUT,-EHOSTUNREACH33 udp_socket(port)0 = ephemeral (≥ 0xC000); -EADDRINUSE34 sendto(fd, buf, len, ip, port)≤ 1472 bytes, -EMSGSIZEabove35 recvfrom(fd, buf, len, *SockAddr, timeout_ms)5th argument in R8; -ETIMEDOUT,-EINTR36 clock_us()microseconds since boot (TSC) 37–43 TCP, shutdown,setenv/getenv/environsee kernel/src/process/syscall.rs44 sigaction(sig, disposition, trampoline)SIG_DFL0,SIG_IGN1 or a handler; old disposition; SIGKILL →-EINVAL45 sigreturn()called by the trampoline when a handler returns (unblocks the next one) 46 thread_spawn(entry, stack, arg)tid; ≤ 64 running further threads ( -EAGAIN); entry/stack must be user addresses; stack 0 = a kernel slot with a guard47 thread_exit(code)ends the thread; in the main thread the process 48 thread_join(tid)exit code as u32; -ESRCHif not a further thread of the caller or already joined49 futex_wait(addr, expected, timeout_ms)-EAGAINif*addr != expected,-ETIMEDOUT,-EINTR50 futex_wake(addr, n)number of threads woken 51 gettid()getpid()returns the process id in every thread52 uname(buf, len)"YAK x86_64", -ERANGEiflenis too small (the example ofdocs/HACKING.md) /bin/edit:buffer.rsis pure logic (tested byedit --selftest),main.rsdraws withESC[r;cH,ESC[K, SGR 7. The text area is all rows but the last; the cursor line and the status bar are redrawn per key, the whole screen only after scrolling or when lines are inserted/removed. Writing the bottom-right cell is safe: the console wraps lazily (only when the next character comes).- Pipes (
process::pipe): bounded queue (64 KiB) + reader/writer counts. A descriptor holds aPipeEnd;Clone(dup, dup2, spawn) increments,Drop(close, process exit) decrements and notifies. Read blocks while empty with writers, returns 0 without; write blocks while full with readers,EPIPEwithout. Waiters sleep onWait::Pipe(id); the pipe lock is only taken with interrupts disabled (descriptors are dropped under the process table lock). Kills abort the wait (EINTR). - Descriptor inheritance:
spawnclones the parent's fds 0–2 into the child (other fds are not inherited). A shell redirects bydup2-ing a pipe end or file over 0/1 beforespawnand restoring its saved copies afterwards. - Process tree:
parent(None = started by the kernel; only the kernel waits for those). When a process ends, finished children are dropped and running ones become orphans, removed from the table at their own exit (nobody waits for them). - Threads (M36):
thread_spawn(entry, stack, arg)starts a kernel thread with the process's CR3 whoseowneris the main thread;task::current_owner()is the pid, sowith_current,getpid, files and signals are per process.Process::threadsmaps further tids toNone(running) / exit code (untilthread_join). The end of a process (exitin any thread, a fatal signal or a fault) setsexiting;signal_pending()is then true for every thread (blocking calls give up) anddeliver_signalssends any thread that is about to enter ring 3 intoexit_current. A further thread just wakes the others and exits; the main thread joins all further threads, then records the status, frees the address space and exits – sowait(which joins the main thread) only returns when every thread is gone. Signals are taken by whichever thread returns to user mode first. - Futexes (
process/futex.rs):futex_wait(addr, expected, timeout)reads the word under theFUTEXESlock (insidewith_current, so no other thread can unmap it meanwhile), queues the thread under the key (pid << 47 | addr) and blocks (Wait::Futex, orWait::Untilwith a timeout);futex_wake(addr, n)moves up to n waiters to thewokenset andtask::wakes them.ulib::sync::RawLock(0 free / 1 locked / 2 contended) only calls the kernel when contended; the user heap uses it,ulib::thread::spawnasks for a kernel stack slot (256 KiB with a guard, see "User layout"). - Signals (M34,
process/signal.rs): per process a disposition per signal (default = end, ignore, handler), a pending mask and anin_handlerflag.send_signalsets the bit andtask::wakes the target; blocking syscalls use abortable waits (input::read_key_or_abort,task::join_or_abort,time::sleep_ms_or_abort, pipes, sockets) that give up whensignal_pending()and returnEINTR. Delivery happens on the way back to ring 3 – at the end ofdispatch(it may changeSyscallFrame.rip/rsp) and in the timer / wake-IPI handlers (they change the CPU-pushedInterruptStackFramewith volatile writes). SIGKILL first, then the lowest pending signal; default →kill_current(reason)(exit statusKilled(reason),waitreports the number viasignal::number_for_reason, exceptions = SIGSEGV). - Handlers without a kernel register frame: the kernel only pushes
sig, handler, rflags, ripbelow the user RSP and jumps to the trampoline registered withsigaction(ulib::signal). The trampoline pushes all 15 GPRs and XSAVEs (or FXSAVEs), calls the handler, callssigreturn, restores everything and ends withpopfq; ret(which also restores RSP). So neither the syscall stub nor the interrupt handlers need a full register frame. Price: no red zone in user code (the target setsdisable-redzone; the kernel writes right below RSP). While a handler runs, further handled signals stay pending (default actions still happen); a signal arriving during the trampoline's epilogue nests safely, because everything below RSP has been consumed by then. The trampoline must be a user address (< USER_END, it becomes asysrettarget).spawnresets handlers but keeps ignored signals (likeexec).FOREGROUND= pid owning the console: set by the kernel shell for its programs, inherited byprocess::wait_childwhile the foreground process waits for that child (handed back when the wait ends);spawnalone never changes it (bug #4). The keyboard and COM1 interrupt handlers recognise Ctrl+C themselves (raw 0x03 / Ctrl tracker on set-1 scancodes, the C key is 0x2E on QWERTY and QWERTZ) and callprocess::interrupt_foreground: the key is swallowed and SIGINT sent (default: killed, "interrupted"), unless the process hasignore_interruptand no SIGINT handler – then the key reaches it as byte 0x03. - init:
kernel_mainrunsprocess::run_init("/bin/sh")(foreground,ignore_interrupt) and waits; when it is missing or exits, the kernel shell runs on thread 0 as before. /bin/sh(user/src/bin/sh/): compiles the kernel'sshell/line_editor.rsandshell/parser.rswith#[path]; both sides provide akeysmodule (Key,KeyEvent) next to them. Pipelines: for each stage, stdin = previous pipe (or<file), stdout = new pipe (or>/>>file), then the stage runs (built-in inline, program spawned); all children are waited for in order, only the last exit code is reported, kills always; an "interrupted" stage kills the rest.read(0)delivers keys as terminal bytes (VT100 sequences for arrows, Home/End, Delete, PageUp/Down, Insert);keys::Decoderturns them back intoKeyEvents. Order of lookup: sh built-ins, functions, kernel built-ins (kshell), program path /$PATH.- sh grammar (M35,
syntax.rs): the text is tokenized into raw words (quotes kept,$((...))one word) and separators, then parsed intoList→AndOr→Pipeline→Command(simple / compound + redirections / function definition). Words are expanded only when a command runs (expand_words: variables, arithmetic, wildcards, then the kernel'sparser::split), so loops see new values. Interactive input that parses asIncomplete(open quote, missingfi/done/}) continues with the prompt>; scripts andsh -care parsed as a whole first. Execution returns aFlow(Next,Exit,Break(n),Continue(n),Return,Interrupted) up the call chain. Compound commands and functions in a pipeline run inside the shell like built-ins.set -eis ignored insideif/whileconditions and for all but the last pipeline of&&/||. Ctrl+C: the shell's SIGINT handler (and a child killed by SIGINT) setsINTERRUPTED, which every list and loop checks; at the prompt the line is cancelled. Wildcards: the expander marks unquoted*/?with private-use characters,glob.rsmatches them per path component. - Command substitution and
"$@"(M55):$(...)stays inside its word when tokenizing (syntax::subst_endskips quotes and nested parentheses). At expansionShell::substituteparses the text and runs it in the shell with stdout on a pipe, while a second thread (ulib::thread) drains the pipe – so the output can exceed the 64 KiB pipe. Afterwards variables, the directory, functions and$@are restored, which gives subshell semantics without a new process (functions keep working inside);exitends only the substitution,$?is its last status. Because expansion produces text thatparser::splitsplits again, substituted output and arguments are inserted quoted: in double quotes as one escaped string, unquoted as one quoted word per field, afterNAME=as one word (no splitting)."$@"closes and reopens the quotes between the arguments; without arguments it produces no word. Variables live in the kernel's per-process environment: values up to 4096 bytes, up to 256 variables – a larger assignment is an error. caseandlocal(M56):;;is its own token; a pattern is a raw word ending in)(an optional leading(, alternatives split at unquoted|), its body a list ended by;;oresac. Subject and patterns are expanded without splitting; unquoted*/?in a pattern are wildcards (glob::matches), no[...]classes.localrecords the variable's old value in the running function's frame (Shell::locals);callrestores the frame in reverse order when the function returns, so nested functions see the caller's locals (dynamic scope, like bash).- Files: each process has a cwd (inherited from the spawner, paths are normalized against it) and up to 32 file descriptors
(
process::Handle): 0 stdin, 1/2 the process'sOutput, then files (path + one offset for reads and writes, shared withdup-ed and inherited copies like a POSIX open file description – the shell's save/restore of fds 0-2 depends on it; writes useFatFs::write_at, orappend_filewith the APPEND flag) and directories (snapshot of the entries atopen). Handlers copy what they need out of the table, do the disk I/O without holdingPROCESSES(onlyfs::ROOT), then update the descriptor.readof a file usesFatFs::read_at, ≤ 64 KiB per call. FS errors map to Linux errno values (ENOENT,EISDIR, ...). - User pointers: every buffer is checked page by page against the process's page tables (present + USER, + WRITABLE for
read) before the kernel touches it; then it is accessed directly (the process CR3 is active) insidecpu::user_access(M37): with SMAP that isstac…clac, the only window in which the kernel may touch user pages (copy_from_user,copy_to_user, the futex read). SMEP keeps the kernel from executing user pages. Both are enabled per CPU bycpu::enable_protectionwhen CPUID has them; QEMU runs-cpu max(TCG) so they are there (KERNEL_CPU=qemu64runs without SMEP/SMAP/XSAVE/AVX - the fallback paths).tests/smap.rschecks that a kernel read and a kernel jump into a user page fault. - User programs (
user/, crateulib+src/bin/*): own targetuser/x86_64-yak-user.json(x86-64 with SSE2 and the normal hard-float ABI, static,code-model=small, red zone allowed – interrupts from ring 3 switch stacks), built with-Zbuild-std=core,alloc(+compiler-builtins-mem; needsrust-srcandgccas host linker for build scripts), linked by GNU ld at0x40_0000(user/linker.ld).make usercopies them tobuild/user/bin;scripts/mkdisk.shand (for the persistent disk)run-qemu.shinstall them into/binviascripts/install-user.sh. - User heap (
ulib::heap): first-fit allocator, free list sorted by address with coalescing, 16-byte units, grows the break by ≥ 64 KiB at a time. No lock (programs are single-threaded). - Demand paging (M33, stack since M35): the heap
[heap_start, align_up(heap_end))and the stack area are only reserved ranges. A user page fault with the present bit clear inside it maps one zeroed RW/NX page (process::handle_page_fault, called first in the page-fault handler) and returns, so the instruction is retried; any other fault, or no free frame, kills the program as before. Syscalls callProcess::populatefor user buffers beforeis_accessible, so the kernel never faults on them. - Environment (M31):
Process::env(BTreeMap), copied into children byspawn; syscallssetenv/getenv/environ./bin/shkeeps no variables of its own – shell variables are environment variables. - Shell: a word that is not a built-in is a program path or a name in
/bin;run <path> [args]does the same explicitly. Programs run in the shell's cwd.ls,catandcpare programs (no built-ins any more). Non-zero exit codes and kills are reported as errors.Shell::capture_programsmakes program output go to the command'sout(unit tests). - FPU/SSE/AVX (
arch::x86_64::fpu): the kernel is soft-float and never uses x87/MMX/XMM/YMM, so user register contents survive syscalls and interrupts untouched. Each thread has a boxed, 64-byte aligned 1 KiBFpuState(initialised from a clean image taken afterfninit+ldmxcsr 0x1F80);schedule()saves the outgoing (unless dead) and restores the incoming thread's state right beforekernel_switch_context. With XSAVE (M37): CR4.OSXSAVE, XCR0 = x87 | SSE | AVX (set byxsetbvon every CPU),xsave64/xrstor64with the XCR0 mask, area size from CPUID 0xD (832 bytes, must fit 1 KiB); without XSAVEfxsave64/fxrstor64. No lazy switching (CR0.TS stays clear), no AVX-512. - User programs and AVX: the user target stays at SSE2, so every program runs on any CPU; code that wants AVX checks
ulib::avx_enabled()(CPUID OSXSAVE + XCR0) and uses#[target_feature(enable = "avx")](fputest,sigtest). The signal trampoline saves withxsave64(header zeroed first) whenulib::XSAVE_ENABLED.
Time
time::ticks(100 Hz, boot CPU's LAPIC timer) for sleeping and timeouts;time::uptime_usfrom the TSC (frequency measured at boot against PIT channel 2) for precise time stamps. All CPUs' TSCs are assumed to run in step.
Network
- Driver (
drivers/e1000.rs): Intel 82540EM as emulated by QEMU, BAR0 MMIO (128 KiB, mapped viaensure_hhdm_mapped). RX ring 32 × 16-byte descriptors (one frame), TX ring 8; DMA buffers 2 KiB (two per frame). RCTL: broadcast accepted, CRC stripped; TCTL: pad short packets. TX descriptors start with DD set so "DD = slot free". Interrupts (M38): no MSI on this model, and the INTA# → I/O APIC wiring would need the ACPI_PRT(AML).net::probe_irqroutes each free PCI input (GSI 16-23, level-triggered, active low) in turn to vector 0x51 and makes the card raise "link status change" by software (ICS); the input where it arrives is kept. The handler readsICRthrough a register base saved at probe time (no NIC lock: the net thread holds it with interrupts on), which clears the causes and releases the level-triggered line, setsRX_PENDINGand wakes the net thread. Enabled causes: RXT0, RXO, RXDMT0, LSC. No interrupt found → polling. - Stack (
net/): configured by DHCP at boot (net/dhcp.rs), static fallback = QEMU's slirp defaults (10.0.2.15/24, gateway and host = 10.0.2.2, DNS 10.0.2.3). TCP (net/tcp.rs) is described in its module docs; packets to our own address go through a loopback queue handled by the net thread. DNS is a user-space library (ulib::dns). Thenetkernel thread runs a round when the card interrupts or a packet is looped back (RX_PENDING+task::wake), at the latest every 10 ms; each round ends withnotify(Wait::Net)– that doubles as the tick for timeouts of everyone waiting on the network and for TCP's retransmission timer. Ping to the gateway: ~0.5 ms (polled: 0-10 ms). - DHCP lease (
net/dhcp.rs): the "dhcp" thread renews at T1 (unicast REQUEST withciaddr), retries after half the remaining time (≥ 60 s), rebinds by broadcast from T2, falls back to the static configuration at the end. - TCP windows (M38): window scaling (RFC 7323, shift 3) when both SYNs carry the option; receive and send buffers
256 KiB, up to 128 segments in flight. ARP: cache learned from requests/replies addressed to us and from IPv4 packets of on-link hosts;
resolvesends up to 3 requests. IPv4: no options sent, fragments dropped, DF set. ICMP: echo requests answered, replies with our id recorded. UDP: ports map to bounded queues (64 datagrams); checksums sent and verified on receipt (a datagram with a wrong checksum is dropped; checksum 0 = "none computed" is accepted, as RFC 768 allows for IPv4). - Network faults (M58):
net::set_loopback_loss(n, phase)drops every n-th packet sent to ourselves (always compiled, off by default – an atomic load per loopback packet);tcp::set_max_retrieslowers the retry limit (default 8),tcp::retransmissionscounts expired retransmission timeouts.kernel/tests/netfault.rschecks with them: with every 10th packet lost (first loss = SYN, SYN-ACK or the first ACK) an echo of 8 KB comes back intact; a connect into silence fails withTimedOutafter its timeout; a peer that goes silent ends the connection after the retries (measured 3.0 s for 3 retries from a 200 ms timeout: 0.2 + 0.4 + 0.8 + 1.6 s). Without an RTT sample (only the handshake was acknowledged) the timeout starts at 1 s – with the default 8 retries a dead peer is noticed after about a minute. DNS (ulib::dns) is checked against a name server that misbehaves (E2E): NXDOMAIN, server failure, an answer without A record, a stray and a truncated reply before the real one, and silence (3 tries of 1.5 s, then "timed out"). - User API:
ping(one echo, waits ≤ timeout),udp_socket→Handle::Udp(Arc<UdpSocket>)(port released when the last descriptor closes),sendto/recvfrom(timeout, abortable by kill),net_info. Small state locks are taken with interrupts off; the NIC lock with interrupts on (it is never needed bywait_forconditions).
Kernel heap
mem::heap::LinkedListHeap: free blocks form a singly linked list sorted by address; each free block storessize+nextin its first 16 bytes.- First fit; a block is split into [front padding][allocation][rest], padding and rest go back to the list. Freeing inserts by address and merges neighbours.
- All addresses and sizes are multiples of 16, so leftovers are either 0 or big enough for a header (no lost bytes).
KernelHeapwraps it in a spin lock (interrupts disabled while held) and grows by mapping ≥64 KiB at the heap end when an allocation fails.- Allocation failure → null → Rust's default alloc error handler panics.
SMP (multiple cores)
arch::x86_64::smp::init(end ofkernel::init): every enabled MADT CPU except the BSP gets INIT, 10 ms, SIPI(s), one at a time. Trampoline: one frame below 1 MiB reserved bypmmat boot (pmm::low_frame), identity-mapped in the kernel PML4 only while APs start. It uses its own tiny GDT (code32/data32/code64/data64), sets PAE, CR3 = kernel PML4 (must be < 4 GiB), EFER.LME + NXE, CR0.PG + WP, and callsap_main(cpu)on a 64 KiB guarded stack.- CPU index:
smp::cpu_index()maps the local APIC ID to 0..MAX_CPUS-1 (BSP = 0; 0 before the LAPIC is set up). Used for the per-CPU TSS. (M26 will move this to a GS-based per-CPU block.) - APs (M25): own GDT/TSS (+ double fault IST stack), shared IDT, FPU/SSE, syscall MSRs, LAPIC (timer masked); they idle with
sti; hltand run jobs from a per-CPU mailbox (run_on, woken by an IPI onWAKE_VECTOR0xF0). Since M26 every AP also runs threads (its start-up context becomes its idle thread, its LAPIC timer preempts). Device IRQs (keyboard, COM1) stay on the BSP.
GDT / TSS
| Selector | Entry |
|---|---|
| 0x00 | null |
| 0x08 | kernel code (64-bit, DPL 0) |
| 0x10 | kernel data (DPL 0), also used for SS/DS/ES/FS/GS |
| 0x18 | user data (DPL 3), selector 0x1B with RPL 3 |
| 0x20 | user code (64-bit, DPL 3), selector 0x23 |
| 0x28 | TSS (16 bytes, occupies 2 slots) |
One GDT and TSS per CPU (gdt::init_cpu), same layout everywhere. The user data/code order is dictated by sysret (SS = STAR base + 8, CS = base + 16). IST1 (gdt::DOUBLE_FAULT_IST) =
20 KiB static stack for double faults. RSP0 = kernel stack of the running thread, rewritten on every context switch.
Interrupt vectors
| Vector | What |
|---|---|
| 0–31 | CPU exceptions (arch/x86_64/interrupts.rs), double fault (8) on IST1 |
| 32 | local APIC timer, periodic 100 Hz (drives time::tick) |
| 33 | ISA IRQ1 – PS/2 keyboard (I/O APIC GSI 1) |
| 36 | ISA IRQ4 – COM1 receive (I/O APIC GSI 4) |
| 39 / 47 | spurious IRQ 7 / 15 of the masked 8259 (ignored, no EOI) |
| 0x50 | AHCI command completion (MSI, M32) |
| 0x51 | e1000 (INTA# through the I/O APIC input found by net::probe_irq, level / active low, M38) |
| 0xF0 | wake-up IPI between CPUs (smp::WAKE_VECTOR) |
| 0xF1 | TLB shootdown IPI (smp::TLB_VECTOR, M41): reload CR3 if running the given address space |
| 0xFF | local APIC spurious vector (ignored, no EOI) |
- Interrupt controllers: local APIC + I/O APIC (since M12). Information comes from the ACPI MADT (
acpi.rs). ISA IRQnis delivered on vector32 + n; the GSI and polarity/trigger come from the MADT interrupt source overrides (QEMU: IRQ0 → GSI 2). All handlers acknowledge withlapic::eoi(). - The 8259 PICs are remapped to 32–47 and then fully masked (never deliver anything). PIT channel 0 is not used; PIT channel 2 (polled via port 0x61) only serves as the reference clock to calibrate the LAPIC timer at boot.
- LAPIC LINT0 is masked (no ExtINT from the 8259), LINT1 = NMI. Registers are reached through the HHDM after
ensure_hhdm_mapped(.., Flags::MMIO). kernel::init()enables interrupts withstiat the very end. Anything holding a lock that a handler also takes must usewithout_interrupts.- Handlers use
extern "x86-interrupt"(nightlyabi_x86_interrupt): the compiler saves registers and emitsiretq. The IDT structure and all gate descriptors are our own code. interrupts::install_exception_handlers(&mut Idt)is separate frominit(); tests build their own IDTs (seetests/page_fault.rs).- Address
0x0000_4000_0000_0000(lower half) is unmapped under Limine and is used by tests as a "guaranteed to fault" address.
Keyboard
- The IRQ1 handler only reads port 0x60 and pushes the byte into a
RingBuffer<128>; translation happens outside the interrupt indrivers::scancode::Decoder. - Scancode set 1 (the 8042 controller translates by default). Layout switchable (
keyboard::set_layout), default DE. QEMU forwards physical keys, so the kernel layout must match the user's real keyboard. keyboard::inject_scancode()feeds bytes without hardware (tests, later e.g. serial input).
Input and shell
- Two input sources, one API:
input::read_key()returnsKeyEvents from the PS/2 keyboard (scancode decoder) or from COM1 (TerminalDecoder: UTF-8, CR/LF, DEL, Ctrl+letter, ANSI arrows/Home/End/Delete). It sleeps race-free withsti; hlt. - The line editor only emits a terminal-compatible byte stream (
\x08= cursor left, characters overwrite,ESC[nA/B/Cfor up/down/right), so the same output works on the framebuffer console and on a serial terminal. Lines up to 4096 characters span several rows (M57): the editor knows the width (set_width: console columns,termsizein/bin/sh) and the prompt's visible width, maps every buffer position to (row, column) and moves across rows withESC[nA/ESC[nB\r+ESC[nC– a backspace never crosses a row. After a character in the last column, terminals keep the cursor there ("pending wrap"); output that ends at a row boundary is followed by" \r", which puts the cursor at the next row's start on every terminal. The unit tests run every edit on two simulated terminals (our console's and xterm's pending-wrap behaviour) and require the same picture. Limit: the width is the console's – a serial terminal narrower than 160 columns shows lines longer than its width wrongly.
- Commands are a static table (
shell::COMMANDS) offn(&mut Shell, &[String], &mut dyn Write) -> Result<(), String>; unit tests run them against aString. Errors are printed in red by the shell, not by the commands. poweroff: ACPI PM1a control port 0x604 (QEMU q35), fallback isa-debug-exit.reboot: 8042 reset line, fallback triple fault.
Output
serial_println!: COM1 only.println!: console and COM1. Both disable interrupts while holding their lock (without_interrupts).- The boot binary's panic handler (
kernel::panic_report) prints to serial and, ifCONSOLE.try_lock()succeeds, in red on the screen. - Console: keeps a
Vec<Cell>character grid (char + colours) for redrawing the cursor cell; cursor = inverted cell. ANSI subset:ESC[2J,ESC[H,ESC[row;colH,ESC[K,ESC[..m(0, 1, 22, 30–37, 39, 40–47, 49, 90–97). - Console: works on a
Surface(32 bpp, stride in pixels). Scrolling copies pixel rows inside the framebuffer (slow on real hardware, fine in QEMU). Font: Terminus 8×16 (PSF2) viainclude_bytes!; Latin-1 characters through the Unicode table, unknown characters become?.
Testing
- Crate layout:
src/lib.rscontains the whole kernel,src/main.rsonlykmain+ panic handler. Every test binary (lib unit tests,tests/*.rs) defines its ownkmain; the linker usesENTRY(kmain)for all of them. - Unit tests:
#[test_case]inside the module. Integration tests:kernel/tests/<name>.rswith their own harness setup (seetests/boot.rs). Tests that must panic or fault getharness = falseinCargo.toml(seetests/should_panic.rs). scripts/run-qemu.shis the cargo runner. A hash suffix in the ELF name means test binary → headless QEMU with a timeout.- Result via the
isa-debug-exitdevice on port 0xF4: value 0x10 → QEMU exit status 33 = PASS, 0x11 → 35 = FAIL. - All test output goes through COM1 (
serial_println!). scripts/e2e-shell.py(make e2e) boots the real kernel with stdin/stdout on COM1, types commands (incl. escape sequences) and checks the output between prompts; the finalpoweroffmust end QEMU.make check=make deps+make test+make e2e+make e2e-uefi.- Robustness (M41):
/bin/syscheckcalls every syscall with bad arguments and races user copies againstbrk;make soak(scripts/soak.py+rootfs/soak.sh) runs a mixed workload forSOAK_ROUNDSrounds;make stressruns the suiteSTRESS_Ntimes with 1/2/4 CPUs,qemu64and UEFI, then a soak run. - User memory is only touched in
copy_from_user/copy_to_user: check and copy under the process table lock, in pieces of ≤ 64 KiB (bug #14). Empty buffers are never dereferenced (bug #12).