GatOS Kernel v2.1.0 (Critical Fixes & Maintenance)

The GatOS Kernel has been updated from v2.0.0 to v2.1.0, following the merge of branch QOL.

This version is a stability, performance, and polish release that touches nearly every subsystem: 99 files changed, ~9,100 insertions and ~5,000 deletions. It fixes several long-standing memory corruption bugs, restores full FPU/SSE code generation across the kernel, and roughly doubles the written documentation.

• Restored full floating point and SSE code generation across the kernel. Instead of building the entire kernel with -mno-sse -mno-sse2 -mno-mmx -mno-80387, the build system now maintains an explicit KERNEL_INTERRUPT_PATH set (interrupt dispatcher, scheduler, timers, keyboard/xHCI IRQ handlers, and the demand-paging path through vmm.c, pmm.c and avl.c) and restricts SSE emission only for those files. Every other translation unit, including drivers, klibc and kernel threads, compiles with full vector support, with the lazy FPU mechanism handling state save and restore.
• Fixed a catastrophic firmware memory corruption bug in kernel_main. Non-MULTIBOOT_MEMORY_AVAILABLE regions were registered as MMIO but then fell through into pmm_populate, handing reserved firmware, ACPI, and MMIO ranges to the physical allocator as free memory. A missing continue was silently arming the kernel to overwrite firmware structures at runtime.
• Rebuilt the PMM around a cleaner buddy allocator with a first-class exclusion table. pmm_exclude_range() registers up to eight physical ranges that must never be handed out, pmm_init derives its order count from the managed range, and population is now firmware-robust across machines with unusual E820/multiboot maps.
• Implemented USB hotplug. The xHCI driver now runs a dedicated hotplug process and thread that polls hub status-change endpoints, handles C_PORT_CONNECTION and C_PORT_RESET events, clears port features over the control pipe, enumerates newly attached devices, and tears down downstream slots with Disable Slot on disconnect. Devices can now be plugged and unplugged at runtime instead of only at boot enumeration.
• Added a full power management subsystem (power.c / power.h). power_off() walks the FADT to the DSDT, parses the _S5_ package for the SLP_TYPa/b values, enables ACPI mode via SMI_CMD, and writes PM1a/PM1b_CNT. reboot() cascades through the 0xCF9 reset register, the ACPI FADT reset register, the PS/2 controller, and finally a triple fault. Intel RAPL energy sampling was added on top, exposing live average package wattage through power_avg_watts().
• Massively improved klibc and ulibc performance. kmemset, kmemcpy, kmemmove, and their userspace twins now align to an 8-byte boundary, run a 64-bit word loop over the bulk of the buffer, and fall back to bytewise copies only for the head and tail, replacing the original one-byte-at-a-time loops used by every subsystem in the kernel.
• Added long and long long parsing support to scanf/vscanf in both klibc and ulibc. The l and ll length modifiers are now honored for %d, %i, %o, %u, and %x, routing through kstrtol/kstrtoul with correct destination widths instead of truncating everything to int.
• Implemented dirty-cell tracking and batched rendering in the console. Each console carries a bitmap of dirty character cells; flush_display now redraws only the cells that actually changed since the last flush, and a deferred-render toggle lets callers batch many writes into a single screen update. Glyph and fill blitting gained a fully unrolled 32bpp fast path that writes eight pixels per glyph row directly into the framebuffer.
• Reworked the syscall ABI and dispatcher. syscall_dispatcher now receives a single cpu_context_t* built on the per-thread kernel stack rather than a raw register array indexed by magic offsets, making argument and return-value handling explicit. MSR_STAR now programs the user base as 0x13 so SYSRETQ produces CS=0x23/SS=0x1B without relying on the CPU forcing SS.RPL=3, a behavior some Intel microarchitectures get wrong. User buffer validation now runs with interrupts disabled so the buffer cannot be swapped between the vmm_check_buffer and the copy, and SYS_MMAP masks user-supplied flags down to VM_FLAG_WRITE | VM_FLAG_EXEC | VM_FLAG_LAZY.
• Rewrote the Kernel Dashboard renderer as a layout engine. Sections, key/value column grids, standalone and paired progress bars, and unified column alignment are now computed from a layout_t derived from console dimensions, instead of hardcoded positioning. The CPU section gained live package power draw from RAPL, and the whole dashboard renders through the deferred-render path so a refresh is a single batched flush.
• Added a MONITOR/MWAIT idle thread to the scheduler, with an automatic hlt fallback when the CPU does not advertise MONITOR/MWAIT. Idle and total tick accounting is exposed through sched_cpu_usage(), giving the dashboard real CPU utilization instead of an estimate.
• Added LAPIC TSC-Deadline timer support. lapic_tsc_arm() programs the LVT timer in TSC-Deadline mode and writes IA32_TSC_DEADLINE directly, replacing periodic-count rearming on hardware that supports it, with lapic_timer_stop() disarming both modes.
• Fixed a TTY teardown leak where threads parked in a TTY's wait queue were left dangling on destroy. tty_destroy now flushes the entire wait queue into the dead queue, and active_tty was made volatile to stop the compiler from caching a stale focus pointer across interrupt-driven focus changes.
• Fixed process creation to allocate fresh physical pages for the userspace BSS and explicitly zero them, rather than reusing whatever the allocator handed back, eliminating a class of nondeterministic userspace startup failures.
• Corrected reserve_required_tablespace parameter passing and added an assertion that required page-table space never exceeds available memory, catching a bad-argument bug that could under-reserve table space on large-memory machines.
• Enabled link-time optimization and stripping in the build pipeline. Kernel objects compile with -flto and link through gcc (so the LTO plugin is actually loaded) with --gc-sections, followed by x86_64-elf-strip; userspace objects build with -ffast-math. The test profile now inherits the fast optimization flags instead of building unoptimized.
• Restored 16-byte stack alignment in generic_interrupt_handler before calling into interrupt_dispatcher, and reworked interrupt-context zeroing to avoid emitting SSE instructions that would clobber the interrupted thread's XMM state.
• Extended CPU feature detection to cover SSE3/SSSE3/SSE4.1/SSE4.2, AVX, VMX, SVM, PAE, and the full CPUID brand string, with cpu_enable_feature wiring up SSE, AVX, SMEP and SMAP at boot and the dashboard reporting each one.
• Added two new documentation chapters, namely C6 (Interrupts, Panics, and Spinlocks) and C7 (PMM and Slab Allocators), ~2,450 lines combined. I also substantially rewrote C0 through C5.
• Sweeping readability and consistency work across the tree: heap.c shed roughly 400 lines through shortened type names (heap_block_header_t -> blk_hdr_t) and de-duplicated statistics helpers, vmm.c, slab.c, process.c, scheduler.c, gdt.c, apic.c and acpi.c were restructured, every test file was tidied, and comments and authorship headers were made consistent throughout.

GatOS Kernel v2.0.0 (Feature Complete)

The GatOS Kernel has just been updated from v1.9.5 to v2.0.0, following the merge of branch decoupling.

This version delivers a massive architectural overhaul with fundamental structural decoupling, algorithmic optimizations, and broadened bare-metal hardware support. It is also the final version of the GatOS kernel (for my thesis purposes).

• Introduced a fully operational eXtensible Host Controller Interface (xHCI) driver. This includes end-to-end PCI/PCIe enumeration, BAR decoding, MSI interrupt configuration, slot/endpoint contexts, and ring buffer management (Command and Event rings).
• Implemented full USB Boot Protocol support. The driver seamlessly parses configuration descriptors to enumerate standard HID keyboards, handles nested USB Hubs, translates keycodes, and dynamically updates hardware LED indicators via SET_REPORT.
• Decoupled the early hardware console from the dynamic memory manager. The console now utilizes a statically allocated 2MB page directory in the BSS section, enabling massive PAGE_HUGE mappings to cover high-resolution framebuffers with minimal TLB pressure and zero heap dependency.
• Integrated an intrusive AVL Tree implementation (avl.c) across critical kernel subsystems, replacing O(N) linear linked lists with O(log N) structures. The VMM now tracks virtual memory areas (VMAs), and the scheduler manages the sleep queue via dynamically balanced trees.
• Rebuilt the vmm_alloc engine to natively support 2MB Huge Pages (PAGE_HUGE). The VMM now intelligently aligns and groups large contiguous virtual allocations (like the physical identity map and userspace binary sections) to drastically reduce page-table depth overhead.
• Implemented Lazy FPU/SSE state loading in the scheduler. By manipulating the CPU's CR0.TS (Task Switched) bit and intercepting INT_DEVICE_NOT_AVAILABLE, the kernel now completely bypasses costly fxsave/fxrstor instructions during context switches unless a thread actually attempts to execute floating-point math.
• Introduced a live, visually polished Kernel Dashboard accessed via CTRL+SHIFT+ESC. It provides a real-time, color-coded breakdown of CPU features, exact physical memory fragmentation, kernel heap pressure, and live thread state tracking (e.g. SLEEPING 500ms, RUNNING).
• Upgraded the TTY multiplexer with "hidden" virtual terminal support. This allows internal diagnostic layers like the Kernel Dashboard to operate silently in the background, fully excluded from the standard ALT+TAB cycling and ALT+F4 closure events.
• Hardened kernel security by enabling Intel Structured Extended Feature Flags for SMEP (Supervisor Mode Execution Prevention) and SMAP (Supervisor Mode Access Prevention). Syscalls now use precise stac/clac inline instructions to explicitly authorize user-memory accesses, closing critical Ring 0 arbitrary execution vulnerabilities.
• Re-engineered the kernel panic handler (panic_c) to operate completely independently of standard rendering pipelines. It now uses a lock-free, scheduler-free pixel-rendering engine to dump exact register states, CPU flags, and detailed page-fault analytics directly to the framebuffer, even during catastrophic system corruption.
• Hardened system ABI compliance and stack integrity. Assembly entry points (syscall_entry.S, ISR.S) now strictly adhere to the System V 16-byte stack alignment convention.

GatOS Kernel v1.9.5

The GatOS Kernel has just been updated from v1.9.0 to v1.9.5, following the merge of branch userspace.

This version delivers a multi-layered system with a robust, standard-compliant userspace environment:

1. Formally decoupled the kernel's internal C library from the userspace runtime. klibc is now a lean, kernel-only utility set, while the new ulibc provides a comprehensive foundation for applications, including full stdio, string, and stdlib implementations.
2. Implemented a sophisticated boundary-tag heap allocator within ulibc that manages memory via the SYS_MMAP and SYS_MUNMAP syscalls. The allocator features arena-based allocation, immediate block coalescing, in-place realloc shrinking, and thread-safety through userspace spinlocks.
3. Integrated FPU and SSE state saving and restoration (fxsave/fxrstor) into the thread context switch path. This ensures floating-point calculations and SIMD operations remain preserved across preemptive scheduling intervals without corruption.
4. Expanded the syscall interface with critical new functionality, including SYS_READ for interactive input, SYS_MMAP/SYS_MUNMAP for dynamic memory management, and SYS_YIELD/SYS_SLEEP_MS for voluntary execution control.
5. Unified terminal operations into a comprehensive SYS_TTY_CTRL multiplexer. This exposes dynamic terminal dimensions (TTY_CTRL_GET_DIMS), screen clearing (TTY_CTRL_CLEAR), and hardware cursor visibility toggles (TTY_CTRL_CURSOR) directly to Ring 3 applications.
6. Overhauled the VGA framebuffer console (console.c) with a dirty-checking differential backbuffer. The driver now dispatches hardware MMIO writes exclusively for pixels that change between frames, reducing system bus traffic by over 80% during animation-intensive workloads.
7. Embedded a lightweight state machine into the TTY line discipline to parse standard ANSI escape sequences (\x1b[H, \x1b[2J), allowing userspace to reposition the cursor and clear the screen without intermediate flushing or kernel intervention.
8. Implemented vmm_check_buffer to rigorously validate userspace pointers and memory ranges during syscall entry. This prevents unauthorized kernel memory access and enforces correct page-level permissions (e.g., verifying write access before servicing a SYS_READ).
9. Introduced ulock_t within ulibc to provide mutual exclusion for shared userspace resources, enabling thread synchronization without requiring constant Ring 0 transitions.
10. Successfully migrated all floating-point math routines from kernelspace to ulibc. This adheres to the microkernel-inspired principle of keeping the core kernel lean while offloading non-essential computations to the application layer.
11. Integrated high-performance userspace demonstration programs that stress-test the new environment. Notably, this includes a fully adaptive, math-optimized 3D spinning ASCII donut that dynamically queries TTY dimensions, scales to fit any resolution, and renders at blistering frame rates using differential framebuffer drawing techniques.
12. Overhauled the kernel's internal test module with hundreds of new validation cases for the VMM, PMM, Slab allocator, and Scheduler, ensuring that the radical changes to memory and execution models remain regression-free.

GatOS Kernel v1.9.0

The GatOS Kernel has just been updated from v1.8.5 to v1.9.0, following the merge of branch threading.

This version introduces significant upgrades across the scheduling subsystem, process model, syscall interface, virtual memory manager, and fault handling:

1. Implemented a core scheduling engine (scheduler.c) with support for thread lifecycle states (READY, RUNNING, SLEEPING, DEAD). The scheduler includes an automated sleep queue and a dedicated idle task that halts the CPU when no runnable threads exist.
2. Introduced formal process_t and thread_t structures (process.c) to represent execution contexts, including per-process virtual memory spaces and per-thread kernel stacks.
3. Replaced legacy interrupt-based traps with the x86_64 syscall instruction path, including the syscall_entry.S assembly stub responsible for GS-base swapping and kernel stack loading during Ring 0 transitions.
4. Implemented a C-side syscall dispatcher (syscall.c) supporting SYS_EXIT, SYS_WRITE, SYS_MMAP, SYS_MUNMAP, and SYS_SET_FS_BASE.
5. Refactored the Global Descriptor Table (GDT) into a fully C-managed subsystem (gdt.c), including a dynamically configured Task State Segment (TSS) with dedicated Interrupt Stack Tables (ISTs) for safe Double Fault and Page Fault handling.
6. Extended the Virtual Memory Manager (VMM) with Demand Paging, allowing the kernel to reserve large virtual regions using VM_FLAG_LAZY and allocate backing physical pages only when a page fault occurs.
7. Updated the linker script (linker.ld) to introduce a .user_text section, which is mapped into every userspace process at a fixed virtual address (USER_CODE_VIRT_ADDR) providing a common runtime entry point (userspace_start).
8. Integrated interrupt-safe spinlocks across the PMM, VMM, Slab, and Heap managers to prevent race conditions under asynchronous thread preemption.
9. Configured the MSR_GS_BASE and MSR_KERNEL_GS_BASE registers to point to cpu_local_t structures, enabling constant-time access to per-core kernel metadata and stack pointers.
10. Added the SYS_SET_FS_BASE syscall to allow userspace runtimes to configure the FS segment, enabling support for thread-local storage and modern C runtime behavior.
11. Expanded stdio.c with a fully featured scanf / vscanf implementation, including scanset support (%[...]) and buffered input integration through the TTY subsystem.
12. Implemented a*thread cleanup mechanism within the scheduler that reaps DEAD threads and automatically destroys parent processes once all member threads terminate.
13. Enhanced the interrupt dispatcher to detect faults originating from Ring 3, allowing the kernel to terminate offending threads with descriptive Segmentation Fault messages instead of triggering a kernel panic.
14. Added vmm_check_buffer to the VMM, allowing the kernel to rigorously validate userspace pointers and lengths before processing syscall arguments, preventing memory safety vulnerabilities.
15. Introduced a multitasking validation module (test_multitasking.c) to stress-test the scheduler and thread lifecycle management.

GatOS Kernel v1.8.5

The GatOS Kernel has just been updated from v1.7.5 to v1.8.5, following the merge of branch 'drivers'.

Due to the breadth of architectural changes introduced in this merge, GatOS skips the release v1.8.0-alpha completely.

This version introduces significant core upgrades across the interrupt architecture, console stack, input pipeline, and timing infrastructure:

1. Implemented a modern APIC-based interrupt architecture, fully phasing out the legacy 8259 PIC.
2. Added full ACPI MADT parsing to dynamically discover Local APIC and I/O APIC topology at boot.
3. Implemented support for Interrupt Source Overrides (ISOs) and Non-Maskable Interrupts (NMIs) to ensure correct IRQ routing on modern hardware.
4. Refactored the IDT and ISR dispatch paths, introducing interrupts_save / interrupts_restore primitives for safe atomic kernel sections.
5. Transitioned the display stack from VGA text mode to a 64-bit UEFI Linear Framebuffer.
6. Implemented a custom PSF font engine (font.c) with a high-performance glyph renderer supporting fast scrolling and ANSI-like foreground/background colors.
7. Introduced a Console Hardware Abstraction layer (console.c) to decouple logical TTY devices from the physical framebuffer.
8. Added a Dynamic TTY Manager supporting multiple virtual consoles (cycle via ALT+Tab).
9. Implemented a Line Discipline (ldisc) layer providing canonical input processing, including backspace handling, line buffering, and echo.
10. Developed a centralized Input Hub coordinating data flow between the PS/2 keyboard driver and the active TTY.
11. Implemented a robust PS/2 controller (i8042) driver with proper initialization, self-test, and port-level primitives.
12. Added a full keyboard driver with scancode-to-ASCII translation and Shift/Caps-Lock state tracking.
13. Introduced high-precision timing infrastructure using HPET and TSC, including automated LAPIC timer calibration.
14. Replaced the legacy VGA stdio with a fully featured stdio.c supporting printf, sprintf, and vsnprintf with floating-point formatting.
15. Implemented scanf / vscanf with scanset support (%[...]) to enable complex interactive input scenarios.
16. Integrated a comprehensive fdlibm-based math library (libm) to support advanced kernel and driver computations.
17. Implemented interrupt-safe spinlocks with optional debug tracking.
18. Integrated spinlock protection across PMM, VMM, Slab, and Heap to prevent race conditions during asynchronous hardware events.
19. Completely reorganized kmain.c into a strict staged initialization pipeline:
    IDT → CPU → PMM → Slab → VMM → Heap → Timers → Console → Input → ACPI → APIC
20. Updated the linker script and compiler flags to enable Dead Code Elimination, significantly reducing kernel binary size.
21. Expanded the kernel validation framework with dedicated test modules for Spinlocks (test_spinlock.c), Timers (test_timers.c), and TTY (test_tty.c).

Note: macOS builds have been fixed as promised in the previous release

GatOS Kernel v1.7.5

The GatOS Kernel has just been updated from 1.7.0 to 1.7.5, by merging branch "project-refactor".

This version introduces several improvements, focusing entirely on structural integrity, cross-platform compatibility, and automated testing, bringing the kernel version to v1.7.5-alpha:

1. Replaced the legacy Makefile-based build system with a robust, cross-platform Python implementation (run.py), eliminating the need for shell scripts like run.sh and setup.sh.
2. Introduced setup.py, a dedicated script that automatically downloads, configures, and installs the architecture-specific toolchains for Windows, Linux, and macOS.
3. Completely refactored the project source tree and header include paths to support a scalable, modular architecture, moving away from the flat structure.
4. Implemented a comprehensive Kernel Test Framework integrated directly into the build system. Tests can now be triggered via python run.py test.
5. Added extensive dedicated test suites for the Physical Memory Manager (PMM), Virtual Memory Manager (VMM), Slab Allocator, and Kernel Heap to ensure strict memory integrity and boundary enforcement.
6. Enhanced run.py to support headless execution and timeout parameters, enabling automated, non-interactive testing sessions.
7. Established a full CI/CD pipeline using GitHub Actions, ensuring that every commit is built and tested against Linux, Windows, and macOS (ARM64) runners.
8. Updated compiler flags to support optimization profiles (fast, vfast) within the Python build scripts.
9. Implemented specific fixes for macOS environments, including handling Gatekeeper warnings and path resolution for the cross-compiler toolchain.
10. Refactored the kernel entry point to support a conditional test-build mode (-DTEST_BUILD), allowing the kernel to run validation suites if the flag is specified, rather than booting into the standard loop.

Note: MacOS builds might fail (I'm waiting on my new Mac Mini to fix them), so until then, please use one of the other platforms. The documentation is also slightly outdated and will hopefully be updated soon.

GatOS Build Toolchain Binaries

Prebuilt, self-contained binaries for GCC, Binutils, GRUB, QEMU (x86_64), xorriso, and mtools.

Includes builds for:

• Windows x86_64
• Linux x86_64
• macOS Universal2 (Intel + ARM)

No dependencies, no installation, automatically pulled by the setup script.

GatOS Kernel v1.7.0

The GatOS Kernel has just been updated from 1.6.0 to 1.7.0, by merging branch "memory". Due to the sheer scale and range of changes, incremental version 1.6.5 has been omitted.

Several improvements are introduced in this version:

• Added optional CFLAGS_FAST to the Makefile for generating very fast kernel images.
• Introduced new helper functions across paging, serial, VGA I/O, misc, and debug modules for improved modularity and maintainability.
• Implemented a Physical Memory Manager (PMM) based on a buddy allocator to reduce fragmentation and improve allocation performance.
• Extended the PMM to support up to 16 TB of memory and added header validation for integrity checks, relying on the physmap to access physical frames.
• Introduced a fully featured Virtual Memory Manager (VMM) with dynamic mapping, resizing, and robust page table management.
• Integrated a Slab Allocator for efficient small-object allocations and tighter control over kernel heap usage.
• Implemented a new kernel heap manager with multi-arena support, non-contiguous regions, and boundary tag coalescing for efficient free block merging.
• Added extensive memory integrity helpers in all allocators (PMM, VMM, Slab, Heap), including red zones, magic values, free list checks, and statistics.
• Refactored the serial subsystem to separate QEMU logs from internal logs for clearer runtime vs debug output.
• Overhauled debug logging, introducing QEMU_GENERIC_LOG, QEMU_LOG, and LOGF for consistent, distinct logging across modules.
• Added kernel panic handlers and integrity checks for stable recovery and clear error reporting during critical failures.
• Introduced CPU feature detection and runtime enablement for PAE, NX, SSE, and AVX for improved hardware compatibility.
• Refactored the ACPI subsystem and other core components for clarity, consistent naming, and easier future expansion.
• Enhanced the build system, boot flow, and ISO generation process, improving output readability and version labeling.
• Each build now generates an ISO based on the current kernel version defined in kernel_main.
• Introduced PANIC_ASSERT for critical kernel assertions.
• When run in QEMU, GatOS now outputs a debug.log containing internal debug logs.
• MMIO region handling is now supported natively in the VMM.
• Moved several definitions between headers for clarity; all headers now use #pragma once.
• Implemented kmalloc, krealloc, kcalloc, and userland equivalents for dynamic memory allocation.

GatOS Kernel v1.6.0

The GatOS Kernel has just been updated from 1.5.5 to 1.6.0, by merging branch "Interrupts".

Several improvements are introduced in this version (versus the previous 1.5.5):

• Implemented hardware exception interrupt handling
• The kernel now fully supports a RSOD (Red Screen of Death) whenever a hardware exception happens anywhere in the code
• Introduced kernel panic handlers
• Implemented ACPI functionality through an API, which will be used to initialize the APIC and hardware devices
• Several refactored naming conventions were introduced - i.e. main.S and main64.S have been renamed to boot32.S and boot64.S accordingly
• Added "DEBUG_GENERIC_LOG" for better debugging functionality, allowing for generic, string only log messages
• Minor multiboot2 API improvements
• Minor improvements to console output

GatOS Kernel v1.5.5

This is the first official release of the GatOS kernel.

Several improvements are introduced in this version (versus the previous 1.5.0):

• The Makefile now compiles a UEFI compliant ISO. We can now use it in both UEFI and legacy BIOS systems*.
• Implemented some new APIs in the multiboot2 parser for physical RAM size calculation.
• Improved code readability and maintainance by introducing new constants for BYTES, MEGABYTES and GIGABYTES.
• SSE (Streaming SIMD Extensions) support has been added, and floating point calculations are now possible.
• Paging functionality has been significantly improved. For more information, check the branch's commit messages.
• A mapping of all physical memory into virtual space (physmap) has been implemented and can be accessed at offset 0xFFFF800000000000.
• GatOS officially supports all memory sizes up to 512GB. Tables are reserved dynamically at runtime.
• Several QOL improvements regarding the code have been introduced. Methods, constants, names and defines have been improved for clarity.
• Linker symbols were tweaked accordingly for readability.