Skip to content

PROGRESS.md

Latest commit

 

History

History
43 lines (38 loc) · 15 KB

PROGRESS.md

File metadata and controls

43 lines (38 loc) · 15 KB
Raw

Flint: progress

Current milestone

M1 through M7 complete and green, plus two post-M7 gap-closure passes against Docs 1-4 (per-process address spaces, then VGA output/guard pages/fault-safe copies/register dumps/log levels/README+Makefile -- see DECISIONS.md for both addenda). M8 (stretch: ELF loader, ramdisk/filesystem, disk driver, SMP) is not started and is explicitly out of scope for this run; see DECISIONS.md.

What boots

  • cargo run boots to long mode via the bootloader 0.9 crate, reaches kmain in src/main.rs, and prints a banner over COM1 serial and the VGA text buffer (src/vga.rs, Doc 2 section 3) (Flint kernel booting... / Flint vX.Y.Z -- boot OK).
  • flint::init() loads the GDT/TSS (with the double-fault IST stack), loads the IDT, remaps and unmasks the PIC, and enables interrupts. The timer (IRQ0) increments a tick counter; the keyboard (IRQ1) decodes scancodes and echoes to serial.
  • Panic handler prints KERNEL PANIC: <info> (plus a best-effort 4-register snapshot) over serial and VGA, then halts (hlt_loop), for the non-test build. Exception-specific panics (a kernel-mode page fault, a general protection fault, a double fault) carry a genuine, at-fault full register dump instead -- see the GPR-capture-trampoline note below.
  • flint::init_memory(boot_info) walks the bootloader's memory map and builds the physical frame allocator (src/memory/frame.rs), an intrusive free list threaded through the free frames' own backing memory. Physical frame 0 is never handed out. Also brings up VGA (vga::init), which needs the same physical-memory offset mapping this call establishes.
  • The same call now also brings up the page table mapper (src/memory/paging.rs, an OffsetPageTable over the bootloader's physical-memory mapping) and the kernel heap (src/memory/heap.rs, a fixed-size-block allocator with a linked-list fallback, installed as #[global_allocator]), so alloc (Box, Vec, BTreeMap) works from here on. Virtual page 0 can never be mapped -- map_page asserts on it -- so a null-pointer dereference always faults.
  • The kernel-mode page-fault handler demand-pages a not-present fault inside a designated "lazy" virtual region (memory::paging::LAZY_REGION_*), mapping a fresh frame and letting the faulting instruction re-run; redirects execution instead (see the fault-safe-copy note below) if the fault happened inside a syscall's user-pointer copy; anything else (a protection violation, or a not-present fault outside those two cases) panics with a full register dump, per Doc 3 section 5 and Doc 4 section 5.
  • The PIT (IRQ0's source) is reprogrammed from its ~18.2 Hz BIOS default to 100 Hz (interrupts::init_pit), giving the scheduler a known 10ms quantum.
  • flint::init_scheduler() starts a preemptive round-robin scheduler (src/task/scheduler.rs) with a placeholder task standing in for the boot thread. task::scheduler::spawn adds kernel threads to the ready queue; the timer handler calls scheduler::timer_tick() (after sending the PIC its EOI, since a switch may not return for a long time) to pop the next ready task and hand-written asm (src/task/context.rs, #[unsafe(naked)]) to actually switch stacks. All kernel threads share the kernel's one address space -- ring-3 processes get their own (see the M6 addendum below); a kernel thread's stack is now a mapped region with an unmapped guard page immediately below it (task::map_task_stack), not a plain heap Box, per Doc 3 section 3.
  • task::scheduler::current_task_id() and a small log_trace!/log_debug!/log_info!/log_warn!/log_error! macro set (src/serial.rs) give every diagnostic line a level and, once the scheduler exists, a [task N] tag, per Doc 4 section 2 -- replacing the ad hoc [user]/[syscall]/[shell] prefixes the original M1-M7 build used.
  • flint::user::setup() maps an isolated user code page (read-only + executable after the demo program's bytes are copied in) and an isolated user stack page (writable + NO_EXECUTE), each in its own separate top-level (PML4) page-table region so neither mapping can ever share -- and therefore silently weaken -- the other's page-table entries at any level. flint::user::enter_user_mode builds an IRETQ frame by hand and drops to ring 3.
  • The syscall path (src/syscall/mod.rs) uses int 0x80 (IDT vector 0x80, gate DPL set to Ring3) with a hand-written naked entry stub (syscall_entry) that saves rax/rdi/rsi, dispatches to normal Rust, and iretqs back. SYS_WRITE and SYS_EXIT are implemented. Every user pointer is validated -- range, mapped, user-accessible, via a real page-table walk (Translate), not a hardcoded address-range check -- before it is ever dereferenced; a bad pointer returns an error code and logs why, and the kernel keeps running. The copy itself (not just the earlier validation) is now also fault-safe: copy_from_user_byte/copy_to_user_byte record a recovery point the page-fault handler redirects to instead of panicking if a validated mapping gets revoked before the copy runs (Doc 3 sections 4/7) -- see DECISIONS.md's gap-closure addendum.
  • src/user/program.rs is Flint's first user program: hand-written position-independent machine code (no ELF loader yet -- M8 stretch) that issues a valid SYS_WRITE, a SYS_WRITE with a pointer into the kernel heap (mapped, but never user-accessible), then SYS_EXIT.
  • M6 addendum (post-M7): per-process address spaces. Each ring-3 process now gets its own PML4 (memory::paging::new_address_space), cloned from the currently-active table so already-populated kernel mappings (kernel image, the physical-memory-offset mapping, the heap) stay identically reachable, while the process's own code/stack/buffer pages are mapped only into its own table and are invisible to any other. memory::paging::activate switches CR3 to it and rebuilds the kernel's single cached page-table mapper to match, so the existing map_page/validate_user_range/demand-paging call sites transparently target whichever address space is active with no call-site changes. Kernel threads (M5) are unaffected -- they still share the kernel's one CR3, since context::switch never touches CR3 and ring-3 processes are not scheduler Tasks (see DECISIONS.md's M6 addendum for the full reasoning, including the accepted "no fork/exec, one process per boot" scope assumption). What M6 delivers and tests, updated: privilege separation (ring 0 vs ring 3 enforced by the GDT/IRETQ), the syscall trust boundary (validated user pointers, hostile pointers rejected), W xor X for the user program's own two pages, and address-space isolation (a real, private top-level page table per process, not a shared one).
  • src/shell/mod.rs: the command set (help, echo, meminfo, ps, ticks/uptime), a pure dispatch(line: &str) -> String function independent of I/O, so it is directly #[test_case]-testable. meminfo reports live frame-allocator stats; ps reports the scheduler's context-switch count; ticks/uptime reports interrupts::ticks().
  • Two new syscalls close the shell's I/O loop: SYS_READ_LINE (blocks on the UART, echoing each byte back as typed, validates the destination buffer as a writable user range -- the copy-to-user half of the checked-copy pattern, which SYS_WRITE alone never exercised) and SYS_SHELL_DISPATCH (validates and reads the line, calls shell::dispatch, prints the response). src/user/shell_program.rs is the ring-3 process: a hand-written asm loop of read -> dispatch -> repeat, ending on exit/quit. cargo run now boots straight into this shell.
  • M7 scope note: the shell process's control flow (the loop, the syscalls it issues, ending on exit) genuinely runs in ring 3, per Doc 2 section 8 ("a user-space process, not a kernel feature"). Command parsing runs in the kernel behind SYS_SHELL_DISPATCH, not in ring-3 code, because Flint has no ELF loader (M8 stretch) and hand-encoding a full string-matching parser directly in raw assembly was judged not worth the added risk this late in the build. See DECISIONS.md.

Harness status

  • cargo test --lib: flint::trivial_assertion, flint::interrupts::tests::test_breakpoint_exception, flint::memory::tests::allocated_frames_are_distinct_and_nonzero, flint::memory::tests::freed_frame_is_reused, flint::memory::heap::tests::{boxed_value_round_trips, large_vec_uses_every_slot, many_boxes_dont_exhaust_the_heap, large_allocation_uses_fallback_path}, flint::memory::paging::tests::page_fault_on_lazy_region_is_handled_and_continues (the PRD's "valid-but-unmapped page fault is handled and execution continues" gate: writes through an intentionally never-pre-mapped pointer, then reads the value back to prove a real frame landed there, not just that the fault was swallowed), flint::vga::tests::printed_text_round_trips_through_the_buffer (writes a known string to VGA, reads the exact cells back out of the buffer), flint::syscall::tests::copy_helpers_return_err_instead_of_panicking_on_a_mid_copy_fault (Doc 3 sections 4/7's fault-safe-copy gate: validates a range, deliberately unmaps it mid-sequence, and asserts the copy helpers return Err -- reaching the assertions at all, rather than a kernel panic, is itself the proof).
  • cargo test --test basic_boot: basic_boot::test_boots_and_prints -- boots and prints.
  • cargo test --test stack_overflow (harness = false): double-fault-on-stack-overflow gate (Doc 3 section 5/7), on the boot thread's own stack.
  • cargo test --test task_stack_overflow (harness = false, new): the same double-fault-on-overflow gate, but for a kernel task's mapped-plus-guard-page stack (Doc 3 section 3) rather than the boot stack -- spawns a self-recursing task and confirms the real kernel double-fault handler catches it (needs the timer/scheduler running to reach the task at all, so it reuses the real IDT rather than a test-local one like stack_overflow.rs).
  • cargo test --test null_page (harness = false, new): dereferences virtual address 0 in kernel mode and asserts the fault is a genuine not-present fault on page 0 caught by a test-local handler, not a crash and not a lucky read of real memory -- the Doc 3 section 3/7 "page 0 is unmapped" gate.
  • cargo test --test register_dump (harness = false, new): Doc 4 section 5's register-dump gate. Loads a known marker value into rax, triggers a genuine, unrecoverable kernel-mode page fault on a deliberately unmapped canonical address, and asserts the panic report's rax= field contains the exact marker -- direct proof the GPR-capture trampolines reflect real register state, not zeros or a stub.
  • flint::task::scheduler::tests::two_tasks_alternate_under_the_timer: the PRD's "two tasks alternate under the timer a known number of times" gate. Spawns two kernel threads, each an infinite loop with no voluntary yield incrementing its own atomic counter; waits until both reach 20. Since neither task can make progress without genuine preemption, and 40+ actual context switches are asserted to have happened, this is direct proof the scheduler and the hand-written context switch both work, not just that the scheduler ran once.
  • cargo test --test user_mode (harness = false): the full M6 demo end to end. Boots, sets up ring 3, jumps to the hand-written user program, and hooks SYS_EXIT to report success and exit QEMU instead of its normal hlt_loop. Serial transcript on a green run: [info] [task 0] private address space: PML4 PhysFrame[4KiB](0x7edc000) (boot PML4 was PhysFrame[4KiB](0x1000)) (direct evidence the process got its own top-level page table, distinct from the boot one -- the PRD's FR-MEM-2/FR-USER-1 gate), [user] hi ring3! (the valid syscall's effect, visible over serial -- the PRD's "user-mode program performs a syscall and returns" gate), [warn] [task 0] SYS_WRITE rejected: ... (the hostile pointer into the kernel heap, rejected -- the "hostile user pointer is rejected and the kernel survives" gate, proven by the fact that execution continues past it), [info] [task 0] exited via SYS_EXIT, then the test's own [ok]. Any fault along the way (a bad IRETQ frame, a wrong syscall offset) surfaces as a kernel panic this test's own panic handler reports as failed, not a silent hang.
  • flint::shell::tests::* (new): help_lists_commands, echo_returns_its_argument, echo_with_no_argument_is_empty, meminfo_reports_frame_counts, ps_reports_a_status_line, ticks_and_uptime_are_synonyms, unknown_command_says_so, blank_line_is_a_no_op -- the command-dispatch logic tested directly, independent of the ring-3/syscall path.
  • cargo test --test shell (harness = false, new): the M7 gate, "the shell echoes input and runs help and one status command." Boots straight into the ring-3 shell loop with SYS_READ_LINE fed a scripted byte sequence (help, meminfo, exit) via syscall::set_scripted_input, so the test is self-contained and deterministic under a bare cargo test rather than depending on external stdin reaching this exact QEMU process. Green transcript: help echoed and answered with the command list, meminfo echoed and answered with live frame stats, exit ends the ring-3 loop via its own SYS_EXIT, hooked to report success.
    • The scripted path exercises the identical ring-3/syscall/validate/echo pipeline real typed input uses -- only the byte source differs. Real interactive input was also manually verified: printf "\nhelp\nps\nticks\nexit\n" | qemu-system-x86_64 -drive format=raw,file=target/x86_64-flint/debug/bootimage-flint.bin -device isa-debug-exit,iobase=0xf4,iosize=0x04 -serial stdio -display none -no-reboot produces the same live back-and-forth over real serial I/O, confirmed by hand during this build (piping stdin through cargo run's own process chain also reaches the UART, but is not used as the automated harness test for the reason above -- see DECISIONS.md). The leading blank line in that command is a warm-up byte for a benign UART startup race (the first byte typed immediately at boot can arrive before the kernel starts polling the UART and get overwritten before it's read); also documented in DECISIONS.md.
  • Bare cargo test runs all of the above plus doctests (0, since no_std) in one invocation, all green.
  • QEMU is run headless (-display none -serial stdio) so serial is what the harness (and a human) actually observes.

Next

  • Nothing required remains: M1 through M7, plus both post-M7 gap-closure addenda, are all green. M8 (ELF loader, ramdisk/filesystem, disk driver, SMP) is stretch and untouched -- see SUMMARY.md for the honest per-requirement status and DECISIONS.md for why each M8 item was left as a documented stub rather than attempted.

Commands

See README.md for the four commands (build/run/test/debug) and their exact form; make debug now wraps what used to be a hand-typed gdb-stub QEMU invocation. One command not in that list, for manually driving a live interactive session over real serial I/O (not the scripted-input test path, and not cargo run's own headless invocation): after cargo build, qemu-system-x86_64 -drive format=raw,file=target/x86_64-flint/debug/bootimage-flint.bin -device isa-debug-exit,iobase=0xf4,iosize=0x04 -serial stdio -display none -no-reboot.