x86-64 OS development from Windows with ChatGPT assistance — real hardware and spec-driven.

[Norppa]

Introduction

I've been doing OS development in NASM on and off for years.

Recently I decided to restart an x86-64 hobby OS project from scratch, with one deliberate difference: I'm using an LLM as a development partner.

This is not an "AI generated an OS in five minutes" project.

The entire development environment currently runs on Windows 11, using NASM and LLVM's lld-link to produce PE32+ UEFI executables.

No Linux host is involved in the current build process.

The idea is to develop the system incrementally and treat the specifications and the actual hardware as the final authority.

My basic rules are:

• Hardware-facing structures, offsets and bitfields must be checked against specifications or reliable references.
• Each development step must actually work before moving on.
• The primary target is physical hardware, not only an emulator.
• Generated code is not accepted simply because it compiles.
• When something fails, the cause gets investigated and corrected before continuing.
• I want to avoid giant generated code dumps containing half a dozen unfinished subsystems.

Current hardware

The current physical development machine is:

Code: Select all

CPU:         AMD Ryzen 5 9600X
             Zen 5 / Granite Ridge

Motherboard: ASUS TUF GAMING B850-PLUS WIFI

RAM:         32 GiB

Graphics:    AMD integrated graphics
             NVIDIA GeForce RTX 5050

Host OS:     Windows 11
Boot:        Native UEFI

Windows is installed on the internal SSD, but the hobby OS currently does not touch that disk at all.


A note about programming languages

The current EFI bootstrap is written in NASM.

I used to be very attached to the idea of writing an OS entirely in assembly, but over the years I reached the point where I was writing increasingly sophisticated NASM macros to imitate higher-level language constructs.

At some point, when you start implementing IF/ELSE constructs, elaborate calling abstractions and pseudo-objects inside an assembler preprocessor, it becomes reasonable to ask why you are still pretending to write assembly.

So my current rule is:

Use assembly when explicitly seeing the machine improves understanding.

Use a higher-level language when the instructions themselves become administrative noise around a more abstract problem.

NASM will therefore remain important for CPU state, special instructions, entry code, interrupt stubs, context switching, certain MMIO operations and other places where the exact machine operations matter.

For more structurally complex subsystems later on, I am no longer imposing an "assembly at all costs" rule.

The language choice will be made module by module.


Milestone 1 — UEFI GOP framebuffer

Status: PASS on physical hardware

The first stage is now working on the actual machine.

Current boot path:

Code: Select all

ASUS UEFI
    |
    v
\EFI\BOOT\BOOTX64.EFI
    |
    v
EFI Boot Services
    |
    v
LocateProtocol()
    |
    v
Graphics Output Protocol
    |
    v
Framebuffer
    |
    v
Direct rendering

The EFI application is built as PE32+ x86-64.

It currently displays a dark gray framebuffer with a small fixed white software cursor.

The cursor is intentionally primitive. Its purpose is simply to prove that:

• the firmware loads the PE32+ image;
• our x86-64 NASM entry point executes;
• EFI_BOOT_SERVICES is accessible;
• LocateProtocol() succeeds;
• GOP is located;
• the framebuffer information is usable;
• and the framebuffer can be written directly.

That entire chain has now been validated on the physical machine.


Booting it on real hardware

There is currently no installer.

Nothing is installed on or written to the internal Windows disk.

The OS is booted as a standard UEFI removable-media application from a dedicated USB flash drive.

1. Format a spare USB flash drive as FAT32.

2. Create the standard x86-64 UEFI removable-media path:

Code: Select all

\EFI\BOOT\

3. Copy the generated executable as:

Code: Select all

\EFI\BOOT\BOOTX64.EFI

For example, assuming the USB drive is X: on Windows:

Code: Select all

mkdir X:\EFI\BOOT
copy /Y build\BOOTX64.EFI X:\EFI\BOOT\BOOTX64.EFI

The resulting drive is simply:

Code: Select all

USB
└── EFI
    └── BOOT
        └── BOOTX64.EFI

No MBR boot sector is involved.

No Legacy BIOS boot path is used.

The firmware discovers BOOTX64.EFI through the standard UEFI removable-media path.


Firmware configuration

My current development configuration is:

Code: Select all

UEFI         ON
CSM          OFF
Secure Boot  OFF
TPM          ON, but unused by the OS

TPM does not need to be disabled. The current OS simply ignores it.

Secure Boot is disabled because the development EFI executable is currently unsigned.


ASUS Secure Boot example

On my ASUS TUF GAMING B850-PLUS WIFI, the relevant setting is:

Code: Select all

UEFI Setup
 -> Boot
 -> Secure Boot
 -> OS Type

The motherboard offers:

Code: Select all

Windows UEFI mode
Other OS

ASUS documents these choices as:

Code: Select all

Windows UEFI mode
    -> Secure Boot enabled

Other OS
    -> Secure Boot disabled

For development I selected:

Code: Select all

Other OS

Windows 11 continues to boot normally afterward.

Windows System Information (msinfo32) reports:

Code: Select all

BIOS Mode:         UEFI
Secure Boot State: Off

So the resulting machine configuration is still completely native UEFI:

Code: Select all

UEFI         ON
CSM          OFF
Secure Boot  OFF
TPM          ON

Disabling Secure Boot does not imply switching back to Legacy BIOS or CSM.

ASUS documents this behaviour here:

ASUS — How to Enable/Disable Secure Boot

There is also no need to clear the Secure Boot key database merely to test this unsigned hobby OS.


Why USB first?

Until the boot path is stable, I prefer keeping the development OS completely separate from the Windows EFI System Partition.

Breaking a disposable USB flash drive is considerably funnier than breaking Windows Boot Manager.


What you should see

If the current build is working correctly, the firmware display should be replaced by a full-screen dark gray framebuffer with a small white triangular cursor near the upper-left corner.

Approximately:

Code: Select all

┌──────────────────────────────┐
│ dark gray background         │
│                              │
│     ◢ small white cursor     │
│                              │
│                              │
└──────────────────────────────┘

The actual cursor is drawn pixel by pixel rather than as a text character, but this is approximately the shape and orientation you should see.

At this stage the cursor is fixed and there is no keyboard or mouse input yet.

So the expected result is simply:

Code: Select all

dark gray framebuffer + small fixed white ◢

If you see that, the following path has succeeded:

Code: Select all

UEFI
 -> BOOTX64.EFI
 -> EFI Boot Services
 -> LocateProtocol()
 -> GOP
 -> framebuffer
 -> direct pixel writes

Source code

I've attached the complete source code for this stage

This is the exact project used for the successful physical-hardware test, including:

• NASM source
• UEFI definitions
• Windows build script
• PE32+ EFI generation
• GOP framebuffer access
• fixed software cursor rendering

OperatingSystem.zip

(6.88 KiB) Downloaded 16 times

Next step

The next target is:

Code: Select all

GetMemoryMap()
    |
    v
Prepare boot information
    |
    v
ExitBootServices()
    |
    v
Continue executing independently
    |
    v
Keep rendering to the existing framebuffer

This is where the project begins transitioning from an EFI application into an autonomous operating-system environment.


Roadmap

The current direction is approximately:

Code: Select all

UEFI
 |
 +-- GOP / framebuffer                    [PASS]
 |
 +-- GetMemoryMap / ExitBootServices
 |
 +-- CPU / memory / interrupts
 |
 +-- ACPI
      |
      +-- RSDP
      +-- XSDT
      +-- MCFG
 |
 +-- PCI Express
      |
      +-- ECAM
      +-- bus/device/function enumeration
      +-- Vendor ID / Device ID
      +-- Class / Subclass / ProgIF
      +-- BAR / MMIO discovery
 |
 +-- review PCIe foundation
 |
 +-- xHCI
      |
      +-- USB
      +-- HID
      +-- keyboard
      +-- mouse
 |
 +-- movable software cursor
 |
 +-- eventually, a graphical environment

I deliberately intend to stop and review the PCIe/BAR foundation before starting actual device drivers.

Current status

Code: Select all

PASS on physical hardware

AMD Ryzen 5 9600X
ASUS TUF GAMING B850-PLUS WIFI

Windows 11
  -> NASM
  -> lld-link
  -> PE32+ EFI
  -> FAT32 USB
  -> physical UEFI boot
  -> GOP
  -> framebuffer
  -> direct rendering

Next:
GetMemoryMap() -> ExitBootServices()

[Norppa]

OperatingSystem.zip

(15.83 KiB) Downloaded 11 times

Milestone 3: custom GDT, IDT and first exception handler working.
Small update, but this one is more interesting than the framebuffer test.

The kernel now leaves the UEFI execution environment and establishes its own minimal CPU environment.

Current boot path:

Code: Select all

UEFI
  ↓
GOP framebuffer
  ↓
GetMemoryMap()
  ↓
ExitBootServices()
  ↓
private kernel stack
  ↓
custom GDT
  ↓
reload CS / SS
  ↓
custom IDT
  ↓
INT3
  ↓
custom #BP handler
  ↓
IRETQ
  ↓
kernel resumes execution

For the test I deliberately use four framebuffer markers:

Code: Select all

green    = ExitBootServices succeeded and private stack is active
magenta  = custom GDT loaded and CS/SS successfully reloaded
white    = #BP handler was entered through my own IDT
gray     = IRETQ returned successfully to the instruction after INT3

All four markers appear on the machine.

The IDT currently contains 256 16-byte entries. All vectors initially point to a generic fatal handler, with vector 3 replaced by a dedicated breakpoint handler.

The breakpoint test is simply:

Code: Select all

    lidt    [IdtDescriptor]

    int3

    ; Reached only if the #BP handler successfully returns with IRETQ.

The handler saves the general-purpose registers, draws the white marker, restores the registers and returns:

Code: Select all

isr_breakpoint:
    PUSH_GPRS

    ; framebuffer diagnostic marker
    ...

    POP_GPRS
    iretq

Interrupts are still disabled (IF=0). I am intentionally not enabling hardware IRQs yet; there is no APIC/interrupt-controller initialization at this stage.

The GDT currently contains only:

Code: Select all

0x00  null descriptor
0x08  64-bit ring-0 code
0x10  ring-0 data

After loading it with LGDT, CS is reloaded with a far return and the data/stack segment selectors are replaced as well.

I also switched away from the stack supplied by the firmware and use a 64 KiB stack contained in the kernel image.

The next step will probably be proper CPU exception stubs for vectors 0–31, including normalization of exceptions with/without error codes, so that a panic handler can display at least:

Code: Select all

vector
error code
RIP

After that I intend to start using the UEFI memory map for physical-memory management before moving on to ACPI/PCIe.

[Norppa]

OperatingSystem.zip

(20.03 KiB) Downloaded 10 times

Milestone 5 completed.
This milestone introduces the project's first physical memory manager (mm\physical.asm).

The kernel now keeps the UEFI memory map, parses it after ExitBootServices(), and builds a simple 4 KiB page allocator from EfiConventionalMemory. For the moment, memory below 1 MiB is intentionally excluded, and freed pages are recycled through a small LIFO free list.

The current self-test checks:
- descriptor parsing
- usable memory accounting
- allocation of multiple distinct pages
- direct write/read access to allocated pages
- free/reallocate correctness
- restoration of the original free-page count

On the current test system, the allocator reports:
- 0x3E memory descriptors
- 0x7C158E000 managed bytes (32 GB RAM)
- 0x7C158E free pages initially

The test then allocates:
- A = 0x00100000
- B = 0x00101000
- C = 0x00102000

After freeing B, the next allocation returns:
- D = 0x00101000

So the recycled-page path behaves as expected. After returning all test pages, the final free-page count again matches the initial one.

So the project now has working:
- UEFI bootstrapping
- framebuffer output
- ExitBootServices()
- private stack setup
- GDT / IDT
- CPU exception handling
- basic physical page allocation

Next milestone: build paging structures and switch to CR3.

[Norppa]

OperatingSystem.zip

(32.45 KiB) Downloaded 10 times

Milestone 6 completed, and Milestone 7 is now partially implemented.

This update replaces the firmware paging environment with page tables owned by HobbyOS and introduces the first virtual-memory primitives.


Milestone 6 — HobbyOS-owned paging
-----------------------------------

The kernel now:

- inspects the paging state inherited from UEFI
- allocates and clears its own paging structures
- builds a bootstrap identity map
- maps the GOP framebuffer when required
- verifies all critical kernel ranges before switching
- loads its own PML4 through CR3
- continues executing and drawing to the framebuffer afterwards
- verifies that the PMM still works under the new CR3

The current bootstrap map covers:

[0, 6 GiB)

using 2 MiB pages.

The PMM is temporarily restricted to conventional RAM inside:

[1 MiB, 6 GiB)

so that all pages returned by the allocator remain directly accessible during
the current bootstrap stage.

This restriction is temporary and will be revisited by the virtual-memory
manager.

On the current Ryzen 5 9600X test system, the CR3 transition was:

firmware CR3 : 0x78001000
HobbyOS CR3 : 0x00101000

The post-switch PMM regression also verifies both fresh sequential allocation
and recycled-page allocation while HobbyOS page tables are active.


Milestone 7 — Virtual memory
----------------------------

The project now explicitly distinguishes physical and virtual addresses.

The following primitives are working:

Code: Select all

    paging_translate()
    paging_map_page()
    paging_unmap_page()
    paging_map_range()

7A — Translation
The kernel can walk its own page tables and translate mapped virtual addresses
back to physical addresses, while rejecting unmapped and non-canonical ones.

7B — 4 KiB mapping
The kernel can create explicit non-identity 4 KiB mappings and verify that
writes through the virtual address reach the expected physical page.

7C — Unmapping
The kernel can remove a 4 KiB mapping, invalidate the TLB entry with INVLPG,
and return the original physical address without freeing the physical page.

7D — Range mapping
The kernel can map and later remove contiguous virtual ranges.

The current hardware test maps three pages:

physical start : 0x0000000140008000
virtual start : 0x0000010000004000

The test verifies translation, read/write aliasing, cleanup, and reports:

VmmRangeChecks = 0x1FF

All nine checks passed on real hardware.

The current memory path is therefore:

Code: Select all

    PMM
        -> physical page ownership

    paging/VMM
        -> virtual mappings

and physical addresses are no longer assumed to be directly interchangeable
with virtual addresses as a general rule.

The next architectural question is how the kernel should access ordinary
physical RAM permanently.

The main candidates are a physical direct map or temporary mappings or a hybrid strategy

Current working milestones:

- UEFI bootstrapping
- GOP framebuffer
- ExitBootServices()
- private kernel stack
- GDT / IDT
- CPU exception handling
- physical memory manager
- HobbyOS-owned paging / CR3
- page-table translation
- 4 KiB map/unmap
- range mapping

[Norppa]

OperatingSystem.zip

(61.32 KiB) Downloaded 6 times

Milestone 7 completed — virtual memory foundation

Since the previous update, HobbyOS gained:

Code: Select all

4 KiB map / query / unmap
2 MiB map / query / unmap
range mapping
virtual -> physical translation
permanent high-half physical direct map
semantic mapping properties
WB / UC cache policy
PMM access above the old 6 GiB bootstrap ceiling

The direct map lives at:

Code: Select all

0xFFFF800000000000

and uses 2 MiB leaves where possible, with 4 KiB fallback.
Mapping policy is now expressed with HobbyOS-level properties:

Code: Select all

MAP_WRITABLE
MAP_EXECUTABLE
MAP_USER

CACHE_WRITE_BACK
CACHE_UNCACHEABLE

rather than exposing raw x86 paging bits to callers.
The kernel also explicitly establishes the paging-related CPU state it depends on, including write protection and NX support.
All of this has been validated on real hardware for both 4 KiB and 2 MiB mappings.
Current state:

Code: Select all

M7A  translation                    PASS
M7B  4 KiB mapping                  PASS
M7C  4 KiB unmapping                PASS
M7D  range mapping                  PASS
M7E  permanent physical direct map  PASS
M7F  remove 6 GiB PMM ceiling       PASS
M7G  mapping policy                 PASS

M7   virtual memory foundation      PASS

1 GiB leaves are detected but intentionally deferred for now.

Bonus: Paging Playground
I also added a small development-only paging_playground.asm.
The first demo maps two virtual addresses to the same physical page, verifies aliasing, then remaps one of them to a different physical page.

Code: Select all

VA_A ----+
         +----> Physical A
VA_B ----+

then:

Code: Select all

VA_A --------> Physical A
VA_B --------> Physical B

All 13 checks pass on hardware:

Code: Select all

PagingPlaygroundChecks = 0x1FFF

It is not part of the normal boot path; it is just a place to experiment with paging mechanisms before they become higher-level architecture.

Next:

Code: Select all

M8 — ACPI