shibedrill/gila
shibedrill/gila
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

Memory investigations
Some checks failed
Continuous Integration / Check (push) Failing after 3m23s
Details
Continuous Integration / Clippy (push) Failing after 2m12s
Details

Browse Source
This commit is contained in:
August 2026-03-16 21:27:47 +00:00
parent a018810c9c
commit f0177d2d4d
Signed by: shibedrill
SSH Key Fingerprint: SHA256:M0m3JW1s38BgO2t0fG146Yxd9OJ2IOqkvCAsuRHQ6Pw
8 changed files with 150 additions and 310 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

83
docs/DESIGN.md
View File

@ -32,7 +32,7 @@ similar to (if not compatible with) D-Bus.
## Boot Process
After being initialized by the bootloader at a random address, the kernel will
perform some memory management work to start allocating memory for the userboot
perform some memory management work to start allocating memory for the Userboot
binary. Userboot is a binary executable that will be loaded as a module, and
it will be initialized as the very first process.
@ -63,60 +63,37 @@ The system, hence, can be configured in two ways:
- The bare minimum drivers needed for disk access are included in the
initramfs, and all other drivers are included in the root filesystem.
## APIs
## Integration as a Full OS
Processes will access services by means of a data bus, possibly similar to D-Bus.
In this model, processes would access information from services by making an
IPC call to the kernel, which would either serve as a D-Bus server or
delegate D-Bus stuff to a special server process. From there, the client
process may establish a connection to the system bus, and use that connection
to request services.
Gila is just a kernel. It does not do much outside of scheduling and memory
management. To make it useful, you must provide a correctly versioned and
formatted initramfs image as a boot module, and include it in the ISO.
For example, if a process wanted to request the kernel version, it could
access the service `site.shibedrill.Gila`, the object path `/site/shibedrill/Gila/Kernel`,
and the property `site.shibedrill.Gila.Kernel.Version`. If the same process
wanted to access the vendor ID of a specific PCI device, it could access
service `site.shibedrill.Pci`, object `/site/shibedrill/Pci/Device/07/00/0`, and
property `site.shibedrill.Pci.Device.Vendor`. This property would be present
in all PCI devices, as it would be defined in an interface common to all PCI
device objects in the service's namespace.
The structure of the initramfs will soon be formally defined, and Userboot will
handle parsing it and confirming it is a valid and compatible version before
initializing the full operating system from its contents.
## Device Drivers
My ideal vision of an OS built on Gila is composed of several "realms", or
privilege levels, which define how information may flow within the system.
Device drivers, in this userspace concept, are initialized as-needed. If a
process requests a service provided by a driver that is not yet running, a
privileged process (or the kernel) will initialize a device driver process.
If the relevant device is present, the kernel will map the necessary portions
of physical memory into the driver's address space, and return information on
the mappings to the new driver. If the device does not exist, the message bus
will return an error.
- Realm 0, "Kernel Realm", consists solely of the kernel.
- Realm 1, "Driver Realm", holds device drivers and protocol/filesystem drivers.
This realm provides a solid hardware abstraction layer which the service realm
may build upon. This is where I would choose to place a memory allocator & any
standard library implementations, developing them as a part of the driver realm.
- Realm 2, "Service Realm", contains components and services that rely on the
driver realm or the standard library. This may include things like access
control, user management, application management, and standard APIs for
applications to access. This is the software abstraction layer, or framework
layer.
- Realm 3, "Application/User Realm", contains processes, applications, and
libraries that are totally untrusted. To perform any functionality they must
correspond with the Service Realm so it can enforce security policies. Anything
which can reasonably be written as an unprivileged application should be one,
to allow for the best isolation and security boundaries.
How the kernel will recognize whether a device is present, is still unknown.
Hopefully, a comprehensive enumeration system can be developed which does not
require device definitions to be built into the kernel. I am considering a
system where device driver binaries have "enumerate" entry points in
conjunction with their "main" entry points, and the "enumerate" function
instructs the driver server to search for compatible devices and fork if any
are found. This removes all device-specific code from the kernel.
## Servers vs. Shared Libraries
Servers and shared libraries serve similar purposes: They make some
functionality usable by any process without code duplication. However, there
are times when processes and developers should prefer one or the other.
A server should be used when:
- The function must somehow acquire a mutually exclusive lock on a resource.
- The function should complete asynchronously.
A shared library should be used when:
- No resources involved need to be mutually exclusive.
- The function is non-blocking and synchronous.
Hence, servers are very important for things like disk drivers and file
system drivers, where non-synchronized writes could cause data loss. It should
also be noted that libraries *can*, and often will, call local procedures
from servers. The functionality of calling a procedure will be handled by the
standard library and shared across processes.
This structure works well with the Biba security model, which dictates how
information must or must not flow between privilege levels to preserve integrity.
No realm may write data to any realm above it, and no realm may read data from
any realm below it except to arbitrate communication between components within
the same realm.

33
docs/DEVELOPMENT.MD
View File

@ -29,9 +29,10 @@ design details can be found in [SECURITY.md](SECURITY.md).
functionality.
- [process.rs](../kernel/src/process.rs): Process types and functions.
- [syscall\_runner.rs](../kernel/src/syscall_runner.rs): Chooses a system call
by its ID and defers actual syscall execution to code in `src/lib/`.
- [libgila/src/](../libgila/src/lib.rs): Library that all Gila's binary programs will be
built against. Some of this code is shared with the kernel.
by its ID and defers actual syscall execution to code in `/libgila/src/`.
- [libgila/src/](../libgila/src/lib.rs): Shared between the kernel and any
binaries built for it. Contains any definitions, structures, and types that will
cross the boundary between userspace and kernelspace, including those sent over IPC.
- [arch/](../libgila/src/arch/mod.rs): Architecture specific functionality like
system call register storing/loading.
- [syscall.rs](../libgila/src/syscall.rs): System call types common to apps and
@ -45,6 +46,11 @@ generated. However, it cannot be booted without installing it in a bootable
Limine filesystem, and it cannot do anything useful without an initramfs
containing system servers, such as the init server and device drivers.
> [!NOTE]
> The ISO build system will be removed once Gila reaches a stable version,
> and instead, bootable image creation will be handled by the system software
> distribution.
This project uses [cargo-make](https://github.com/sagiegurari/cargo-make) to
handle building ISOs and managing files not associated with Cargo. You need to
install it before you can build an ISO automatically. To do so, you can run
@ -82,7 +88,7 @@ and `iso` tasks will automatically be rerun if their input files change.
> [!NOTE]
> The `-p {profile}` argument must go between `cargo make` and the task
argument.
> argument.
### Targets
@ -138,12 +144,19 @@ compression is disabled. It must also be changed in
## Writing Programs for Gila
Gila's system calls will soon be fully defined in `libgila`. The library is
developed in tandem with, and is used within, the kernel. As Gila does not
currently support any kind of Rust or C standard library, you must compile
your programs for the same architecture as Gila itself, bare metal. Userspace
programs must not use any privileged instructions that would cause an
exception if running in Ring 3 or any other least-privileged mode of a
processor.
developed in tandem with, and is used within, the kernel. As such, it is
treated as part of the kernel. Programs built against the wrong version of
`libgila` will not work due to mismatched definitions.
As Gila is merely a kernel, and its userspace will heavily depend on whatever
servers and drivers are included alongside it, no standard library for any
language will ever be issued as part of the kernel's software package. Instead,
this will be deferred to the system software developer who designs and
integrates the userspace.
To compile software for Gila, simply write your program as if it were a `no_std`
binary for the kernel's processor architecture. By linking against `libgila`,
the program may access things like system calls, error types, and IPC data formats.
## Contributing

40
docs/SECURITY.md
View File

@ -73,34 +73,24 @@ introduces a serious security risk, as any compromise or failure of the login
server could result in security failures, resource exhaustion, denial of
service, or other undesirable events, affecting all other users on the system.
### Realms
### Boot Security
A complete OS using the Gila microkernel will consist of four "realms". Each
realm depends on the realm preceeding it.
Since Gila is just a kernel, the complexity of its userspace is up to the
operating system developer. The developer/integrator may choose to include some
or all of their software in the initramfs, rather than store it on a disk. By
properly hashing all boot modules + the config file, embedding the config hash
in Limine, and enabling Secure Boot, a chain of trust rooted in the hardware
can be established.
- Kernel Realm: The kernel itself, providing MMIO pages to the Driver Realm
- Driver Realm: Driver processes supplying interfaces to the Service Realm
- Service Realm: System services, supplying services like filesystems to the
User Realm
- User realm: User services/apps, running at the least privileged level
### Userspace Security Model
Typically, for a secure deployment of an OS designed for a focused purpose,
one has no real need to dynamically modify the Service Realm, Driver Realm, or
Kernel Realm. So to simplify the deployment, the Driver and Service realms can
be contained in the kernel's initramfs.
For embedded use, or for virtual appliances, the User Realm can additionally be
embedded in the initramfs. This eliminates the need for verifying or decrypting
a hard disk, or loading information from the network. Of course, volatile
information can still be stored on a hard drive, but the mission critical
pieces of code can be stored in a read-only format until the system needs to be
updated.
Furthermore, the signatures of the kernel and any kernel modules can be
calculated and stored in the Limine config file, whose hash can be embedded in
the bootloader itself for use with Secure Boot. This quickly establishes
hardware-based root-of-trust for the kernel and all realms stored in the
initramfs.
Most of how information will flow within a full OS is up to the integrator.
Gila as a kernel enforces strong boundaries between processes, and requires
processes to have the correct capabilities before they may access a resource.
The kernel will never *trust* or *act on* information gleaned or sent from
processes, aside from very limited functionality regarding system calls, to
enforce the Biba security model at the boundary between kernelspace and
userspace.
### Namespaces

1
kernel/src/arch/x86_64/mod.rs
View File

@ -7,5 +7,4 @@ pub mod cpuid;
pub mod gdt;
pub mod intel_virt;
pub mod interrupts;
pub mod paging;
pub mod serial;

114
kernel/src/arch/x86_64/paging.rs
View File

@ -1,114 +0,0 @@
// Copyright (c) 2025 shibedrill
// SPDX-License-Identifier: MIT
#![allow(dead_code)]
use crate::constants::KERNEL_BUILD_PROFILE;
use crate::memory::alloc::boxed::Box;
use crate::{LOGGER, LogLevel, format, log_info, log_trace, memory::HHDM_RESPONSE};
use free_list::{PAGE_SIZE, PageLayout};
use lazy_static::lazy_static;
use spin::Mutex;
use x86_64::{
PhysAddr, VirtAddr,
registers::control::*,
structures::paging::{PageTable, PageTableFlags},
};
lazy_static! {
pub static ref KERNEL_PML4: Mutex<Option<Box<PageTable>>> = Mutex::new(None);
}
pub fn switch_ptable() {
let allocated_region = crate::memory::FREELIST
.lock()
.allocate(PageLayout::from_size_align(PAGE_SIZE, PAGE_SIZE).unwrap())
.expect("Could not allocate pages for new page table!");
log_info!("Got region for new PML4: 0x{:x}", allocated_region.start());
//let pml4_start_vaddr = allocated_region.start() + HHDM_RESPONSE.offset() as usize;
//let pml4_ptr = pml4_start_vaddr as *mut PageTable;
/*
Dropping this PML4 causes a deadlock. Here is why:
- The allocator, Talc, did not allocate the memory we built the PML4 in.
- When this function exits, the allocator is *locked*.
- The allocator is told to drop this PML4, and panics, because it was never "allocated" to begin with.
- The allocator panics.
- The panic function tries to call the logging code.
- The logging code tries to acquire lock on the allocator, but it will never get it, since it is locked
until the function finishes exiting. Which it won't, because the allocator panicked.
*/
//let mut pml4 = unsafe { crate::memory::alloc::boxed::Box::from_raw(pml4_ptr) };
//*pml4 = PageTable::new();
//log_info!("Initialized page table at 0x{:p}", pml4);
}
pub fn get_mappings() {
log_info!("Paging: {}", Cr0::read().contains(Cr0Flags::PAGING));
log_info!(
"Protection: {}",
Cr0::read().contains(Cr0Flags::PROTECTED_MODE_ENABLE)
);
log_info!(
"Physical Address Extensions: {}",
x86_64::registers::control::Cr4::read().contains(Cr4Flags::PHYSICAL_ADDRESS_EXTENSION)
);
log_info!(
"Page Size Extensions: {}",
Cr4::read().contains(Cr4Flags::PAGE_SIZE_EXTENSION)
);
log_info!(
"Paging mode: {}",
match crate::memory::PAGING_REQUEST.get_response().unwrap().mode() {
limine::paging::Mode::FOUR_LEVEL => "Four-Level",
limine::paging::Mode::FIVE_LEVEL => "Five-Level",
_ => unreachable!(),
}
);
log_info!(
"Page-Level Write-Through: {}",
Cr3::read().1.contains(Cr3Flags::PAGE_LEVEL_WRITETHROUGH)
);
log_info!(
"Page-Level Cache Disable: {}",
Cr3::read().1.contains(Cr3Flags::PAGE_LEVEL_CACHE_DISABLE)
);
if KERNEL_BUILD_PROFILE == "debug" {
let pml4_phys_addr = Cr3::read_raw().0.start_address();
log_info!("Page Map Level 4 Address: 0x{:x}", pml4_phys_addr.as_u64());
let pml4: &'static mut PageTable = table_from_phys_addr(pml4_phys_addr);
iter_table(pml4, 4);
}
}
fn table_from_phys_addr(address: PhysAddr) -> &'static mut PageTable {
let virt_addr: VirtAddr = VirtAddr::new(address.as_u64() + HHDM_RESPONSE.offset());
let ptr: *mut PageTable = virt_addr.as_mut_ptr();
unsafe { &mut *ptr }
}
fn iter_table(table: &'static mut PageTable, level: usize) {
if level == 1 {
for page_entry in table.iter() {
if page_entry.flags().contains(PageTableFlags::PRESENT) {
log_trace!(
"Page entry: 0x{:x}, flags: {:?}",
page_entry.addr(),
page_entry.flags()
);
}
}
} else {
for table_entry in table.iter() {
if table_entry.flags().contains(PageTableFlags::PRESENT) {
log_info!(
"Table entry: 0x{:x}, flags: {:?}",
table_entry.addr(),
table_entry.flags()
);
iter_table(table_from_phys_addr(table_entry.addr()), level - 1);
}
}
}
}

14
kernel/src/main.rs
View File

@ -17,6 +17,8 @@ mod panic;
mod process;
mod util;
use core::ptr::addr_of;
use arch::x86_64::interrupts::IDT;
use arch::x86_64::serial::SerialPort;
use boot::{BASE_REVISION, params, *};
@ -25,11 +27,13 @@ use log::*;
use memory::alloc::{boxed::Box, format, string::String, vec};
use params::*;
use x86_64::VirtAddr;
use x86_64::structures::paging::PageTable;
use lazy_static::lazy_static;
use limine::firmware_type::FirmwareType;
use spin::mutex::Mutex;
use crate::arch::x86_64::cpuid::{CPUID, virt_supported};
use crate::{arch::x86_64::cpuid::{CPUID, virt_supported}, memory::{HHDM_RESPONSE, paging::{PML4, active_level_4_table, iter_table}}};
lazy_static! {
pub static ref SERIAL_3F8: Mutex<SerialPort> = Mutex::new(
@ -138,8 +142,6 @@ unsafe extern "C" fn main() -> ! {
memory::log_memory();
memory::log_address();
arch::x86_64::paging::get_mappings();
if let Some(string) = CPUID.get_processor_brand_string() {
log_info!("CPU brand string: {}", string.as_str());
}
@ -151,8 +153,10 @@ unsafe extern "C" fn main() -> ! {
}
log_info!("Virtualization provider: {:?}", virt_supported());
arch::x86_64::paging::switch_ptable();
log_info!("Made it this far");
log_info!("Physical address of current L4 table: 0x{:x}", x86_64::registers::control::Cr3::read().0.start_address());
log_info!("Virtual address of current L4 table: 0x{:x}", x86_64::registers::control::Cr3::read().0.start_address().as_u64() + HHDM_RESPONSE.offset());
//iter_table(4, &PML4);
panic!("Finished boot, but cannot start init because processes not implemented!");
}

106
kernel/src/memory.rs → kernel/src/memory/mod.rs
View File

@ -1,12 +1,13 @@
// Copyright (c) 2025 shibedrill
// SPDX-License-Identifier: MIT
const PAGE_SIZE: usize = 4096;
use crate::boot::modules::MODULE_RESPONSE;
use crate::boot::params::EXECUTABLE_FILE_RESPONSE;
use crate::{LOGGER, LogLevel, format};
use alloc::string::String;
use free_list::{AllocError, FreeList, PAGE_SIZE, PageRange};
use lazy_static::lazy_static;
use limine::response::MemoryMapResponse;
use limine::{
@ -14,13 +15,11 @@ use limine::{
request::{ExecutableAddressRequest, HhdmRequest, MemoryMapRequest, PagingModeRequest},
response::HhdmResponse,
};
use spin::Mutex;
use talc::*;
pub extern crate alloc;
pub static FREELIST: Mutex<FreeList<32>> = Mutex::new(FreeList::<32>::new());
pub static HW_FREELIST: Mutex<FreeList<32>> = Mutex::new(FreeList::<32>::new());
pub mod paging;
lazy_static! {
pub static ref MEMMAP_RESPONSE: &'static MemoryMapResponse = MEMMAP_REQUEST
@ -28,94 +27,6 @@ lazy_static! {
.expect("Bootloader did not supply memory map");
}
pub fn round_up_to(multiple: usize, input: usize) -> usize {
if input.is_multiple_of(multiple) {
input
} else {
input - (input % multiple) + multiple
}
}
fn round_down_to(multiple: usize, input: usize) -> usize {
if input.is_multiple_of(multiple) {
input
} else {
input - (input % multiple)
}
}
pub fn deallocate_usable() -> Result<(), AllocError> {
init_statics();
let mut any_error: Option<AllocError> = None;
let mut bytes_deallocated: u64 = 0;
for entry in MEMMAP_RESPONSE.entries() {
if (entry.entry_type == EntryType::USABLE)
| (entry.entry_type == EntryType::BOOTLOADER_RECLAIMABLE)
{
if let Err(error) = deallocate(**entry) {
any_error = Some(error);
}
bytes_deallocated += entry.length;
}
}
log_trace!("Deallocated 0x{:x} bytes", bytes_deallocated);
if let Some(error) = any_error {
Err(error)
} else {
Ok(())
}
}
pub fn deallocate_hardware() -> Result<(), AllocError> {
let mut any_error: Option<AllocError> = None;
let mut bytes_deallocated: u64 = 0;
for entry in MEMMAP_RESPONSE.entries() {
if (entry.entry_type == EntryType::ACPI_NVS)
| (entry.entry_type == EntryType::RESERVED)
| (entry.entry_type == EntryType::FRAMEBUFFER)
{
if let Err(error) = {
let base: usize = round_down_to(PAGE_SIZE, entry.base as usize);
let mut length = entry.length as usize + (entry.base as usize - base);
length = round_up_to(PAGE_SIZE, length);
let range = PageRange::from_start_len(base, length);
match range {
Ok(range_inner) => unsafe { HW_FREELIST.lock().deallocate(range_inner) },
Err(err) => {
log_error!(
"Failed to convert range 0x{:x} @ 0x{:x} (original: 0x{:x} @ 0x{:x}): {}",
length,
base,
entry.length,
entry.base,
err
);
Ok(())
}
}
} {
any_error = Some(error);
}
bytes_deallocated += entry.length;
}
}
log_trace!(
"Deallocated 0x{:x} bytes of hardware reserved memory",
bytes_deallocated
);
if let Some(error) = any_error {
Err(error)
} else {
Ok(())
}
}
pub fn deallocate(entry: limine::memory_map::Entry) -> Result<(), AllocError> {
let range = PageRange::from_start_len(entry.base as usize, entry.length as usize)
.expect("Could not convert map entry to range!");
unsafe { FREELIST.lock().deallocate(range) }
}
pub fn log_memory() {
if !MEMMAP_RESPONSE.entries().is_empty() {
let mut log_msg: String = String::from("Memory map:");
@ -173,16 +84,6 @@ pub fn log_memory() {
"Memory report: \n\tFree: 0x{usable:x}\n\tAvailable: 0x{reclaimable:x}\n\tUsable: 0x{:x}\n\tHardware: 0x{hardware:x}\n\tUnusable: 0x{unusable:x}\n\tTotal: 0x{total:x}",
usable + reclaimable,
);
log_trace!("Deallocating available memory...");
let _ = deallocate_usable();
log_trace!("Deallocating hardware reserved memory...");
let _ = deallocate_hardware();
let free_bytes = { FREELIST.lock().free_space() };
log_info!(
"Freelist: 0x{:x} bytes ({} pages)",
free_bytes,
free_bytes / PAGE_SIZE
);
} else {
panic!("Memory map contains no entries");
}
@ -195,6 +96,7 @@ pub fn log_address() {
resp.physical_base()
);
log_info!("Kernel virtual start address: 0x{:x}", resp.virtual_base());
log_info!("Kernel physical start address (calculated): 0x{:x}", 0i64 - resp.virtual_base() as i64 + resp.physical_base() as i64);
} else {
log_warning!("No kernel address response provided.");
}

69
kernel/src/memory/paging.rs Normal file
View File

@ -0,0 +1,69 @@
use x86_64::PhysAddr;
use x86_64::structures::paging::PageTable;
use x86_64::VirtAddr;
use lazy_static::lazy_static;
use x86_64::structures::paging::PageTableFlags;
use crate::log::*;
use crate::memory::format;
use crate::memory::HHDM_RESPONSE;
pub unsafe fn active_level_4_table()
-> &'static mut PageTable
{
use x86_64::registers::control::Cr3;
let (level_4_table_frame, _) = Cr3::read();
let phys = level_4_table_frame.start_address();
let virt: VirtAddr = VirtAddr::new(HHDM_RESPONSE.offset() + phys.as_u64());
let page_table_ptr: *mut PageTable = virt.as_mut_ptr();
unsafe { &mut *page_table_ptr }
}
lazy_static! {
pub static ref PML4: &'static mut PageTable = unsafe { active_level_4_table() };
}
pub fn iter_table(level: usize, table: &PageTable) {
for (i, entry) in table.iter().enumerate() {
if !entry.is_unused() & entry.flags().contains(PageTableFlags::PRESENT) {
log_info!("L{} Entry {}: {:?}", level, i, entry);
if (level > 1) & !entry.flags().contains(PageTableFlags::HUGE_PAGE) {
let phys = entry.frame().unwrap().start_address();
let virt = phys.as_u64() + HHDM_RESPONSE.offset();
let ptr = VirtAddr::new(virt).as_mut_ptr();
let new_table: &PageTable = unsafe { &*ptr };
iter_table(level - 1, new_table);
}
}
}
}
/*
let l4_table = unsafe { active_level_4_table() };
for (i, entry) in l4_table.iter().enumerate() {
if !entry.is_unused() {
log_info!("L4 Entry {}: {:?}", i, entry);
// get the physical address from the entry and convert it
let phys = entry.frame().unwrap().start_address();
let virt = phys.as_u64() + HHDM_RESPONSE.offset();
let ptr = VirtAddr::new(virt).as_mut_ptr();
let l3_table: &PageTable = unsafe { &*ptr };
// print non-empty entries of the level 3 table
for (i, entry) in l3_table.iter().enumerate() {
if !entry.is_unused() {
log_info!(" L3 Entry {}: {:?}", i, entry);
}
}
}
}
*/