Thank you for the wiki

All off topic discussions go here. Everything from the funny thing your cat did to your favorite tv shows. Non-programming computer questions are ok too.
Post Reply
cardboardaardvark
Posts: 20
Joined: Mon Nov 18, 2024 10:06 pm

Thank you for the wiki

Post by cardboardaardvark »

Hi,

I'm new. New to the forum and new to OS development. I wanted to say thank you very much to the people who contributed to the wiki. It was extremely helpful for getting this system administrator with a lot of Perl experience and some C++ experience to the point of having the start of a toy kernel implemented in C++ with runtime features actually ticking away counting uptime and showing a clock on the VGA screen.

It wasn't a quick process to get here but the wiki was so much of a time saver. Perfect way to get started with the x86 GDT/IDT/PTD/PTE so I can actually see things happening with out having to wrap my head around the official Intel references which can be a little, ahem, overwhelming at the very start. I never went to college for CS (I studied welding instead) and I swear I got about a quarter's worth of education out of the wiki in only a few weeks.

So thanks again from this grizzled sysadmin starting his new hobby of trying to come up with an OS design that isn't just another Unix clone.

Thanks again. And here's a screenshot of TalOS (yes, I've been playing too much Skyrim) ticking away inside qemu: Image
thewrongchristian
Member
Member
Posts: 425
Joined: Tue Apr 03, 2018 2:44 am

Re: Thank you for the wiki

Post by thewrongchristian »

cardboardaardvark wrote: Mon Nov 18, 2024 10:28 pm So thanks again from this grizzled sysadmin starting his new hobby of trying to come up with an OS design that isn't just another Unix clone.
Welcome to the club. Be prepared for it to suck your time out of your life if you stay interested!

BTW, you can't go much wrong with being just another Unix clone. As you do stuff, you'll get "a-ha" moments as you realize why Unix does things the way it does (you'll also get many many WTF moments as well, Unix isn't perfect!) And you'll have a ready source of user software to port, which may not be the case if you're using a non-Unix like API.
cardboardaardvark
Posts: 20
Joined: Mon Nov 18, 2024 10:06 pm

Re: Thank you for the wiki

Post by cardboardaardvark »

thewrongchristian wrote: Tue Nov 19, 2024 5:30 am BTW, you can't go much wrong with being just another Unix clone. As you do stuff, you'll get "a-ha" moments as you realize why Unix does things the way it does (you'll also get many many WTF moments as well, Unix isn't perfect!) And you'll have a ready source of user software to port, which may not be the case if you're using a non-Unix like API.
Aye, I've thought about the lack of existing software to port if I don't supply a POSIX API and how that is going to be a detriment. Probably going to wind up with no one but me using it. But also, even if I do make another Unix clone, probably no one but me will ever be using it.

I saw a post around here yesterday where the OP was pointing out most people just get a clock going and walk away. And if anyone goes past that they wind up making a tiny Unix implementation and then walk away. No one actually tries anything different.

It's likely not as bad as "no one" ever tries to do something different but I'm not surprised to see that it is rare. I've got some ideas and it is really what I wanted to explore. As I've been reading about kernel design and from the little I know about Windows I find that I'm more inclined to think in terms of how Windows/VMS solves problems than how Unix does it. Which is rather surprising to me seeing as I've been using Unix as my primary workstation since 1994 and owned my own personal Sun Ultra 5 which I also used as my primary workstation.

I'm interested to see where a Windows/Unix Frankenstein winds up. One thing I very much want to do better than POSIX is how permissions work. The base model is too simplistic, extended permission attributes on files don't go far enough, and all the MAC choices for Linux and FreeBSD leave a lot to be desired.
User avatar
eekee
Member
Member
Posts: 881
Joined: Mon May 22, 2017 5:56 am
Location: Kerbin
Discord: eekee
Contact:

Re: Thank you for the wiki

Post by eekee »

Good to see another OS which isn't just another Unix clone. :) I've been thinking about the hardware Unix first ran on; 32KB RAM & 3 1GB disks in the PDP-11, and wondering quite how much of modern Unix's weirdness goes back to that strange ratio. Parent processes cleaning up after dead children might be part of it, it makes sense to put that task in userspace which could be swapped out. They didn't write that particular oddity into Plan 9, but it had plenty of other weirdness in pursuit of small code size for some reason.

I'm going for a unique OS design myself, and I've been looking at odd languages in the hope of speeding development: Forth and "plain English programming". Neither has worked out too well for me, though they might for others. I was drawn to Forth by someone who said it was much faster than C++ coding. Professionally, he wrote libraries for C++ systems, but he wrote them in Forth in well under half the expected time. I however struggled to manage strings and other data structures, thinking the minimal facilities the language provided were enough. They weren't, and I needed to learn how to design my own. You need to understand how to design a lot of different things to be a fast Forth programmer. I would have learned faster if I'd used a Forth system with more modern facilities, seeing how they implemented things.

Plain English programming with CAL-4700 is essentially compiled pseudocode. Here, my problem is that I don't think the same way as the compiler authors, and thus it's taking time to learn to espress my self in their pseudocode. On the positive side, it was very easy to learn to read the code of the IDE and compiler; a big set of real-world examples.
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
cardboardaardvark
Posts: 20
Joined: Mon Nov 18, 2024 10:06 pm

Re: Thank you for the wiki

Post by cardboardaardvark »

eekee wrote: Mon Nov 25, 2024 4:08 am I've been thinking about the hardware Unix first ran on; 32KB RAM & 3 1GB disks in the PDP-11, and wondering quite how much of modern Unix's weirdness goes back to that strange ratio.
A while ago I started a project that needed a microcontroller to drive a machine I was building. I grabbed an Arduino, tossed out their IDE/runtime, and targeted it directly with gcc and avr-libc. I was quite surprised to see so much Unix coming out of avr-libc. Then it clicked: this $5 microcontroller I'm working with is a substantial fraction of the hardware that Unix was originally created for. Pretty amazing.
Good to see another OS which isn't just another Unix clone.
I'm giving it a shot anyway. I'm still at the phase where it is a drastically over complicated clock. I'm moving ahead though. The kernel main loop now pulls jobs out of a queue and if no job is present does a hlt. I multiplexed the PIT and wrapped it with an interface that lets you create timers with arbitrary initial and reoccurring delays with a precision of 1ms. In the interest of minimizing the amount of time where interrupts are disabled the PIT ISR checks to see if it already has a pending job and if not puts one in the queue to check for expired timers then exits. In the interest of future concurrency if expired timers are found they too are submitted as jobs to run.

I broke the status bar in the console into two parts: memory info and the clock and the memory info updates more frequently than the clock. From left to right the memory info goes: size of kernel program break, available physical pages, total physical pages:

Image
I've been looking at odd languages in the hope of speeding development
I've got the same goal. What I really want is a kernel that lets me play around with experimental APIs in user space and I'm happy to sacrifice performance and a bunch of other things to get there. I don't expect my first try at making a kernel to wind up anywhere beyond proof of concept quality, not even prototype, so even if successful at coming up with a new user space API I like I fully expect to throw the kernel away in the future anyway. Might as well embrace that.

At the moment I've got newlib and GCC 14.2's libstdc++ running in hosted mode in the kernel and I'm running GCC with C++23 as the language version. It definitely lets me move a lot faster than C. I'm traditionally a Perl programmer and lately more of a Python programmer and, unfortunately, I definitely can't move as fast with C++ as I can with those languages.

I am very very tempted to run an experiment and see how hard it would be to get Perl running in kernel space. Though while I'm still down at such a low level I don't yet see any benefit with Perl beyond std::shared_ptr, std::string, std::vector, std::list, etc. Maybe in the future.

My current task is getting ready to have the kernel be able to execute user mode processes. I've implemented the trampoline described over here in assembly and got it running with the linker placing it at the normal place. I've built a linker script to park the trampoline only at the top 1 megabyte of the address space as talked about here (the trampoline when stripped of debug info is less than 1KiB, which amuses me).

My current sub-task is finding the ELF section header for the trampoline so I can get the physical and virtual addresses for it and map them in the page table directory. It's been more difficult than I expected to find the section name strings but I expect to solve that soon, get the trampoline mapped, and start finding the new things that will be more difficult than I expected.

Once I got the PIT multiplexed I decided to run it at a silly frequency (5khz, I might turn it up more) to expose reentrancy issues early. Wow. Never in my life have I seen malloc() go reentrant. That gave me a good laugh.

I've already got spinlocks implemented and present in the interest of future concurrency. So going reentrant isn't catastrophic. It does a cause a deadlock but also leaves a nice easy to debug stack trace for GDB. I solve the reentrancy issue at the moment by disabling interrupts in sections of code that are common inside and outside of ISRs.

After I solve spreading my trampoline around in the address space I think I'll go back and make a class of spin lock specifically for parts of the kernel that are common inside and outside of ISRs and have that mutex assert on locking that interrupts are disabled.

This is certainly a fun hobby.
cardboardaardvark
Posts: 20
Joined: Mon Nov 18, 2024 10:06 pm

Re: Thank you for the wiki

Post by cardboardaardvark »

cardboardaardvark wrote: Wed Nov 27, 2024 2:23 pm a class of spin lock specifically for parts of the kernel that are common inside and outside of ISRs and have that mutex assert on locking that interrupts are disabled.
This worked out quite well. I couldn't use assert() because the logging system itself uses a mutex so what I did instead was go into a silent infinite loop. It causes the kernel to hang if it happens and leaves a nice backtrace to figure out what's going on with GDB.

To implement this I now have 2 spin lock types: a general purpose one and one for use when an ISR might be doing the locking. In my interrupt handler I set a flag using an atomic variable that is true the whole time the interrupt handling system is running. The general purpose mutex checks for this flag and hangs if it is true. The ISR mutex checks the interrupt enabled bit in the CPU and hangs if it is true. This trivial implementation won't survive SMP but it's good enough for now.

All the known instances showed up immediately when I started using this and it'll catch this accidentally happening in the future as changes are made with the benefit of not having to wait for the race condition to happen.
Post Reply