OSDev.org

Hello, I'm not sure if this is the appropriate place to ask this question since it's not purely technical, but it's still about os development.

I've decided to make a small OS, probably something very similar to Linux which ideally will be POSIX/LSB compliant. My main goal is to learn about how a system works so that I can tweak it, control it, and even better, upgrade it. If I manage that, I'd like to try to make an hardened system or a very customized one. I'm also interested in "tips" to make a OS based on security, which would be harder to attack.

Now, I realize this is going to take me a lot of time. I am familiar with assembly, can code in C, C++ and other language and have some background on this all.

What are your advices to stay motivated and productive ? Avoid distractions, get coding done...

I've been thinking of making the OS open source, even if at first it's some mix between copy pasta and ugly code, so that I can make release often and so people can see it and comment. It can be a motivator, getting feedbacks and all.

Staying productive... I'm not sure if I'll manage to stay hours and hours on a problem (like program for 4 hours non-stop), to sort it out correctly. Go where it's difficult, not where it's easy and spoonfed. This require discipline, how can I get more of it ? What are your advices on all those issues ?

As to the core of your question, unfortunately you're asking a question that all of the self help books in the world haven't been able to solve in the history of humanity.

If you find some magic steps, then let me know. There are certainly things that help, like time management, organization, and scheduling techniques that can help. But as far as I can tell, as with anything else it's one of those things you can only get good at with lots of time and as part of maturing (a trait which I myself have much to improve upon, as well), and, unfortunately, some people are just naturally more suited to it than others to begin with (but don't let that keep you from focusing on improving).

If you want quick tips, the one I hear most often and that seems most realistic and core to me is one that is generally quoted by aspiring writers: Write for some amount of time every single day, even if the only thing you can do is copy text from a book you love (or in this case, copy down some code from a piece of OSS software you really admire, I suppose).

The idea is that this keeps you in a rhythm that turns what can be a chore into a habit, and even if you don't have it in you to be creative/productive on a given day -- which happens all the time to even the most brilliant, dedicated people -- by copying down something you admire it keeps your brain used to operating in that space and you'll also end up learning something intrinsic by reconstructing someone else's work that you can't really learn and internalize in any other way.

Honestly, not that I'm great with this, but I feel like trying to do that and also putting together a to-do for your project (which will turn your big project into smaller, less daunting packets of work that you can pop off your list in your work time each day) are all the tools you need to get yourself 99% of the way there.

And keep in mind that you will probably miss a day, or miss a week, or generally fall off of the wagon. And when you do, it's completely fine and completely human and the most important thing is that you don't let that discourage you from getting back on. The more times you fall off, the faster you get at getting back up and on with business. Trust me.

You can search this and other forums, wikis, books, source code, tutorials... in search of tricks that you can really understand, searching for complete information. Review post by post, topic by topic, manually. If a message doesn't have anything useful for you, just go to the next. The same with every Google result. Anything interesting and immediately useful by you, just use it and save the rest as much as you want. Even an artificial intelligence, which in reality is mindless and doesn't know what it's doing, can progress in this way. For example, the first post of this forum is http://f.osdev.org/viewtopic.php?p=9

When a message has complete and accurate information, and is actually directed to teach you something without delay, you will feel that it immediately captures your attention as you get to understand what it's saying in a way that you can reuse immediately, just like excellent reference library content. If not, you will just feel that you are just browsing in an errant way, without feeling tired, with attention and interest in browsing (looking for the good messages), but without really getting much done.

Extremely concrete, narrow and simple goals every day.
The computer is really intended to reduce every problem and logical concept or formula down to a level where it can no longer be simplified, to a level where no other unrelated simultaneous logics of other tasks for a given application are present, to a level where each stand-alone bit of logic is cleanly distinguishable, to a level where each bit of code you write is actually an additional CPU instruction, reusable in the same stable and assembly-like way, for maximum logical portability, with no real need to use fancier language elements on its own - that's another unrelated layer. The CPU shows you that routines written in the style of the actual CPU instructions are the ones that last longer than any other, for decades to come and go and no computer without them, so why not implement code with that style?

No computer feedback for human debugging at all times.
The first thing that will potentially demotivate you is not having any sort of human input or feedback to know how you are doing. Other people might know answers but won't solve them for you at all times. At least you can still write human-readable debug messages for each step of your program, and functions to input data and to ask you whether to continue or not depending on whether your program is producing the right values or not. In that sense, it's the machine itself that can tell you in a written form readable for any human, if it's getting the results you are expecting, and at a logical level, is like asking an actual somebody else who knows best how to implement anything and directed by you.


Share the isolated tricks you learn and your doubts with the intention to clear and solve them immediately one more step.
After all of that you'll want to share what you've done with others so that things become more entertaining based on what you have been doing. The most fun for an open environment like this is based on not hiding your code or documented knowledge, but instead showing it as soon as you achieve something, so that others can make use of it, put it to the test and add to it for everyone's benefit, to make your worldwide development environment better with your contributions and shape the set of its good resources with your intellectual structure. Everyone is currently learning more in this way, so why not keep talking in terms of actual code, in terms of concrete routines for something specific, and do that every day to learn how to implement useful things every day? It's guaranteed to yield more answers than otherwise.

Every time you want to do something, write it down in terms of an extremely simple and extremely concrete, immediately usable goal that you can understand, that does as much as a CPU instruction would. In this way you won't get stuck if you complete the "daily tasks", or if you don't know how and become frustrated. There's no such "daily tasks" to be optimum, it's better to have extremely simple tasks every time you want to do something, not the whole thing at once. It can even give enough activity as to naturally start doing more things than you expected.

Dear orion40,

I am sincerely sorry that you've had tilde respond to your thread.
I myself have entirely avoided reading any of the above post in order to save my eyes and my soul.
It is recommended that you do the same for the sake of your continued sanity.

Best wishes, hope your brain doesn't melt,
jojo

Hi,



In my experience, it's not possible to stay motivated/productive for the entire (10+ year) period it takes to write a usable OS.

It makes more sense to plan for breaks. For example; stay motivated/productive for 11 months (then spend a month doing something else), then stay motivated/productive for another 6 months (then spend 2 weeks doing something else), then ...

Planning for breaks means doing things like making sure your code is maintainable and has adequate documentation; to minimise the amount of time you lose trying to reacquaint yourself with parts of the OS you will forget. Initially this might seem "less important" (because the OS project is small and there's less to forget), but as the OS grows it gets much more important (e.g. you might spend 12 months working on a GUI, then take a break when GUI "works", then after the break you'll come back and try to do something on kernel code that you haven't really looked at for 2 years).


Cheers,

Brendan

Thanks everybody for your replies. I think I'll break big stuff into smaller stuffs to don't be discouraged too easily, and try to participate here as much as possible. Taking a break as well sometime I guess.



What was your point with the first post of the forum ?



Well, okay. Thanks for the laugh.



10 year ? How long would it take for someone who know how to use a Linux system, understand basic assembly, C, to build something usable ? I've seen other guys here having they own OS with full GUI (not something that interest me that much though) and I doubt they've been at it for ten years.

That depends on your goals, what you define as usable, your experience and your approach.

Most hobby OS that have a nice GUI and run nice user space programs are relative straightforward UNIX clones. Porting programs to them is easy. Note that they are still not usable in the common definition of that word: Many hobby OS will simply break on many machines as they lack the drivers and infrastructure to support their hardware.

If you want to try out novel ideas at the kernel or system level it might even take a few years to get something as simple as bash running. I spend a year until I could run bash and another 2 years rewriting core components until I found an API design that I'm satisfied with. Now I can run much more than just bash and my OS can boot natively from USB. Plus I'm somewhat confident that I have the fastest IPC mechanism of any OS . It will still take me at least another year until I get a beautiful GUI (e.g. bash in xterm) running.

Hi,



First; let's split "writing an OS" into several categories:

• Writing the boot code (boot loader, any other pieces after boot loader but before kernel is started, and any pieces after kernel is started - e.g. including things like "init scripts" if you have any)
• Writing the "core kernel functionality" (physical and virtual memory management, scheduler, IPC, etc)
• Writing the "system code" (device enumeration, device drivers, file systems, TCP/IP stack, etc). Note that these may or may not be in the kernel ("monolithic vs. micro-kernel vs. exo-kernel")
• Writing the user-space stuff that's part of the OS (this includes things like the GUI and/or terminal emulator, fdisk utility, file explorer, help system, something for updates and dependencies, etc)
• Writing some kind of OS installer (an OS is useless if you can't install it). This is typically some kind of image you download and boot, that creates partitions, copies files, installs boot code, asks you what time zone you're in, etc.
• Writing third-party developer documentation (e.g. so that developers are able to write things like applications and games for your OS). This can include documentation for third-party device driver developers (and probably should because you're not going to be able to do all device drivers yourself).
• Writing end user documentation; including some kind of user manual containing instructions on how to install and setup the OS, and including content for the built-in help system (e.g. animated "how to" tutorials for showing how to use various wizards/dialog boxes/tools, the "help" pages for each utility, etc).

Now consider the word "usable". Let's define "usable" as "good enough for normal people (who aren't developers) to use for normal stuff". This means that all the pieces mentioned about are good enough for actual use (including functionality, stability, security, error reporting, etc).



Well, no, you probably haven't. You've probably seen people create a fraction of an OS; often without enough drivers and lacking other features, and often not writing major pieces of an OS at all (e.g. boot code, OS installer, etc); and even if you ignore all that still not (yet) reaching the state of "usable".

For a relatively extreme example; so far I've spent about 20 years (very sporadically, not continuous full time) and I'm still figuring out parts of the OS design. I've reached the stage where I'm not even sure if the current implementation is a prototype or part of the actual OS (which is a sign of progress - when I was "less experienced" I used to be sure my current implementation was part of my actual OS - hindsight is painful sometimes).


Cheers,

Brendan

Let's see. I worked on PDCLib, which is "just" a standard C library implementation -- no kernel-space coding, no device drivers, nothing fancy but just implementing what ISO/IEC 9899:1999 specified very cleanly, without any fancy optimization, and with the help of Plaugher's book "The Standard C Library" telling me in great detail how it could be done.

But as with any hobby project, it was an on-and-off affair, took second place to real life happening and interests shifting, then came back to being my primary pastime.

From November 2003 to February 2006, it took me well over two years.

Just for the freestanding headers and the "easy" parts of <string.h> and <stdlib.h>.

Getting <stdio.h> even halfway right took another four years.

Of course, had this been a nine-to-five assignment, I'd have been much faster. But it wasn't, and so I wasn't. If you want this to be fun, and not something you later consider to have been a waste of life-years, expect this to be rather typical. You squeeze it in between your day job and other activities. Means you'll pick up a particular ticket multiple times before you get anywhere, because the train arrives, your girlfriend shows up for a visit, or your buddy invites you over for a BBQ.

You'll drop the ball altogether for months at a time, and that is perfectly OK. It's a hobby. It shouldn't take over your life.

I learned lots from working on this project, and the feedback I got from people actually using my code was exhilarating. I'm happy I'd done it. I finished some other project since, which ate five years of labor, which also taught me lots. I'm sad I never finished PDCLib. I might, some day, or I might not, as my life has gotten interesting in several other ways.

It was a good time. If you can say the same later on, you'd been productive enough.

For me, OS development and related things like low level HTML5/JavaScript, graphics and archiving file formats, emulators, compilers..., typing recorders, have given me the best times since 2004 to date. I haven't stopped to center in OS level development but I have switched to many personal projects that are now all helping me in my system now that I finally have an expandable shell.

Probably you should ensure to have a kernel with a shell to which you can easily add new commands and from which you can run external programs from your OS. From there you are guaranteed to advance also with the help of what you learn in any other projects.

I'm currently thinking about considering the standard PC with x86 portable (to 16, 32 and 64-bit) manually written assembly code a platform to port to, and I want to port existing programs in C, C++, Delphi, etc., directly to OS-less versions of existing programs (Firefox, VLC, Audacity, SQLite3, C library and headers made in pure assembly, GCC, NASM, ReactOS, Linux/BSD/UNIX and the like.....), so as to have truly optimized assembly code native for the PC, in addition to the original generically portable code. In the end what we would get would be the raw and usable knowledge, easy enough and clean as to move that much assembly code intellectually by hand. We would get to understand how everything is implemented knowing how file formats, executables, etc., work, by not relying on linkers but doing those structures ourselves. We could then later automate more but based on a technology made by hand. I think that it has to be done to make the knowledge, understanding and intellectual technology of the developers on top of the advances and not just the unexplained code and binaries reach more normal levels as ours so that we can also get to understand what is known in the top notch edge.

I only wished that there could be more people who answered immediately based on code implementations as much as I myself do. That's currently the rarest thing to find and what people who look to stay productive without getting bored is really looking for.

The IRC chat at FreeNode is very good, with people often mentioning very good tricks, but I feel that technical talks could be much faster and richer than in the IRC channel.

______________________________
______________________________
______________________________
I think I'm a PC person. It looks like what PC people really want is to get to build software and hardware by ourselves. The PC is full of interesting and valuable open source snippets and tricks so it has always kept us motivated. Just like with any other platforms and the people who are expert at them, PC people will want it to be as native and optimized as the first PCs showed it had intentions to become, and more. See the enthusiasm at this, the PC really starts the most attempts, motivation and inspiration at making open source OSes and programs. Even Linux started in the most standard PC platform.

Damn, well I never thought I'd be done in a month, but for it to take several years ?

Let's say I'm making an UNIX clone, or at least something where the standards are well defined.

Would it still take more than several years ? Assuming I "skip" some stuff, like the bootloader (GRUB would do the job) ?

In the end my goal is to learn how it works, so I can tweak most existing systems, and understand them well. Is OS dev still a good idea, or is there something better ? Not necessarily easier, but at least more efficient ?

Hi,



If you skip the boot loader (and just use GRUB) and skip the kernel (and just use Linux) and skip all of the user-space (and just use a plain "open source and GNU" user-space), then (assuming you're working alone) it'll probably still take 10 years to build a repository and test all the packages properly.



If your goal is just to learn how existing stuff works and "tweak"; then it might be more efficient to read a book and join an existing "army of beige" unix distro (Ubuntu, RedHat, FreeBSD, illumos, whatever - it's mostly all just the same set of recycled left overs with superficial differences).


Cheers,

Brendan

I joined this forum in 2015 as a complete noob that didn't understand how to compile a cross compiler. Now 2 years later my knowledge has grown exponentially, all thanks to my inspirational drive. If you really want to learn something then you are going to learn just that. Just try hard enough, not because you are forced to, but because you want to. Currently I am working on the 4th revision of my OS, which means 4 revisions in 2 years, each one getting better and better. You will find yourself in a situation where you just want to toss everything again and start over. Do it! Working on an unstable core for years is the worst thing that can happen. If you need to start over, do so, but this time work harder and try not to copy paste as much (if done before). GRUB is just fine, it has everything you need, mostly (not talking about x64). You main goal is to learn and expand your knowledge. When you lack inspiration you have a 200+ pages topic "What does your OS look like?" just take a look and get back to work. OS deving is a good idea if you have enough time and will to learn. Also Intel manuals will help you a lot, sometimes. Don't resist manuals and reading the documentation, that is why it exists, to be read and applied. If you are stuck, come here and ask other people. Just don't post 10 different topics in a day. Nobody is going to write an OS for you, that is a cross you are accepting to carry along the way. Also another friendly suggestion, never ever use James Molloy's paging code. You will understand this sentence in a year, once you figure it out yourself you will see how bad it is. I am running out of time, just imagine a picture of you sitting in front of your monitor browsing on your own operating system. Then install your favorite IDE all the tools and get to work. When you get bored, just relax and rest, don't push yourself too hard.

Will probably take you a year of pretty much constant work to get the baiscs down, if that's what you're interested in. Anything past that piles up fast.