OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Aug 05, 2021 12:11 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 13 posts ] 
Author Message
 Post subject: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 8:52 am 
Offline
Member
Member

Joined: Sat Feb 20, 2021 3:11 pm
Posts: 94
After struggling to easily modify my OS, I decided to rewrite all of the build system(makefiles). I read through a couple of makefile tutorials and I have no idea how to write a normal makefile for an OS(I use cmake for my other projects and also they are easier to build) or any other large project, actually I couldn't even understand how to write a makefile for any project that consists of more then one directory. What I came up with is very broken - either I have a root makefile which does "make -C dir" to start other makefiles, but in this case the variables from root makefile are not passed to other makefiles and I need to copy then over and over and over again even though they are 100% the same; the second thing I thought of is to include all of the make files from other directories, but then all of them would have project root as their working directory and so I would need to add /dir/dir before every file and every directory in every makefile. So how should I write makefiles for multi-directory projects? Any resources on how to use makefiles(except the offical doc which is 100% useless unless you need to find a specific feature or understand how some specific feature works) for larger projects? How do you do it in your OS?

_________________
https://github.com/NGX123/tos


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 9:10 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1560
ngx wrote:
actually I couldn't even understand how to write a makefile for any project that consists of more then one directory.
The same way.
Code:
(output): (input files)
        (commands to be executed to create output from input files)
That simple. Input files can be in another directory as well, and output can be too.

If you have submodules, eg. the directory has it's own Makefile, then do:
Code:
dir/submodule.a:
        make -C dir all
This will enter the "dir" directory and call "make all" there. You should set the output to a file that whis submodule produces, so that if that file exists, the submodule's make will be skipped when you run "make all" in the project's directory.

Here's a simple example in USBImager. Here for example "bzip2" is in a subdirectory and has it's own Makefile. So I have
Code:
bzip2/libbz2.a:
   @make -C bzip2 libbz2.a
When I run "make all" in the project's directory, and ./bzip2/libbz2.a doesn't exists, this will run Makefile in the "bzip2" directory too.

As for the variables, you could include a Makefile.env from both your project's Makefile and from the submodule's Makefile. For example:
Makefile.env:
Code:
CFLAGS = -g

Makefile:
Code:
include Makefile.env

dir/out:
      make -C dir all

dir/Makefile:
Code:
include ../Makefile.env

...whatever


Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 9:42 am 
Offline
Member
Member

Joined: Sat Feb 20, 2021 3:11 pm
Posts: 94
bzt wrote:
Code:
(output): (input files)
        (commands to be executed to create output from input files)
That simple. Input files can be in another directory as well, and output can be too.

I know, I had that at the start, but that makes a giant monolothic makefile managing all of the dirs which is very large and not really extendable.

bzt wrote:
If you have submodules, eg. the directory has it's own Makefile, then do:
Code:
dir/submodule.a:
        make -C dir all
This will enter the "dir" directory and call "make all" there. You should set the output to a file that whis submodule produces, so that if that file exists, the submodule's make will be skipped when you run "make all" in the project's directory.

If it is normal calling of make file with make -C then it is my second idea, but the problem is - how do I pass variables to the submakefiles

bzt wrote:
As for the variables, you could include a Makefile.env from both your project's Makefile and from the submodule's Makefile. For example:
Makefile.env:
Code:
CFLAGS = -g

Makefile:
Code:
include Makefile.env

dir/out:
      make -C dir all

dir/Makefile:
Code:
include ../Makefile.env



I also have thought of this, but if I will have deeper subdirectories then it will look like ../../../../… and if i change something it would not be obvious how many ../.. to remove

_________________
https://github.com/NGX123/tos


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 2:15 pm 
Offline
Member
Member
User avatar

Joined: Sun Feb 18, 2007 7:28 pm
Posts: 1517
Hi,
Quote:
I also have thought of this, but if I will have deeper subdirectories then it will look like ../../../../… and if i change something it would not be obvious how many ../.. to remove
What we do is use a system environment variable for this. E.g. from a component makefile:
Code:
TARGETNAME  = nasm
TARGETTYPE  = PROGRAM
TARGETPATH  = $(SDK)/bin
INCLUDES    = $(STDINC)
SOURCES     = main.c\
   table.c\
...
!INCLUDE $(NBE_ENVDIR)/makefile.def
We don't use recursive make so won't comment on it.

_________________
OS Development Series | Project Neptune | Repository | Wiki
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 2:28 pm 
Offline
Member
Member

Joined: Sat Feb 20, 2021 3:11 pm
Posts: 94
neon wrote:
Hi,
Quote:
I also have thought of this, but if I will have deeper subdirectories then it will look like ../../../../… and if i change something it would not be obvious how many ../.. to remove
What we do is use a system environment variable for this. E.g. from a component makefile:
Code:
TARGETNAME  = nasm
TARGETTYPE  = PROGRAM
TARGETPATH  = $(SDK)/bin
INCLUDES    = $(STDINC)
SOURCES     = main.c\
   table.c\
...
!INCLUDE $(NBE_ENVDIR)/makefile.def
We don't use recursive make so won't comment on it.


Thanks, can you please exapand, I didn't really understand the solution, should we have a variable with a directory and append it before paths?

_________________
https://github.com/NGX123/tos


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 2:43 pm 
Offline
Member
Member
User avatar

Joined: Sun Feb 18, 2007 7:28 pm
Posts: 1517
Hi,

The basic idea is just to set it in an environment variable i.e. [Windows]. This would allow the path to be anywhere on the system and you can just use that variable in your makefiles to include and access environment makefiles i.e. !INCLUDE $(NBE_ENVDIR)/makefile.def

_________________
OS Development Series | Project Neptune | Repository | Wiki
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 2:46 pm 
Offline
Member
Member

Joined: Sat Feb 20, 2021 3:11 pm
Posts: 94
neon wrote:
Hi,

The basic idea is just to set it in an environment variable i.e. [Windows]. This would allow the path to be anywhere on the system and you can just use that variable in your makefiles to include and access environment makefiles i.e. !INCLUDE $(NBE_ENVDIR)/makefile.def


Oh, do you mean like set a variable in the shell environment (e.g. zsh/bash/cmd) and add it(e.g. $(env_var)/path) before every pathin the makefile?

_________________
https://github.com/NGX123/tos


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 2:54 pm 
Offline
Member
Member
User avatar

Joined: Sun Feb 18, 2007 7:28 pm
Posts: 1517
Hi,

If it makes sense, sure. I use a master makefile so only do it once per component makefile. I was just addressing a way to include a location without needing all those "../../" etc.

_________________
OS Development Series | Project Neptune | Repository | Wiki
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 3:07 pm 
Offline
Member
Member

Joined: Sat Feb 20, 2021 3:11 pm
Posts: 94
neon wrote:
Hi,

If it makes sense, sure. I use a master makefile so only do it once per component makefile. I was just addressing a way to include a location without needing all those "../../" etc.


Okay thanks, but you said that you are doing it only once per makefile, how?

_________________
https://github.com/NGX123/tos


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 8:38 pm 
Offline
Member
Member
User avatar

Joined: Sun Feb 18, 2007 7:28 pm
Posts: 1517
Hi,

We have a master set of makefiles ("environment") and per-project makefiles ("component"). What was posted above was an example of a "component" makefile. Each "component" makefile includes the master makefile which defines all the targets and inference rules. The following is a snippet from the master include file, $(NBE_ENVDIR)/makefile.def:
Code:
!if "$(TARGETTYPE)" == "LIBRARY"
$(TARGET) : $(OBJECTS)
   $(LIB) $(LIBRARIAN_FLAGS) $(OBJECTS)
!else
$(TARGET) : $(OBJECTS) $(TARGETLIBS)
   $(LINK) $(LINKER_FLAGS) $(OBJECTS) $(TARGETLIBS) -out:$(TARGET)
!endif

clean :
   del "$(O:/=\)\*.obj"
   del "$(TARGET:/=\)"
Per project component makefiles just need to include this file which defines all of the targets so we can call make on them.

The last line of all component makefiles include this file once: they set up what is needed and let the environment do the rest.

_________________
OS Development Series | Project Neptune | Repository | Wiki
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Sun Jun 13, 2021 10:04 pm 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 960
If you know CMake, why don't you just use CMake?

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Tue Jun 15, 2021 12:51 pm 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1560
ngx wrote:
I also have thought of this, but if I will have deeper subdirectories then it will look like ../../../../…
And what's the matter with that? You only have to write them once. Not using absolute paths has the great benefit that anybody can clone your repo anywhere on their disks and it will Just-Work (TM) without Anything-Crazy-Cmakeish-or-Sconsish-or-Automakeish-Ninjaish magic requiring many additional dependencies with incompatible versions in your toolchain. It just works as it should. Relative paths are good. Relative paths are useful.

ngx wrote:
and if i change something it would not be obvious how many ../.. to remove
Yes, it is pretty obvious, you should be knowing your own directory structure, so should be no probs there. And how often do you reorganize your entire project structure? Chances are good that once you've settled with the structure you'll never ever change it again, max. you'll add some new directories later when you implement a new driver for example, in which case - I bet - you'll just copy an existing Makefile and update the target in it. No need to touch the paths and the include.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Writing a makefile for an OS
PostPosted: Wed Jun 16, 2021 1:49 pm 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 216
ngx wrote:
After struggling to easily modify my OS, I decided to rewrite all of the build system(makefiles). I read through a couple of makefile tutorials and I have no idea how to write a normal makefile for an OS(I use cmake for my other projects and also they are easier to build) or any other large project, actually I couldn't even understand how to write a makefile for any project that consists of more then one directory. What I came up with is very broken - either I have a root makefile which does "make -C dir" to start other makefiles, but in this case the variables from root makefile are not passed to other makefiles and I need to copy then over and over and over again even though they are 100% the same; the second thing I thought of is to include all of the make files from other directories, but then all of them would have project root as their working directory and so I would need to add /dir/dir before every file and every directory in every makefile. So how should I write makefiles for multi-directory projects? Any resources on how to use makefiles(except the offical doc which is 100% useless unless you need to find a specific feature or understand how some specific feature works) for larger projects? How do you do it in your OS?


I wrestled with this for a while as well.

At my previous job, we used imake to generate Makefiles using the C pre-processor macros, and to be fair, it was quite useful, as our product was cross platform (Linux, Solaris, HP-UX, AIX, Windows), and we could abstract a lot of platform specifics behind CPP macros, but it was a horror to understand the subtleties, especially as you had Makefile subtleties obfuscated with the CPP macros. It also didn't help that we used the platform make, rather than a single cross platform make (such as gmake.)

I pondered cmake and even ant, but for my kernel project, I decided to use a single top level Makefile, and include per-subdirectory fragments. I did this after reading papers like "recursive make considered harmful"

I define a TOP directory so I can address files relative to the project root, without relying on '..', snapshot here

Basically, the structure is (Makefile):
Code:
all::

TOP=$(CURDIR)

ARCH=i386

OBJS=$(SRCS_S:.S=.o) $(SRCS_C:.c=.o)
SRCS_C :=
SRCS_S :=
SYS_H := \
include/sys/times.h \
include/sys/stat.h \
include/sys/time.h \
include/sys/types.h \
include/sys/errno.h \
include/sys/unistd.h

subdir := build
include $(subdir)/subdir.mk
subdir := libk
include $(subdir)/subdir.mk
subdir := kernel
include $(subdir)/subdir.mk
subdir := posix
include $(subdir)/subdir.mk
subdir := fs
include $(subdir)/subdir.mk
subdir := drivers
include $(subdir)/subdir.mk
subdir := build
include $(subdir)/tools.mk
subdir := arch/$(ARCH)
include $(subdir)/subdir.mk
subdir := user
include $(subdir)/subdir.mk
subdir := initrd
include $(subdir)/subdir.mk


The each of the subdirectories listed has a subdir.mk fragment that defines what's going on in that directory (here's kernel/subdir.mk in full):
Code:
SRCS_KERNEL_C := $(subdir)/main.c  $(subdir)/core.c  $(subdir)/pci.c  $(subdir)/printk.c  $(subdir)/panic.c $(subdir)/thread.c $(subdir)/sync.c $(subdir)/check.c $(subdir)/vm.c $(subdir)/vfs.c $(subdir)/dev.c $(subdir)/timer.c $(subdir)/input.c $(subdir)/intr.c $(subdir)/device.c
SRCS_C += $(SRCS_KERNEL_C)


subdir.mk can include other subdir.mk fragments (fs/subdir.mk):

Code:
fsdir:=$(subdir)

subdir:=$(fsdir)/tarfs
include $(subdir)/subdir.mk
subdir:=$(fsdir)/devfs
include $(subdir)/subdir.mk
subdir:=$(fsdir)/fatfs
include $(subdir)/subdir.mk
subdir:=$(fsdir)/procfs
include $(subdir)/subdir.mk


At the end of all the includes, SRCS_C and SRCS_S variables contain all the source files that need to be compiled/assembled, and the usual make rules kick in to do the compiling.

The result is only a single set of rules have to be included, the only relative paths are subdirectory paths (no '..' parent directory walking), there are no recursive make invocations complicating passing information, and make is blindingly fast and totally capable of parallel builds (-j option).

And I can easily re-arrange the directory structure, as a sub-directory subdir.mk will have the subdir variable pointing to the containing sub-directory, and TOP pointing to the root source directory.

I'm not sure how portable the above is, it may need tweaking on different make type, but I just use gnu make (why write a portable Makefile when you can use a portable make).


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot], Majestic-12 [Bot], rdos and 32 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group