OSDev.org

Abstraction. In case of a file handle, I do not expect userspace applications to know their way around my storage devices, file systems, and volume abstractions, and I do not allow them to access the hardware directly. Instead, I provide the abstraction known as a "file", which has numerous properties, and applications can write to them in a standard way, and they don't have to know about M.2, USB3, SATA, LVM, LUKS, MBR, GPT, ext2/3/4, btrfs, &c. pp.

But even aside from files, userspace applications want to access hardware in a standardized way, and handles have been shown to be the best way of handling those. At work, I have to work with a userspace device driver for a thing working over SPI. The kernel abstracts the details of the SPI bus interaction away, and allows the driver to access the device it is talking to largely without regard for the precise workings of the SPI host device.

Handles are useful for creating uniform interfaces to devices or functions. They become less useful when the OS tries to cram them into a single type (Posix file handles).

But, what is the usage of handles if user space applications can't even access the memory where the file is ?

Do you mean like a handle makes it easier to track the privileges of each process on an object (for example), instead of redoing the same thing on each structure and bloating the code.

For the application, the handle is an opaque cookie. Doesn't really matter if the file handle has value 4 or 0xabcd1234. It only hands the file handle to the OS in the appropriate places. So the application only saves it in a variable and gets it out whenever it needs something. Sometimes there are special values. Like on POSIX systems, file handles 0, 1, and 2 are reserved for standard input, standard output, and standard error output. But for the most part, the handle is opaque.

And yes, by having a standard handle type, you can avoid writing the same thing over and over again. In Linux, you can open all sorts of file descriptor, and you can close all of them with the close() system call. Same for CloseHandle() in Windows. And you can duplicate all of them using dup()/dup2()/DuplicateHandle(), and bestow them to child processes as the standard streams.

Let's look at POSIX FileID's. 0 is standard input, 1 is standard output, 2 is standard error. But that is only how it is at the start of your program. You can close FileIDs, you can reopen them (e.g. writing your stdout to a file), you can open further streams. Somehow the OS is keeping track of all this. Obviously at some place, there is a data structure with all the necessary information on a given stream in it. Since you can have many files, there's an array of those data structures somewhere. The FileID is an index into that array (to which you don't have any access, but which is helpful for you and the system to communicate with each other).

The C standard library's FILE * works a little differently, because what you get from fopen() is not an integer (being an index into an array), but the data structure holding the necessary information itself. My own C standard library holds in FILE the OS file handle (in case of POSIX, the FileID we had just been talking about), a pointer to the stream's buffer, counters for buffer size, buffer index, and buffer end (used for read buffers) as well as the position in the file, an array of ungetc()'ed characters, a counter of those characters, an integer of status flags, a mutex for accessing the file and structure, the file name, and a pointer to the next FILE structure as I administer them as a linked list.

But the nice thing is, you still don't have to bother with any of that. This structure is not part of the API. What you get is an opaque FILE * from fopen() which you then pass around to fread() and ftell() and fprintf() and whatnot, untill you fclose() the stream again. You never bother with what that FILE * actually points to. It's a handle.

It doesn't have to be files. It doesn't have to be C. If you work with any object-oriented programming (OOP), an object instance is similarly a handle. You retrieve an instance of class MyStorage, let's call it ms, and now you can ms->read() or ms->write() or ms->delete(). But you never actually bother with what is behind ms itself. At least, as long as you are a well-behaved OO programmer. Here, ms is a handle to an instance of MyStorage.

And even if you are not using an OOP language, as soon as you are using the pattern of "get a handle from one function, use the handle with several other functions, release the handle with one function", that is basically object-oriented programming. Regardless whether you are looking at FILE * from the standard library, Image * from ImageMagic, MyClass x in C++ (or Java, or C#, or ...), all of that is a handle. (Indeed it is very easy to "wrap" a OOP API for a non-OOP programming language. The constructor becomes the function creating the handle, the destructor becomes the function releasing the handle, and all the classes' member functions become functions using that handle. If you feel really ambitious, you can even recreate class inheritance using nothing but C structs as handles. AmigaOS Exec did that back in the 80's...

A handle is a piece of information that allows you to take the thing you identified in some earlier call, and then work on it in some way. It's just a (very) convenient programming pattern.

Except, real OO programming doesn't have a root class with read and write methods. For the obvious reason that some objects doesn't support read or write operations (for instance, an audio stream). Also, things like standard input and output doesn't have positioning, so having GetPos and SetPos as virtual methods doesn't make any sense for those. Keyboard input should deliver key press and key release events, not stream data. Standard output should have positioning on the screen and colors.

So, if you want to do it right, you define a handle type, define the methods it supports and then code those as syscalls that takes the handle and any additional parameters. Then you have done something that is really OO.

I made a simple object manager (handle manager) today. Can you please see if that's what you guys are talking about.
This is my own design, and it's a little bit inspired by windows (atleast the function namings).

Here is the directory :
https://github.com/NXTdevosc1/New-Opera ... src/nos/ob

Completely missed the point, as usual.

Guys can you please look at my repo and tell me if my object manager (or handles subsystem) is good. Because I often have to rewrite the whole API because I did not understand it well before.

My point was that Posix file handles are not object-oriented, and never will be. Therefore, that is a really bad example of how to do OO with handles.

And I said that your argument is absolutely missing the point. As usual.

A group of functions that's working on a common handle can trivially be implemented as a class (using the handle as 'this' pointer). A class can trivially be converted to a group of functions working on a common handle. What the class / group of functions is doing or not or whether the architecture makes a lot of sense to begin with is irrelevant. We're talking about "what is the purpose of handles".

Examples for handles are POSIX file id's, standard C FILE *, and / or OOP objects. I also mentioned Image * from ImageMagick.

And no-one claimed that POSIX file id's are OOP, other than you doing the old straw-man shtik. They are handles.

Yes, and I said that having a base class with virtual read and write methods, and no other classes, is not OO programming.

OO programming is when you create standard classes for distinct functions. For instance, you could implement an USB class for USB devices. This handle/class then can operate regardless if the particular USB device the handle refers to is connected to UHCI, OHCI, EHCI, XCHI or an USB hub. This is a reasonable use of handles. However, if you associate this handle with Posix file handles, then it is not reasonable. USB devices does not have generic read/write methods.

The "handles" implemented in C are not handles in my OS, and they cannot be used with any syscall. You can create a file handle (by doing a OpenFile syscall that returns a handle), and then use it with ReadFile and WriteFile, but you cannot use this handle with C standard handles. The OpenFile syscall only accept physical files in the FS, not devices or other Unix cludges. You cannot open stdin (keyboard) or stdout (console) with OpenFile.

Dude... at some point you really need to get it into your head that other people know their stuff as well. And you should stop making straw-man arguments. It's really annoying. At no point did I suggest making "a base class with virtual read and write methods". You pulled that one out of your backside, because you took a simple foo / bar example and read an architectural suggestion into it. And now you're talking about USB drivers in a thread on "what is a handle". Seriously...

Abstracting USB devices as classes and assigning them handles actually is a good example for how to use handles and illustrates the concept very well since there is no read/write interface involved.