OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 2:59 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 105 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7
Author Message
 Post subject: Re: unite to make an OS
PostPosted: Mon Feb 20, 2023 2:20 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
eekee wrote:
That ties together the two topics of this post, because how would a DOS shell redirect program output to memory? By temporarily replacing the software interrupt vectors for output.
That was how DOS did a lot of things. For example, keyb.com is a program that adds support for different keyboard layouts (the default BIOS keyboard handler can only deal with American keyboards). It does that by overriding interrupts 15h and 16h, and also a couple of others. Or subst.com basically just overloading interrupt 21h, rewriting DOS system calls.

This is also DOS's biggest weakness: Once a TSR is added to the system, it cannot be removed again in a standard way. On Linux, you can kill a hanging process (unless it is hanging in non-interruptible sleep, but that is more of a hardware issue), and you can even unload kernel modules, but on DOS, a TSR is here to stay.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Mon Feb 20, 2023 3:43 pm 
Offline
Member
Member
User avatar

Joined: Mon May 22, 2017 5:56 am
Posts: 812
Location: Hyperspace
nullplan wrote:
This is also DOS's biggest weakness: Once a TSR is added to the system, it cannot be removed again in a standard way. On Linux, you can kill a hanging process (unless it is hanging in non-interruptible sleep, but that is more of a hardware issue), and you can even unload kernel modules, but on DOS, a TSR is here to stay.

Good point. I (somehow) never had a hanging TSR, but had only good ones which had their own unload mechanism. Actually, that might not be true. It was so normal to reboot DOS computers that a TSR which didn't work well didn't make much of an impression. I would just try one and if it made the computer lock up, I would hit ctrl-alt-del and not try it again. As for the normality of rebooting, I've had to reboot FreeDOS many times because my text editor got funny. I had hopes of having fun with FreeDOS, even writing an OS with it, but a text editor which gets memory corruption if you move the mouse pointer while a key is down was one of the things which changed my mind.

_________________
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Tue Feb 21, 2023 1:54 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 3191
eekee wrote:
Ew! I don't like that definition of autism either. It's more applicable to Asperger's syndrome because people with Asperger's syndrome can learn appropriate social behaviour, but Asperger's syndrome is no longer counted as part of the autistic spectrum.


I think a more accurate statement is that nowadays people that used to get the Asperger's syndrome (AS) label instead get an autism diagnosis. Which is a good thing since there never was a difference between the AS label and the autism label, and both are on the autism spectrum sharing similar issues.

eekee wrote:
Some people with some varieties of higher-functioning autism can can learn appropriate social behaviour too, but certainly not all.


Maybe some don't want to be appropriate? :-)


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Tue Feb 21, 2023 7:06 am 
Offline
Member
Member

Joined: Sat Nov 21, 2009 5:11 pm
Posts: 852
Quote:
That ties together the two topics of this post, because how would a DOS shell redirect program output to memory? By temporarily replacing the software interrupt vectors for output.

The proper way to do this in DOS would be to use a temporary file residing on a RAM drive, instead of attempting to override a bunch of INT 21H functions.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Thu Feb 23, 2023 1:26 pm 
Offline
Member
Member

Joined: Sat Jan 28, 2023 11:41 am
Posts: 31
Location: Belarus
kerravon wrote:
Alexey1994 wrote:
But if you dynamically calculate the relative address through the EIP register, then it will be possible to launch several processes, and one process will switch another through this same piping, that's when it becomes possible.

I don't understand this. Can you elaborate?


This is where I added the first implementation of what I had in mind https://github.com/Alexey1994/BelOS/com ... 836b286a31

This is not a fast implementation that switches the context of the process when reading or writing a single character. It also doesn't give a very good idea of the process, but this is only the first working implementation.

eekee wrote:
Code:
cmd1 | cmd2 | cmd3

becomes:
Code:
cmd1 >%TEMP%\cmd###1.tmp
cmd2 <%TEMP%\cmd###1.tmp >%TEMP%\cmd###2.tmp
cmd3 <%TEMP%\cmd###2.tmp

where % delimits environment variable names.


At first I wanted to do this, but I was motivated to do it right and now I can adapt programs to work with large amounts of data


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Thu Feb 23, 2023 4:30 pm 
Offline
Member
Member

Joined: Fri Nov 17, 2006 5:26 am
Posts: 232
Alexey1994 wrote:
kerravon wrote:
Alexey1994 wrote:
But if you dynamically calculate the relative address through the EIP register, then it will be possible to launch several processes, and one process will switch another through this same piping, that's when it becomes possible.

I don't understand this. Can you elaborate?


This is where I added the first implementation of what I had in mind https://github.com/Alexey1994/BelOS/com ... 836b286a31

This is not a fast implementation that switches the context of the process when reading or writing a single character. It also doesn't give a very good idea of the process, but this is only the first working implementation.

Thanks. I understand now, but note that I didn't see any mention of "EIP". Also note that "previous" doesn't have an "e" on the end. :-)

Quote:
eekee wrote:
Code:
cmd1 | cmd2 | cmd3

becomes:
Code:
cmd1 >%TEMP%\cmd###1.tmp
cmd2 <%TEMP%\cmd###1.tmp >%TEMP%\cmd###2.tmp
cmd3 <%TEMP%\cmd###2.tmp

where % delimits environment variable names.


At first I wanted to do this, but I was motivated to do it right and now I can adapt programs to work with large amounts of data

Would it be correct to say that your way is right (or perhaps ideal/good) for a single-tasking system? Actually, you've defacto created a scheduler, haven't you? Wouldn't it be better to formalize that rather than putting it in the piping process?


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sat Feb 25, 2023 4:47 am 
Offline
Member
Member

Joined: Sat Jan 28, 2023 11:41 am
Posts: 31
Location: Belarus
kerravon wrote:
Thanks. I understand now, but note that I didn't see any mention of "EIP".

Here https://github.com/Alexey1994/BelOS/blob/main/binary/BIOS32/components/programs/load/main.c#L16
I calculate the offset relative to the EIP and further use it to calculate the absolute address of the data.

kerravon wrote:
Actually, you've defacto created a scheduler, haven't you?

No, I don't have a schedule, just a context switch that launch only when needed (some event like alt+tab or lack of data in the pipe)


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sun Feb 26, 2023 1:18 am 
Offline
Member
Member

Joined: Fri Nov 17, 2006 5:26 am
Posts: 232
Alexey1994 wrote:
No, I don't have a schedule, just a context switch that launch only when needed (some event like alt+tab or lack of data in the pipe)

Ok, so scheduling and context-switching are apparently different concepts.

But regardless, it probably doesn't belong in the piping mechanism.

I was thinking that when you have:

processA | processB | processC

that an efficient way of doing things would be for processA to do it's normal fwrite() which turns into a write() of BUFSIZ, although it may not detect that it is buffered, so it would be line buffered because it is stdout, and thus not be a full BUFSIZ. But we can start to address the situation where the 1-byte nature that you have implemented (as proof of concept I think) can start to be enhanced so that it would at least be line buffered. So it is at the point that write() or similar is called that you have multiple bytes, and the OS needs to determine what to do with them.

At this point, the OS has control, it hasn't wasted any effort doing buffering, and knows that this is a pipe scenario, and is in a position to do a context-switch to processB.

ProcessB would be initiated at this stage, and when it comes to do a read() of BUFSIZ, the data that was presented by ProcessA can be directly copied to satisfy its requirement.

The same "complication" of line buffering vs fully buffered would exist, plus the fact that the BUFSIZ may be different for the different processes (different compiler or compile options or even language).

And then presumably multi-processing/tasking/contexting/threading/whatever buzzword people keep using would sit on top of that. And a scheduler goes in somewhere.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sun Feb 26, 2023 9:30 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 3191
My understanding of piping is that standard input is redirected to standard output of the previous stage. How the intermediate stages are implemented is up to the OS. In my design they are processes, but that is not necessary.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sun Feb 26, 2023 12:51 pm 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
That’s not strictly true. Pipes (in Unix at least) just provide a means of communication between processes via file descriptors. They are commonly used to replace the stdout descriptor of one process and the stdin of another, but that’s not the only way to use the connection. (For example, you might want to redirect stderr rather than stdout, or you might just want a connection totally separate from stdout and stdin.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sun Feb 26, 2023 1:48 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 3191
iansjack wrote:
That’s not strictly true. Pipes (in Unix at least) just provide a means of communication between processes via file descriptors. They are commonly used to replace the stdout descriptor of one process and the stdin of another, but that’s not the only way to use the connection. (For example, you might want to redirect stderr rather than stdout, or you might just want a connection totally separate from stdout and stdin.


Yes,that is my understanding too. I don't think pipes through file descriptors is an effective means of process communication. I have my own IPC mechanism that is far more efficient.

The main issue with file based communication is that files are stream-oriented, while communication typically is message oriented. There is no way to signal message boundaries with files, so various tricks are used. Particularly sockets are missing the "push" call that is a message mechanism, and typical socket implementations rely on timeouts instead of signalling that data should be sent.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sun Feb 26, 2023 3:27 pm 
Offline
Member
Member

Joined: Sat Jan 28, 2023 11:41 am
Posts: 31
Location: Belarus
iansjack wrote:
That’s not strictly true. Pipes (in Unix at least) just provide a means of communication between processes via file descriptors. They are commonly used to replace the stdout descriptor of one process and the stdin of another, but that’s not the only way to use the connection. (For example, you might want to redirect stderr rather than stdout, or you might just want a connection totally separate from stdout and stdin.

If you take the UNIX approach that almost everything is a file, then yes, stdin stdout is the only available means of communication.

But for me, a file is something that has a certain size and certain data as long as it does not change, so I do not think that it can be considered a file, for example, /dev/random, it is better to have a program that writes to some standard output, a pipe, arbitrary data.

I also believe that such a concept at the OS level as redirection of streams is not needed, since it is well done with the help of one program.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Sun Feb 26, 2023 8:12 pm 
Offline
Member
Member

Joined: Fri Nov 17, 2006 5:26 am
Posts: 232
rdos wrote:
There is no way to signal message boundaries with files, so various tricks are used. Particularly sockets are missing the "push" call that is a message mechanism, and typical socket implementations rely on timeouts instead of signalling that data should be sent.

Can fflush() be used? Which perhaps informs the OS via an fseek of 0 bytes from SEEK_CUR. Or alternatively, if you detect that you have switched from writing to a socket to reading from the socket, then that causes the "push"?

Not sure if it is relevant, but under PDOS I can do fopen("com1", "r+b"). And I do an fseek of SEEK_CUR 0 between reading and writing. C90 requires an intervening seek, of any kind.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Mon Feb 27, 2023 2:37 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 3191
kerravon wrote:
rdos wrote:
There is no way to signal message boundaries with files, so various tricks are used. Particularly sockets are missing the "push" call that is a message mechanism, and typical socket implementations rely on timeouts instead of signalling that data should be sent.

Can fflush() be used? Which perhaps informs the OS via an fseek of 0 bytes from SEEK_CUR. Or alternatively, if you detect that you have switched from writing to a socket to reading from the socket, then that causes the "push"?

Not sure if it is relevant, but under PDOS I can do fopen("com1", "r+b"). And I do an fseek of SEEK_CUR 0 between reading and writing. C90 requires an intervening seek, of any kind.


I wrote my own socket API that includes the push function and that does not have an implicit timer.

I don't particurly like the idea that "everything is a file", so I created my own APIs and then (when useful) link them to file handles to achieve a bit of Posix compliance.


Top
 Profile  
 
 Post subject: Re: unite to make an OS
PostPosted: Fri Mar 17, 2023 8:43 pm 
Offline
Member
Member
User avatar

Joined: Mon May 22, 2017 5:56 am
Posts: 812
Location: Hyperspace
rdos wrote:
eekee wrote:
Some people with some varieties of higher-functioning autism can can learn appropriate social behaviour too, but certainly not all.

Maybe some don't want to be appropriate? :-)

:P :mrgreen:

rdos wrote:
The main issue with file based communication is that files are stream-oriented, while communication typically is message oriented. There is no way to signal message boundaries with files, so various tricks are used. Particularly sockets are missing the "push" call that is a message mechanism, and typical socket implementations rely on timeouts instead of signalling that data should be sent.

In Plan 9, communication over pipes and virtual files preserves message boundaries; each write() makes a message. It's a great feature! It has a limit: Writes larger than 8KB will be split into multiple messages. (The OS uses 8KB buffers in many places. )

_________________
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 105 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 24 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group