OSDev.org

The Place to Start for Operating System Developers
It is currently Tue Oct 15, 2019 9:14 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: TTYs / PTYs
PostPosted: Mon Sep 16, 2019 1:39 pm 
Offline
Member
Member
User avatar

Joined: Wed Jan 19, 2005 12:00 am
Posts: 94
One area that I've kind been stalled on with my OS is how to properly implement TTYs. I think I get the gist of them, but not the details, and even my books on OS dev kinda just gloss over the topic without getting into code very much on the subject. I can imagine lots of ways that might work "good enough", but I'd like to actually feel like I "get it" before putting together some hacking thing.

Here's my understanding:

I have a keyboard driver, which for each keyboard even fills up a queue of events, reading from /dev/keyboard will get those events.

A TTY is in principle basically a Screen/Keyboard combination. Assuming it is available in the VFS as /dev/tty, the writing to it would put chars on the screen, reading from it would get characters from the keyboard, typically blocking if no keys have been typed. Simple enough.

First question:

How is /dev/tty's input typically connected to /dev/keyboard's? Is it a case of just having the TTY open /dev/keyboard when it is created and deferring the reads to it as needed? I feel that this is too simplistic and won't be robust when I want to implement pipes and redirection.

I have a similar question for PTYs and how their I/O is "wired" up to the underling TTY.

I am imagining a "tree" where each node is a process and each edge is a queue, but this seems like an awful lot of indirection between the user typing and the process actually seeing the event. Are there any straight forward resources I can look at the get a better understanding of this?

Second Question:

For implementing Job control (ex Ctrl+Z to background a job), who "sees" the Ctrl-Z? Assuming a process tree where "shell" is running a process "do_stuff", what is the chain of events?

Does "do_stuff"'s next read trigger, shell to do a read, who sees the Ctrl+Z and then handles it? Or is the Ctrl-Z "pushed" to the controlling process more directly so that this can happen even if it doesn't happen to be doing a read at the time?

I've read http://www.linusakesson.net/programming/tty/index.php, which is great for the high level concepts and history, but I feel my knowledge gap is somewhere int he details not mentioned there.

Any help and/or simple examples would be very appreciated.
Thanks!


Top
 Profile  
 
 Post subject: Re: TTYs / PTYs
PostPosted: Mon Sep 16, 2019 6:44 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 314
In the beginning, there were terminals. These were physical devices attached to serial lines. When they received characters from the computer, they displayed them on a screen. When a user typed on their keyboards, the terminal would send text to the computer. But we moved away from physical terminals to virtual terminals - terminal emulators.

Quote:
How is /dev/tty's input typically connected to /dev/keyboard's? Is it a case of just having the TTY open /dev/keyboard when it is created and deferring the reads to it as needed? I feel that this is too simplistic and won't be robust when I want to implement pipes and redirection.


There are, essentially, three sides to the terminal equation. Two of them are more obvious, and they have been commonly known as the master and slave end.

On one end is the application running in the terminal. Its output and input are the TTY and when it reads it receives keyboard input, and when it writes it prints to the terminal. This is the slave end of the TTY.

The opposite end is the terminal emulator (or the real terminal attached to a serial line). When it reads from the TTY it gets the characters the application wants to print, and what it writes to the TTY is the results of the keyboard key presses. This is the master end of the TTY.

But there is a middle to the TTY as well. This lives in the kernel. This is where line buffering, interrupts (^C, etc.), job control, etc. are managed.

The terminal emulator receives key presses from some API - probably a windowing API, rather than reading directly from the keyboard. It turns each key press into the appropriate sequence of bytes, so when you press Shift+A you get the value "A" and when you press PageUp you get the escape sequence ^[[5~. These are then written to the master side of the TTY and received by the kernel. The kernel needs to process these based on the TTY configuration, which is controlled by termios in the POSIX world, so that it can interpret things like ^C or ^Z and act accordingly. It may also need to do things like convert line endings, provide line buffering (if enabled), etc. After this process is done, eventually some bytes are going to end up in the queue of available data for the slave end to read - that's our application. Maybe it's a shell, so it reads those characters and executes a command. Now an application is going to output bytes to the slave end, so we go back to the kernel and handle output processing. This is not where control sequences are interpreted, but it is where more line ending conversion may happen. Those bytes end up in the queue of available data for the master to read, our terminal emulator reads them, interprets whatever control sequences it understands, and displays the output.

In other words:

Keyboard -> UI -> Terminal Emulator -> Text and escape sequences -> TTY master -> Kernel TTY layer (line conversion, control key handling, job control, line buffering, input echo) -> TTY slave -> Application

And:

Application output -> TTY slave -> Kernel TTY layer (line ending conversion) -> TTY master -> Terminal Emulator (escape code processing) -> Display

_________________
toaruos on github | toaruos.org | gitlab | twitter


Top
 Profile  
 
 Post subject: Re: TTYs / PTYs
PostPosted: Mon Sep 16, 2019 7:31 pm 
Offline
Member
Member

Joined: Mon Jul 25, 2016 6:54 pm
Posts: 183
Location: Adelaide, Australia
I'll try and answer these, I'm working on this in my OS at the moment so it'll be a good exercise for me.

First, the underlying piping in from /dev/keyboard and out to your graphics context is performed by the tty driver. This can be in the kernel or a usermode process. It creates the character input and output streams and abstracts away the difference between hardware configuration (keyboard/mouse, actual teletype, serial line or network connection). On Linux the tty driver opens /dev/ptmx (psudo-terminal master) which automatically creates a corrosponding /dev/pty# file. User processes read and write to /dev/pty# and the kernel routes the calls to the tty driver via it's open /dev/ptmx fd.

When a user connects a process spawns for them with file descriptors 0 1 and 2 set to read and write to the /dev/pty# file for the hardware device they are using to connect. This corrosponds to stdin, stdout and stderr so the <stdio.h> library is already available for the user process.

In Linux system keyboard commands are caught after the tty driver converts the hardware input into a character stream, but before characters reach the user mode process. They are caught by a part of the driver stack called the line disipline. This stage actually does a lot, including converting LF into CRLF and capturing system key commands. For Ctrl+C and Ctrl+Z, and sends corrosponding signals to the process in the terminal. Since the shell will be the process running in an interactive terminal, it receives the signals from the line disipline. It traps these signals and decides what to do with them, either forwarding them to processes it started or doing some job control. Job control is managed by the shell itself. See Klange's better answer.

I found this useful https://www.oreilly.com/library/view/li ... /ch18.html


Last edited by StudlyCaps on Mon Sep 16, 2019 9:45 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: TTYs / PTYs
PostPosted: Mon Sep 16, 2019 8:15 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 314
StudlyCaps wrote:
For Ctrl+C and Ctrl+Z, and sends corrosponding signals to the process in the terminal. Since the shell will be the process running in an interactive terminal, it receives the signals from the line disipline. It traps these signals and decides what to do with them, either forwarding them to processes it started or doing some job control. Job control is managed by the shell itself


This is inaccurate. The shell informs the kernel of a foreground process group, and that process group receives the signals sent from the TTY - including both the interrupt signal (^C) and the suspend signal (^Z) - not the shell.

When a process receives the suspend signal, it is suspended by the kernel - this is the default action for the signal and it's one of the few signals that have a special action that isn't just "die". When the process is suspended, a process that was waiting on it will have its wait call return with appropriate status bits set that say the process was suspended and still exists. This waiting process is generally going to be the shell, which will handle this situation by marking the "job" as suspended and providing another shell prompt (and setting itself as the foreground process group) so a new job can be run (or a "fg" or "bg" command can be used to resume the suspended job, with it set as the foreground process or not).

_________________
toaruos on github | toaruos.org | gitlab | twitter


Top
 Profile  
 
 Post subject: Re: TTYs / PTYs
PostPosted: Mon Sep 16, 2019 9:51 pm 
Offline
Member
Member
User avatar

Joined: Wed Jan 19, 2005 12:00 am
Posts: 94
Thanks guys!

I think that looking through all of the links into toaruos.org is going to be VERY helpful.

I think this is a case of something that will click as I just try to implement it based on what I know and seeing examples, so this definitely helps put me in the right direction!


Top
 Profile  
 
 Post subject: Re: TTYs / PTYs
PostPosted: Tue Sep 17, 2019 10:00 am 
Offline
Member
Member

Joined: Tue May 13, 2014 3:02 am
Posts: 268
Location: Private, UK
klange wrote:
In the beginning, there were terminals. These were physical devices attached to serial lines. When they received characters from the computer, they displayed them on a screen.


Well, in the beginning, there were teletype machines (that's where the name "TTY" comes from). They were basically electronic typewriters (to the point that they often shared major components), but rather than have the printer connected directly to a keyboard as an electronic typewriter, they'd have a communications port such that anything typed would be sent over the port and anything received would be printed. Teletype machines actually pre-date computers, being used for various forms of text-based communication for decades before someone realised a computer could interface with that communications port and provide a ready-made "interactive terminal".

Terminals with screens (or "Glass TTYs" as they were often called in their earliest days) were the second generation devices, first appearing in the mid-late 1960s, with paper-based units continuing to be common (because they were cheaper) well into the early UNIX era.

Anyway, the whole idea of a "TTY" or "PTY" these days is very much a legacy UNIX concept that I'd strongly suggest not copying verbatim in a new-build OS (if you're just writing a classical UNIX clone, you can stop reading here). IMHO, out-of-band signalling (i.e. have "control" calls entirely separate to "write to the display" calls) is far superior to the old concept of "escape sequences" (they were invented solely because there was no other way to send commands to a physical terminal connected via a single serial communications port) for controlling a text-based interface and it's full of legacy cruft (the "terminfo" system, used to attempt to make applications less dependent on the exact type of terminal in use contains literally thousands of definitions for slightly different types of terminals; I'd estimate that well over 99% of them haven't been seen outside of a museum/collection for 20+ years).

My OS uses a much more modern terminal concept; out-of-band control (no "native" escape sequences), graphics mode support and built-in pointing device support. My GUI runs entirely within a full-screen terminal, as could any graphical application. I even have a userspace library to provide emulation of an old-fashioned escape-sequence controlled terminal entirely within a process that wishes to output that way (i.e. nothing in kernelspace knows anything about such sequences).

As for how to connect a particular keyboard and pointing device to a particular display (remember that a PC can have multiples of all of those), currently my OS uses a simple technique; after the initial hardware detection the "boot manager" checks to make sure a keyboard, pointing device and video device have been detected and sets up the initial configuration before the terminal subsystem is loaded. This will likely need to be revisited at some point; especially once I have USB support and the possibility of multiple physical displays.

_________________
Image


Top
 Profile  
 
 Post subject: Re: TTYs / PTYs
PostPosted: Tue Sep 17, 2019 4:17 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 314
mallard wrote:
Well, in the beginning, there were teletype machines (that's where the name "TTY" comes from). They were basically electronic typewriters (to the point that they often shared major components), but rather than have the printer connected directly to a keyboard as an electronic typewriter, they'd have a communications port such that anything typed would be sent over the port and anything received would be printed. Teletype machines actually pre-date computers, being used for various forms of text-based communication for decades before someone realised a computer could interface with that communications port and provide a ready-made "interactive terminal".

Terminals with screens (or "Glass TTYs" as they were often called in their earliest days) were the second generation devices, first appearing in the mid-late 1960s, with paper-based units continuing to be common (because they were cheaper) well into the early UNIX era.

The first computer terminal had a keyboard and blinkenlights; teletypes found their way to computers after that and were also known as hard-copy terminals, which technically makes glass TTYs the third generation, but this history lesson is an unnecessary tangent.

Quote:
Anyway, the whole idea of a "TTY" or "PTY" these days is very much a legacy UNIX concept that I'd strongly suggest not copying verbatim in a new-build OS (if you're just writing a classical UNIX clone, you can stop reading here). IMHO, out-of-band signalling (i.e. have "control" calls entirely separate to "write to the display" calls) is far superior to the old concept of "escape sequences" (they were invented solely because there was no other way to send commands to a physical terminal connected via a single serial communications port) for controlling a text-based interface and it's full of legacy cruft (the "terminfo" system, used to attempt to make applications less dependent on the exact type of terminal in use contains literally thousands of definitions for slightly different types of terminals; I'd estimate that well over 99% of them haven't been seen outside of a museum/collection for 20+ years).

In-band signaling has a lot of benefits, such as being able to pass things with control sequences all the way through a Unix pipeline, and the ability to treat something like a socket as an equivalent bytestream to a terminal output. Personally, I would have liked to have more in-band signalling for TTYs, but we ended up with out-of-band control of bitrates, echo modes, etc. The biggest benefit, in my mind, is that no synchronization is needed if you have only one in-band output stream.

Terminfo and the former lack of standardization in control sequences was an unfortunate reality of having dozens of manufacturers building new products in a highly competitive landscape, but as someone who has built terminal applications in the modern era that employ raw escape sequences, I've found the problem doesn't really exist anymore. There's a standard set of DEC sequences popularized by Xterm that everyone is expected to implement now.

Quote:
My OS uses a much more modern terminal concept; out-of-band control (no "native" escape sequences), graphics mode support and built-in pointing device support. My GUI runs entirely within a full-screen terminal, as could any graphical application. I even have a userspace library to provide emulation of an old-fashioned escape-sequence controlled terminal entirely within a process that wishes to output that way (i.e. nothing in kernelspace knows anything about such sequences).

You're fighting a bogeyman here, as this is never something a kernel should concern itself with. (That a handful of OSes implement terminal emulators in their kernels is an amusing aside, but even in Linux there have been efforts to remove the kernel's "console" emulator)

_________________
toaruos on github | toaruos.org | gitlab | twitter


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot] and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group