TTY/Poll synchronization issues: BusyBox vi unresponsive in custom x86 kernel

[cksfafwefasdf]

Hello everyone,

I am developing a custom x86 hobbyist kernel called MagicBox. I've reached a point where I can run BusyBox (ash & vi) via musl-libc, but I am encountering a specific synchronization issue with vi.

The Symptom

• In ash: Interactive commands work as expected.

• In vi:
  • The editor starts and renders the file content or the empty buffer correctly.
  • Commands like :q work fine if entered immediately after startup.
  • The Issue: Any other input (e.g., pressing a, i, or arrow keys) causes vi to stop responding. sys_read() captures the characters, but vi never issues a subsequent sys_write() or sys_writev() to refresh the terminal.

Current Implementation Details

1. The "Stub" Poll

Currently, my poll() implementation is a synchronous "stub" to satisfy BusyBox requirements:

Code: Select all

// Simplified logic
static int32_t do_poll(struct pollfd* fds, uint32_t nfds) {
    for (uint32_t i = 0; i < nfds; i++) {
        if (fds[i].fd <= 2) { // TTY FDs
            fds[i].revents = fds[i].events; // Faking "always ready"
        }
    }
    return (int32_t)nfds; // Returns immediately without blocking
}

I intended to delegate the blocking logic to the subsequent read() call.

2. Semaphore-based TTY Read

My TTY driver uses a semaphore (line_sem) for synchronization:

• Input Handler: In non-canonical mode, every received byte calls sema_signal(&line_sem).

• TTY Read: It performs a sema_wait(&line_sem) and then enters a while loop to try and fill the user-requested count from the ring buffer.

3. Ioctl Support

I have implemented TCGETS, TCSETS, TIOCGPGRP, and TIOCSPGRP. vi seems to successfully switch the terminal to non-canonical mode.

My Hypotheses

I suspect the issue lies in the interaction between my "fake" poll() and the blocking read():

• Desync: poll() claims data is ready. vi enters read(). If read() tries to fill a large count but the buffer only has 1 byte, my current loop might be causing an unintended block.

• POSIX Semantics: My implementation lacks strict VMIN/VTIME logic. I suspect vi's state machine relies on poll() timing out to trigger UI refreshes, which my current poll() doesn't support.

Request for Direction

I would appreciate any advice on:

• Is a proper Wait Queue implementation for poll mandatory for editors like vi to function?

• Does the semaphore-based "one signal per byte" approach fundamentally break stream-oriented programs?

• POSIX Compliance in TTY: Since my current TTY implementation is quite "lazy," should I prioritize a more strict adherence to POSIX TTY semantics (e.g., proper termios line discipline) to support complex editors? Specifically, I suspect my lack of VMIN/VTIME handling is a major blocker.

GitHub Repository: https://github.com/cksfafwefasdf/MagicBox

• Poll Logic: https://github.com/cksfafwefasdf/MagicB ... cpt.c#L585

• TTY Driver: https://github.com/cksfafwefasdf/MagicB ... r/tty_bk.c

A Note on Comments: I apologize that there are many non-English comments in my source code, as English is not my primary language. I have tried to ensure that function names and the overall logic remain self-explanatory in English.

Thanks in advance for any insights!

[nullplan]

I suspect your poll() implementation is an issue. The VMIN/VTIME thing not so much, since busybox happens to use those in the mode you support (it sets VTIME to 0 and VMIN to 1, requesting to block until data is available, indefinitely). It is indeed entirely possible that the issue is that poll() claims that data is available all the time, but then read() blocks. So vi thinks that new commands are available from STDIN all the time and never updates the screen since it only needs to quickly grab the command available there, but there is actually no data there.

Is a proper Wait Queue implementation for poll mandatory for editors like vi to function?

That is entirely possible.

Does the semaphore-based "one signal per byte" approach fundamentally break stream-oriented programs?

No. That is indeed what I'd expect from a tty connected to the user. There is typically never more than 1 byte available (unless a special key was pressed, and even then the number of bytes can be very small if a serial line is in the middle).

POSIX Compliance in TTY: Since my current TTY implementation is quite "lazy," should I prioritize a more strict adherence to POSIX TTY semantics (e.g., proper termios line discipline) to support complex editors? Specifically, I suspect my lack of VMIN/VTIME handling is a major blocker.

Why "suspect" anything? The busybox source code is available, and you can just read it or debug it to figure out what is happening. From my glance at the source code, it seems to use poll() for the timing quite a lot, so adding actual blocking and timing support would be number 1 on my list.

[Octocontrabass]

vi never issues a subsequent sys_write() or sys_writev() to refresh the terminal.

Busybox vi waits until poll() says there is no more data to read before refreshing the terminal. (This part here.) Your poll() says there is always data to read, so the terminal will never be refreshed.

[cksfafwefasdf]

I suspect your poll() implementation is an issue. The VMIN/VTIME thing not so much, since busybox happens to use those in the mode you support (it sets VTIME to 0 and VMIN to 1, requesting to block until data is available, indefinitely). It is indeed entirely possible that the issue is that poll() claims that data is available all the time, but then read() blocks. So vi thinks that new commands are available from STDIN all the time and never updates the screen since it only needs to quickly grab the command available there, but there is actually no data there.

Is a proper Wait Queue implementation for poll mandatory for editors like vi to function?

That is entirely possible.

Does the semaphore-based "one signal per byte" approach fundamentally break stream-oriented programs?

No. That is indeed what I'd expect from a tty connected to the user. There is typically never more than 1 byte available (unless a special key was pressed, and even then the number of bytes can be very small if a serial line is in the middle).

POSIX Compliance in TTY: Since my current TTY implementation is quite "lazy," should I prioritize a more strict adherence to POSIX TTY semantics (e.g., proper termios line discipline) to support complex editors? Specifically, I suspect my lack of VMIN/VTIME handling is a major blocker.

Why "suspect" anything? The busybox source code is available, and you can just read it or debug it to figure out what is happening. From my glance at the source code, it seems to use poll() for the timing quite a lot, so adding actual blocking and timing support would be number 1 on my list.

Thank you for the reply! Your analysis is very helpful.

I was actually considering diving into the vi source code to trace this, but I wasn't quite sure where to start looking. Your explanation of the poll() and read() interaction makes sense—my "fake ready" status is essentially trapping vi in a blocking read() before it can reach its refresh logic.

Now I have a direction. I'll focus on implementing a proper blocking mechanism for poll() and wait queues for my TTY driver.

Thanks again for the guidance!

[cksfafwefasdf]

vi never issues a subsequent sys_write() or sys_writev() to refresh the terminal.

Busybox vi waits until poll() says there is no more data to read before refreshing the terminal. (This part here.) Your poll() says there is always data to read, so the terminal will never be refreshed.

Thank you for the detailed analysis and the source code reference. This is a very strong lead.

I will focus my debugging on this interaction between poll() and the input loop. Having the specific code path in vi to look at is a huge help.

Thanks again for the guidance!

[sortie]

Yeah. You want to implement poll properly, or you're going to have issues like this. Always setting POLLIN will lead to exactly this problem.

The good news is that poll is not too hard to implement properly with some good structure. Personally for Sortix, I have poll node and poll channel concepts, where sys_poll makes N nodes, and then each one is registered on the relevant file's poll channel. Each driver has a function that returns the poll bits currently set for a file, and if the events already happened, then the node is awakened immediately. Otherwise the node is registered on the channel. Whenever the file events change, the channel is informed of the new event bits, and the appropriate nodes are awakened. Back in sys_poll, the syscall enters a wait loop on all the nodes (or a timeout). When it wakes, it unregisters each node, and determines which events happened and copies the poll revents out to user-space and returns the number of events that happened. There are more complexity too with regard to signals and signal masks.

The bad news is that it can be quite hard to know what exactly the right poll semantics are. POSIX is rather silent on the issue and the choice of poll bits can be rather hard to know, and actually varies wildly between operating systems in different situations.

That's why I built os-test, which tests a lot of standard and non-standard behaviors, and it also has a bunch of poll tests:

https://sortix.org/os-test/
https://gitlab.com/sortix/os-test/

I do need to write even more poll tests. There's a lot more to study. But the tests I already have might help you.