It has been a while since I posted an actual update to this thread. I wanted to talk briefly about the new support for VirtualBox Guest Additions I just finished up. I added support for "Auto-resize Guest Display" and "Mouse Integration", and while working on that added support to Yutani for general display resizing.
I also wrote a wiki article if you want to try to implement these features in your own OS.Essentially, VirtualBox provides a device that takes commands over a memory/io port device and can tell you when the host window has resized / what the optimal resolution is with interrupts. The display device in VirtualBox is an implementation of the Bochs graphics adapter (which QEMU also uses), but it has a different PCI device ID, and it supports arbitrary resolutions through a very simple interface. If you haven't written drivers for this "device", you should - it's very easy, and gets you arbitrary resolutions in multiple emulators. Auto-resize in VirtualBox uses an interrupt to tell you when the optimal display size has changed, to which you respond by requesting the display adapter switch to that resolution. Mouse Integration is similar - an interrupt fires every time the mouse moves and you can read the absolute position of the mouse from the guest device. It doesn't include button information, so you still need to read your relative mouse (whether that's PS/2 or USB), which won't include movement information (unless you miss one of the bits in the mouse configuration, in which case VirtualBox injects some "jiggle" into the X coordinates of your relative mouse...).
Most of the work here was in my userspace, as Yutani (my compositor) had not been designed to support the display resolution changing while it was running. I had to also update some of my applications (wallpaper, panel) as they didn't expect to be resizing after startup either. Mouse Integration was a bit simpler, but still required some additional code in Yutani to read the new absolute mouse device file.
I'm interested in looking into VirtualBox's seamless mode, but it will require a lot more work in the compositor. Clipboard and drag+drop might be easier to look at. I'm also going to look at the shared folders later on.
You can try out the VirtualBox integration with
this ISO.
I also have largest screenshots of
Linux,
OS X, and
Windows versions of VirtualBox running ToaruOS.