Gui events

[AlexHully]

Hi,

I would like a clarification:

when I click on a portion of the screen, the window manager looks where the click happened (x and y), looks for the window currently on top at this coordinate in a linked list or a struct array, looks inside that window struct for a corresponding button and callbacks as needed to execute the function.

Is my vision of the whole process OK?

Thanks

[BASICFreak]

Hi,

I would like a clarification:

when I click on a portion of the screen, the window manager looks where the click happened (x and y), looks for the window currently on top at this coordinate in a linked list or a struct array, looks inside that window struct for a corresponding button and callbacks as needed to execute the function.

Is my vision of the whole process OK?

Thanks

A very simple flow chart (not fully accurate)
Mouse Event -> Window Manager -> Application Event Handler -> [whatever action is taken] -> Back to waiting for an event

You do not worry about what in the application you clicked, that's for the application's event handler - you just pass x, y, button (or similar) to application

And for something that captures mouse (such as any modern game) you could make a way to bypass the Window Manager (don't have to, but it would be less IPC)

Hope that helps some.

NOTE: Some of the above is assumption. (it's my next task after after porting my old modules, so I have yet to implement it)

[max]

Heyho,

well it depends on what structure you store your components in. A tree structure makes a lot of sense for window management, as it basically is all tree-like; a button on a window moves with the window etc..

Dispatching an event is easy through a tree; you just give it to the topmost component (in the Ghost WM this is the "screen"), which then looks for the topmost component at the events location within its list of children. This would be a window then for example. Then the relative position of the matching component is substracted from the events location and the event is dispatched further down. Like this, each component receives an event containing the relative position of the event within itself, avoiding the need to possible calculate the events relative position by iterating up in the tree. Each component may then decide within its handle method if it wants to handle the event itself, or dispatch it further down.

This gives the best performance as you don't have to iterate over a big list of components to find the one that is actually hit. Also, it is automatically assured that the topmost component is hit, not a button that accidentally has the same position but lays in the window behind the active one

You can take a look here for inspiration.

Greets,
Max

[Brendan]

Hi,

when I click on a portion of the screen, the window manager looks where the click happened (x and y), looks for the window currently on top at this coordinate in a linked list or a struct array, looks inside that window struct for a corresponding button and callbacks as needed to execute the function.

Is my vision of the whole process OK?

Maybe.

Typically the GUI is in one process and applications are separate processes, and one process can't call another process, so callbacks can't work between GUI and applications. To work around that you need some sort of IPC (Inter-Process Communication).

There's many different types of IPC (shared memory, shared files, pipes, sockets, RPC, messages, etc). For events; mostly you want a variable sized data structure for each type of event (e.g. a key press event might say which key, whether it was pressed or released, if it corresponds to a Unicode/ASCII character, shift/alt/control state, caps lock/scroll lock/num lock state, etc). Some forms of IPC are "less good" for this (e.g. pipes/streams) because it's harder to figure out where one event ends and the next begins. Also note that often you want a thread to block waiting for an event, which means that the IPC tends to have a heavy influence on scheduling (e.g. threads are unblocked/scheduled when an event arrives); and this blocking/unblocking makes some forms of IPC (shared memory with nothing else) awkward because there's no way to notify scheduler.

Inside a thread, you might have a loop like this:

Code: Select all

    while(running) {
        get_event();
        handle_event();
    }

The "handle_event()" is the piece that might (or might not) use call-backs.

when I click on a portion of the screen, the window manager looks where the click happened (x and y)

That works, but only when windows have a regular shape, and makes it difficult to handle various compositing effects. For example, if you feel like mapping an application's window onto a curved surface (or if there's multiple windows with "holes" in them, or...) it becomes a nightmare. Basically, you've got an (x, y) screen co-ord and need a way to map that back to a visible object, where that visible object might be a GUI button, or a monster in a 3D world, or anything else. The most flexible way of doing this is called "picking".

How "picking" works that instead of drawing a 1024*768 pixel screen you draw a 1*1 pixel screen where the mouse was clicked and see what "colour" that pixel happens to be; except that instead of drawing things using colours you use IDs (e.g. one button might be "ID = 1", another button might be "ID = 2", etc; and if the pixel happens to be "ID = 2" you know which button was clicked). The advantage of this is that it works with any/all transformations done by anything, recycles a lot of the rendering code, and works reliably regardless of how convoluted things get (e.g. you can literally have multiple windows with holes that have been warped/transformed in insane/bizarre ways, and it still works perfectly).


Cheers,

Brendan

[SpyderTL]

How "picking" works that instead of drawing a 1024*768 pixel screen you draw a 1*1 pixel screen where the mouse was clicked and see what "colour" that pixel happens to be; except that instead of drawing things using colours you use IDs (e.g. one button might be "ID = 1", another button might be "ID = 2", etc; and if the pixel happens to be "ID = 2" you know which button was clicked). The advantage of this is that it works with any/all transformations done by anything, recycles a lot of the rendering code, and works reliably regardless of how convoluted things get (e.g. you can literally have multiple windows with holes that have been warped/transformed in insane/bizarre ways, and it still works perfectly).

Or, if you have enough memory, just have all of your draw calls also "draw" to an offscreen 1024*768 buffer with the window ID, and then when the user clicks the mouse, just look up the window ID in the offscreen buffer.

But, if you're always going to have rectangular windows/controls, finding the top-most control that contains the mouse point is probably quicker.

[AlexHully]

Thanks everyone,

@SpyderTL: i thought about this solution but the memory used makes me fear

@Brendan: i searched the web but could not find good resources on the picking method.
Your idea interests me a lot (if it is not ram hungry).

How may I know the color/id? How do we map a pixel to an id? That part is not clear to me.

To make things simpler, to explain/understand let's start a beginner example: i have a rectangular window. Three buttons in it. I click one button.

How do I know which one, with picking? I am not asking for code, just theory/logic/starter. I will make the rest of the process myself.

Thanks a lot

[Ready4Dis]

While these might not be the numbers you would use... you you basically create a 1x1 screen (single pixel). You then render your window + buttons into that screen at the proper location.

Your window gets an assigned ID of 1, the first button = 2, second button = 3, 3rd button = 4.

When you click, you render the window with a value of 1, then render the buttons with 2, 3, 4. Whatever 'color' (ID) that special 1x1 window is, is the current object ID.

Of course, this is highly simplified, but that's the basics.

[AlexHully]

then render the buttons with 2, 3, 4. Whatever 'color' (ID) that special 1x1 window is, is the current object ID.

I am sorry, but I still don't get it.

So I create multiple 1x1 pixels with the id of the button concerned, and that for each button? So that when i click on an area I will read the id concerned?

The 0's are non interest zones (non button) in my app window
Like so?
-------------------------------
l0000000000000000000000000000|
l2222220000000000000333300000|
l2222220000000000000333300000|
l0000000000000000000000000000|
l0000000000000000000000000000|
l0000000000004444000000000000|
l0000000000004444000000000000|
l0000000000000000000000000000|
--------------------------------

I am pretty slow to understand that one. Feel sorry about that.

[Chandra]

Hi,

I would like a clarification:

when I click on a portion of the screen, the window manager looks where the click happened (x and y), looks for the window currently on top at this coordinate in a linked list or a struct array, looks inside that window struct for a corresponding button and callbacks as needed to execute the function.

Is my vision of the whole process OK?

Thanks

In most cases, a generic approach is to setup an event handler. Every visible window has an event handler registered to the window manager or, the OS kernel, or whatever you've setup to dispatch events.. When an event occurs, for e.g. a button click, the default handler for this device forwards the event information to the event handler registered for the window. Which event handler receives the event is basically dictated by one or more parameters, such as currently active window. What action to execute when an event occurs is based on how the window control has been setup. For e.g. consider an OK button. This control holds attributes such as (x,y) co-ordinate relative to the current position of the window, width & height and action associated for each button click on the mouse, etc. Based on this information, it is relatively straight forward to determine whether the click has been made within the control perimeters and if indeed it is, execute the associated action.

[Brendan]

Hi,

then render the buttons with 2, 3, 4. Whatever 'color' (ID) that special 1x1 window is, is the current object ID.

So I create multiple 1x1 pixels with the id of the button concerned, and that for each button? So that when i click on an area I will read the id concerned?

The 0's are non interest zones (non button) in my app window
Like so?
-------------------------------
l0000000000000000000000000000|
l2222220000000000000333300000|
l2222220000000000000333300000|
l0000000000000000000000000000|
l0000000000000000000000000000|
l0000000000004444000000000000|
l0000000000004444000000000000|
l0000000000000000000000000000|
--------------------------------

Like that; but instead of drawing the full screen full of pixels you only draw a 1*1 pixel area, like this:

Code: Select all

               ---
               |4|
               ---

Note that for most graphics systems you have a large virtual space (e.g. with a 360 degree field of view) and a "view port" that only shows part of that virtual space at a time (e.g. with an 85 degree field of view). Things are clipped to the edges of the view port. For example, if you've got a widget or something where half of it is past the edge of the screen then you have to discard the not visible part of it. By making the view port 1024*768 and drawing everything you might get a full screen image with very little clipped to the screen edges, and by making the view port 1*1 pixel and drawing everything almost everything will be clipped without being drawn (which makes rendering faster).

Also; don't forget that good graphics systems are resolution independent. In that case you don't even need to draw a whole 1*1 pixel - you can draw an area that's a tiny fraction of a pixel (and have sub-pixel mouse pointer resolution, where a pixel might be right on the edge of a button and if you click on the left side of the pixel you hit the button and if you click on the right side of the same pixel you miss the button). This doesn't really change anything (as far as implementation goes).


Cheers,

Brendan

[AlexHully]

But to get that single pixel, i need to calculate it. So I need my map/ buffer wih all the information already there, ready to be discriminated, right?

So i will end up with my scene + 1 hidden color buffer of that screen anyway.

If it is wrong, then please direct me to a resource that will show me the process.

Thanks

[Combuster]

First of all, for most 2D UIs that approach is overkill, and sticking with rectangles is easier and faster. In fact if you have a touch display, you often want larger responding areas than what's actually being drawn and this method just doesn't cut it.

If you still insist on picking, you only need a second rendering when you actually want to process an event, and you certainly don't need to render the entire screen to get the pixel value you want.

[AlexHully]

YES!

I finally got it. Spent more time on 3d forums and opengl.

Yes picking is not for basic 2D Uis, but in a 3d "perspective", I get the functioning now.

Thanks a lot

[Ready4Dis]

It can be used for 2d or 3d, and it can be used with a mouse or touch screen. For touch screen, you could render an image 10x10 pixels, or any arbitrary number of pixels that makes sense, and then figure out from there what is most central in the image and which ID dominates the area to determine what was meant to be pressed. As mentioned, this allows things like non-rectangular windows, windows with holes in them, applying any sort of effects to be applied and the handler will work flawlessly. You can rotate, zoom, make it look like it's leaning back in 3d, and everything will still "pick" properly. It takes a little longer to get working, but once it works, you don't have to extend it any more. If you want to add support for a new type of drawing (maybe a rotating 3d cube for selecting apps?) it just works. If you go the traditional route of just checking (object.x1 < mouse.x < object.x2) and (object.y1 < mouse.y < object.y2) then it won't work for any other cases besides a rectangle.

Glad you understand now, and hopefully you can see how useful it can be, even if you're just doing a 2d gui with some effects, and if you ever decide you want to render something in 3d to click on, it will work without any changes.