Win32 interactive resizing proof-of-concept #1296
Conversation
Basically works by setting up a callback to draw a new frame which we do when the WM_PAINT event arrives, which is how Windows tries to tell us it's time to redraw the window due to a resize etc. Unlike the WM_SIZE event, it only happens when we've consumed the whole message queue, so it should be pretty similar to the regular situation where a frame happens after `pollEvents` is called. This approach is necessary in the first place because Win32 window messages can be reentrant, so the handler for one event, like a click on the corner of a window, can in turn fire a series of extra messages for the drag and resizes and release, without first returning.
|
Done a quick review and it really feels like an obscure non local solution for a platform specific problem. I can't imagine many engineers would be able to see the code and understand what it's doing and why. We could put lots of comments in the code to explain why this hacky code exists but far better would be to come up with a localized solution to the deal with the oddities of Windows API behavior rather than pollute the wider code with hacks. |
|
The 'proper' solution is to draw a new frame when we get a The other two potential hacks would be running the message pump on another thread (which would mean creating the window on another thread and synchronising a lot of stuff, which I'd expect to end up much more hacky) or providing our own handlers for the recursive events which aren't recursive (which probably wouldn't match native look and feel unless we lifted code from WINE/Proton/ReactOS, but they're GPL, so we can't). I could have a go at some alternative implementations and then it'd be easier to judge which is least bad. |
|
I'm tidying up cppcheck issues today in prep for making a 1.1.8 dev release. I will pop over into Windows and see what I can learn about this resize issue. Don't worry about coming up with alternatives, once I properly understand the issue I should be in a better place to review your existing changes and perhaps spot a cleaner way to navigate through this. |
|
Unfortunately I've been drowned in cppcheck issues fixing today so have only briefly been able to look at the resize issues. I'll need to return this when I get back from my trip. To test I ran vsgviewer models/openstreetmap.vsgt and recording an animation path then ran vsgviewer with this animation path so that it's clear when the viewer is rendering frames and when it's not. If I don't resize the animation plays correctly but as soon as I start to resize the window the frames stop rendering and the view in the window just gets resized by Windows. This is clear that something in the Windows API is stopping the usual frame rendering from happening during resize. I don't know where it's getting stuck, and it's possible this has already been explained in the various threads, but this looks to be crux of the issue. Having callbacks might be a workaround for Windows stopping the viewer from rendering as it should be doing, but it's hacky so I'd rather not go polluting other parts of the VSG to workaround to this Win32 pecularity. |
|
I wouldn't really call it a Win32 peculiarity. It's the only surviving mainstream native desktop windowing API that uses the approach, but there's only Win32, X.org, Wayland and the MacOS one (it's either Quartz or Core Graphics but it's unclear which) left, so it's hardly an odd duck in a sea of uniformity. Plenty of non-native frameworks (e.g. Qt, which we've already discussed) use the same approach as Win32 where an event handler will dispatch child events without returning, and may not return until an input action's entirely finished. It's just VSG being set up under the assumption that that'll never happen that causes the problem. |
|
I don't recall this behaviour being reported in the earlier VSG days let
alone OSG, it feels like relatively recent changes to Windows has caused
this issue.
…On Tue, 8 Oct 2024, 18:50 Chris Djali, ***@***.***> wrote:
I wouldn't really call it a Win32 peculiarity. It's the only surviving
mainstream native desktop windowing API that uses the approach, but there's
only Win32, X.org, Wayland and the MacOS one (it's either Quartz or Core
Graphics but it's unclear which) left, so it's hardly an odd duck in a sea
of uniformity. Plenty of non-native frameworks (e.g. Qt, which we've
already discussed) use the same approach as Win32 where an event handler
will dispatch child events without returning, and may not return until an
input action's entirely finished. It's just VSG being set up under the
assumption that that'll never happen that causes the problem.
—
Reply to this email directly, view it on GitHub
<#1296 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKEGUC2VPWEQQDGMGMTRCLZ2QLP7AVCNFSM6AAAAABPQO4IXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBQGQ3TONBQGA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
That's just a matter of no one reporting/noticing it. Windows has worked this way since before X.org, Wayland, and whatever-MacOS'-current-API-is-called were invented. The Win32 API doesn't change behaviour, hence binaries from before I was born still running fine on modern versions of Windows, and people justifiably claiming that Win32 Is The Only Stable ABI on Linux. |
|
You might want to look up the dates of when X11 and Apple started with graphical user interfaces... X11 was released 8 years before Win32. I was developing X11 and Motif applications in 92, 3 years before Windows 95 was released and MS finally got into the 32 bit computing era. |
|
I said Windows worked this way since then, not the Win32 API. This predated Win32, which was largely backwards-compatible with Win16, and a subset of Win32 was already usable on Windows 3.1 in 1993 (despite being a 16-bit OS, applications could run in 32-bit mode), or what was at the time the whole thing on Windows NT 3.1 the same year. If you look at an example for Windows 1.0 that'd build and run in 1985, it's got some major similarities to the current |
|
I was writing event driven applications before Win32 existed on X11/Motif/OpenGL, it's not anything new. Event driven applications are perfect fine for interactive applications. They aren't appropriate for real-time applications where the frame rate needs to be kept at a constant rate and never interrupted by events, the OSG/VSG heart land is vis-sim so out of the box it's frame driven. Sure you can do even driven just fine with the OSG/VSG as well, but that's for interactive applications nor real-time ones, for the later we'll always need to be frame driven. |
|
Even if things are event-driven, you can still make things real-time by having a budget (either time or counter) to cap the number of events you'll handle before ignoring any more and drawing a frame anyway. That's more-or-less orthogonal to whether things are event-driven or not, as not having a budget when consuming incoming events as part of a frame loop, like VSG does now, means a flood of events can cause a missed frame deadline. There's a counter in the XCB windows's Anyway, that's not really particularly relevant to the root cause of the problem. The real cause is that event handlers are free to remain on the stack and emit their own child events as long as they run their own event loop to take over from the default one. For example, a mouse button down handler might look something like: void onMouseButtonDown(event)
{
onDragStart();
while (innerEvent = blockUntilEvent(); innerEvent.type != mouseButtonUp)
{
handleEvent(innerEvent);
if (innerEvent.type == mouseMove)
onDrag();
}
onDragEnd();
}If an event handler might not return until some other event arrives later, then you need some way to still draw frames before it returns, whether that's doing it on another thread, or drawing in an event handler (potentially after also adding an event source based on a timer so you don't skip frames when no events are generated for a while). Some of Windows' default message handlers and Qt's event handlers are set up like this. It doesn't cause problems with vsgQt as frames are drawn in response to a timer event and don't care what's already on the stack, whereas it does cause problems with Win32_Window because it can't draw frames until all event handlers have returned. |
|
Head in hands, no you can't make real-time graphics application that are event driven, professional simulators just don't and shouldn't work that way. Sure write games that don't really care whether a frame drops or comes in early, or interactive applications that are event driven, but please keep the event driven far away from real-time graphics simulators and VR, even fudging things doesn't cut it. The OSG and VSG strive to be professional grade APIs that can do real-time graphics and VR from the ground up. This is a pretty fundamental technical point, trying to push event driven as OK to someone with years more experience and knowledge of the topic is really stupid, your ego has got ahead of your understanding. I appreciate your input on trying to get this problem resolved but please stick to what you know, not what you think you know. |
|
I'm wondering if maybe we're using different definitions of real-time (in which case, I'd appreciate a clarification so I can stop looking like an idiot). Every time I've come across the term in a computing context, it's either meant:
Clearly we're not using the second definition here as it's silly. There are plenty of event-driven real-time frameworks for programming microcontrollers, so the first definition can't be fundamentally incompatible, either. As I thought we were using the first definition, I felt it was worth mentioning that the current approach has the same problem a naive event-driven approach would, as a burst of events can delay a frame for an arbitrarily long time as all the |
|
I found some discussion with an eventual fix (that I've not looked at yet) when SDL ran into the same problem and took years to find a nice way to fix it: libsdl-org/SDL#1059. One of the things that was mentioned is that MacOS has an analogous problem where pressing and holding one of the window buttons will block for basically the same reasons, so it's just X and Wayland that don't have this problem. |
|
Thanks for the SDL discussion link. It's an interesting read. One thing missing is that folks are being frustrated with SDL but in fact no one is pointing the finger at the real culprint, Microsoft. This bit of the Win32 API is clearly a big fuck up, I'm not saying this lightly, it's really bad design. All the problems that have folks chasing their tales, wasting weeks work on these errors are all down to shoddy design & implementation that shouldn't got into the final release. However, Microsoft has long been able to ignore better engineered prior art like unix and X11 and come up with their own bodge together shit show but through force of marketing and manipulative practices forced it's way to to top, despite poor engineering from the bottom up. It's become such a pervasively perverse situation that competent engineers have been so used to bad design from MS they normalize it and weave it into their own understanding of how to do things and not realize they are bound to the same engineering shortsightedness. The reason why the OSG, VSG and SDL's polling of events is being tripped up by windows is not because polling of events is flawed, it because Win32 has elements of really bad design and implementation. Polling of events should never take more than a fraction of millisecond, it should never break a single frame let alone completely stall the calling application. It's a MS fuck up through and through. The fact the Apple also have this buggy bad design doesn't make either parties approach excusable. It's just really bad. Neither operating system should be deployed in anything mission critical or real-time if they can't get even really basic things right. An not, moving to event driven application doesn't fix these fundamental problems w.r.t real time applications, it just hides the glaring obvious MS bad design highlight by resize, you simply can't time a frame begin/swap buffers around when an OS decides it's time for a idle or timer event. For real-time graphics app the thing that drives the frame is the swap buffers of the display, potentially across multiple displays across multiple machines. The VSG doesn't yet have support for the later out of the box as it'll require extensions and some hardware to work with, but the fundamental design is ready for it. |
|
Even for web applications, where there's no way for Windows details to pollute the environment, lots of UI frameworks opt into the same general approach as Win32. It's definitely not the best thing to make the only option (as you can't feasibly build a classic frame loop on top of it, but you can build it on top of a classic frame loop), but people do like it and opt into it when they don't have to forty years later. I don't like it, either, and using this approach for window resizing in particular is egregiously obtrusive, but it's not as simple as being a terrible decision everyone hates made for stupid reasons that no one understands. It's pretty clear that Microsoft weren't expecting to maintain binary backwards compatibility for Windows 1.0 applications until Windows 10 when they designed it, and how it would work (or as it turned out, not) for multimedia applications a decade before machines running Windows were typically capable of full colour was clearly not a priority. It'd certainly be less hassle to make a frame loop if there was a As a specific nitpick, X11 isn't prior art here. X as a whole was initially released in mid-1984, pretty late into Windows' development (with the 1.0 release coming a year later) and had frequent breaking changes until X11 was released in 1987, two years after Windows. It's also notorious for not being well-engineered, hence being unmaintained for twelve years, and an inordinate amount of ongoing effort towards developing Wayland and porting everything over to it.
This is:
As evidenced by the comparative size of those two bullet points, it's obviously way less hassle and hoop-jumping with X11, but it shouldn't be insurmountable on Windows, and even if I'm missing something that you've thought of and I don't know about, it should at least be possible to get pretty close. |
|
The problem with the Win32 API is not that it supports event driven applications, it's that it breaks any sensible frame driven or similar non event driven applications, it's why the design and implementation is broken. API should support both event driven and non event driven usages. It should be trivial to do but it isn't, and causes really ugly glitches like we have, glitches that are hard to track down and fix, and ones that spill over to application/end user domain as demonstrated by the SDL thread. I am bit lost on the whole process a subset of events if there isn't time budget to do it. The getting of all the events from the windowing API should be trivially fast. If application won't to process those events in expensive ways then it's up to the applications on how to handle them. Interactive applications won't care about frame drops in the same way a visual simulation or VR application will, for the later the application won't be resizing the windows etc. at critical times in the simulation so these type of expensive window related events will be a none issue. What I'd like for us to achieve is to avoid the frame blocks that are happening in Win32_Window.cpp during the window resize. I want us to get to a place that on all platforms things just work, whether event driven like a Qt app or frame driven like a simulation/VR application should be. Things already work how they should work on X11/Xcb so it's a case of bring Win32 and macOS if it's not up to scratch. This is not to say say that X11/Xcb are superior for all purposes, it's just that they work well for how simulation/VR application need windowing and event handling to function i.e. do what we ask from them and otherwise keep out of our way. |
|
The way I'd be inclined to do things (after a visit to the pub, so it's not to be taken as gospel) would be to give
Part of why I mentioned this is that I've meddled with things running on microcontrollers where I had to cap the number of characters I'd read from a serial connection otherwise I'd miss my chance to poll pins (the Arduino Uno only has two interrupt-capable pins) or send a pulse to a stepper motor driver. There's a much bigger gap in computer graphics between the cost of draining the event queue and drawing a whole frame than there is between reading a command from a serial connection and the number of cycles between motor encoder pulses on a microcontroller, so it's a silly concern here. The other part is that when |
|
I'm back in the office again after a being away since last Wednesday. I will tag a dev release today to at least give us a stable reference point for future work, even if it doesn't solve all the outstanding issues. After posting these dev releases I'll work on the viewport sizing state issue and Win32_Widow.cpp issues with event handling. For the later I'm now inclined toward having a thread per vsg::Win32_Window, creating this thread in the constructor when the window is created locally and needs to handle the events. I think for cases like the vsgQt usage which provides the window and event handling this won't be required. With the threading of the event loop, the easiest implementation would be to have a thread per vsg::Win32_Window object, but would we need a single event loop thread per application that handles multiple windows? If the later do the events have user data about which window they are associated so we can direct it at the appropriate Win32_Window's event queue? |
|
The first parameter to the |
|
Do you think a single thread for creating the native windows and running an event loop that gets the events and then adds them to the appropriate Win32_Window event queue is appropriate or have a single thread per Win32_Window? |
|
Both are probably viable. I'd hope that the nested message pumps run by message handlers when they don't immediately return would be capable of receiving and dispatching messages for other windows - at least in Wine, the |
Basically works by setting up a callback to draw a new frame which we do when the WM_PAINT event arrives, which is how Windows tries to tell us it's time to redraw the window due to a resize etc.
Unlike the WM_SIZE event, it only happens when we've consumed the whole message queue, so it should be pretty similar to the regular situation where a frame happens after
pollEventsis called.This approach is necessary in the first place because Win32 window messages can be reentrant, so the handler for one event, like a click on the corner of a window, can in turn fire a series of extra messages for the drag and resizes and release, without first returning.
This PR seems to work fine on Windows and shouldn't have changed anything on other platforms, but I've not tested it on other platforms, and it lacks some polish as I wasn't sure how best to accomplish the
advanceToNextFramesplitting, so thought I'd wait for your opinion before doing more than the minimum.