Refactor overlay support to use SubViewportContainer

[vilhalmer]

Previously, overlay support was limited to a single overlay at a time and hijacked the XR rendering pipeline. I've refactored this to allow as many overlays per app as you want (up to the SteamVR limit). I haven't actually attempted it yet, but they should also be able to exist alongside an XR-pipeline scene in a scene application.

To create an overlay, place an OpenVROverlayContainer into your scene just as you would a normal SubViewportContainer. Its SubViewport will then additionally render in VR. Easy! There's a separate toggle for the overlay visibility, as well as controls for its size and position in the VR scene. It can be attached to a device. I also exposed all of the flags that SteamVR provides for overlays, though I don't know what all of them do because they never write any documentation.

Things I'm not sure about:

• The separate absolute position and device-relative position properties. The intent here was that if you move an overlay between floating in the world and being positioned relative to a tracked device, the two positions will always need to be different and so it makes sense to keep track of them separately to make it easier to swap. However, they could be combined and the change left up to the developer to handle.
• The use of ObjectID to track the overlays within openvr_data. Is there a better way to do this?
• I think there must be a property hint to make the position UI in the inspector nicer?
• Is my comment "We have no way of knowing when our SubViewports' textures are actually updated" actually true?

This code has months of testing in development of my current project and has been very stable. I'm running two overlays at once, as well. It's possible I injected a bug while cleaning it up for posting, but I tested again and it still seems good.

[vilhalmer]

P.S. This goes really well with the feature added in godotengine/godot/pull/94412 to keep the overlays rendering with the main window minimized:

DisplayServer.register_additional_output(OpenVRInterface.get_interface())

I'll add this to a demo when I get to making one.

[fire]

This looks amazing, is it easy to set up a demo project, or can one be provided?

[vilhalmer]

I want to write a demo, I'm not sure the best way to include two demos since the project gets compiled directly into the existing one. I could add an overlay to it, but then the code is tangled.

It should be really easy to try out though, just need the same basic code from the existing demo to initialize openvr, and a scene with

• OpenVROverlayContainer
  • SubViewport
    • Any controls you want

Put this code in your main script to avoid starting a whole scene app:

ovr = OpenVRInterface.get_interface()

ovr.set_application_type(2) # Set to OVERLAY MODE = 2, NORMAL MODE = 1
ovr.set_tracking_universe(1) # Set to SEATED MODE = 0, STANDING MODE = 1, RAW MODE = 2

if not ovr.initialize():
    print("Failed to connect to OpenVR")

I'll try to get a demo project posted somewhere even if it doesn't get merged into this repo yet.

[beniwtv]

Looks like interesting changes, especially minimized support! Small correction though: Multiple overlays were possible before, just not really well documented by myself ;)

[vilhalmer]

Ah good to know, I never actually attempted getting things working in Godot 3.x because I knew I wanted to use 4, so I didn't explore a ton.

[vilhalmer]

@beniwtv or @BastiaanOlij will you have some time to look this over in the near future? Doesn't have to be super granular since we aren't in a hurry to release, but I want to make sure I'm not going in a completely wrong direction somewhere before we're stuck with it.

[BastiaanOlij]

Looks good to me, ben is probably the best of to give an in depth review as I've not used overlays myself. Will definately give it a spin once you have a demo available somewhere. But codewise it looks good.

As for your outstanding questions, the only thing I can really comment on is the ObjectID usage, I think that is fine. It shouldn't change once a node is instantiated. We used the RID of the viewport originally because that was already known to the renderer.

You might want to do something if someone is crazy enough to attach two viewports to a container.

[fire]

I and @Toxic-Cookie couldn't figure out how to start the overlay or make a demo.

[beniwtv]

@vilhalmer I'll have some time tomorrow to check it, been curious to try it, but work is currently quite busy.

[vilhalmer]

No worries, if you want to wait a little longer I'm going to try to get a demo scene put together today and pushed into this PR.

[vilhalmer]

Done, added as a separate scene from the main demo.

[beniwtv]

When running the demo, I am currently getting a segmentation fault (program closes) in the standard C library. Gonna need to check why that is.

[Toxic-Cookie]

When running the demo, I am currently getting a segmentation fault (program closes) in the standard C library. Gonna need to check why that is.

Out of curiosity, could you share reproduction steps and other information like a crash log, hardware, OS, and how your project is setup?

[beniwtv]

Sure, to reproduce: Just run either the normal or the overlay scene (crash so far does not seem to relate to overlay being enabled or not). Hardware is AMD Radeon RX 7900 XTX on Linux, using Forward+ renderer (Linux).

The exact line of code it crashes is (trial method with print(s) to see the problem):

vr::VRCompositor()->Submit(vr::Eye_Left, &texture_left, &bounds, vr::Submit_VulkanTextureWithArrayData);

Not entirely sure yet, even in debug backtrace is not really helpful:

================================================================
handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.3.stable.arch_linux
Dumping the backtrace. Please include this when reporting the bug to the project developer.
[1] /usr/lib/libc.so.6(+0x3d1d0) [0x71da3bd5d1d0] (??:0)
-- END OF BACKTRACE --
================================================================

EDIT: For completeness, in overlay, it crashes here:

vrerr = vr::VROverlay()->SetOverlayTexture(overlay, &overlay_texture);

[vilhalmer]

I've been testing in Windows pretty much exclusively because I don't have a linux machine with a real GPU, but I've been putting one together so I'll try to reproduce soon.

Is this with the release build from Actions renamed to debug? Or did you build it?

[beniwtv]

I built the latest code myself. I looked a bit further into this last night, but there is sparse documentation on Vulkan for SteamVR.

My hunch is that there is something wrong with the texture format, which results in vrclient writing to a memory region it's not supposed to. This could be driver-dependend, e.g. Godot selecting another format for this driver in it's Vulkan renderer.

However I'm not really very familiar with Vulkan as of yet.

[vilhalmer]

Me neither, that part is all severely out of my depth 😅

I know that Vulkan has optional components, perhaps we need to be checking which are available. The OpenVR API asserts that a bunch are present already iirc.

[fire]

There's a related comment on Discord.

Bastiaan "Mux213" Olij — Yesterday at 02:23

@xrworkout when we designed the foundation for Godot 4 we were focusing only on Vulkan, and with Vulkan multiview is well supported.
The problem of also supporting dual render passes is that it requires far more destructive changes to the rendering pipeline, while its a far slower approach.

I really wish more vendors would add multiview support to their OpenGL drivers, especially if hardware is used in the XR space. AMD for instance, as I understand it, supports it but only on their window drivers, but indeed not on linux

[vilhalmer]

I assume the implication of this is "can't do this"? Or we'll have to copy each frame to a new texture at the very least?

[beniwtv]

Hmm, not sure that relates as I think the comment about supporting it was related to OpenGL and not Vulkan right?

[BastiaanOlij]

Hmm, not sure that relates as I think the comment about supporting it was related to OpenGL and not Vulkan right?

Yeah, that just applies to OpenGL where multiview isn't well supported outside of NVidea and Qualcomm.
If we're using Vulkan here than Godot should just render the content fine. Vulkan is very demanding when it comes to setting the right flags when creating textures you render too, so it's indeed very possible that SteamVR is expecting certain things to be enabled that are not.