Any Chance of a Pipewire + XDG-Desktop-Portal Capture Method for Wayland

[d-air1]

I'm not entirely sure of how the entire wayland stack works, but I was wondering if it would be possible to capture the screen the same way that obs studio does. I believe on my machine it uses nvenc as the encoder, pipewire for video capture, and xdg-desktop-portal to request what to capture.

I was reading through the documentation and it seems we have wlr for wlroots based compositors, but that doesn't work for the other two major compositors, nvfbc requires patches, kms works, but there seems to be ongoing work for virtual monitors. Pipewire and xdg-desktop-portal seem to be universally supported across all major desktop focused wayland compositors. Eg: kwin, mutter, wlroots.

Again I'm not sure about the details of any of this, I just saw that obs was able to open a dialog box thing and let me choose any monitor or create a virtual monitor and record that. I believe this would move some of the logic behind detecting monitors and virtual desktops away from sunshine.

[d-air1]

I notice that if I used kde's screenshare portal to create a virtual monitor. Sunshine will detect it, but it isn't listed in kms monitors list and isn't selectable.

[cgutman]

Yes, we want to move to Pipewire-based video capture for systems that support it. We just haven't written the code yet.

[d-air1]

Cool, sounds great.

[himekifee]

That would be brilliant, have been waiting for a pipewire-based screen capture for so long. Would make a virtual display really handy.

[yes91]

This will be especially important for NvFBC support on Wayland. From the NVIDIA 560 changelog:

• Added a PipeWire backend to NvFBC that allows it to work on the Wayland compositors that support screencasting via XDG Desktop Portal.
• This new interface will be available through an upcoming Capture SDK release.

[Quackdoc]

Expanding on this, It would be really nice if it isn't restricted to XDG. While XDG is nice, It would be great to be able to directly use pipewire nodes. One use case for this is gamescope headless capture. The usecase I have in mind would be a small SBC with video in and USB out (so a very low latency KVM) for remote gaming for my consoles. This would be "camera capture" but implementing but doing up a portals implementation for the host for applications like this doesn't make a whole lot of sense.

[valpackett]

This, especially since gamescope does not implement wlr-export-dmabuf-unstable-v1 (I'm looking into that right now but oof, with the way gamescope bypasses like 90% of wlroots, it's tough to integrate wlroots's built-in protocol implementations) (and kinda cannot do it? it's designed with a strong separation of clients, there's no consistent view of outputs between clients, sunshine really shouldn't be a client)

[garnacho]

FWIW, I have some WIP code towards XDG Desktop Portal support. So far, I have the most interesting bits mostly working (i.e. streaming through Pipewire). A possible further improvement is to handle input injection through libei for mouse/keyboard/touch to work ootb without uinput, but I'll look to clean up the changes and make a PR available within the coming week(end)s.

[garnacho]

FWIW, there is now a PR at LizardByte/Sunshine#2507

[Quackdoc]

Is this PR XDG-portals only? I took a quick look and it seems like so. It would be nice if the pipewire streaming bits were abstracted out so it could be set up so we could directly provide node info for capturing. This would be necessary for a headless gaming streaming setup with gamescope.

for context, gamescope directly creates a pipewire node you can capture from using pipewires' gstreamer's pipewiresrc as an example you can capture gamescope using gst-launch-1.0 pipewiresrc target-object=gamescope ! videoconvert ! xvimagesink

This would be really nice to expose

[parkerlreed]

Agreed. Generic Pipewire node capture would be great for this. I'm compiling the PR now to test on Arch Linux with Plasma 6.

[Docmine17]

Capturing the display via pipewire would be very good, I recently tried to use sunshine on Fedora Silverblue via flatpak, and I was unable to capture video via KMS, I need to use rpm directly...

Pipewire also supports window capture.