Support for image format swizzles

[kd-11]

A major hurdle porting rpcs3 to mvk is the lack of format swizzles; even the single component textures cannot be swizzled to read R to all outputs. Are there any plans to support this? The old readme mentioned only 2 format swizzles were supported.

[billhollings]

This is indeed a Vulkan compatibility limitation.

The problem is that Metal does not support specifying per-texture component swizzling.

An alternate solution could be to modify shaders to handle configurable swizzles on the fly...but that might be incredibly cumbersome...AND slow down performance for the vast majority of shaders that do not require component swizzling.

Community ideas and suggestions for alternate fixes and workarounds are welcome.

[cdavis5e]

I considered using specialization constants for this, but that would require us to recompile the shader anytime an image view was set with a swizzle. It also wouldn't work below Metal 1.2 (iOS 10, macOS 10.12). I also considered hardcoding the swizzle into the generated MSL, which is what Wine does for OpenGL without GL_ARB_texture_swizzle, but then that would not only require recompilation, but going through the SPIRV-Cross transpilation process all over again!

[kd-11]

I ended up just doing the decode in the shader by using custom texture instruction with a channel decoder to emulate this. This is usable for emulators, but for general API use, I'm not sure if the extra performance hit is worth it.

[cdavis5e]

Well, unfortunately, we (CodeWeavers) need this. D3D12 has general swizzles now (which we will have to support), and we also need some swizzles in particular (e.g. putting the stencil value from a stencil-only view of a depth/stencil image in green). I hear talk that DOOM also wants this. (And I wanna rip and tear demons on my Mac! I don't care if it's a little bit slower, I just want it to work at all!)

[hrydgard]

Metal 3 has support for custom texture swizzle.

[billhollings]

Just a reminder for anyone who wants to add Metal 3 features...please do so on the Metal-3.0 branch.

[kanerogers]

Stubbed my toe on this one today! Now that more Metal 3 features are supported out of the box, it would be great to get this one on the list. Happy to help with implementation if you can point me in the right direction!

[billhollings]

Stubbed my toe on this one today! Now that more Metal 3 features are supported out of the box, it would be great to get this one on the list. Happy to help with implementation if you can point me in the right direction!

MoltenVK does support swizzling. But we may not be advertising all formats accurately.

See for instance the concurrent PR #2383 to fix that for a couple of formats.

What formats are you running into trouble with?

[kanerogers]

Oh, excellent! The specific format was VK_FORMAT_R8, if that helps!

[kanerogers]

Unrelated, there are a few open issues in the repo that could probably be closed off now as they don't reflect the current state of the project. Would it be helpful to have a second set of eyes go over these issues and flag them? I'm mindful of spamming your inbox or if you've already a process in place for this sort of thing.

MoltenVK is invaluable to our game (we have a large and enthusiastic macOS player base), so I'd love to contribute anything I can back to help - even if it's just busywork like this. 😆

[cdavis5e]

Unrelated, there are a few open issues in the repo that could probably be closed off now as they don't reflect the current state of the project. Would it be helpful to have a second set of eyes go over these issues and flag them?

That would be very much appreciated.

[billhollings]

MoltenVK is invaluable to our game (we have a large and enthusiastic macOS player base), so I'd love to contribute anything I can back to help - even if it's just busywork like this. 😆

Wonderful to hear! Are you able to share the name of the game? No worries if you can't or don't want to, but it's always great to hear how MoltenVK is being used!

[kanerogers]

Wonderful to hear! Are you able to share the name of the game? No worries if you can't or don't want to, but it's always great to hear how MoltenVK is being used!

We're in a closed beta at the moment, can say more publicly early next year. 😄 If you'd like to know more in the meantime just shoot me an email at kaneDOTmDOTrogers at Google's mail service. 😆

[kanerogers]

That would be very much appreciated.

Fabulous! I think this issue is a good candidate for closure given (a) the feature has now been implemented (b) there's now a documentation trail explaining why a particular swizzle may not yet be supported, and what to do if it isn't.