obs-nvenc: Fix incorrect CUDA array size allocation #11924
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix incorrect CUDA array size allocation.
Motivation and Context
The obs-nvenc module was using a double-sized CUDA array as a shared texture with the P010 format.
The original code snippet:
According to the cuArray3DCreate documentation,
the previous descriptor requested 4 bytes per element, which was incorrect.
How Has This Been Tested?
The changes were tested on Ubuntu 24.10 with an RTX 3060 Ti GPU.
To force obs-nvenc to use shared textures as if it were running on a different GPU,
in the
hevc_nvenc_create
function within nvenc.c,nvenc_create_base
was called with texture=false.The frontend configuration used the P010 color format with the NVENC HEVC encoder.
The following recording scenarios were tested:
Types of changes
Bug fix (non-breaking change which fixes an issue)
Checklist: