Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Game: Core: SharedLib: While WoWScreenDXGI is used, every image processing sourced from DXGI instead of using some part of GDI #547

Merged
merged 1 commit into from
Oct 25, 2023

Conversation

Xian55
Copy link
Owner

@Xian55 Xian55 commented Oct 25, 2023

From now on when DXGI reader type is used, every part of the application sources the Bitmap data respectively.

GDI based Graphics.CopyFromScreen can be called anytime anywhere, however the downside is that, its going to block the thread for a pretty long time around ~20-30ms. During this time possible one or two frames might get lost while running the game in high FPS scenario.

DXGI based approach however can produce frame data around roughly every 10ms and AcquireNextFrame has the option to don't await for the next frame. Copying the bitmap data from the GPU to the system memory takes around 1-2ms(at least on my test machine), however this can be significantly reduced by using shaders in the future.

DXGI has one big limitation, there can be only instance of IDXGIOutputDuplication so the previous setup wasn't ideal.

Where every major bitmap extracting process had its own Thread(AddonThread, ScreenShotThread, MinimapThread) all these logic had to merged together. WoWScreenDXGI has exclusive access to IDXGIOutputDuplication. When a new frame is present its going to copy the bitmap data from the GPU memory to system memory. This can be really taxing, so instead only the AddonData is obtained every frame, and the whole Screen and Minimap only acquired with a discrete intervals this way the CPU usage kept low as possible.

There was one major problem with the current setup. WoWScreenDXGI acts as IAddonDataProvider and IWoWScreen as well. So two or more threads are now fighting for the same Bitmap resource.

GDI Bitmap has one downside, at a singular moment, there can be one and only thread access to the Bitmap. Sadly the ReaderWriterLockSlim doesn't help at all. During this time even accessing any kind of property for reading will throw 'object used in elsewhere' exception. LockBits out of the question. Each place where the Bitmap could be accessed, have to acquire the lock first, including the BlazorServer Frontend, Core.ScreenCapture etc...

After many trial and errors, managed to come up a solution which only produce a really small amount of code duplication in the BotController, however maximises the CPU utilization as much as possible but keeps CPU usage also low compare to the GDI solution while using DXGI.

While NpcNameFinder is enabled the CPU usage should remain as low as possible.

The first major benefit which DXGI brings is that, the Game client window pixel data is now available to the program. On the other hand opens up the possibility to migrate NpcNameFinder and MinimapNodeFinder image processing algorithms to the GPU probably as Compute Shaders.

Secondly it has a higher chance to avoid frame skipping. Not to mention, it could lower the latency between actions. However this needs to be balanced, it has a trade off between CPU time and sleep.

From CoreTests project the NpcNameOverlay has been pulled into Core which means now HeadlessServer and BlazorServer has the possibility during runtime to visualize the NpcNameFinder state on the screen with an Overlay, this can be handy for debugging purposes. Can be enabled by CLI or editing the appsettings.json.

Also fixed an index by one issue in the NpcNameFinder after filtering out the Empty Rects.

…ssing sourced from DXGI instead of using somepart of GDI
@Xian55 Xian55 added refactor This ticket concerns the possible simplification of code/data. enhancement This pull request implements a new feature. labels Oct 25, 2023
@Xian55 Xian55 changed the title Game: Core: SharedLib: While WoWScreenDXGI is used, every image processing sourced from DXGI instead of using some part of GDI Refactor: Game: Core: SharedLib: While WoWScreenDXGI is used, every image processing sourced from DXGI instead of using some part of GDI Oct 25, 2023
@Xian55 Xian55 changed the title Refactor: Game: Core: SharedLib: While WoWScreenDXGI is used, every image processing sourced from DXGI instead of using some part of GDI Refactor: Game: Core: SharedLib: While WoWScreenDXGI is used, every image processing sourced from DXGI instead of using some part of GDI Oct 25, 2023
@Xian55 Xian55 merged commit 7dc6f63 into dev Oct 25, 2023
1 check passed
@Xian55 Xian55 deleted the refactor/wowscreendxgi-threading-bitmap-locks branch October 25, 2023 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement This pull request implements a new feature. refactor This ticket concerns the possible simplification of code/data.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant