PPU LLVM: Memory Consumption Enhancements

[elad335]

PPU LLVM's memory manager was not able to handle more than 512MB of data, but at same time instead of reporting an error on outage it reused memory buffers. This resulted in corrupted PPU LLVM cache files for affected titles.

Fixes PES games with their humongous executable file.
Fixes #11469 (you need to remove broken PPU LLVM and SPU LLVM cache)

Additional impovements

• Reduces RPCS3 RAM memory consumption by 100MB-300MB ingame and more than twice this amount during PPU LLVM compilation. This is because PPU analyzer information is stored much more efficiently, size of data members has been reduced and PPU LLVM compilation no longer makes copies of this memory.

Before (Demon's Souls)

After - Around 160MB reduction in memory!

[elad335]

Maybe the proper solution would also require splitting huge executable files caches.

[elad335]

I need testers to report the size of their EBOOT.BIN and its PPU LLVM module count for taxing games

[digant73]

Removed all caches and started Resistance 1. It freezes at the the end of Applying PPU Code with the following error, also reported on the GUI:

F {PPU[0x1000000] main_thread} VM: Access violation writing location 0x28ec8348 (unmapped memory)

RPCS3.log

[Satan86]

PES 2017 crashes to desktop after Compiling PPU Modules. It has 347 modules and the EBOOT.BIN is 16.996 KB.
RPCS3.log
The log says this at the end: ·F 0:05:53.309380 {PPU[0x1000000] main_thread} JIT: Out of memory (size=0xd1a78, align=0x10)

[bigol83]

I don't know if this is relevant but FF XIII still crashes when applying PPU modules

The game has 119 modules and the eboot is 8.150 KB

[Darkhost1999]

I don't know if this is relevant but FF XIII still crashes when applying PPU modules

I was thinking about adding a comment #16112
Because this PR didn't introduce the issue.
If this PR did introduce the issue I would've commented here.

FFXIII has gone through ~3 or 4 regressions experiencing close to desktop on a clean cache. If the llvm cache version is bumped again then only older builds will be able to play for now. But it seems we can compile on the older build, update, and play just fine on the current master.

[digant73]

PPU LLVM's memory manager was not able to handle more than 512MB of data, but at same time instead of reporting an error on outage it reused memory buffers. This resulted in corrupted PPU LLVM cache files for affected titles.

is that an issue we could see just at boot? or something that can occur during the game (e.g. access on unmapped memory etc.)?

[elad335]

I don't know why it regressed but it's because the firmware ps2 emulator, I disabled its compilation altogether. RPCS3 can't launch it in the first place.

[Darkhost1999]

Thanks, I just observed the Windows build canceled so I checked out, compiled, and tested as working.

[woj1993]

Hi. So I tested this build and got 1 weird case when I finished playing Scogger HD [SCOG00001] game I tried to turn on second game (Zuma [NPUA30003]) and It wouldn't start. So I exited emulator and tried to restart it.
Apparently it was running in background and consuming gigabytes of memory:

Logs:
log very large memory.zip
After that I played ASURA'S WRATH DEMO [NPEB90381] and it was playable. Also I'm not sure but I think clothes and hair got physic. Is this possible? Maybe I'm searching for differences too hard.
Then I played Dragon Ball: Raging Blast 2 Demo [NPEB90287] and it was playable with no visible differences.
If you want and if this would be useful to you I can play also these games:
Dead Space™ Dismemberment Demo [NPUB90187]
Ni no Kuni: Wrath of the White Witch – Journey to Another World [NPEB90452]
Ratchet and Clank® Future: Tools of Destruction™ - Demo Version [NPUA98153]
But probably tomorrow I'm not sure if I will have time today.

[Darkhost1999]

There was another user on Discord but with Macos to receive an error with RPCS3 using 41GB of memory when their system has 8GB.
However, they were not utilizing this Pull Request. It doesn't seem that we can pin the blame here and will need to figure out how to replicate the issue.