Question about Cuda

[Nem-z]

Hey,

I finally get the webui to launch on Windows with the last commit (thank you for that).

Now I have a question:
I see in the System Info tab that my Cuda version is 11.8:

When running nvidia-smi it shows I have Cuda 12.1 installed:

Why the Cuda version differs? Do my performances may be lower because of that? Because if I remember correctly when I benchmarked with A1111 I got ~40 it/s at max and in this version I hit ~30 it/s.
I tried to re-installing Cuda toolkit as stated in #107 discussion.

[vladmandic]

your system cuda libs are 12.1, but torch is compiled against 11.8, so it reports that.
normally mixing major versions is a no-no, but this is a known good combo, so no issues.

and yeah, rtx4090 should give you >40it/s, but only at higher batch sizes as it cannot get saturated enough otherwise. 30it/s is about as expected at batch size 1.

[Nem-z]

Thank you for the quick answer.
As for my performance issue I re-launched a test on A1111 and it gave me ~30it/s this time around. So I must find where my problem is but it seems it's not on the WebUI side.

[rbaechtold]

Good to know, I was asking myself the same. I only get around 25it/s with my RTX4090 (at different batch sizes) so I guess I have to keep digging for a solution too.

[gad-ass]

Woah, I have same issue, on 4090 I have less then 20it/s.
Was testing on different models & sampling methods.

[AlUlkesh]

@gadaszewski
To better be able to compare speeds, try the benchmark from the System Info tab with the stock 1.5 model.

On my 4090 I get:

Batchsize 1: 24.66
Batchsize 2: 35.05
Batchsize 4: 40.58

[ExecuteAI]

Can anyone help me understand this? I have uninstalled all and started over from Nvidia drivers to Nvidia tool kit 11. And 12.1. Anaconda new install, Vlad Diffusion 3 times. Why will it not find my CUDA I am on a 2080 Ti and it worked with the old Stable Diffusion.

[ExecuteAI]

I removed the first command line, but it still does not work.

[ExecuteAI]

And suddenly the CUDA does not respond again?

[ExecuteAI]

It worked when I made the path. Now it is not available. LOL

[vladmandic]

I'm asking about this. --precision full --no-half

in UI settings -> CUDA params.

[ExecuteAI]

It finally worked after the fifth install.
You write that the CUDA has to be installed, but you made me make a path instead.
Look under your installation instructions. The path worked.

[ExecuteAI]

[ExecuteAI]

or does it have to be under system Variable?

[vladmandic]

system variable PATH always exists on windows. you should just add C:\Windows\System32 to it, not create new variable.

[ExecuteAI]

Ok I did. Thanks ill try now.