why is forge faster?

[blacklig]

Anyone knows the answer?

Do they use some optimalisations like triton out of the box?

but on the other hand, when bigger batch, I noticed the oposite..

[vladmandic]

before asking why, let's establish a baseline.

on which platform and hardware and settings? what is the performance?
and note full end-to-end times of generation, not just what's reported in progress bar.

in my tests, forge was only faster on low end hardware when compared to sdnext with lowvram as it's more efficient with aggressive model offloading. but slower in pretty much all other scenarios.

[blacklig]

mine is RTX 3090, I am using it with controlnets so where I am on like 2.2s for some 512x512 (original backend) on SD.NEXT I might be seeing like 1.7 on forge etc..

but for batch of 4 next was quite faster.... I wasnt testing it that much thoroughly though as I noticed next.sd works way better with IP adapter model (but that might be becaue of mess of authors with like different preprocessors, different models etc.. somehow just next.sd gives better results and I was not able to get same on forge even though trying numerous settings there..)

If you have in mind some testing ideas I might try it here to see what goes better..

Anyways, how about triton? I dont think you are using it, right? I dont know if forge does, but I noticed it was being installed..
Triton can speed up things considerably.

[vladmandic]

comparing sdnext original backend to forget is not fair. original backend is compatibility for a1111 and any performance will be same as there, no better or worse. you want to compare performance in any meaningful way, use diffusers backend. and i would be surprised if it didnt end up as much higher.

and yes, triton is available as an optional compile backend, so are many many others.
but again, only when using diffusers backend.

diffusers backend is default in sdnext and gets a lot of new stuff almost daily.
original backend is for a1111 backward compatibility only. no major work has been done on it in a long time, only break/fix items.

[blacklig]

Isnt forge running something as "original backend" anyways?

I am talking about original control nets for SD1.5 so it cannot run on diffusers backend in SDXL, or can it?

I would like to be on diffusers but as it doesnt show control net as was in original, it is diffucult for me.. in forge there is not even any such choice so I dont even know what it runs in, prolly diffusers as well but working with CNs?

or is it possible to use all "legacy" CNs for SD 1.5 with diffusers backend and SD 1.5 models on SD Next?

[vladmandic]

forge is a1111 backend plus new model offloading methods to swap in and out of vram.
controlnet extension is written for a1111 and works only on apps using that backend.
controlnet models can be used by anyone and sdnext has its own implementation of how to manage them.
if you want to use old controlnet extension, you need to run in backend original, but then don't compare performance, its pointless.

[blacklig]

but can I use old controlnets with diffusers backend on SD.Next? I think I cant, but maybe I am wrong?
And I compared Forge and "Original" SD.Next backend when I tested those

[vladmandic]

like i said, no. and yes, forget is more optimized than a1111 or sdnext when running in original backend.
but like i said, original backend is for backwards compatibility and no performance optimization has gone into it in months.
if you care about performance, use sdnext with diffusers backend and native controlnet functionality.