Why does batch generation complicate reproducibility?

[afbagwell]

I've talked about this issue before, but lately I've revisited the issue with a few image tests.

Here's the basic problem.
Set up image generation with a prompt. For this experiment I found one online.

Prompt: Action scene, dynamic pose, firm stance, combat stance, Focus on (Assassin Striking from the shadows with a deadly precision, Wizard, Short, Toned, Oval Face, Dark Skin, Silver Hair, jade Eyes, Narrow Nose, Thick Lips, Receding Chin, Garibaldi, Buzz Cut, Spiky, An ancient, mystical ruin, hidden deep in a dense, overgrown jungle) fighting (Gnoll), in a mystical forest at noon, illuminated by bioluminescent plants and inhabited by mythical creatures
Negative prompt: Bad anatomy, (Long hair, 1girl, Woman, feminine, breasts, girl, lady, female:1.4)

Steps: 30 | Seed: 1 | Sampler: DPM++ 3M SDE Karras | CFG scale: 7 | Size: 768x512 | Batch: 4x1 | Index: 1x1 | Parser: Full parser | Model: Illustro_v3_full | Model hash: 8d820d7e86 | VAE: vae-ft-mse-840000-ema-pruned | Backend: Original | Version: ccae1ef | Operations: txt2img; face; hires | Hires steps: 30 | Hires upscaler: Lanczos | Hires upscale: 2 | Hires resize: 0x0 | Hires size: 1536x1024 | Denoising strength: 0.4 | Latent sampler: DPM++ 3M SDE Karras | Image CFG scale: 6 | Face restoration: CodeFormer | CFG Rescale phi: 0

I ran this prompt three times with different batch configurations so that each time I generated 4 images.

Batch: 4x1

Batch: 2x2

Batch: 1x4

You can already see the problem. Even though its the same parameters used to generate each image, the images change slightly depending on the batch configuration. They don't change if you re-run the same batch configuration repeatedly.

To make matters worse, adding a Lora makes the changes between batch configs even more pronounced. I used the same prompt above but replaced the phrase
in a mystical forest at noon
with
in a mystical forest at dusk <lora:epiNoiseoffset_v2:1.0>

Batch: 4x1

Batch: 2x2

Batch: 1x4

And finally, here's the issue I keep running into. I use dynamic prompts plugin to create templates. Some of these templates will plug in elements that use Loras, making some images in the same batch that use Loras and others that don't. We can achieve the same effect here with the dynamic prompts extension and changing the text once again to:

in a mystical forest at {noon|dusk <lora:epiNoiseoffset_v2:1.0>}

The effects can cause completely different results on each run.
Batch: 4x1

Batch: 2x2

Batch: 1x4

What's happening here (after a lot of testing) is a bit sneaky.

I will typically generate a 1x4 batch that can come out like this: ... image1(Lora), image2, image3(Lora), image4

Then I want to recreate image2. But just running its saved metadata through Text2Image again, results in a totally different image. Instead you must add the Lora from image1 to the beginning of the prompt in image2 to replicate it. This Lora is NOT in the original image2's metadata.

---

Conclusion:

Currently the only way to run images with any guarantee of being able to reproduce an image exactly is to always set your Batch Size to 1.

Does anyone know why this happens? Is it a Stable Diffusion thing based on how batches are processed? Or is this a bug in how the web UI is handling image generation?

Long post, I know. Thanks for reading.

[acorderob]

All the images in a batch are generated at the same time, using the prepared model. This model includes the loras, that are taken from the first image prompt. Thus, all the images in the batch will include the loras set for the first image, and only those. Like you, i use Dynamic Prompts and i have to use batch size 1 due to this inherent limitation. Nothing we can do about it.

Regarding the differences when loras are not part of the ecuation, i don't know what is happening there, but i'm guessing that either the sampler or the optimizations used have some small randomization effect.

[vladmandic]

notes from @acorderob are correct.
regarding batch-size vs batch-count, it also impacts how random generator behaves. basically, seed is used to initialize random generator (that is the whole point of seed) for each image individually, but then order or random sequence pulled from generator may be different with different combinations of batch-size vs batch-count. 2x2 is not the same as 4x1 or 1x4.

[afbagwell]

Thanks, both of you!

[anothertal3]

I've stumbled upon this topic after falling into the same trap myself when using Dynamic Prompts + LoRAs and batches. I really hoped that there would be some sort of solution other than having to force batch to be 1. I'm currently doing the same but this does introduce a noteworthy overhead on my lower-end machine while searching for upscale-worthy images.

Just to clarify:

1. It doesn't matter whether you use 1x4 batches or 4x1 batches. The images will never be absolutely reproducible.
2. All images may be subject to such issues. It's not something like "the impact increases with batch size/count and 2 is ok-ish while 4 might be not".

Is my understanding correct? And there's really no way to mitigate the behaviour?

[vladmandic]

there is a way by fully reinitializing random generator for every batch count and disabling batch size (as random generator cannot be disabled inside batch inner loop) - doable, but non-trivial and likely to cause number of other issues. at the moment, i do not plan anything on this topic - but contributions are welcome.

[anothertal3]

Understood, thank you for the clarification.