Second pass high-res fix not working? Force Hires is required to use high-res fix?

[weirdlighthouse]

Without forced hires, I see no difference in image quality other than an increase in resolution. is this is how Second pass supposed to work now?

normal image

after Second pass

[vladmandic]

because hires was originally intended to work with latent upscalers only.
then people wanted to start using 3rd party upscalers.
then people wanted to start using 3rd party upscalers AND hires.
ok, no issue. but then you should select force hires, otherwise there is no possibility to run just upscaler.
and if you want to set it enabled as default, do so in ui defaults.

[anothertal3]

If I understand correctly second pass is currently meant to work as follows:

• Latent upscaler
  • Always uses hires fix and will upscale in latent space
• Non-latent upscaler (e.g., Ultrasharp) and no "Force Hires"
  • Will only execute an upscaling call using the given upscaler and target resolution. Nothing else is done afterwards.
  • This is probably identical to simply executing an upscaling inside "Process Image"?
• Non-latent upscaler (e.g., Ultrasharp) and "Force Hires":
  • Will first execute an upscaling step identical to the previous item using the given upscaler
  • Will then take the resulting, upscaled image and execute an "img2img"-operation for it using the hires denoise and steps and resizing set to 1 (or none)
  • In summary, this is more a quality-of-life improvement and the result should be identical to upscaling via "Process Image" and executing "img2img" afterwards manually?

I'm uncertain, though, if those assumptions are correct.

Due to VRAM limitations I recently wanted to establish a semi-automatic batch-workflow. I therefore batch-upscaled all images via the "Process Image" tab. Afterwards I processed the batch via "img2img". Alas, in my experiments this would always return waaay more blurry and soft images compared to using the same settings directly with "Force Hires Fix".

So, could someone please verify or correct this understanding?

[vladmandic]

yes, no issues with those assumptions. hires fix was never more than that to start with, people thought it was something different than basically img2img pass after initial txt2txt pass.
regarding blurry/soft images, its most likely due to params - with hires, it uses params specified in second pass. with explicit img2img it uses params specified in primary pass (as thats the only pass).

[anothertal3]

Thanks a ton for the quick response and confirmation.

Sorry to bother you further but could you please elaborate on the parameters issue?

Let's say I've generated an image using the following parameters:

• Prompt / Negative Prompt are set
• Seed
• Sampler: DPM++ 2M Karras
• Resolution: 512x768
• Steps: 20
• CFG: 7
• Full Quality: on
• Tiling: off
• Face restoration: off
• (No changes to advanced tab)
• 2nd pass:
  • Force hires: on
  • Secondary Sampler: DPM++ 2M Karras
  • Hires steps: 13
  • Upscaler: Ultrasharp
  • Denoise: 0.35
  • Upscale: 2
  • 2nd prompt / 2nd negative prompt: nothing, because hidden

-> This would result in a nice looking picture A_hires.

Now I repeat the same procedure but omit everything regarding 2nd pass:

1. This would result in the image B_original which is a smaller version of A_hires.
2. I would then process (upscale) B_original using the following settings which results in B_upscaled:

• Upscaler: Ultrasharp
• Upscale: 2
• GAN/... : Off

3. Now I switch to img2img and use the following settings:

• Source Image: B_upscaled
• Prompt: same as B_original
• Negative Prompt: same as B_original
• Seed: same as B_original
• Denoise: 0.35 (same as hires attempt)
• CFG: 7 (same as hires attempt)
• Resize: None (B_upscaled is already upscaled)
• Sampler: DPM++ 2M Karras (same as hires attempt)

This would result in B_final.

It would now make a lot of sense to me if A_hires and B_final would be very similar and of equal quality. But as I've mentioned before: B_final will usually appear way more blurry.

Are those all the parameters you were talking about or is there more that I need to be aware of?

LoRAs probably shouldn't change the outcome if they are used in the exact same fashion during all related prompts, I guess?

[vladmandic]

yes, that's how it should work (you're missing steps param).
i'll check on the quality issue, just really short on time right now.

[anothertal3]

Right: "steps = 13" during img2img.

No worries, you're doing an impressive job and it's insane that you find the time to respond that quickly and helpful at all!

Now that I know it should in fact work, I will be happy to take another look myself once more and try to figure out if there's something I'm still missing.

My GPU is on the lower end (6 GB VRAM) so I'm trying to find an efficient workflow for "try to generate good stuff" (quite quick) and afterwards batch-resize the best (veeery slow, ~8-10 min/picture). As hires fix doesn't offer batch-support for known images/seeds (to my knowledge) the latter way using a two-step process of upscaling and then batch-img2img seemed like a reasonable approach when combined with this script AUTOMATIC1111/stable-diffusion-webui#4938 (comment) in order to set prompts/settings per image.