[Issue]: Inpaint with FluxFillPipeline fails with "Input is in incorrect format." #3699
Open
2 tasks done
Labels
cannot reproduce
Reported issue cannot be easily reproducible
question
Further information is requested
Comments
vladmandic
added
backlog
|
i cant reproduce - how are you loading flux fill model?
|
vladmandic
added
question
|
I have seen the same error trying to use flux fill. Selected the Flux fill model from the model drop-down list (I know it's not selected in the screenshot right now, just illustrating)
|
|
you cannot select flux fill from dropdown as its NOT a base model, you need to select it from scripts so its properly initialized - that's why I've posted my screenshot! |
I was going to assume you would say that (and it makes sense), so I've tested it and it blows out of VRAM (Nvidia 4090) |
|
yes, its a horribly inefficient model - but it works fine if you enable a) balanced offload, b) on-the-fly quantization |
|
@vladmandic Not sure what I'm going wrong, but now getting other error. Do I understand correctly that I select Flux.1 Dev as Base model and enable Flux Tools and select Fill?
|
|
yes, that is correct, but please simplify - is torch compile relevant for this issue? if it works without compile, then this issue is about torch.compile compatibility with flux-fill. and if it doesn't work with compile, well, reproduce with clean debug log without compile. also, using flux without offloading is really expensive, not sure why you have it disabled. and stablefast is not a flux thing. |
|
any updates here? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue Description
Trying to use inpaint or img2img with FluxFillPipeline but it fails with error.
I simply upload an image into inpaint, mask area, add prompt and try to generate
Version Platform Description
Relevant log output
00:19:09-209237 INFO Base: pipeline=FluxFillPipeline task=TEXT_2_IMAGE batch=1/1x4 set={'guidance_scale': 6, 'generator': 'cuda:[750230034, 750230035, 750230036, 750230037]', 'num_inference_steps': 20, 'output_type': 'latent', 'width': 1264, 'height': 1688, 'parser': 'native', 'prompt': 'embeds'} 00:19:09-219323 ERROR Processing: args={'prompt_embeds': 'cuda:0:torch.bfloat16:torch.Size([4, 512, 4096])', 'pooled_prompt_embeds': 'cuda:0:torch.bfloat16:torch.Size([4, 768])', 'guidance_scale': 6, 'generator': [<torch._C.Generator object at 0x7f6d5f1d1230>, <torch._C.Generator object at 0x7f6d5f1d2750>, <torch._C.Generator object at 0x7f6d5f1d0ff0>, <torch._C.Generator object at 0x7f6d5f173570>], 'callback_on_step_end': <function diffusers_callback at 0x7f6e297a60c0>, 'callback_on_step_end_tensor_inputs': ['latents'], 'num_inference_steps': 20, 'output_type': 'latent', 'width': 1264, 'height': 1688} Input is in incorrect format. Currently, we only support <class 'PIL.Image.Image'>, <class 'numpy.ndarray'>, <class 'torch.Tensor'> 00:19:09-786059 INFO Processed: images=0 its=0.00 time=0.64 timers={'gc': 0.56, 'process': 0.29, 'post': 0.28, 'init': 0.05} memory={'ram': {'used': 5.18, 'total': 30.91}, 'gpu': {'used': 31.97, 'total': 44.52}, 'retries': 1, 'oom': 1}Backend
Diffusers
UI
Standard
Branch
Master
Model
FLUX.1
Acknowledgements
The text was updated successfully, but these errors were encountered: