Lycoris Pretrained Text Encoder?

[TheOnlyHolyMoly]

Hey, just stumbled across this... https://civitai.com/models/83622?modelVersionId=88870

Can anyone help me understand how it is supposed to work?

[vladmandic]

interesting concept.

well, unlike lora, lyco can be used to replace (nearly) any layer in the model, so it seems that this is basically a model fine-tune (although not a small fine-tune) and then he simply chose to extract text-encoder weights from that model and package it as lyco so it can be loaded on top of any other model.

btw, more common use of the "extract" capability is to create lyco out of dreambooth so you can have small lyco instead of carrying full model around, but no reason why you couldn't extract anything.

[TheOnlyHolyMoly]

is that similar to what we did with the dict-on-top approach?

[vladmandic]

dict method is non-selective, it loads one model and then loads another on top of it, so if second model has any non-initialized parts, it doesn't get initialized with blanks and instead you get bleed-through.

[TheOnlyHolyMoly]

understood, thank you Vlad