Lora doesn't look like the character I want

[ShinobiiSpartan]

I'm having issues trying to get my Lora of a character to look like the character fully, as well as get the same clothes the character wears. In terms of the prompts, I'm using all the correct ones and I'm training with 200+ images so I know I must be doing something wrong with the settings. If someone can help I'd really appreciate it as I'm still fairly new to all this. Thank you.

[ShinobiiSpartan]

Also I'm not sure if this is the right spot to write this, if that's the case, that's my bad🙏

[hollowstrawberry]

Hey, I moved this to Discussions.

Could you share your training settings?

[ShinobiiSpartan]

First off thank you. Here are the exact settings.

Images: 227
resolution: 512
flip_aug: off
shuffle_tags: enabled
activation_tags: 1
num_repeats: 2
epochs: 10
train_batch_size: 2
unet_lr:4e-4
text_encoder_lr: 1e-2
lr_scheduler: cosine_with_restarts
lr_scheduler_number: 3
lr_warmup_ratio: 0.05
network_dim: 24
network_alpha: 12
conv_dim: 8
conv_alpha: 4

[hollowstrawberry]

The text encoder lr is too high. Try 8e-5 (a fifth of the unet).

That might solve it. You may also share an image from your dataset and its corresponding tags, so I can have a look.

Also, what model are you using?

[ShinobiiSpartan]

Sorry for my late reply, I will definitely try the text encoder lr change, thank you very much. Here's an example of one my dataset images and my tags would be:
hood, hoodie, layered sleeves, long sleeves, short over long sleeves, hood down, short sleeves, purple shirt, shirt, shorts, toeless footwear, boots, black footwear, brown shorts

The model I'm using is Anylora

[ShinobiiSpartan]

The text encoder lr is too high. Try 8e-5 (a fifth of the unet).

I tried this text encoder but still, the lora doesn't look as close to the original character. Is there a "perfect" setting where you can get an almost 1:1 look of a character from the Lora training?

[ArmyOfPun1776]

maybe try training for longer? like maybe try going for 15-20 Epochs... I don't do anime, but my Realist models always come out almost 1:1. I just have to adjust the weight of the LoRa during gen.

Have you thought about maybe refining and reducing your dataset as well? more doesn't always mean better I've found.

[ShinobiiSpartan]

Hm will most definitely be trying what you are saying and good to know that more images doesn't equal better. I'll get back to you, thank you.

[ShinobiiSpartan]

maybe try training for longer? like maybe try going for 15-20 Epochs... I don't do anime, but my Realist models always come out almost 1:1. I just have to adjust the weight of the LoRa during gen.

Have you thought about maybe refining and reducing your dataset as well? more doesn't always mean better I've found.

I just tried it and it worked!! I appreciate you and hollowstrawberry a lot. What I did was shorten the dataset to pictures that were only showing my character with the same look as her original self and I upped the epochs to 15 as you suggested. Thanks again🙏