A few questions about SDXL Lora Training

[walkingclark]

Hi holo, and guys,

Thanks greatly for responding to my posts. I didn't expect the response to be so soon.

I have a few questions about this SDXL Colab which often confuses me in different branches of this project. It would be very helpful if anyone could answer:

1. about install_dependencies:

Are the warnings or errors fatal to the results?
It usually involves the version of jax or opencv-python

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behavior is the source of the following dependency conflicts.
orbax-checkpoint 0.5.23 requires jax>=0.4.26, but you have jax 0.4.23 which is incompatible.

WARNING: The candidate selected for download or install is a yanked version: 'opencv-python' candidate (version 4.7.0.68 at https://files.pythonhosted.org/packages/07/e2/405fd71f433960b0ce6a546536b05a26d85508df7eea98850784e10323e9/opencv_python-4.7.0.68-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.org/simple/opencv-python/) (requires-python:>=3.6))
Reason for being yanked: Deprecated, use 4.7.0.71

Should I do anything in the future if I see these errors or warnings?

2. about Optimizers:
It is recommended to adopt AdamW8bit or Prodigy in the Colab. I wonder if there are suggested args for other optimizers. Just in case if I want to try them out some time to get away from issues or problems. Some said AdamW8bit may cause problems, I wonder if that's so.

3. about fp16/bp16
It is recommended to turn on "bp16" while using A100 GPU. However, in config_dict() > "saving_arguments" > "saving precision", it is written "fp16" statically. Should I change this accordingly?

4. The network_dim and network_alpha
It is recommended 8-4 in SDXL Colab, while it is recommended 16-8 to 32-16 in SD1.5. I wonder what are these numbers based on? Because I tried different numbers in SD1.5 (such as 16-4, 16-1, 64-32, etc.) and they work fine. But I don't know why it is limited to be under 32-32 in SDXL Colab.

5. How do I know if the Loras are "Full" or "Pruned"?

These may be silly questions, however, They troubled me for a long time and cost me tons of time and computing units. Some answers would be much appreciated.

[hollowstrawberry]

1. These two errors can be ignored. Keeping all the dependencies clean is simply a bit too hard with every update to google colab itself. As long as it works it's fine.
2. AdamW8bit is probably the way to go every time. Some people used to say it didn't work for XL, but that was debunked long ago.
3. Saving precision should always be fp16 for compatibility reasons with other graphics cards. bf16 can be beneficial during training however.
4. It affects the filesize. If I allowed 64 dim on SDXL they would be several hundred megabytes. This is unnecessary given that the complete model is "just" a few gigabytes. Basically, dim8 on SDXL can hold as much information as dim32 on SD1.
5. I think "pruned" is a concept only applied to checkpoints. It shouldn't matter anyway.

Hope that helps!

[walkingclark]

Great helps. I'll keep trying based on the information I learned. I'll try to avoid unnecessary attempts and experiments, cause it's really time consuming. Hopefully I'll get one SDXL right. You wouldn't know how many months I've been stuck with it. Thank you again for the reply.