i keep on getting voluptuous.error.MultipleInvalid: extra keys not allowed kept on deleting whatever the error shows it starts with bucketing i think so i removed it from toml it keeps on adding another error and now its saying voluptuous.error.MultipleInvalid: extra keys not allowed @ data['learning_rate']

[DeFaker]

                WARNING  ignore following options because config file is found: train_data_dir,    sdxl_train.py:146
                         in_json / 設定ファイルが利用されるため以下のオプションは無視されます:
                         train_data_dir, in_json
                ERROR    Invalid user config / ユーザ設定の形式が正しくないようです               config_util.py:367

Traceback (most recent call last):
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 950, in
train(args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 178, in train
blueprint = blueprint_generator.generate(user_config, args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 401, in generate
sanitized_user_config = self.sanitizer.sanitize_user_config(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 364, in sanitize_user_config
return self.user_config_validator(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 272, in call
return self._compiled([], data)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 595, in validate_dict
return base_validate(path, iteritems(data), out)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 433, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['cache_latents']
Traceback (most recent call last):
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE_main.py", line 7, in
sys.exit(main())
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\python.exe', 'C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py', '--config_file', 'C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041357.toml']' returned non-zero exit status 1.
04:14:18-652454 INFO Training has ended.
04:15:41-065450 INFO Start Finetuning...
04:15:41-068443 INFO Validating lr scheduler arguments...
04:15:41-072431 INFO Validating optimizer arguments...
04:15:41-074428 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.toml
existence... SUCCESS
04:15:41-076421 INFO Validating C:/Users/Jassi/Desktop/5mimg existence... SUCCESS
04:15:41-078415 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs existence and writability... SUCCESS
04:15:41-080410 INFO Validating stabilityai/stable-diffusion-xl-base-1.0 existence... SUCCESS
04:15:41-082405 INFO Dataset config toml file used, skipping caption json file, image buckets, total_steps,
train_batch_size, gradient_accumulation_steps, epoch, reg_factor, max_train_steps creation...
04:15:41-084399 INFO Max train steps: 1600
04:15:41-086394 INFO lr_warmup_steps = 0
04:15:41-090383 INFO Saving training config to C:/Users/Jassi/Desktop/kohya_ss/outputs\last_20250106-041541.json...
04:15:41-093375 INFO Executing command: C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE launch
--dynamo_backend no --dynamo_mode default --mixed_precision bf16 --num_processes 1
--num_machines 1 --num_cpu_threads_per_process 2
C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py --config_file
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041541.toml
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
2025-01-06 04:15:59 INFO Loading settings from train_util.py:4528
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041541.
toml...
INFO C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041541 train_util.py:4547
WARNING clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません sdxl_train_util.py:351
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: clean_up_tokenization_spaces was not set. It will be set to True by default. This behavior will be depracted in transformers v4.45, and will be then set to False by default. For more details check this issue: huggingface/transformers#31884
warnings.warn(
2025-01-06 04:16:01 INFO Load dataset config from sdxl_train.py:142
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.t
oml
WARNING ignore following options because config file is found: train_data_dir, sdxl_train.py:146
in_json / 設定ファイルが利用されるため以下のオプションは無視されます:
train_data_dir, in_json
ERROR Invalid user config / ユーザ設定の形式が正しくないようです config_util.py:367
Traceback (most recent call last):
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 950, in
train(args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 178, in train
blueprint = blueprint_generator.generate(user_config, args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 401, in generate
sanitized_user_config = self.sanitizer.sanitize_user_config(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 364, in sanitize_user_config
return self.user_config_validator(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 272, in call
return self._compiled([], data)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 595, in validate_dict
return base_validate(path, iteritems(data), out)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 433, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['gradient_accumulation_steps']
Traceback (most recent call last):
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE_main.py", line 7, in
sys.exit(main())
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\python.exe', 'C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py', '--config_file', 'C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041541.toml']' returned non-zero exit status 1.
04:16:03-633010 INFO Training has ended.
04:16:43-497119 INFO Start Finetuning...
04:16:43-499087 INFO Validating lr scheduler arguments...
04:16:43-502077 INFO Validating optimizer arguments...
04:16:43-504071 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.toml
existence... SUCCESS
04:16:43-506065 INFO Validating C:/Users/Jassi/Desktop/5mimg existence... SUCCESS
04:16:43-508061 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs existence and writability... SUCCESS
04:16:43-510056 INFO Validating stabilityai/stable-diffusion-xl-base-1.0 existence... SUCCESS
04:16:43-512050 INFO Dataset config toml file used, skipping caption json file, image buckets, total_steps,
train_batch_size, gradient_accumulation_steps, epoch, reg_factor, max_train_steps creation...
04:16:43-514044 INFO Max train steps: 1600
04:16:43-516039 INFO lr_warmup_steps = 0
04:16:43-520029 INFO Saving training config to C:/Users/Jassi/Desktop/kohya_ss/outputs\last_20250106-041643.json...
04:16:43-523021 INFO Executing command: C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE launch
--dynamo_backend no --dynamo_mode default --mixed_precision bf16 --num_processes 1
--num_machines 1 --num_cpu_threads_per_process 2
C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py --config_file
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041643.toml
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
2025-01-06 04:17:02 INFO Loading settings from train_util.py:4528
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041643.
toml...
INFO C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041643 train_util.py:4547
WARNING clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません sdxl_train_util.py:351
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: clean_up_tokenization_spaces was not set. It will be set to True by default. This behavior will be depracted in transformers v4.45, and will be then set to False by default. For more details check this issue: huggingface/transformers#31884
warnings.warn(
2025-01-06 04:17:02 INFO Load dataset config from sdxl_train.py:142
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.t
oml
WARNING ignore following options because config file is found: train_data_dir, sdxl_train.py:146
in_json / 設定ファイルが利用されるため以下のオプションは無視されます:
train_data_dir, in_json
ERROR Invalid user config / ユーザ設定の形式が正しくないようです config_util.py:367
Traceback (most recent call last):
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 950, in
train(args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 178, in train
blueprint = blueprint_generator.generate(user_config, args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 401, in generate
sanitized_user_config = self.sanitizer.sanitize_user_config(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 364, in sanitize_user_config
return self.user_config_validator(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 272, in call
return self._compiled([], data)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 595, in validate_dict
return base_validate(path, iteritems(data), out)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 433, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['learning_rate']
Traceback (most recent call last):
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE_main.py", line 7, in
sys.exit(main())
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\python.exe', 'C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py', '--config_file', 'C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-041643.toml']' returned non-zero exit status 1.
04:17:05-138303 INFO Training has ended.
04:23:02-340135 INFO Start Finetuning...
04:23:02-342129 INFO Validating lr scheduler arguments...
04:23:02-344124 INFO Validating optimizer arguments...
04:23:02-345121 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.toml
existence... SUCCESS
04:23:02-347117 INFO Validating C:/Users/Jassi/Desktop/5mimg existence... SUCCESS
04:23:02-349111 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs existence and writability... SUCCESS
04:23:02-351106 INFO Validating stabilityai/stable-diffusion-xl-base-1.0 existence... SUCCESS
04:23:02-353101 INFO Dataset config toml file used, skipping caption json file, image buckets, total_steps,
train_batch_size, gradient_accumulation_steps, epoch, reg_factor, max_train_steps creation...
04:23:02-355095 INFO Max train steps: 1600
04:23:02-357090 INFO lr_warmup_steps = 0
04:23:02-361079 INFO Saving training config to C:/Users/Jassi/Desktop/kohya_ss/outputs\last_20250106-042302.json...
04:23:02-364071 INFO Executing command: C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE launch
--dynamo_backend no --dynamo_mode default --mixed_precision bf16 --num_processes 1
--num_machines 1 --num_cpu_threads_per_process 2
C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py --config_file
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042302.toml
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
2025-01-06 04:23:22 INFO Loading settings from train_util.py:4528
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042302.
toml...
INFO C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042302 train_util.py:4547
WARNING clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません sdxl_train_util.py:351
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: clean_up_tokenization_spaces was not set. It will be set to True by default. This behavior will be depracted in transformers v4.45, and will be then set to False by default. For more details check this issue: huggingface/transformers#31884
warnings.warn(
2025-01-06 04:23:23 INFO Load dataset config from sdxl_train.py:142
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.t
oml
WARNING ignore following options because config file is found: train_data_dir, sdxl_train.py:146
in_json / 設定ファイルが利用されるため以下のオプションは無視されます:
train_data_dir, in_json
ERROR Invalid user config / ユーザ設定の形式が正しくないようです config_util.py:367
Traceback (most recent call last):
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 950, in
train(args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 178, in train
blueprint = blueprint_generator.generate(user_config, args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 401, in generate
sanitized_user_config = self.sanitizer.sanitize_user_config(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 364, in sanitize_user_config
return self.user_config_validator(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 272, in call
return self._compiled([], data)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 595, in validate_dict
return base_validate(path, iteritems(data), out)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 433, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['learning_rate']
Traceback (most recent call last):
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE_main.py", line 7, in
sys.exit(main())
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\python.exe', 'C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py', '--config_file', 'C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042302.toml']' returned non-zero exit status 1.
04:23:26-026761 INFO Training has ended.
04:23:44-547177 INFO Start Finetuning...
04:23:44-549211 INFO Validating lr scheduler arguments...
04:23:44-551195 INFO Validating optimizer arguments...
04:23:44-553189 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.toml
existence... SUCCESS
04:23:44-555157 INFO Validating C:/Users/Jassi/Desktop/5mimg existence... SUCCESS
04:23:44-557148 INFO Validating C:/Users/Jassi/Desktop/kohya_ss/outputs existence and writability... SUCCESS
04:23:44-559144 INFO Validating stabilityai/stable-diffusion-xl-base-1.0 existence... SUCCESS
04:23:44-561141 INFO Dataset config toml file used, skipping caption json file, image buckets, total_steps,
train_batch_size, gradient_accumulation_steps, epoch, reg_factor, max_train_steps creation...
04:23:44-563166 INFO Max train steps: 1600
04:23:44-565158 INFO lr_warmup_steps = 0
04:23:44-569117 INFO Saving training config to C:/Users/Jassi/Desktop/kohya_ss/outputs\last_20250106-042344.json...
04:23:44-572109 INFO Executing command: C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE launch
--dynamo_backend no --dynamo_mode default --mixed_precision bf16 --num_processes 1
--num_machines 1 --num_cpu_threads_per_process 2
C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py --config_file
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042344.toml
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
2025-01-06 04:25:07 INFO Loading settings from train_util.py:4528
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042344.
toml...
INFO C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042344 train_util.py:4547
WARNING clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません sdxl_train_util.py:351
C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: clean_up_tokenization_spaces was not set. It will be set to True by default. This behavior will be depracted in transformers v4.45, and will be then set to False by default. For more details check this issue: huggingface/transformers#31884
warnings.warn(
2025-01-06 04:25:09 INFO Load dataset config from sdxl_train.py:142
C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.t
oml
WARNING ignore following options because config file is found: train_data_dir, sdxl_train.py:146
in_json / 設定ファイルが利用されるため以下のオプションは無視されます:
train_data_dir, in_json
ERROR Invalid user config / ユーザ設定の形式が正しくないようです config_util.py:367
Traceback (most recent call last):
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 950, in
train(args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\sdxl_train.py", line 178, in train
blueprint = blueprint_generator.generate(user_config, args)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 401, in generate
sanitized_user_config = self.sanitizer.sanitize_user_config(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\sd-scripts\library\config_util.py", line 364, in sanitize_user_config
return self.user_config_validator(user_config)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 205, in call
return self._compiled([], data)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 549, in validate_dict
return base_validate(path, data.items(), out)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\voluptuous\schema_builder.py", line 382, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['learning_rate']
Traceback (most recent call last):
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Jassi\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\accelerate.EXE_main.py", line 7, in
sys.exit(main())
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\Jassi\Desktop\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\Jassi\Desktop\kohya_ss\venv\Scripts\python.exe', 'C:/Users/Jassi/Desktop/kohya_ss/sd-scripts/sdxl_train.py', '--config_file', 'C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-042344.toml']' returned non-zero exit status 1.
04:25:11-662845 INFO Training has ended.

[Sylsatra]

Please provide your config file!

[DeFaker]

Please provide your config file!

{
"adaptive_noise_scale": 0.00375,
"additional_parameters": "",
"ae": "",
"apply_t5_attn_mask": false,
"async_upload": false,
"batch_size": "4",
"block_lr": "",
"blocks_to_swap": 0,
"blockwise_fused_optimizers": false,
"bucket_no_upscale": true,
"bucket_reso_steps": 64,
"cache_latents": true,
"cache_latents_to_disk": false,
"caption_dropout_every_n_epochs": 0,
"caption_dropout_rate": 0,
"caption_extension": ".txt",
"caption_metadata_filename": "meta_cap.json",
"clip_g": "",
"clip_l": 0,
"clip_skip": 1,
"color_aug": false,
"cpu_offload_checkpointing": false,
"dataset_config": "C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.toml",
"dataset_repeats": "1",
"debiased_estimation_loss": false,
"disable_mmap_load_safetensors": false,
"discrete_flow_shift": 3,
"double_blocks_to_swap": 0,
"dynamo_backend": "no",
"dynamo_mode": "default",
"dynamo_use_dynamic": false,
"dynamo_use_fullgraph": false,
"epoch": 240,
"extra_accelerate_launch_args": "",
"flip_aug": false,
"flux1_cache_text_encoder_outputs": false,
"flux1_cache_text_encoder_outputs_to_disk": false,
"flux1_checkbox": false,
"flux1_clip_l": "",
"flux1_t5xxl": "",
"flux_fused_backward_pass": false,
"fp8_base": false,
"full_bf16": false,
"full_fp16": false,
"full_path": true,
"fused_backward_pass": false,
"fused_optimizer_groups": 0,
"generate_caption_database": false,
"generate_image_buckets": false,
"gpu_ids": "",
"gradient_accumulation_steps": 6,
"gradient_checkpointing": true,
"guidance_scale": 3.5,
"huber_c": 0.1,
"huber_scale": 1,
"huber_schedule": "snr",
"huggingface_path_in_repo": "",
"huggingface_repo_id": "",
"huggingface_repo_type": "",
"huggingface_repo_visibility": "",
"huggingface_token": "",
"image_folder": "C:/Users/Jassi/Desktop/5mimg",
"ip_noise_gamma": 0,
"ip_noise_gamma_random_strength": false,
"keep_tokens": 0,
"latent_metadata_filename": "meta_lat.json",
"learning_rate": 5e-05,
"learning_rate_te": 1e-05,
"learning_rate_te1": 1e-05,
"learning_rate_te2": 1e-05,
"log_config": false,
"log_tracker_config": "",
"log_tracker_name": "",
"log_with": "",
"logging_dir": "",
"logit_mean": 1,
"logit_std": 1.29,
"loss_type": "l2",
"lr_scheduler": "constant",
"lr_scheduler_args": "",
"lr_scheduler_type": "",
"lr_warmup": 0,
"lr_warmup_steps": 0,
"main_process_port": 0,
"masked_loss": false,
"max_bucket_reso": "1024",
"max_data_loader_n_workers": 0,
"max_resolution": "1024,1024",
"max_timestep": 900,
"max_token_length": 75,
"max_train_epochs": 240,
"max_train_steps": 1600,
"mem_eff_attn": false,
"mem_eff_save": false,
"metadata_author": "",
"metadata_description": "",
"metadata_license": "",
"metadata_tags": "",
"metadata_title": "",
"min_bucket_reso": "1024",
"min_snr_gamma": 5,
"min_timestep": 100,
"mixed_precision": "bf16",
"mode_scale": false,
"model_list": "custom",
"model_prediction_type": "sigma_scaled",
"multi_gpu": false,
"multires_noise_discount": 0,
"multires_noise_iterations": 0,
"noise_offset": 0.0375,
"noise_offset_random_strength": false,
"noise_offset_type": "Original",
"num_cpu_threads_per_process": 2,
"num_machines": 1,
"num_processes": 1,
"optimizer": "PagedAdamW8bit",
"optimizer_args": "",
"output_dir": "C:/Users/Jassi/Desktop/kohya_ss/outputs",
"output_name": "last",
"persistent_data_loader_workers": false,
"pretrained_model_name_or_path": "stabilityai/stable-diffusion-xl-base-1.0",
"random_crop": false,
"resume": "",
"resume_from_huggingface": "",
"sample_every_n_epochs": 0,
"sample_every_n_steps": 0,
"sample_prompts": "",
"sample_sampler": "euler_a",
"save_clip": false,
"save_every_n_epochs": 240,
"save_every_n_steps": 0,
"save_last_n_epochs": 0,
"save_last_n_epochs_state": 0,
"save_last_n_steps": 0,
"save_last_n_steps_state": 0,
"save_model_as": "safetensors",
"save_precision": "fp16",
"save_state": false,
"save_state_on_train_end": false,
"save_state_to_huggingface": false,
"save_t5xxl": "",
"scale_v_pred_loss_like_noise_pred": false,
"sd3_cache_text_encoder_outputs": false,
"sd3_cache_text_encoder_outputs_to_disk": false,
"sd3_checkbox": false,
"sd3_fused_backward_pass": "",
"sd3_text_encoder_batch_size": false,
"sdxl_cache_text_encoder_outputs": true,
"sdxl_checkbox": true,
"sdxl_no_half_vae": true,
"seed": 1234,
"shuffle_caption": false,
"single_blocks_to_swap": 0,
"skip_cache_check": false,
"split_mode": false,
"t5xxl": "",
"t5xxl_device": "bf16",
"t5xxl_dtype": 1,
"t5xxl_max_token_length": 512,
"timestep_sampling": "sigma",
"train_batch_size": 2,
"train_blocks": "all",
"train_dir": "",
"train_text_encoder": false,
"use_latent_files": "No",
"v2": false,
"v_parameterization": false,
"v_pred_like_loss": 0,
"vae_batch_size": 0,
"wandb_api_key": "",
"wandb_run_name": "",
"weighted_captions": false,
"weighting_scheme": "logit_normal",
"xformers": "xformers"
}

[DeFaker]

Please provide your config file!

if possible can you share a config and a version that is working for sdxl fine tuning

[DeFaker]

adaptive_noise_scale = 0.00375
bucket_no_upscale = true
bucket_reso_steps = 64
cache_latents = true
cache_text_encoder_outputs = true
caption_extension = ".txt"
clip_skip = 1
dataset_config = "C:/Users/Jassi/Desktop/kohya_ss/outputs/config_finetune-20250106-040314.toml"
dataset_repeats = 1
dynamo_backend = "no"
enable_bucket = true
gradient_accumulation_steps = 6
gradient_checkpointing = true
huber_c = 0.1
huber_scale = 1
huber_schedule = "snr"
in_json = "/meta_cap.json"
learning_rate = 5e-5
learning_rate_te1 = 1e-5
learning_rate_te2 = 1e-5
loss_type = "l2"
lr_scheduler = "constant"
lr_scheduler_args = []
max_bucket_reso = 1024
max_data_loader_n_workers = 0
max_timestep = 900
max_token_length = 75
max_train_epochs = 240
max_train_steps = 1600
min_bucket_reso = 1024
min_snr_gamma = 5
min_timestep = 100
mixed_precision = "bf16"
no_half_vae = true
noise_offset = 0.0375
noise_offset_type = "Original"
optimizer_args = []
optimizer_type = "PagedAdamW8bit"
output_dir = "C:/Users/Jassi/Desktop/kohya_ss/outputs"
output_name = "last"
pretrained_model_name_or_path = "stabilityai/stable-diffusion-xl-base-1.0"
resolution = "1024,1024"
sample_prompts = "C:/Users/Jassi/Desktop/kohya_ss/outputs\sample/prompt.txt"
sample_sampler = "euler_a"
save_every_n_epochs = 240
save_model_as = "safetensors"
save_precision = "fp16"
seed = 1234
train_batch_size = 2
train_data_dir = "C:/Users/Jassi/Desktop/5mimg"
wandb_run_name = "last"
xformers = true

[Sylsatra]

Please provide your config file!

if possible can you share a config and a version that is working for sdxl fine tuning

Sorry i only do flux for now!

[Sylsatra]

Your config is very confusing so I have a few questions to ask before helping you.

1. What method do you want to train with? Dreambooth, Lora, or Finetuning? For SDXL, I recommend Lora training in the Lora tab.
   Read some guides about SDXL Lora training on Civit.ai if you want to train a Lora. If you still need help, feel free to contact me.
2. Which GitHub branch are you using? main, sd3-flux,...
   Since you are training with SDXL, don't use the sd3-flux branch. It is a bit too advanced to start with. I suggest using the main branch.
3. Where did you get this config? It is very weird.
4. What is your system spec? What is your GPU(eg Nvidia4060)? How much RAM do you have? Because your config is using a lot of computing power.

[DeFaker]

not lora im trying to do full finetune not dreambooth or lora , ill be renting 8xh100 also im using sd3-flux,1 branch but i kept getting errors on runpod so i tried to do it locally first to debug issues since on windows i hit the same errors as well , appreciate your help tho :)

[Sylsatra]

not lora im trying to do full finetune not dreambooth or lora , ill be renting 8xh100 also im using sd3-flux,1 branch but i kept getting errors on runpod so i tried to do it locally first to debug issues since on windows i hit the same errors as well , appreciate your help tho :)

I suggest you train using dreambooth method instead of finetuning (dreambooth is better for now)