Adding new v7

.gitignore

@@ -1 +1,2 @@
-venv
+venv
+mytraining.ps

README.md

@@ -77,7 +77,7 @@ my_sks_dog_dreambooth
 Edit and paste the following in a Powershell terminal:
 
 ```powershell
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
+accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v7.py `
     --pretrained_model_name_or_path="D:\models\last.ckpt" `
     --train_data_dir="D:\dreambooth\train_bernard\train_man" `
     --reg_data_dir="D:\dreambooth\train_bernard\reg_man" `
@@ -99,12 +99,12 @@ accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
 If you would rather use model finetuning rather than the dreambooth method you can use a command similat to the following. The advantage of fine tuning is that you do not need to worry about regularization images... but you need to provide captions for every images. The caption will be used to train the model. You can use auto1111 to preprocess your training images and add either BLIP or danbooru captions to them. You then need to edit those to add the name of the model and correct any wrong description.
 
 ```
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6-ber.py `
-    --pretrained_model_name_or_path="D:\models\v1-5-pruned-mse-vae.ckpt" `
+accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v7-ber.py `
+    --pretrained_model_name_or_path="D:\models\alexandrine_teissier_and_bernard_maltais-400-kohya-sd15-v1.ckpt" `
     --train_data_dir="D:\dreambooth\source\alet_et_bernard\landscape-pp" `
     --output_dir="D:\dreambooth\train_alex_and_bernard" `
     --resolution="640,448" `
-    --train_batch_size=8 `
+    --train_batch_size=1 `
     --learning_rate=1e-6 `
     --max_train_steps=550 `
     --use_8bit_adam `
@@ -113,9 +113,13 @@ accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6-ber.py `
     --cache_latents `
     --save_every_n_epochs=1 `
     --fine_tuning `
-    --fine_tuning_repeat=200 `
+    --dataset_repeats=200 `
     --seed=23 `
     --save_half
 ```
 
 Refer to this url for more details about finetuning: https://note.com/kohya_ss/n/n1269f1e1a54e
+
+## Change history
+
+* 11/7 (v7): Text Encoder supports checkpoint files in different storage formats (it is converted at the time of import, so export will be in normal format). Changed the average value of EPOCH loss to output to the screen. Added a function to save epoch and global step in checkpoint in SD format (add values if there is existing data). The reg_data_dir option is enabled during fine tuning (fine tuning while mixing regularized images). Added dataset_repeats option that is valid for fine tuning (specified when the number of teacher images is small and the epoch is extremely short).

train.ps

@@ -1,106 +0,0 @@
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
-    --pretrained_model_name_or_path="D:\models\v1-5-pruned.ckpt" `
-    --train_data_dir="D:\dreambooth\train_bernard\train_man" `
-    --reg_data_dir="D:\dreambooth\train_bernard\reg_man" `
-    --output_dir="D:\dreambooth\train_bernard" `
-    --prior_loss_weight=1.0 `
-    --resolution="512,512" `
-    --train_batch_size=1 `
-    --learning_rate=1e-6 `
-    --max_train_steps=3000 `
-    --use_8bit_adam `
-    --xformers `
-    --mixed_precision="fp16" `
-    --cache_latents `
-    --gradient_checkpointing `
-    --save_every_n_epochs=1 
-
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
-    --pretrained_model_name_or_path="D:\models\bernard\asd man-3000-remgb-sd15.ckpt" `
-    --train_data_dir="D:\dreambooth\train_bernard\train_man" `
-    --reg_data_dir="D:\dreambooth\train_bernard\reg_man" `
-    --output_dir="D:\dreambooth\train_bernard" `
-    --prior_loss_weight=1.0 `
-    --resolution="512,512" `
-    --train_batch_size=1 `
-    --learning_rate=1e-6 `
-    --max_train_steps=1500 `
-    --use_8bit_adam `
-    --xformers `
-    --mixed_precision="fp16" `
-    --cache_latents `
-    --gradient_checkpointing `
-    --save_every_n_epochs=1
-
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
-    --pretrained_model_name_or_path="D:\models\v1-5-pruned-mse-vae.ckpt" `
-    --train_data_dir="D:\dreambooth\train_bernard\train_man" `
-    --reg_data_dir="D:\dreambooth\train_bernard\reg_man" `
-    --output_dir="D:\dreambooth\train_bernard" `
-    --prior_loss_weight=1.0 `
-    --resolution="512,512" `
-    --train_batch_size=1 `
-    --learning_rate=1e-6 `
-    --max_train_steps=4500 `
-    --use_8bit_adam `
-    --xformers `
-    --mixed_precision="fp16" `
-    --cache_latents `
-    --gradient_checkpointing `
-    --no_token_padding `
-    --save_every_n_epochs=1
-    
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
-    --pretrained_model_name_or_path="D:\models\v1-5-pruned-mse-vae.ckpt" `
-    --train_data_dir="D:\dreambooth\source\alex\train" `
-    --output_dir="D:\dreambooth\train_alex" `
-    --prior_loss_weight=1.0 `
-    --resolution="448,640" `
-    --train_batch_size=8 `
-    --learning_rate=1e-6 `
-    --max_train_steps=4500 `
-    --use_8bit_adam `
-    --xformers `
-    --mixed_precision="fp16" `
-    --cache_latents `
-    --save_every_n_epochs=1 `
-    --shuffle_caption
-
-# -fine_tuning
-
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6.py `
-    --pretrained_model_name_or_path="D:\models\v1-5-pruned-mse-vae.ckpt" `
-    --train_data_dir="D:\dreambooth\source\alex\train\50_portrait-pp" `
-    --output_dir="D:\dreambooth\train_alex" `
-    --resolution="448,640" `
-    --train_batch_size=8 `
-    --learning_rate=1e-6 `
-    --max_train_steps=4500 `
-    --use_8bit_adam `
-    --xformers `
-    --mixed_precision="fp16" `
-    --cache_latents `
-    --save_every_n_epochs=1 `
-    --fine_tuning `
-    --shuffle_caption
-    
-Resume:
-
-accelerate launch --num_cpu_threads_per_process 6 train_db_fixed_v6-ber.py `
-    --pretrained_model_name_or_path="D:\models\v1-5-pruned-mse-vae.ckpt" `
-    --train_data_dir="D:\dreambooth\source\alet_et_bernard\landscape-pp" `
-    --output_dir="D:\dreambooth\train_alex_and_bernard" `
-    --resolution="640,448" `
-    --train_batch_size=8 `
-    --learning_rate=1e-6 `
-    --max_train_steps=550 `
-    --use_8bit_adam `
-    --xformers `
-    --mixed_precision="fp16" `
-    --cache_latents `
-    --save_every_n_epochs=1 `
-    --fine_tuning `
-    --fine_tuning_repeat=200 `
-    --seed=23 `
-    --save_half
-    

train_db_fixed_v6-ber.py

@@ -2,6 +2,9 @@
 # The license of this script, like train_dreambooth.py, is Apache License 2.0
 # (c) 2022 Kohya S. @kohya_ss
 
+# v7: another text encoder ckpt format, average loss, save epochs/global steps, show num of train/reg images, 
+#     enable reg images in fine-tuning, add dataset_repeats option
+
 from torch.autograd.function import Function
 import argparse
 import glob