Style LoRA Training guide for Stable diffusion 1.5 and SDXL Concepts Results and Conclusion

#stablediffusion #A1111 #AI #Lora #koyass #sd #sdxl #style #styletraining also see realistic Character training for SD 1.5 and SDXL https://youtu.be/vA2v2IugK6w you can download SD 1.5 model (regularized) from https://civitai.com/models/130320?modelVersionId=142935 This video is about taring a Style for stable diffusion 1.5 and SDXL with LoRA, its concepts, how it is different from object or character training in terms of data and training parameters, and how can we determine if our style is trained well or not, and what to expect from it. 00:00:00 introduction for style training includes results showcase 00:01:17 data preparation 00:02:16 folder preparation for Kohya training script this includes class name, instance name 00:03:50 captioning images 00:06:08 Kohya ss LoRA training settings for SD 1.5 without regularization 00:10:43 initial review of results 00:11:38 Kohya ss LoRA training settings for SD 1.5 without regularization images 00:12:50 SDXL LoRA training parameters 00:17:07 testing results and comparisons in A1111 show xyz comparisons, different prompts, LoRA models, regularization and without the principles explained here apply to any style regardless of what it was, art, clothing or anything else. we will also see difference between using regularization images and without and determine which option produced the better results. the style I am going to train is a simple black and white sketching style more like hand drawing, which is used in many illustrations. now this style like many is already learned by SDXL, and could be produced by SD 1.5 with the right prompts, but using a LoRA can make it easier and straight forward. Now regardless of how useful this style is, the principles are the same to create your own style LoRA. Conclusion: 1- to learn a style we must have large number of different images from different classes that only have the style in common. 2- 100 and up to 400 images are good number for style training, the more the better. 3- lower number of repeats is very important 1 and up to 4 depending on how many images you have, for 400, 1 or 2 is more than enough...1600 steps worked for simple style, a lot more could be required for complex styles, this is different from one dataset to another. 4- captioning must include everything except the style details, style details must be removed from captions. 5- regularization improves results of a style just like with characters. 6- simple styles don't need more than 32 network dimension for SD 1.5 or 16 for SDXL, complex styles could require a lot more. 7- SDXL contains too many styles already, unlikely that you need to train any new art style! 8- Noise value 0.0357 might be useful for SDXL training in advanced settings. 9- --network_train_unet_only for SDXL didnt improve results in this example, better to test with and without for each dataset 10- Regularization is strongly recommended to increase model flexibility and quality, better to test with and without and choose the better option 11- style is successful if it runs well on weight 1, and could run at higher weights 1 and up to 2, if it doesnt learn the training data too, only the style itself, and can mix with other LoRAs without corrupting their output. 12- styles will affect the output to a certain degree despite how light it is. other useful info about how stable diffusion works in general and some tips can be seen at https://youtu.be/1MkZB2YEiy4 Beginners guide to stable diffusion in Automatic1111 at: https://youtu.be/RtjDswbSEEY cloth/object training https://youtu.be/wJX4bBtDr9Y Koyass for lora training https://github.com/bmaltais/kohya_ss Class images, you can generate them from SD for example... I used these which you can download (800 style class images size 1024x1024 https://huggingface.co/datasets/AIHowto/460RegImages1024x1024/resolve/main/jpg%20style%20class%20images%201024x1024.7z ) another data set containing 1200 class images (768x768 is https://huggingface.co/datasets/AIHowto/460RegImages1024x1024/resolve/main/jpg%20768X768%20STYLE%20class%20images.7z) Computer Specs: Laptop: Legion 5 Pro Processor :AMD Ryzen 7 5800H , 3201 Mhz System RAM: 16.0 GB Graphics GPU: NVIDIA GeForce RTX 3070 Laptop GPU 8GB