Home
Artists
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register
Partychan Telegram Find&Fuck Waifus ThePornDude
Home Artists Posts Import Register

The Very Best Kohya GUI Workflow & Config For SD 1.5 Based Models DreamBooth / Full Fine Tuning (Patreon)

Published:
2024-01-28 22:58:44
Imported:

Downloads

Content

Patreon exclusive posts index

Join discord and tell me your discord username to get a special rank : SECourses Discord

I have made over 70 full DreamBooth trainings for over 7 days and meticulously analyzed their results to the find very best training hyper parameters.

120 amazing quality images with their prompt info posted on CivitAI

We have 3 configs. 

Tier 1 is best quality. Don't use xFormers.

Tier 2 is second best quality. Uses xFormers to reduce VRAM.

xFormers : reduces VRAM, increases speed, reduced quality

Gradient Checkpointing : reduces VRAM, reduces speed, quality sam

All Tier 2 are equal quality and only speed and VRAM usage changes.

Since OneTrainer supports EMA, it is better than Kohya.

OneTrainer config : https://www.patreon.com/posts/97381002

You can download configs:

tier_1_quality_slow_22GB.json

tier_2_lowVRAM_10_2GB.json

tier_2_speed_11_8GB.json

There are 2 strategies of training. Stylized vs Realism.

To find out very best models for both realism and stylization models, I have made 161 models comparison recently if you remember : https://youtu.be/G-oZn4H-aHQ

Models Downloader Script And The Patreon Post Shown In The Video ⤵️ https://www.patreon.com/posts/1-click-download-96666744

1st:

Training for realism. For this training strategy I have chosen the Hyper Realism V3 model from CivitAI. The config file will download it automatically from Hugging Face or alternatively you can give the local path.

2nd:

Training for stylization like 3d render of yourself. For this task I have chosen RealCartoon-Pixar V8 from CivitAI.

To use this model the key change you need to make is, making Clip skip 2 in Advanced Settings.

I used 15 training images and trained 150 repeat 1 epoch.

My used training images are as below (they are at best medium quality)


For RealCartoon-Pixar V8 hopefully I will add Regularization images to this post soon. 

For realism, use our very best real unsplash collected regularization images ⤵️

https://www.patreon.com/posts/massive-4k-woman-87700469

I trained both 768x768 and 1024x1024. 768x768 training works better than 1024x1024. Moreover, generating 1024x1024 works better than 768x768. When fixing faces with ADetailer extension, make the ADetailer extension resolution 768x768 even if you generate images in 1024x1024.

If you don't know how to load configs and use here a tutorial : https://youtu.be/EEV8RPohsbw



Comments

Anonymous

Question, are the reg images/traning images aspect ratio important/considered on this config? Should I crop my input images to be square or leave them in any resolution, thanks!

Furkan Gözükara
>>128224557

I suggest to crop all images into 768x768 for SD 1.5 including reg images. if you are using our reg images we already have that resolution

Anonymous
>>128224557

Thanks! a further question, im really new at kohya, where do I set the repeats? For reference ive got 191 images

Furkan Gözükara
>>128224557

please watch this tutorial gui settings part. where I set training dataset folders : https://youtu.be/16-b1AjvyBE - 13:23 Which parameters you need to set on Kohya GUI before starting training

Anonymous

Noob question how do I run the scripts?

Furkan Gözükara
>>128291063

hello here you can see i have shown : https://youtu.be/EEV8RPohsbw and if you want learn more about kohya here https://youtu.be/sBFGitIvD2A

Anonymous

no buckets??? 🤯

Furkan Gözükara
>>128387164

Well i prefer to use all same resolution but of course you can try bucketing too. i would compare though bucket vs no bucket

Arcon Septim

when you used hyper realism, what checkpoint you used for training? fp16 or fp32? also what vae it uses?

Anonymous

something wrong with the first preset tier_1_quality_slow_22GB.json, took me 2 minutes to finish with an rtx 4090 24 gb from runpod and also when I loaded the config, in the training parameters name it's was the name of the third .json

Steve Bruno

Any thoughts on Prodigy over Adafactor? Been reading a lot more stuff from Civitai creators none of the YT "influencers" are pushing it.

Furkan Gözükara
>>128933081

I tested it and I didnt get better results. I think it depends on other parameters too. So if you have a pre-shared config you can compare

Art

apart from the EMA stuff, this is replicable on Kaggle Kohya notebook to train loras?

Furkan Gözükara
>>129101576

Ah you say Kaggle. SD 1.5 is almost same as in Kaggle but not SDXL. SDXL requires BF16 for better performance. or FP32

Anonymous

I have a question in regards to the "tier_2_speed_11_8GB.json" file. The default LR is 7e-7. I have 24 decent images, no captions and using an img folder name of "75_oshw woman". I also have your reg set for the woman 768x768 being used. When I train, the sample images never ever converge into a likeness of what I'm training on (like normally it would). I think the training rate is way too low, or in your config are you doing something closer to 15,000 repeats? Normally the LR is something between 1e4 and 1e5 so I just want to make sure. I'm running still in kohya as I've only got a 3070 8GB and none of the OneTrainer settings gets me below 10GB.

Furkan Gözükara
>>129441898

actually i trained my model with only 15 images and 150 epoch for this config. so when reg images included it was total 15 * 150 * 2 = 4500 steps. so with 24 images 75 repeating and 1 epoch might not be sufficient. try 150 repeat and it may even become overtrained

Anonymous

Thank you for that advice, I'll go give it a try. I had noticed the epoch was set to the value 1 in the config file and had thought maybe it was one of those setups where the highest quality was running at 1 epoch and setting the steps high by way of repeats (and then just saving every 500 steps). I'll go give that a try.

Furkan Gözükara
>>129449697

well epoch doesnt mean much since kohya utilized repeating logic. so as you increase repeat you are actually increasing epochs in reality. more repeating better since it uses more reg images. so 8 epoch 40 repeat vs 1 epoch 320 repeat, 320 repeat better

Anonymous

Thank you again for the clarification. So in your first example to my above question with 15 images and 150 epoch, that would be a good setup for a run without reg images. Are you setting the epoch high in that scenario to make it easier to setup more checkpoint saves? Even with some of my settings off, your base setup is starting to work for me. It's been an obsession of mine for months with some hit and miss results. Your work here is wonderful.

Khoa Vo

What do you use as the prompt for the dataset? Is it something like 150_ohwx_man?

Anonymous
>>129461540

if you were doing the prompt you are mentioning, in Kohya_ss it would be 150_ohwx man Notice the space after ohwx. The had confused me in the beginning. The images folder in kohya points to something like this: \YourAITrainingDirectory\Subject\Images But the actual images of your topic live one folder deeper: \YourAITrainingDirectory\Subject\Images\150_ohwx man

Furkan Gözükara
>>129461540

true it is 150_ohwx man. there is no _ between ohwx and man. so we are training with ohwx rare token and man class token

Anonymous

I hate to bug you again. I've tried all sorts of ways to get your "tier_2_speed_11_8GB.json" file to work, with the settings recommended. I've already got some good success with other methods, so took a decent dataset where the reality of likeness is around 90% and I used it with the settings. I am training on an Irish woman, the folder path repeats and token being "1_owsh woman" and I'm doing 150 epochs. There are 21 decent quality images. It should way over train with that number. Instead what is happening is during the training, in the sampling (with a prompt of "photo of a owsh woman") I sample every 50 steps and see the system showing me a sea wall, a harbour, then a car and eventually settles onto an older asian woman which has no likeness. I can confirm the folder paths are good, as I can change the LR from 7e-07 to 1e-05 and start to see a decent likeness. I'm using the custom model you recommend in the settings and more importantly the learning rate which you have set to 7e-07. I've tried a variety of image sets that I had prepared, no captions, just the folder tags. To fix it, I just changed the LR to 1e-05 for example. I am raising this more because I think its possible the base example has the LR value set wrong and was going to ask you to look at it as I suspect if you do a training at the exact base settings from the json (with of course modifications for the repeats/epoch combination you choose), you'll find it way undertrains. Unless 7e-7 is for very large datasets? Or do I have to set epochs towards 1000? Something is wrong with the config. It occurs to me now to ask. Maybe I'm using the configs in the wrong location. I'm loading them in kohya_ss in the LoRA tab, are these configs for fine tuning or other? I noticed some learning rate differences between the: tier_2_speed_11_8GB.json and tier_2_lowVRAM_10_2GB.json file, namely in the lowVRAM there are additional training settings that are missing from the tier_2_speed_11_8GB.json such as : "learning_rate_te": 7e-07, "learning_rate_te1": 3e-06, "learning_rate_te2": 0.0, I know the setup us using Adafactor as the optimizer, but the LR Scheduler is setup as "constant" instead of allowing the Adafactor to optimize the LR. Is this intentional? I really liked the results you have in your 6 pages of showing what you can do and am doing my earnest to make your config work. Thanks for any further advice. I'll try a number of settings and report back what I find.

Anonymous
>>129496333

The "tier_2_lowVRAM_10_2GB.json" is the config I grabbed from this page in relation to the SD 1.5 Koyha best settings training. I am training against SD1.5 as I don't have enough memory to train SDXL locally yet. If you do a quick check on the "tier_2_lowVRAM_10_2GB.json" file you'll notice the LR is "7e-07" and the Unet and TE are 0. I'm not sure if its intentional, but just raising the question as I suspect the values might be set too low. It's not the SDXL config, but the SD1.5 one. Specifically this setting from the json above is what I'm inquiring about: "learning_rate": 7e-07 Am I correct in that I load your above config into the LoRA tab of Koha_ss, or this intended for the fine tuning tab?

Furkan Gözükara
>>129496333

this is Kohya config and I just loaded and all set to 7e-7 accurately . this is not LoRA this is DreamBooth that is your mistake.

Anonymous
>>129496333

Sorry, I just re-read the title of this wiki. It is for Dreambooth training and not LoRA training. I'm switching over to loading the config in dreambooth tab and trying again, sorry for the trouble. Even in the lowmem dreambooth, I can't get below 10 GB, which means a 17 hour train. I guess runpod will be the option. Did you ever find that you could get the Dreambooth training in your low mem config as low as 8 GB (or slightly lower?)

Furkan Gözükara
>>129496333

sadly not yet. therefore i suggest you to use Kaggle notebook. it is free 30 hours training every week. sd 1.5 works best settings there but of course runpod would be easiest and for SDXL runpod is much better. but for SD 1.5 kaggle works same quality as runpod https://www.patreon.com/posts/kohya-sdxl-lora-88397937

Anonymous

Do captions make a difference with this setup? I already have a well captioned dataset which gives me good results with LoRA training but I want to see if Dreambooth Fine tuning works better. Should I include my captions or leave them out?

Furkan Gözükara
>>129690025

i would compare this. use captions vs non caption only ohwx man or woman. and see which one performs better

Anonymous

there are so many video tutorials on your yourtube channel. Which one should I follow to use your .json file SD 1.5 lora. And how to get best output form automatic1111?

Anonymous
>>130630690

yeah it honestly is confusing with the amount of videos there are

Furkan Gözükara
>>130630690

you guys are right. because of that I prepared this article (1 week ago). please check it out and let me know if anything is still not clear : https://www.patreon.com/posts/full-workflow-sd-98620163

Furkan Gözükara
>>130630690

you guys are right. because of that I prepared this article (1 week ago). please check it out and let me know if anything is still not clear : https://www.patreon.com/posts/full-workflow-sd-98620163

Furkan Gözükara
>>130630690

you guys are right. because of that I prepared this article (1 week ago). please check it out and let me know if anything is still not clear : https://www.patreon.com/posts/full-workflow-sd-98620163

saint-mares gérard

maybe i'm wrong but when i'm loading tier1 config it uses xformers and model output name is completed by tier2

Furkan Gözükara
>>130821331

hello just fixed the folder naming issue. also you are loading as dreambooth tab right? xformers not selected i verified

Ella Bang

Says that the configs include the models hyper realism v3 to be downloaded from huggingface but it seems to be missing.

Furkan Gözükara
>>131664013

actually it downloads it is auto set. i uploaded it here as diffusers : https://huggingface.co/MonsterMMORPG/sd15_best_realism

Dmitrii

hyperrealism is 2GB size, but my trained output model is 4GB. Why is that? how to make output 2GB as well?

Profile Photo

What number of epochs did you input? 150 epochs ? Because I chose 150 epochs with 15 photos and I have 72 hours of execution which is too much

Furkan Gözükara
>>133954429

that means it is using shared vram. you must be doing something inaccurate. are you using regularization images?

min min

amazing work!can you touch us how to make the reg datasets?what's the thought!

saint-mares gérard

In your opinion, is there a good way to train online today ? I used colab but all kohya scripts are outdated