All The LoRA FLUX Training Experiments I Have Done So Far (Patreon)

I have been using a machine on RunPod that has 8x RTX A6000 over a week now. I have done all the following experiments with each one being 3000 steps.

I will hopefully update this post as I do more.

All findings, grids, and best configs and 1-click installers for Windows, RunPod and Massed Compute shared here : https://www.patreon.com/posts/kohya-flux-lora-110293257

You can see saved safetensors files in below repos. Click load more button at the end to see entire repo files

https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research/tree/main

https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part2/tree/main

https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part3/tree/main

https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part4/tree/main

All Tested Experiments List So Far

1) 16GB_Config.safetensors

2) 1e_04_bf16_128_rank-000170.safetensors

3) 1e_04_bf16_128_rank_full_bf16.safetensors

4) 1e_04_fp16_128_rank-000180.safetensors

5) 1e_04_fp16_128_rank_full_fp16-000180.safetensors

6) 1e_04_fp8_32_rank-000170.safetensors

7) 1e_04_fp8_bf16_accelerate_32_rank-000160.safetensors

8) 1e_04_fp8_bf16_accelerate_full_bf16_32_rank-000160.safetensors

9) 1e_04_fp8_full_fp16_32_rank-000170.safetensors

10) 2e_04_fp8_bf16_accelerate_full_bf16_32_rank.safetensors

11) 3e_04_fp8_bf16_accelerate_full_bf16_32_rank.safetensors

12) 6e_05_best_additive_sigma.safetensors

13) 6e_05_best_additive_uniform.safetensors

14) 6e_05_best_max_grad_norm_0.safetensors

15) 6e_05_best_raw_sigma.safetensors

16) 6e_05_best_raw_sigmoid.safetensors

17) 6e_05_best_sigma_scaled_sigma.safetensors

18) 6e_05_best_with_reg_images.safetensors

19) 6e_05_bf16_128_rank_full_bf16.safetensors

20) 7e_05_bf16_128_rank_full_bf16.safetensors

21) 8e_05_bf16_128_rank_full_bf16.safetensors

22) 8e_05_fp8_bf16_accelerate_full_bf16_32_rank.safetensors

23) 9e_05_bf16_128_rank_full_bf16.safetensors

24) 9e_05_fp8_bf16_accelerate_full_bf16_32_rank.safetensors

25) Best_v1.safetensors

26) Best_v1_3e_5.safetensors

27) Best_v1_4e_5.safetensors

28) Best_v1_512px.safetensors

29) Best_v1_512px_Lowest_VRAM.safetensors

30) Best_v1_5e_5.safetensors

31) Best_v1_5e_5_max_grad_norm_0.safetensors

32) Best_v1_Lowest_VRAM.safetensors

33) Best_v1_T5_attention_mask.safetensors

34) Best_v1_cfg_3_5.safetensors

35) Best_v1_max_grad_norm_0.safetensors

36) Best_v1_reg_0_1_percent.safetensors

37) Best_v1_reg_prior_lost_0_3.safetensors

38) Best_v1_reg_prior_lost_0_6.safetensors

39) Best_v1_reg_prior_lost_1_0_apply_t5_attention_mask.safetensors

40) Best_v2.safetensors

41) Best_v2_1280.safetensors

42) Best_v2_1280_256rank.safetensors

43) Best_v2_1536.safetensors

44) Best_v2_1536_256rank.safetensors

45) Best_v2_FP8.safetensors

46) Best_v2_FP8_LowVRAM.safetensors

47) Best_v2_Rank16.safetensors

48) Best_v2_Rank256.safetensors

49) Best_v2_Rank32.safetensors

50) Best_v2_Rank4.safetensors

51) Best_v2_Rank64.safetensors

52) Best_v2_Rank8.safetensors

53) Best_v2_Split_QKV.safetensors

54) Best_v2_joycaption.safetensors

55) Best_v2_joycaption_Split_QKV.safetensors

56) CivitAI_FLUX.safetensors

57) Best_v2_shift_6e_05

58) Best_v2_shift_4e_05

59) Best_v2_shift_3e_05

60) Best_v2_shift_2e_05

61) Best_v2_shift_1e_05

62) Best_v2_shift_9e_06

63) Best_v2_only_OHWX

64) Best_v2_shift_value_1