Comprehensive Different Version and Precision FLUX Models Speed and VRAM Usage Comparison (Patreon)

I just updated the automatic FLUX models downloader scripts with newest models and features. Therefore I decided to test all models comprehensively with respected to their peak VRAM usage and also their image generation speed.

Automatic downloader scripts : https://www.patreon.com/posts/109289967

 Patreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.

Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord

Please also Star, Watch and Fork our Stable Diffusion & Generative AI  GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)

Testing Results

• All tests are made with 1024x1024 pixels generation, CFG 1, no negative prompt

• All tests are made with latest version of SwarmUI (0.9.2.1)

• These results are not VRAM optimized - fully loaded into VRAM and thus maximum speed

• All VRAM usages are peak which happens when finally decoding with VAE after all steps completed

• Below tests are on A6000 GPU on massed Compute with FP8 T5 text encoder - default

• Full tutorial for how to use locally (on your PC on Windows) and on Massed Compute (31 cents per hour for A6000 GPU) is at below

• SwarmUI full public tutorial : https://youtu.be/bupRePUOA18

Testing Methodology

• Tests are made on a cloud machine thus VRAM usages were below 30 mb before starting SwarmUI

• nvitop library is used to monitor VRAM usages during generation and peak VRAM usages recorded which usually happens when VAE decoding image after all steps completed

• SwarmUI reported timings are used

• First generation never counted, always multiple times generated and last one used

Below Tests are Made With Default FP8 T5 Text Encoder

flux1-schnell_fp8_v2_unet 

• Turbo model FP 8 weights (model only 11.9 GB file size)

• 19.33 GB VRAM usage - 8 steps - 8 seconds

flux1-schnell 

• Turbo model FP 16 weights (model only 23.8 GB file size)

• Runs at FP8 precision automatically in Swarm UI

• 19.33 GB VRAM usage - 8 steps - 7.9 seconds

flux1-schnell-bnb-nf4 

• Turbo 4bit model - reduced quality but VRAM usage too

• Model + Text Encoder + VAE : 11.5 GB file size

• 13.87 GB VRAM usage - 8 steps - 7.8 seconds

flux1-dev

• Dev model - Best quality we have

• FP 16 weights - model only 23.8 GB file size

• Runs at FP8 automatically in Swarm UI

• 19.33 GB VRAM usage - 30 steps - 28.2 seconds

flux1-dev-fp8

• Dev model - Best quality we have

• FP 8 weights (model only 11.9 GB file size)

• 19.33 GB VRAM usage - 30 steps - 28 seconds

flux1-dev-bnb-nf4-v2

• Dev model - 4 bit model - slightly reduced quality but VRAM usage too

• Model + Text Encoder + VAE : 12 GB file size

• 14.40 GB - 30 steps - 27.25 seconds

FLUX.1-schnell-dev-merged

• Dev + Turbo (schnell) model merged

• FP 16 weights - model only 23.8 GB file size

• Mixed quality - Requires 8 steps

• Runs at FP8 automatically in Swarm UI

• 19.33 GB - 8 steps - 7.92 seconds

Below Tests are Made With Default FP16 T5 Text Encoder

• FP16 Text Encoder slightly improves quality and also increases VRAM usage

• Below tests are on A6000 GPU on massed Compute with FP16 T5 text encoder - If you overwrite previously downloaded FP8 T5 text encoder (automatically downloaded) please restart SwarmUI to be sure

• Don't forget to select Preferred DType to set FP16 precision - shown in tutorial : https://youtu.be/bupRePUOA18

• Currently BNB 4bit models are ignoring FP16 Text Encoder and using embedded FP8 T5 text encoders

flux1-schnell_fp8_v2_unet

• Model running at FP8 but Text Encoder is FP16

• Turbo model : 23.32 GB VRAM usage - 8 steps - 7.85 seconds

flux1-schnell

• Turbo model - DType set to FP16 manually so running at FP16

• 34.31 GB VRAM - 8 steps - 7.39 seconds

flux1-dev

• Dev model - Best quality we have

• DType set to FP16 manually so running at FP16

• 34.41 GB VRAM usage - 30 steps - 25.95 seconds

flux1-dev-fp8

• Dev model - Best quality we have

• Model running at FP8 but Text Encoder is FP16

• 23.38 GB - 30 steps - 27.92 seconds

My Suggestions and Conclusions

• If you have a GPU that has 24 GB VRAM use flux1-dev-fp8 and 30 steps

• If you have a GPU that has 16 GB VRAM use flux1-dev-bnb-nf4-v2 and 30 steps

• If you have a 12 GB VRAM or below GPU use flux1-dev-bnb-nf4-v2 - 30 steps

• If it becomes too long to generate images due to your low VRAM, use flux1-schnell-bnb-nf4 and 4 to 8 steps depending on speed and duration that you can wait

• FP16 Text Encoder slightly increases quality so 24 GB GPU owners can also use FP16 Text Encoder + FP8 models

• SwarmUI is currently able to run FLUX as low as 4 GB GPUs with all kind of optimizations (fully automatic). I even saw someone generated image with 3 GB GPU

• I am looking for BNB NF4 version of FLUX.1-schnell-dev-merged model for low VRAM users but couldn't find yet

• Hopefully I will update auto downloaders once I got 4bit version of merged model