Home
Artists
Search
Recent
Random
Posts
DMs
Tags
Random
Importer
Import
FAQ
Account
Register
Favorites
Login
Stable WarpFusion v0.9 Nightly - MultiControlNet - Changelog (Patreon)
Content
Sort of a disclaimer: don't dive headfirst into a nightly build if you're planning to use it for your currect project which is already past its deadline - you'll have a bad day :D
ATTENTION! MULTI-CONTROLNET (with at seast 2 controlnets) REQUIRES 16GB OF VRAM @ 720p, IF YOU ARE RUNNING LOW ON VRAM, BE PREPARED TO USE GOOGLE COLAB HOSTED ENV OR TRY RUNNIng AT YOUR OWN RISK :D
Happy International Women's Day!
No better way to celebrate it than by warping! :D
Notebook download - https://www.patreon.com/posts/79679221
Changelog:
- add MultiControlNet
- add MultiControlNet autodownloader/loader
- add MultiControlNet order, weight, start/end steps, internal/external mode
- add MultiControlNet/Annotator cpu offload mode
- add empty negative image condition
- add softcap image range scaler
- update model guidance fn
MultiControlNet
Added MultiControlNet.
Consider using multicontrolnet by default even if you only need 1 model. It adds more options, that are otherwise unavailable in non-multi controlnet mode, like weight or controlnet start/end steps.
How to use:
Init Settings
Got to Load up a stable -> define SD + K functions, load model -> model_version -> control_multi
use_small_controlnet - True
small_controlnet_model_path - leave empty
download_control_model - True
force_download - Enable if some files appearto be corrupt, disable if everything is ok.
You can then specify a path to your cusom v1.x checkpoint, and pick one or more controlnet models via checkboxes below. Those models will be downloaded if they are not available locally. You can redefine the list and order of controlnet models later in stable-settings cell.
Controlnets and their annotators will be automatically loaded, unloaded, downloaded, etc. depending on your settings in stable-settings cell.
Runtime settings
controlnet_multimodel_mode
External or internal. Internal - sums controlnet output values before feeding those into diffusion model, external - sum outputs of one controlnet conditioned diffusion model External seems slower but smoother, uses less VRAM
External mode:
controlnet1 -> diffusion -> output1
controlnet2 -> diffusion -> output2
weighted sum(output1 + output2) -> final result
Internal mode:
weighted sum(controlnet1 + controlnet2) -> diffusion -> final result
controlnet_multimodel settings
This is a dictionary containig a list of controlnet models. Order doesn't really matter, as their results are summed up.
Format example:
controlnet_multimodel = {
"control_sd15_depth":{
"weight":1,
"start":0,
"end":0.8
},
"control_sd15_canny":
{
"weight":0,
"start":0,
"end":1
}
}
weight (only available in internal mode) - weight of the model predictions in the output
start - % of total steps at which this controlnet begins working
end - % of total steps at which this controlnet stops working
This way you can: limit effect of certain models, mix more controlnets than you can fit into VRAM by making sure only a limited numer of models runs at a given step.
- controlnet steps are counted relatively to total steps
- You have 50 steps, 0.3 style strength. Controlnet start 0.2 end 0.8 will run from step 10 to step 40, overlapping with the actual steps taken at steps 35-40
- [||||||||||] 50 steps
[-------|||] 0.3 style strength (effective steps - 0.3x50 = 15)
[--||||||--] - controlnet working range with start = 0.2 and end = 0.8, effective steps from 0.2x50 = 10 to 0.8x50 = 40
Empty negative image condition
img_zero_uncond
By default image conditioned models use same image for negative conditioning, like if you specified the same text in both positive and negative prompt. (i.e. both positive and negative image conditionings are the same)
You can use empty negative condition by enabling this
Softcap image range scaler
do_softcap
Softly clamp latent excessive values. Reduces feedback loop effect a bit
softcap_thresh
Scale down absolute values above that threshold (latents are being clamped at [-1:1] range, so 0.9 will downscale values above 0.9 to fit into that range, [-1.5:1.5] will be scaled to [-1:1], but only absolute values over 0.9 will be affected)
softcap_q
Percentile to downscale. 1-downscale full range with outliers, 0.9 - downscale only 90% values above thresh, clamp remaining 10%)
Notebook download - https://www.patreon.com/posts/79679221
A reminder:
Changes in GUI will not be saved into the notebook, but if you run it with new settings, they will be saved to a settings.txt file as usual.
You can load settings in misc tab.
You do not need to rerun the GUI cell after changing its settings.
Local install guide:
https://discord.com/channels/973802253204996116/1067887206125015153/1067888756910215278
https://github.com/Sxela/WarpFusion/blob/main/README.md
Youtube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq
For tech support and other questions please join our discord server:
https://discord.gg/sayE6j2sdP
Discord is the preferred method, because it is nearly impossible to provide any decent help or tech support via Patreon due its limited text formatting and inability to add screenshots or videos to comments or DMs.
Error reports in comments will be deleted and reposted in discord.
