Stable WarpFusion v0.20 - controlnet xl (Patreon)

Local install guide: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv.

Make a new env and grab a fresh install.bat from that repo.

A100 or 24gb card is highly recommended.

Managed to make diffusers ControlNets work thanks to ComfyUI loader.
Really memory-intensive stuff, 16 GB can barely fit 2cns at 1280x720. Probably a memory leak somewhere will have to test it later.

Changelog:

v0.20.2, 18.08.2023:

• add sdxl diffusers controlnet loader from ComfyUI
• add sdxl controlnets
• save annotators to controlnet folder
• hide sdxl model load spam
• fix sdxl tiled vae errors (still gives black output on sdxl with vanilla vae)
• fix cc_masked_diffusion not being loaded from gui

v0.20.3, 19.08.2023:

• fix rec noise for control multi sdxl mode
• fix control mode

v0.20.4, 20.08.2023

• add cell execution check (kudos to #soze)
• add skip diffusion switch to generate video only (kudos to #soze)
• add error msg when creating video from 0 frames
• fix control_multi annotator folder error

v0.20.5, 21.08.2023:

• add control-lora loader from ComfyUI
• add stability ai control-loras: depth, softedge, canny
• refactor controlnet code a bit

v0.20.6, 21.08.2023:

• fix tiled vae for sdxl
• stop on black frames, print a comprehensive error message

v0.20.7, 22.08.2023:

• add controlnet tile to controlnet_multi (v1.x not sdxl)
• remove single controlnets from model versions
• fix guidance for controlnet_multi (now work with much higher clamp_max)

v0.20.8, 22.08.2023:

• fix instructpix2pix not working in newer versions (kudos to #stabbyrobot)
• fix AttributeError: 'OpenAIWrapper' object has no attribute 'get_dtype' error

v0.20.10, 23.08.2023:

• fix clip import error

v0.20.11, 24.08.2023:

• fix ModuleNotFound: safetensors error
• update comfyui repo

v0.20.12, 24.08.2023:

• fix cv2.error ssize.empty error for face controlnet

v0.20.13, 27.08.2023:

• temporarily disable reference for sdxl
• move to L tier

v0.20.14, 27.08.2023:

• fix flow preview

v0.20.15 2.09.2023

• fix gitpull error

v0.20.16, 3.9.2023

• add dummy model init for sdxl - won't download unnecessary stuff. For this to work drop control_multi_sdxl.dummypkl into warp root folder. Note that visually results will differ.

v0.20.17, 5.9.2023:

• disable dummy init for non-sdxl, which load relatively fast

26.09.2023

• move to M tier

14.10.2023, v0.20.18:

• fix xformers version

16.10.2023, v0.20.19:

• fix UnidentifiedImageError

20.10.2023, v0.20.20:

• fix pytorch dependencies error
• fix zoe depth error
• move installers to github repo

SDXL - Controlnets are downloaded automatically from huggingface. For now, 5 are supported:

"control_sdxl_canny":"https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0/resolve/main/diffusion_pytorch_model.fp16.safetensors",

"control_sdxl_depth":"https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0/resolve/main/diffusion_pytorch_model.fp16.safetensors",

"control_sdxl_softedge":"https://huggingface.co/SargeZT/controlnet-sd-xl-1.0-softedge-dexined/resolve/main/controlnet-sd-xl-1.0-softedge-dexined.safetensors",

"control_sdxl_seg":"https://huggingface.co/SargeZT/sdxl-controlnet-seg/resolve/main/diffusion_pytorch_model.bin",

"control_sdxl_openpose":"https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0/resolve/main/OpenPoseXL2.safetensors"

Also added depth, canny, and sketch Stability-ai control-loras from here: https://huggingface.co/stabilityai/control-lora

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Tutorials:
https://youtu.be/HkM-7wxtkGA
https://www.youtube.com/watch?v=FxRTEILPCQQ
https://www.youtube.com/watch?v=wqXy_r_9qw8
https://www.youtube.com/watch?v=VMF7L0czyIg
https://www.youtube.com/watch?v=m8xaPnaooyg

Youtube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

For tech support and other questions please join our discord server:
https://discord.gg/YrpJRgVcax

Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Error reports in comments will be deleted and reposted in Discord.