Stable WarpFusion v0.13 Nightly - New consistency algo, Reference CN (changelog) (Patreon)

Sort of a disclaimer: don't dive headfirst into a nightly build if you're planning to use it for your currect project, which is already past its deadline - you'll have a bad day. This is not a production-ready user-friendly software :D

Make sure you've tried awesome free solutions like Deforum or TemporalKit before subscribing and submitting yourself to the warp.

ATTENTION! 16GB VRAM NVIDIA GPU is recommended for local run, or colab pro with a high ram env. You may be able to run it on a smaller amount of VRAM (8Gb being the minimum), but I haven't tested it and can not confirm how low you'll have to go with your settings, so please join discord and ask around before subscribing.

A first step at rewriting the 2015's consistency algo.

Download: https://www.patreon.com/posts/83087458

Changelog:

• add alternative consistency algo (also faster)
• auto skip install for our docker env
• clean some discodiffusion legacy code (it's been a year :D)
• add controlnet default main model (v1.5)

upd 21.05.2023:

• add reference controlner (attention injection)
• add reference mode and source image
• skip flow preview generation if it fails
• downgrade to torch v1.13 for colab hosted env
• save schedules to settings before applying templates
• keep pre-template settings in the GUI
• add gui options to load settings, keep state on rerun/load from previous cells
• fix schedules not kept on gui rerun
• rename depth_source to cond_image_src to reflect it's actual purpose

Upd 22.05.2023 @ v0.13.10:

• fix outer not defined error for reference
• remove torch downgrade for colab
• remove xformers for torch v2/colab
• add sdp attention from AUTOMATIC1111 to replace xformers (for torch v2)

Upd 24.05.2023 @ v0.13.11:

• fix reference controlnet infinite recursion loop

Upd 24.05.2023 @ v0.13.12:

• fix prompt schedules not working with "0"-like keys

Upd 26.05.2023: available as a nightly build

New consistency algorithm

The new algo is cleaner and should reduce missed consistency mask replated flicker

Consistency is now calculated simultaneously with the flow.

use_legacy_cc:
The alternative consistency algo is on by default. To revert to the older algo, check use_legacy_cc in Generate optical flow and consistency maps cell.

missed_consistency_dilation:
Missed consistency mask "width". 1 - default value

edge_consistency_width:
Edge consistency width. Odd numbers only, default  = 11

Reference controlnet (aka attention injection)

by By Lvmin Zhang (https://github.com/lllyasviel)

https://github.com/Mikubill/sd-webui-controlnet

Added attention injection. You can mix attention data from your reference image and the one that's being generated. Runs 2x slower as it basically samples 2 images in parallel (the stylized and reference).

Works with any model, not only controlnet multi, as it's just a hack on attention layers. We still caall in controlnet to honor its author's naming decision.

Reference controlnet (attention injection) ->

use_reference: Check to enable

reference_weight: strength of reference image vs current image

reference_source: source of reference image. Options: ['None', 'stylized', 'init', 'prev_frame','color_video']
None - off
stylized - use current input image
prev_frame - previously stylized frame
init - raw video frame
color_video - frame from color video

reference_mode:
Options: ['Balanced', 'Controlnet', 'Prompt']
Defines what should affect the result more, prompt or reference.

Notebook download:
https://www.patreon.com/posts/83087458

Local install guide:
https://discord.com/channels/973802253204996116/1067887206125015153/1067888756910215278
https://github.com/Sxela/WarpFusion/blob/main/README.md

Tutorials:
https://youtu.be/HkM-7wxtkGA
https://www.youtube.com/watch?v=FxRTEILPCQQ
https://www.youtube.com/watch?v=wqXy_r_9qw8
https://www.youtube.com/watch?v=VMF7L0czyIg
https://www.youtube.com/watch?v=m8xaPnaooyg

Youtube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

For tech support and other questions please join our discord server

Discord is the preferred method, because it is nearly impossible to provide any decent help or tech support via Patreon due its limited text formatting and inability to add screenshots or videos to comments or DMs.
Error reports in comments will be deleted and reposted in discord.