Stable WarpFusion v0.23 - dwpose, controlnet preview (Patreon)

Local install guide: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv.

A100 or 24 GB card is highly recommended for sdxl.
Make a new env and grab a fresh install.bat from that repo.

Changelog:

• add dw pose estimator from https://github.com/IDEA-Research/DWPose
• add onnxruntime-gpu install, (update env for dw_pose)
• add dw_pose model downloader
• add controlnet preview, kudos to #kytr.ai, idea - https://discord.com/channels/973802253204996116/1124468917671309352
• add temporalnet sdxl - v1 (3-channel)
• add prores thanks to #sozeditit https://discord.com/channels/973802253204996116/1149027955998195742
• make width_height accept 1 number to resize the frame to that size keeping the aspect ratio
• add cc masked template for content-aware scheduling
• add reverse frames extraction
• move looped image to video init cell as video source mode
• fix settings not being loaded via the button
• fix bug when cc_masked_diffusion == 0
• add a message on missing audio during video export / mute exception
• go back to root dir after running rife

18.09.2023, 0.23.6:

• add gdown install

19.09.2023, 0.23.8:

• add deflicker for win
• add experimental deflicker from https://video.stackexchange.com/questions/23384/remove-flickering-due-to-artificial-light-with-ffmpeg
• fix linear and none blend modes for video export
• detect init_video fps and pass it down to video export with respect to the nth frame
• do not reload already loaded controlnets
• rename upscaler model path variable
• make mask use image folder correctly as a mask source

19.09.2023, 0.23.10:

• fix gui for non controlnet-modes (getattr error in gui)
• fix video_out not defined

26.09.2023

• moved to L tier

6.10.2023, v0.23.11:

• fix dwpose 'final_boxes' error for frames with no people

14.10.2023, v0.23.12:

• fix xformers version

14.10.2023, v0.23.13:

• fix flow preview error for less than 10 frames

16.10.2023, v0.23.15

• fix pillow errors (UnidentifiedImageError: cannot identify image file)
• fix timm import error (isDirectory error)
• deprecate v2_depth model (use depth controlnet instead)

20.10.2023, v0.23.16:

• fix pytorch dependencies error
• fix zoe depth error
• move installers to github repo

11.11.2023:

• move to M

DW Pose

to select dw_pose, go to gui -> controlnet -> pose_detector -> dw_pose

Download new install.bat or install via !pip install onnxruntime-gpu gdown manually

Width_height max size

You can now specify a single number as width_height settings, and it will define the max size of the frame, fitting the output inside that size, keeping the aspect ratio. For example, if you have a 1920x1080 video, width_height=1280 will downscale that video to 1280x720

Controlnet Preview

Added controlnet annotations (detections) preview. Thanks to #kytr.ai for the idea.

To enable, check only_preview_controlnet in Do the Run cell. It will take the 1st frame in frame_range from your video and generate controlnet previews for it.

Prores

add prores codec thanks to #sozeditit https://discord.com/channels/973802253204996116/1149027955998195742

Create video cell -> output_format -> prores_mov

It has better quality than h264_mp4, and a smaller size than qtrle_mov

Video Init Update

Moved looped image init to video init settings. To use init image as video init, select video_source -> looped_init_image

Added reverse option

Extract frames in reverse. For example, if a person moves into the frame, it's easier to reverse the video so that the video starts with the person inside of the frame.

Detect fps

fps will now be detected from your init_video, divided by extract nth frame value and used in your video export if, you set your video export fps to -1.

Example: your video has 24fps. you extract every 2nd frame. The suggested output fps will be 24/2 = 12fps. If you set your video export fps to -1, it will use predicted fps = 12.

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Guides made by users:

• 05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ
• 11.05.2023, v0.11 How to use Stable Warp Fusion
• 13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration
• 14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking
• 23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install
• 15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)
• 17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation
• 21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion
• 21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial
• 04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)
• 15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!

YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

For tech support and other questions please join our discord server:
https://discord.gg/YrpJRgVcax

Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Error reports in comments will be deleted and reposted in Discord.