[AI] SVD with AnimateDiff Refiner - More Details (Patreon)

Stable Video Diffusion plus AnimateDiff Refiner Beta Release

*Green Nodes are Input Nodes,

*Purple Nodes are Image Scaling Nodes

Use SVD as usual, This workflow add animate diff refiner pass, if you used SVD for refiner, the results were not good and If you used Normal SD models for refiner, they would be flickering. So AnimateDiff is used Instead.

HOW TO USE :

SVD Setup

• Checkpoint- SVD for 14 Frames | SVD_XT for 25 Frames. https://comfyanonymous.github.io/ComfyUI_examples/video

• SVD FPS - 6-8 is good

• SVD Motion Bucket - Under 100 For portraits, 100 - 200 for Others. Test till 500 ( In the Initial Still Image if there elements captured in Motion like, fire, snow, smoke etc - Use 80 - 180)

• SVD Video Frames - 14 Frames for SVD | 25 For SVD_XT

• SVD Augmentation Level - Add Extra noise in SVD, 0.1-0.3 seems stable. You can Test other values.

• Global Seed - Control All Sampler's Seed, Changing this also changes the motion.

• Load Starting Image - Choose the Image you want to Experiment with

• Upscale Image By - Rescale as Desired. See SVD Height and Width Input Checks and keep them Under 1k resolution

• Enable Custom Resolution - 0 For Disabled | 1 for Enabled

• SVD Custom H and W - Enter your custom height and width.

• Refiner UpScale - Use 1.2 - 1.5 for best detailed result. (Keep Above 1k) See Refiner Resolution Checks and Keep above 1k for sharp and fine result.

• Refiner Denoise - Use 0.4 - 0.6 for best result.

AnimteDiff Setup

• Load AD Checkpoint - Load Model according to your Picture - Realistic, Anime, Cartoon. Try Experimenting, Epic Realism Model was seen to give best result in Realistic Photos.

• Lora Stack - Load your desired Loras. Artifacts like blue lines are seen when using loras

• Prompts - Change prompts according to the picture you see, and loras.

Tips:

SVD
- When You Seen no motion in video or panning motion, try changing seed and Motion Bucket.
- If you see too much motion/blur, lower the motion bucket and change seed.
- Elements like flowing water, wind, clouds , etc, elements which are captured during motion is seen to produce better motion after render.

AnimateDiff
- If you see no details in the final render then you need to increase refiner's upscale value to 1.5 or something near, to give some space to add details
-Loras work as usual, don't over do loras.

Installation :

1) The Workflow File Is Attached Below also you can download from here

2)  AnimateDiff Evolved 

3) SVD   with svd - Model 1 and svd_xt - Model 2

4) Rest Install Custom Nodes with Comfy Manager - "Install Missing Custom Nodes" or install from below:

• https://github.com/WASasquatch/was-node-suite-comfyui

• https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes

• https://github.com/shiimizu/ComfyUI_smZNodes

• https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet

• https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved

• https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite

• https://github.com/giriss/comfy-image-saver

• https://github.com/kijai/ComfyUI-KJNodes

• https://github.com/YMC-GitHub/ymc-node-suite-comfyui

• https://github.com/Stability-AI/stability-ComfyUI-nodes

• https://github.com/pythongosssss/ComfyUI-Custom-Scripts

• https://github.com/LucianoCirino/efficiency-nodes-comfyui

• https://github.com/wolfden/ComfyUi_String_Function_Tree

• https://github.com/Stability-AI/stability-ComfyUI-nodes

• https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet

• https://github.com/FizzleDorf/ComfyUI_FizzNodes

• https://github.com/mcmonkeyprojects/sd-dynamic-thresholding

• https://github.com/giriss/comfy-image-saver

---------------------------------------------------------------------------------------------

Warning : If you want to Edit the workflow, don't delete GetNode or SetNode, just break the connections and keep them aside, they have a bug when deleting,  the comfyui workspace will stuck and won't let you click anything else, the solution is to clear everything from the Browser Cache and Clear Data of the Past hour from browser settings.

For Someone Getting this Python Scalers Error :

Delete this Dynamic Threshold Node from the workflow, rest will work fine.

__________________________________________________________________________

My Discord Server : https://discord.gg/z9rgJyfPWJ

____________________________

Download Beta Workflow :