AI ART: Animations (Patreon)

These were a few quick experiments with AI animations:

LAZY & DIRTY APPROACH: DIRECTLY IN VAM
I think the proper way to do AI animation is to record a normal video and then have the AI convert it frame by frame. I felt too lazy to do all that so in Alive v51 I just added a new time slider that allows slowing down time more than vam allows it (I think there are some free plugins that do that too). Making things go like 1 frame every few seconds. Then I just spam-clicked the AI cam and it generated these frames and stitched them together.


RECOMMENDED APPROACH: AUTOMATIC1111
For people very interested in doing AI animation properly, it's possible to do stuff like this through AUTOMATIC1111. In img2img mode you normally give A1111 a starting image and it generates a new image starting from that. That's what Alive is doing in the AI Cam normal mode too. But you can also select the Batch tab, and set there an input directory that contains frames (jpg/png files).

When generating in this mode, A1111 will run each of the frame through the AI and generate new frames. You can install ffmpeg as a tool to convert mp4 to jpg frames and then later stitch them up again.

FFMPEG is a command line utilty, it doesn't have an UI so I created a few simple script that I can just double click and convert things, rather than type in commands. Attached are 3 such scripts. If you used my TextAudioTool for Alive speech, you probably installed ffmpeg already because it's something the speech to text component (Whisper) needs to work.


If you add ffmpeg-mp4-to-jpg.bat to a folder that has some mp4 files and if ffmpeg was installed correctly, when running this script it will create a new folder for each mp4 found in that folder:

Andthe folder will have the frames extracted as individual images:

Then you can use one or a few of those frames to test things in img2img in Automatic1111. Prepare a prompt that works for what you want to generate and check it on a few frames before running it for all of them. Then you can switch to Batch and put the folder with the frames as the input folder. When you hit generate, Automatic1111 should then generate new images for each frame individually.

When done, to stitch them up into an mp4, you can use my other script: ffmpeg-jpg-to-mp4 (works for pngs too). You can copy it in the folder that has the output images. When you run that script, it will stitch together all the jpg/png images found in the same folder and will create a file slideshow.mp4.

Bonus: ffmpeg-mp4-to-gif can be used to convert mp4 to gifs in the same way. Just add it to a folder where there is some mp4s, and where running it, it will generate gifs in the same folder.

These scripts have some hardcoded variables: framerate as 20 fps, gif only as wide as 600px, etc. You can edit the files in notepad and change those if you need higher or lower framerate or other settings.