AI ART: VAM textures with Stable Diffusion (Patreon)

Somebody asked if it's possible to do vam textures with StableDiffusion and I gave it a quick try at face textures.  I don't think it's possible to create textures 100% from scratch. The way to go I think is to use existing textures as input images and using the AI to alter them but to also keep some areas intact so that the new textures will still fit in VAM.

STEPS

1.  Pick a starting image & resize it

I picked one of the awesome free skin by Riddler as a starting point (Skin6) and resized it to 600x600px. Something between 512-1024px is recommended for input images, bigger than that you'll probably start getting errors. I don't think it makes a huge difference the input image size.

2. Click Inpaint & select everything

With inpaint, I selected the biggest Brush and most transparency and drew over the whole image

3. Inpaint erase eyes, mouth & seam

After I selected everything, in Inpaint I then clicked Erase, picked a smaller Brush Size and full opacity. The transparency mechanic is a bit weird but the gist of it is that it's just a visual thing in the UI. When you Draw/select, it's best to keep the brush transparent so that you still see the image behind. When you Erase it's best to keep it opaque, so that it actually erases that area and you don't have to click multiple times to erase it.

Then I erased some areas so that the AI keeps them the way they are.

Using Inpaint to keep the area around the eyes intact will prevent the AI from trying to do actual eyes in the image. If it does, the eyes will very rarely align with the texture and you'll get lots of visible artefacts on the skin.

The same goes for the area of the lips, if you let the AI have full control over the mouth it will sometimes do the might slightly open, teeth, and it will look weird in vam. Forcing the AI to keep that center area where the lip texture connects should provide much better results. 

Finally, keeping the seams intact, the neck area, will make the face texture fit better with the body texture 

4. Generate images

I then generated a few images using these settings:

Prompt: skin texture, female, Sampler: ddim, Inference Steps: 25, Guidance Scale: 3.1, Model: HassanBlend1.5, VAE: vae-ft-mse-840000-ema-pruned, Negative Prompt: lowres, drawing, blurry, freckles, dirty, ugly, Prompt Strength: 0.25

There are definitely ways to make it better but I wanted to keep it simple. Adding more details to the prompt and more stuff to avoid to the negative prompt might produce better quality textures and more customization for brows & lips.

For the size I used 768x768, and hit Upscale on the images generated that I liked to make them 4x larger. Making them at 1024x1024 and then hitting Upscale will result in 4k textures. 

Increasing the the Guidance Scale & Prompt Strength levels will let the AI get more creative with the details but it will also mess up the texture more often and some will not be usable. With these settings all textures generated were usable for me.

SCREENSHOTS

a few examples in vam (most are low quality 768x768, didn't upscale them)

and the original skin for reference:

EXAMPLE TEXTURES:

CREDITS:

Riddler's Skin 6 used as starting face texture