Continuing Character Customization Part 2 (Patreon)

The backend for custom characters is almost done! I just reached a milestone with the pipeline, complete model import at runtime! I used an MMD Miku model as a test. I will now demonstrate the current process to import a custom character into the game. It's subject to change as the pipeline matures over time. Some modeling experience may be required to understand the following but I'll do my best to explain it in layman's terms:

First I went on the internet and downloaded a raw MMD .pmx file. It doesn't have to a .pmx, as long as you can open it in Blender. Easy enough, anyone can do that. Then I used CATS Blender addon to import the .pmx into an empty Blender scene.

The model of course came with a lot of useless stuff that is meant to be used for Miku Miku Dance. So first, I had to delete all the bones and crud that I didn't need. Because I'm only supporting custom heads (for now), I started by just removing the body parts of the mesh.

This Miku model came with Japanese names for the bones and other data, so I used CATS's one button click to translate all that information. It made it easier to see what I needed to get rid of. 

Then I deleted the bones that I wasn't going to use:

This is the completed base mesh that I want to use. The next step was to make the materials it uses compatible with my game. For character cards, I have decided to combine the hair and skin textures into 1 card (previously I said there would be 5 cards per character, now there will be 3). Therefore Miku's skin and hair textures need to be combined into 1 mega texture. CATS also has a very handy 1 button click to do this. I set my atlas texture resolution to 1024x1024 (like clothing), and created the atlas. Result:

1 model = 1 texture. So far so good. At this point the workflow is really easy. Merely delete the useless stuff and then click 2 or 3 CATS buttons. This is ideal for making lots of characters fast and intuitively. The last part is probably the hardest, mapping Miku's facial expressions to my in-game Shinobu's. The reason this is a little bit harder is because Miku does not share the same facial expression sliders as my Shinobu:

All of Shinobu's animations were animated with the 28 facial expression shapeKeys shown above. That means the Miku model needs to have the same facial expression sliders to properly match the in-game animations. This was unfortunately not the case and its different on a model by model basis.

Some shape keys can be salvaged. Like "Grin". It will work for Shinobu's "smile". One problem however is that most of Shinobu's shape keys have left and right lip side sliders. Meaning the "smile" needs to split into "smile_r" and "smile_l". This can be time consuming especially for the rest of the needed shape keys so I spent some time and implemented Automirror support for my export script. Meaning you now only have to create "smile" with both sides set and my script will automagically split it up for you into the two "_r" and "_l" variants. That brought down the necessary shapekeys from 28 to 18. A lot of time saved. Keep in mind that you don't have to match every single expression shape key. You can always ignore some if you think it will fit your particular character better (perhaps she has her mouth covered or something).

And that's it! The model will be packaged into a .viva3d file that my game can read at runtime. The export process takes care of everything from deformations:

to shape keys:

and to bone hierarchy:

This is a very powerful tool because it opens up tons of runtime customizability. Custom furniture, items, clothing, etc. is now possible in the future. The Blender script is very fast and exports the at sub 500ms meaning you can almost instantly see your changes in the Unity game. This is what an ideal customization pipeline is all about.

One of the last features that I need to implement is adding support for tweaking model settings. This will be handy for calibrating your model's eyes to the in-game logic so she can look around properly. So far it's been really fun working on this tech. I will begin the open sourcing of my Shinobu model and the WIP Blender script next devpost for Supporter Tiers so you can test out the pipeline.

Stay tuned.