eromancer

August 2018 Progress Update! (Patreon)

Published:

2018-08-02 07:26:03

Edited:

2018-08-02 07:35:14

Imported:

2021-05

Content

Hey everyone!

It started out clunky but this month turned out pretty well, especially toward the end! I’ll start out by going over some of the highlights.

One of the biggest milestones in our conversion from Daz to Maya happened just last night: TK and I successfully completed a full test conversion of Malise to Maya. This means that rigging and joint corrective morphs for all clothing and body parts and hair were brought over and hooked up via TK’s scripts. A key requirement of this process was that we be able to do it very quickly since a lot of conversions will be needed (mainly for character variations and adjustments/fixes), and with an actual conversion time of something like ten minutes we definitely met that. Reaching this point was tough, as a lot of obstacles presented themselves this month. I’ll talk more about those below and how we overcame them. I’ll also discuss the remaining threads we need to tie off here as well as future optimizations we can make.

AltairPL had a tough month due to illness in his family, but he pulled through it with some solid coding progress on the new engine! He is currently nearing completion of a working prototype for his game archive implementation, something that’s critical for being able to make releases with the engine.

Late in the month we had a new team member join us, and I’m already pretty certain his work will become a staple in our design process. Limbo Limbo is a concept artist, but also has the benefit of being a skilled mechanical and hard surface designer and modeler – something critical to the aesthetic of our project. Limbo will be pumping out character designs for the foreseeable future, but he will likely help with modeling assets as well. Mr. Kittyhawk has been upping his game with clothing by learning Marvelous Designer, and will no doubt stay busy with the amount of designs Limbo is putting out :D.

Lastly, Uber, Mr. Kittyhawk and I have put a lot of effort into designing and creating Onyx’s weapon shop , as well as converting weapon props to use throughout the game to Maya.

Read on for the details!

Topics for this Update

Here’s the list of major topics we’ll be discussing in this month’s update. Check out the Moving Ahead! section at the bottom for an overview of our progress toward Battle Test 4 and the next major release. Topics listed with the ‘***’ have been significantly updated or are new since the last Inner Circle Update.

Maya Conversion Progress Part 1
Maya Conversion Progress Part 2
Pose Library Implemented in Maya***
Soft-Body and Hair Physics on Rigged Characters
Fluid R&D: Texture Projection
Limbo’s Concepts Part 1: Splicers***
Limbo’s Concepts Part 2: Nephilim Runts
Syndicate Outfit Work-In-Progress
Redshift Weapon Conversions
Onyx’s Weapon Shop Part 1
Onyx’s Weapon Shop Part 2***
Map Sprite Perspective Change
AltairPL’s URGE Engine Progress***
Moving Ahead!***

Maya Conversion Progress Part 1

The first thing to know about where we stand with our Maya conversion is that I’ve completed basically all my conversion tasks for materials and so forth that will be required for Battle Test 4. However, I can’t move forward with posing until we have TK’s complete (rigged) conversions of Malise, Neon, and our enemy cop buddy. This is why that news at the beginning of this post about our first successful full test conversion of Malise is a big deal :D. I’ll get into the details on that in Part 2, but first I’d like to share all the other progress we’ve made.

To start off, the police enemy is now converted to Genesis 3. I did the body and UV conversion, while TK refitted the armor to the new body, including the new leg armor, shown here in a Daz mock-up. While he was at it, he also remodeled Malise’s gloves and boots to work in Maya. If you recall from the monthly update, they utilized Daz’s HD morphs, something not compatible with Maya. The last thing the cop needs are some custom body model corrections to fix errors generated by the Genesis 1 to Genesis 3 conversion process, and then he should be ready for Maya conversion. Neon’s list of fixes for her Daz scene is shorter than expected, so she’s probably only a day’s worth of work away from being ready to go to Maya as well.

TK and I spent quite a bit of time early in the month on genitals, which has taken some more effort than anticipated. TK had to come up with a process for remapping the UVs for the girly parts, and after that I was able to get some pretty good results with the materials. We can definitely do better with some more time, but right now working with materials is a bit like wading through a swamp. There’s a lot of overhead and setup time I have to do, especially for polished renders, since we’re not quite ready to be working with the rigged figures yet. On that note, there are some definite rigging improvements TK will be able to make to the gens that will improve the shape, scale, and stretching.

Here are the full-size renders shown above :D. I was planning on doing more stuff with liquid for these, but I figured there are more important things to do right now.

For the male gens, we are probably good to go with the system we are using now for the cop enemy, but for more exposed characters we’ll need to switch to another system that uses geo-grafts, which actually connects the geometry to the body. Right now, we’re using a system that uses opacity textures to blend the genitals geometry with the body, and this creates a nasty discolored seam due to subsurface scattering where the two overlap. This shouldn’t be a very big inconvenience though.

And on that note, here’s a render of the material conversion for the male gens :D.

Maya Conversion Progress Part 2

This second part of our Maya conversion update is a lot heavier (will try to trim the fat), but it also explains where a significant amount of time went this month.

TK ran into a series of issues this month with the Daz export process that would have really blown up the time required to make a character conversion to Maya. Without a solution, it would have taken a few days of manual work for a single conversion. Since we plan to continue using Daz for human character model creation and setup, and since characters need repairs and variations pretty often, this wasn’t practical.

Basically, Daz was dumping out tons of empty morphs to the FBX file (the intermediary file that we import into Maya to rebuild a character). These empty morphs would get read in by Maya and eat up all our RAM (it blew through my 64 GB). At the time, our only solution was to break down conversions into small chunks – like a piece of clothing at a time – leading to the above-mentioned wrench in the machine.

Since it can be exported in plain-text, I managed to reverse engineer the FBX file a bit, and we figured out that the bogus data could indeed be cleaned out of the file before it gets imported into Maya. No small feat, considering these files can be nearly 2 GB. 2GB of text means it can’t even be opened in a text editor. This is where we pulled AltairPL in, and after updating him on the problem he determined he could write an FBX cleaner that should work in all situations. APL and TK spent the next few days on this.

We’ve been using APL’s script now for over a week, and it’s working great. TK spent an evening teaching me the full procedure for converting a character, and I’ve been using a test conversion of Malise I did with no problems. Since our systems were previously getting bogged down by the above issue, we never actually got to see a conversion work with all the required morphs. Once we could, we found some issues with TK’s conversion script that needed some attention.

Last night’s test suggests these issues are all cleared up, and that everything is working great. The test we did required that we make a 4000 line file that included all the morph export rules for all of Malise’s parts and clothing, and the FBX file ended up being 650 MB. We’d like to eventually automate the process of creating the morph export rules file, but now that TK has done the research on how to make it, it only takes a few hours per character (and only needs doing once if there are no changes), and therefore it’s pretty low priority.

Pose Library Implemented in Maya***

This is admittedly something I overlooked in previous “we need to do this immediately” lists, largely because it’s such a basic feature in Daz I took it for granted.

A pose library consists of being able to save and load poses, as well as a UI for doing that and related functions efficiently. Maya doesn’t have one of these by default, so we looked to third party plug-ins as a starting point. After striking out with one, we settled on Red 9’s StudioPack toolset, which comes with a Pose Saver but even more importantly a Hierarchy Control UI, which allows us to tailor pose settings on the fly to different rigs. It comes with a HumanIK preset, something we need for our humans converted from Daz, which is where the first plug-in we tried fell short.

One critical feature we’ve become used to however was not supported, and that’s the ability to use one pose across all similar characters. This is something Daz does very well. I therefore put on mah coding gloves and got down to business reverse engineering the plug-in. The fact that I’s open source and in Python is great, and is something I’m really appreciating about working in Maya. Initial testing suggests I succeeded in adding the required functionality, but we have to do some practical work to know for sure.

Overall this solution should be more future proof than anything we had in Daz, and should work nicely with all the wacky rigging requirements we will need once we get into really unique monsters / robots / etc.

Soft-Body and Hair Physics on Rigged Characters

While we haven’t had full conversions sorted for long, being able to do partial conversions as of earlier this month allowed us to get things like hair rigging working in Maya. Utilizing that, I’m able to test some new things.

A big benefit of using Maya is that we can use physics simulation on the fly. Two types that will really improve H artwork are hair physics and soft-body collisions (squishies!). I showed some soft body testing before when I first experimented with Maya’s nCloth a while back. I’m taking what I learned then and am now applying it to a rigged character – something we haven’t done before.

Even though nCloth was designed as cloth simulation, it’s become very popular for both soft-body and hair physics. The past couple of days I’ve been experimenting with settings in an attempt to build presets that work in most situations. I’m still dealing with some complications, but I’m already able to produce some pretty solid results for some use cases. The ones I wanted to tackle first were breast/butt squishes, and hair draping across a surface. Previously, haven’t been able to do either of these very reliably, and they come up often in H artwork. The biggest thing I wanted to test though is if soft-body collisions will work *with a clothed figure*. The potential for clipping here is high since now we’re talking about multiple simulations running simultaneously, but it’s worked very well in my tests so far!

Here’s a set of images showing some of the various results I achieved with soft-body physics. Keep in mind that gravity isn’t on until the simulation starts, so the breasts fall into place at the start.

And here are some different results I got with hair physics. Ignore the floaty bits around the ears, I didn’t do a great job painting the physics parameters since I’ll have to do it again later :D. Also, the hair material looks kind of messed up due to that Redshift opacity issue I keep mentioning. It will look more like hair and less like a bird’s nest once Redshift 3.0 is out.

There’s a whole lot of opportunity to expand on this, and one idea that Uber has suggested is using a low poly physics rig to drive the high poly models. I actually spent part of today doing some testing with it, and the concept appears to work, but it’s a pretty advanced undertaking that requires some meticulous weight painting. The benefit though is that once it’s set up it’s done, and in my tests the performance was great -- I was getting 60 FPS physics n’ jiggles while posing. It’s hard to say whether the quality is comparable yet to what I have shown above, but it’s worth considering.

Fluid R&D: Texture Projection

While we’re on the topic of new tech stuff, I thought it’d be worth mentioning a method Uber came up with for speeding up liquid compositing. I mentioned previously that to get liquid looking really pro we needed up to three separate render passes and composite them in post – something that can get time consuming. Uber’s method is one used in the film industry often and would allow us to get the same or better results with just one render pass.

The basic idea is instead of mapping textures to the liquid model, we’d project them from the camera and mask them with the liquid’s geometry. What this means is I can take a really quick test render, then use it to paint a mask to remove liquid or wetness, then project it as an opacity map from the camera. Even better, we can use it as an extinction map to make liquid more translucent but retain reflections – something more realistic than can be attained by editing in post.

For the best results, the method will take a bit of finagling to fit smoothly in our workflow, since making it hassle-free it depends on having the projection nodes set up and ready to introduce into materials quickly. This is something that will be easier once we have our materials management system set up in Maya.

Splicers Concepts

I've been working with our new concept artist, Limbo Limbo, for the past week or so now on these designs, and I am really blown away by his abilities. I found that we would often burn a lot of time redoing things in 3D due to a lack of design references, and since I’m not a great concept artist I really felt the need to find someone solid. After all, it’s way easier to redo a sketch than it is to redo a 3D model. Limbo has been able to capture a lot more detail in his sketches than we’ve been capable of in the past, which makes me far less nervous about moving forward with the designs since I can just point to a picture instead of having to work with Uber, Mr. Kittyhawk, TK, or whomever on every little detail.

Anyways, the first thing I've got to show you includes some of the designs for the new Splicers. Check them out below:

I had shown you guys some of the assets we've got for them already, as well as some rough 3D concepts, but Limbo perfectly captured that mix of Akira and Mad Max I really wanted for these guys. If you liked the old look, then don't be alarmed, we’ve got something for you as well!

We have a number of assets we showed of previously that will go great with these designs, with the overall plan being to create a variety of looks to make them feel like a real street gang as opposed to a bunch of clones.

So yeah, what do you think of the adjustments we’ve made? There will likely be some adjustments made still, so feel free to give us feedback!

Nephilim Runts

The second design I want to show off is one I'm considering using for a set of enemies from a faction I haven't discussed at all called the Nephilim. These guys will encompass the all-out crazy designs that don't fit within the other groups, and can best be described as cyberdemons similar to what you might see in Doom. While this group will largely be composed of fearsome, hulking fusions of flesh and steel, these guys will be the runts of the pack -- quick little zergling-type bastards that you'll love to hate.

Let us know what you think :D. I hope you're as excited as I am about Limbo helping out with the project.

Syndicate Outfit Work-In-Progress

Mr. Kittyhawk normally does clothes in Zbrush but has been pushing it to the next level this month by diving into Marvelous Designer. His first project has been to revamp the Syndicate design for the new art style.

He’s done really well learning the tools in Marvelous so far. It took some effort to figure out the subtleties and workflow to produce good seams while maintaining bulk for thicker garments, but his results are looking promising. Below are some previews of the Syndicate guys’ coat he’s been working on. Aside from a process to add thickness in ZBrush, everything here was done in Marvelous. Once the design is finalized he will begin adding details to the geometry in ZBrush, then we’ll work on a material. There are a ton of good ZBrush brushes out there for clothing, so we’ll need to start assembling a library since we will likely be making more and more of our own clothes from here on out.

Redshift Weapon Conversions

If you recall, last month we had started converting weapon props over to Redshift to use both as props in maps as well as for enemies and during story scenes. Mr. Kittyhawk took over for me on this and pumped out a ton of variations. The following list is incomplete, but it shows some of the progress that was made anyhow. There’s also a custom MP5 that Uber is working on that will likely get used by the Splicers.

Onyx’s Weapon Shop Part 1

Here’s the latest look at Onyx’ weapon shop!

A lot of effort has gone into this map. Not only will it be used for a couple of story scenes throughout the game, but it’s our first real experience doing a “lived-in” interior. The number of props needed for this kind of thing is pretty surprising, and we’ve been looking into ways we can circumvent some extra work on the details in these kinds of interiors down the road. One possibility, as Uber tested out early on in July, is photogrammetry.

Ubercharge was away for the first week of the month and didn’t have access to his 3D tools, so instead he ran some photogrammetry tests to see if we could utilize it for quickly generating interior clutter / props. This is basically the art of making 3D scans of real world objects using photos that get turned into a point cloud using special software. The point cloud is then turned into a useable 3D mesh, and the texture info from the scans can be used to create materials. He’s been quite successful at using it for outdoor props for his schoolwork. It’s less forgiving on inorganic objects, and the detail isn’t good enough to use for anything close to the camera, but it could end up being useful for maps, which are rendered at distance and can simply have errors smoothed out / painted over.

Tests aside, all of the work on this map was straight manual. Mr. Kittyhawk and I started with some work on the hard surface details around Onyx’s counter. He created a chain link tile set while I did a bunch of kitbashing and made a test scene with Onyx so that I could get some camera framing for story shots. Here are some renders of that:

I then created the blockout for the map. I’ll leave it to you to figure out what the heck is going on in that scene, but look below for a hint :D.

Next, I worked on a shader for holograms (could still use some improvement), which I intend to place throughout the store, maybe with signage or prices hovering in front of display cases / above counters. The plan is to animate them to glitch out every once in a while, to add some animation to the map. This will definitely get used for other maps as well.

After that, Uber fleshed out the blockout with hard surface details and polished up some of Mr. Kittyhawk’s designs.

Onyx’s Weapon Shop Part 2***

So, we were hoping to have the weapon shop map complete by this update, but we still have a couple days on it. Mr. Kittyhawk had to take a day this week to learn about and create the Redshift proxies we need to place a bazillion guns everywhere. Proxies are basically instances of geometry that don’t utilize extra resources, meaning the scene will be about 700 MB instead of 5 GB. It also won’t bury the viewport under tens of millions of polygons, meaning we’ll actually be able to navigate the scene for any additional edits we need.

While he was doing that, Uber spent some time working in this little guy :D. This happens to be our first animated map event in the new art style and will be used in one of the story events. Here’s the full-size GIF.

Map Sprite Perspective Change

This has been brought up before, but for the first time I’m strongly considering changing the perspective we use for the map sprites to match that of the game map. Or more accurately, we’d be using two sets of sprites – one for the top-down perspective and keep the current one for side-view. In the retro JRPGs ours is styled after, it’s common to use only a front orthogonal projection for sprites like we’ve been doing, but retro RPGs don’t have the level of detail the new art style allows. What this means is that we’d be severely limiting the extent to which the characters can interact with the environment since their sprites would clash with the map’s perspective.

Here’s an image to demonstrate what I mean.

In this example, the colored sprites show the perspective at which we currently render them. The other models show what they would look like with the new perspective. Poses like the one Onyx has behind the counter, or the guys in the hallway, wouldn’t be possible with the current sprite perspective, and the same goes for a whole lot of event ideas we have. Beyond opening up a lot of opportunities in this regard, I suspect the sprites will look way more grounded in the world than they currently do. Aside from the simple shadows changing shape, the only other thing this idea breaks is reflections. In the image you’ll notice reflections in the current sprite perspective are exact mirror images, meaning we can render reflections with some simple code. This isn’t the case with the map perspective sprites. They would require their own sprite sheet. This would’ve been very tough to render in Iray, but it isn’t an issue with Redshift since we can turn off shadows and render reflections separate from the character itself.

It’s a bit hard to say without materials on the models, but I think it should look better. We got good feedback from the inner circle crowd on this one, but what do you guys think?

AltairPL’s URGE Engine Progress Update

Inner Circle readers, skip to the dotted lines for this past week’s update

July started really badly for me - I've been dealing with an illness in the family and it's still kicking my butt. The game archive was supposed to be long done, and it would be if not for the aforementioned problems.

Not counting finishing game archive notes, I didn’t get much accomplished in the first week of July, since it was around this time the shit hit the fan.

The situation started improving a bit in a second week, so I finally managed to make some progress on the archive stuff. For the most part, prototype of game archives creation was implemented and working as planned, including creating multiple archives when size of the data is nearing limits for 32-bit applications. I also did a test using most/all of the 0.05 data that would go to the archive. I'm pretty happy with the speed of the archive(s) creation - it was created in about 34 seconds, which is way better than 240 seconds I got from my Ruby implementation for 0.05 release. It's hard to compare it to the vanilla RPG Maker archive creation time, since the archive creation is immediately followed by making the .zip archive of the entire game. On top of the archive stuff, I've also fixed one of my earlier screw-ups related to keyboard input, which reared its ugly head only in certain circumstances, so even though I don't remember why, I had sent it to the rest of the dev team for testing, I'm glad I did.

The remainder of the July was a little bit more normal, and even though I was constantly interrupted, I made a lot of progress.

Since I could create test archive(s), it was then time to code reading them. When doing this, I had identified a few problems with the whole system - omissions, miscalculations, etc. I've fixed most critical ones on the spot, but some of them I had to leave for later. Most of them will be fixed before the first URGE-only BattleTest is released. The rest can wait for a bit, but I made notes whenever possible. An example of this is the patch/hotfix part of the system, which in URGE will be an integral part of the engine, instead of a hack I had to do to implement this functionality in RPG Maker via a Ruby script. Considering that the BattleTest releases are relatively small in size, I can leave this for later. Though, I would love to deal with it ASAP and be done with this archive stuff once and for all (yeah, right).

The next thing I did was to help out TK with some insanity he was facing. The problem he was having is that stuff exported from Daz had a lot of weird crap included, which was causing his and Ero’s systems to run out of RAM when those files were imported in Maya. TK or Ero probably included gory details in their part of the update, and I don't know them all, so I'm not gonna say more. Anyway, TK needed something to clean up files exported from Daz before importing them in Maya. The example file was relatively small, so I made a quick and dirty regular expression to use in notepad++ search function. But after seeing another example file which was much more complex and learning that one regular expression was not enough, I'd decided to take a break from the archive stuff, and make a not so quick and dirty Ruby script for TK to use. Well, TK wasn't very happy that he has to install Ruby, but I think he was/is pretty happy with how the script works, so mission accomplished.

I made a bit of a detour around that time... was thinking about coding something with the archive(s) creation in mind, when someone commented on something similar in a completely unrelated context. I'm talking about an ASCII/text progress bar, similar to what you can see mainly in Linux/Unix console apps like package managers or few Windows console apps like boot-time checkdisk. Best thing about progress bar like this is that it's a clear indication of whether long process, like archive(s) creation, is actually doing something or hanging. Another nice thing is that it's using only one line of the debug console, instead one line for every file or something, so it's easier to sift through the console contents. Anyway, it required some research and cursing on my IDE and Windows, but I finally managed to make it work... excluding IDE console, which is seriously stupid.

Testing the ASCII/text progress bar reminded me of a series of console issues I was having in some circumstances, so I decided to fix those as well. Another bit of research and curses and I finally got it under control - there's still one minor issue left, but it's so insignificant, that I won't even try to figure out why it happens or how to fix it.

---------------------------------------

After that it was time to go back to the archive stuff, and to start integrating the archive system with the part of URGE used by the game. It was a long and bumpy road. First, I had to fix some text encoding problem, which surfaced almost out of the blue, and then I was flooded with a lot of problems related to the Ruby scripts. When I was previously coding stuff in Ruby with RPG Maker in mind, I had to make a lot of weird stuff to make things more usable than RPG Maker allowed. And now, most of those "fixes" were causing problems. So, I had to dive into Ruby scripts to make extra conditions and processing for URGE. The bad news is that I completely forgot about Ruby game scripts and I was not prepared for dealing with this stuff now, so it took a bit more time than it should. The good news is that I was able to make it work, and in the process I've fixed few completely unrelated issues and made a lot of notes about things I previously coded in Ruby, which could/should become integral part of the engine, which would make them faster and better.

I still have a lot of stuff to do to make the archive stuff ready for testing, both internal (dev team) and external (patrons), but it seems to be working just fine in my tests, so I'm pretty happy with it.

Unfortunately, I expect real life distraction to continue for the near future, but I hope I'll be able to spend more and more time on this stuff. The first order of business is to make a more or less final version of Battle Test for internal testing, in order to identify and exterminate bugs that don't occur on my rig. Then, I'll concentrate on implementing all the stuff absolutely necessary for the patron Battle Test, and then refine it as much as possible before next one is actually released when Ero has enough of the new visual stuff done.

I haven't really thought about what to tackle after archive stuff, but at the moment I'm not quite sure how soon it will be ready.

Moving Ahead!

Last month we had a section at the end of our update by the same name, and I’d like to first reiterate what was said at the beginning of that one. The timeline has been getting progressively more confusing with the Maya conversion, but our next milestone is still Battle Test 4 / the Enty launch. From a player’s perspective, here’s what that release will contain:

Full lust system enabled, including all stages and armor damage for both characters
At least one example H skill chain
Potentially the new combat user interface

Here’s an updated list of goals we need to accomplish to make that happen:

Maya Conversion Stuff

COMPLETE: Convert materials for genitals
COMPLETE: Convert the police enemy from Genesis 1 to Genesis 3. Genesis 1 uses TriAx rig weightmapping, which is completely incompatible with any modern 3D app.
COMPLETE: TK’s rigging conversion (Daz to Maya) pipeline and scripts need to be functional and relatively bug free.
COMPLETE: Implement a pose library into Maya. I had overlooked this one last month, but it was really critical. We still need to do additional testing, but I’m pretty comfortable labeling this one as complete.
Update Neon’s base Daz scenes to match her model we’ve been using for the recent battle tests. Need to do this before converting to Maya.
Get Malise, Neon, and the police enemy rigs working in Maya (this shouldn’t be a problem at this stage, but again we haven’t gotten the opportunity to test TK’s scripts on different characters yet). This includes props like hair/genitals. The full Malise test from last night falls into this one. Its success suggests we should be able to accomplish this for the other characters as well at this point, but until the characters are in Maya and tested I won’t stamp this one :D.
Convert Malise/Neon’s battle poses from Daz to Maya and make fixes where necessary. This one requires we have the rigged versions of Malise, Neon, and the Police enemy in Maya.
Lay the groundwork for portrait rendering automation in Maya and get a basic system working (this will be a huge timesaver, so it’s worth investigating as soon as possible).
Render all the portraits up through Battle Test 3 in Redshift and get them hooked up to our lighting system. Will likely knock out a script for simplifying that last part, since any future full re-renders will be impractical to set up by hand.
NEW: Get joint rotational limits working in Maya. By this I mean we need to put limits on things like elbows so they don’t bend backward when posing. I had believed this info was being pulled from Daz with the export scripts, but that wasn’t the case. It’s definitely something we can add to TK’s scripts.

Other Tasks for Battle Test 4

Finish Malise’s 3D armor damage
Pose/render the later lust stage / armor damage portraits
Develop H skill chain and artwork for police / both characters
Potentially revamp combat user interface

I did a write-up on our goals further out, including some info on the next major release, in last month’s ‘Moving Ahead’ section, so I definitely recommend checking that out if you want more info there :D. And, obviously, feel free to ask questions.

That’s about it for this month! Thanks again for all your support guys, and I look forward to your feedback!

Content

Files