May 2022 Update! (Patreon)

May has been a very productive month, and our two new coders have helped accelerate things tremendously. 

Most of these innovations have happened behind the scenes.

The biggest source of performance woes in a game like ours comes from the JobDriver, Pathfinding & Navmesh Rebuilds/Path Cost Recalculation, Lighting, and Sprite Memory.

Solving those big tech risks is among the first huge Full Production hurdles past the prototype, which we're tackling now.

To help our maps remain huge, and not devour memory and hard drive/save file space, Curtis and Kent are working separately on two huge innovations:

Kent has our Symbol system for saving and loading, and Curtis has his Dynamic Atlasing, Shader Based Tilemap Terrain, and Data Streaming system. Together these result in tiny map file sizes, and map with texture coordinates rather than huge pre-baked levels.

First to tackle is our new Tile Map Accelerator / Shader Based Terrain. This is going to allow us to do something pretty amazing in Morningstar: have large maps full of activity, and keep memory overhead low. Zooming out should remain very smooth, even as characters are roaming around complex maps, by sampling the pixels on our terrain tiles and rendering them as colors on a shader, which takes over as the camera zooms. 

It's mind boggling when you realize, each square is about 1.5x the height of a person, and at full zoom, each pixel is a square, then each pixel is several squares.

This map is probably 5x bigger than our maps in game will be. Simply because the time to walk across that 1000 tile map corner to corner is probably 20 minutes in real life. Ideally we want maps to be traversed in a reasonable amount of time when you send a caravan away, or have a trader / raider come for a visit. 

But it's a good performance benchmark.

Powering this is a Dynamic Atlas, which can handle Texture Atlases (combining many images into just 1 big image) on a per-region basis. This way we don't always need a ton of images loaded into memory, just the ones this instance of our worldmap uses in this biome. And if we need another atlas for even more mixing, such as multiple biomes on one map, we can plan for that in memory in advance. 

The GPU can accelerate display far faster than the CPU ever can, so this is a huge boon.

This handles both the memory and display issues our maps need to overcome. With thousands of tiles on our maps, baking maps is impossible. The resulting images would be billions of pixels across and tall. So Curtis has been working on a system that will allow all that data to stream in only as needed around the player camera, and still allow us to zoom way out and reveal large areas with lower levels of detail.

In this gif the player's screen is that box in the middle, shown bottom right as what you'd see in game if you are zooming around super fast.

With 1024x512 terrain tiles, the visual memory issues we introduce are gargantuan. This is not an easy problem to solve at all. 

If you look at StarCraft from 1998, or at Baldur's Gate, Fallout -- those old isometric games had all their tileset sprites baked into sheets that were half the size of ONE of our individual tiles. And they never let you zoom in/out!

To remain futureproof and look good on a 4k Ultra Widescreen Monitor, and a classic 4:3 CRT Computer Monitor, Morningstar has to handle huge files in a crazy performant way.

We may still cut down our raw sprites and save memory for other applications, and low spec computers that are very old, but if we get lucky with Curtis's tests, we can remain at our established asset resolution and still zoom in really close to our characters. 

We may need to put in play some constraints for older computers, like limiting the number of transitions on tilemaps and culling them sooner, preventing the camera from getting as close to the ground or as far back, etc. There are other risks involved in supporting older hardware too, and we have plans to tackle those as they come up.

After we re-impliment tile transitions (where tiles blend into each other at the edges instead of these hard diamond edges) and procedural rendering layers, we'll be off to the races in generating maps to playtest the new Animation Framework we described last month, so characters can wander the map and attack each other. 

That is where James's work comes in. 

Pathfinding on these gigantic maps is a struggle for most games. It certainly was for us on SOS2 for RimWorld, and in other games I've worked on. 

To innovate over our inspirations and attempt to avoid the same major slowdowns, Morningstar is going to look to some scientific research on the subject of efficient path finding: https://webdocs.cs.ualberta.ca/~mmueller/ps/hpastar.pdf

(Square or Isometric, for this test it doesn't matter much. They work the same in the end.)

The fastest route between two points is a straight line. 

Sure. 

But what if you need to check if the line is straight, for each segment of that line? What if there are obstacles? What if the end location isn't accessible because it either A: is already in a closed room, B: just became closed by a locking door or a cave collapse? 

Suddenly that line isn't so straight and it's not fast. 

So what James has done is break up our map into a set of regions containing a number of square tiles, like the Area Codes of a phone number, and checked the edges of each region. Each region has a ring of tiles inside and outside its edges into adjacent regions. These edge tiles we check whenever the map is told to update a region, such as when players are building structures, or caves are collapsing. Then we draw points between regions like a series of highway on/off ramps.

You can see the green dots are "start" and red dots as "stop," and this map dynamically reacts to changes in possible path samples. It essentially builds roads that it expects will be the most traveled, and passesthat to the AI kind-of like google-earth. 

Suddenly a massive map full of obstacles to avoid and bad guys chasing you is a lot more manageable.  You only need to care about the precise, high detail movement on individual tiles when characters are close to their target, instead of worrying about every issue along the way between regions. They're just simplifying the number of instructions per second to the intersection of roads and off ramps, instead of assessing every tile along the way. 

Because this can be broken up into regions, it can also be threaded for big events, like a nuke destroying multiple regions of tiles at once (this will always hitch a little bit). And we can pre-calculate preferred pathing by getting data from the tiles they are crossing, such as sand, concrete, or tar, and find the real shortest point between two regions. This should give the illusion of intelligence, as our characters stick to roads and avoid slow tiles when crossing big maps, all without losing performance to complex pathing decisions. 

You can tell there are more inspirations/PTSD from SOS2's vacuum pathing at work here. 

Where this may challenge us is when the whole map is getting an update, such as when snow is falling. Suddenly every outdoor tile is getting a speed penalty, and there is no one region that isn't being updated. Now we're recalculating the whole map as the snow piles up. And that is next month's big hurdle to tackle: what happens if many regions are being recalculated at once? What is that performance impact, especially if there are characters crossing the map for jobs or while in combat? 

The next hurdle after that is a Threat Heat Map, which will create zones of avoidance as a layer of data attached to our tiles and calculated on the GPU again. This lets us have a rapidly updating system that tells characters "don't go here, avoid this scary thing" in realtime. So that Scary Creature roaming in the northwest corner of the map isn't recalculating the whole region as it moves, just a bubble of pixels around itself. 

That should allow us to have a popup to warn players, "hey, this part of my path is dangerous because of the {Scary Creature's Name}. I can't avoid it by any other route. Should I still follow this path?"

This Threat Heat Map will also help characters decide whether to attack a target from behind, or head on. They will see an opening on their target facing away from them and move in for the kill, avoiding the cone of danger, because the Threat Cone coming off an armed character can rapidly update their local path awareness. 

This should be able to take place simultaneously with dozens more characters all searching the index for grass as cows graze, or soldiers looking to purge the last enemies on the map, and choosing the shortest path to the target long distance.

Because our game is manually dictated by the player, aiming and attacking need to be FAST. It affects the accuracy of the player, and thus their feeling of satisfaction and skill. So the Pathfinding performance and our ability to smartly turn on/off Jobs is going to be vital to gameplay. 

If it were turn based or just raw strategy this wouldn't matter. You can drop under 200fps, who cares? But for a game where play skill matters (if you choose to play this way instead of as a leader giving orders far away) then we need optimal performance. 

Really glad James and Curtis have come onboard to help tackle these fundamental issues.

Finally, from Kent, he has begun working on an Expectations system that tells characters that are in the middle of a job, such as "haul corn to the cattle trough,"  to look at their Hierarchy of Needs, and check against the pathing cost and their expectation of fulfilling their needs while on that path. 

If there is a character closer to the feed stock, and closer to the tough, a pawn that is starving can just pass the job to the character idling nearby with nothing to do, with a lower pathing cost, and a lower risk of starving to death or falling asleep. 

This is one of those major head scratchers every game like this is forced to face: how smart is your AI, really?

Because any game that needs AI to be smart to succeed is doomed. :p 

Dumbing things down to a bare minimum, if a character is allowed to look at their needs hierarchy and say: 

"I have 50% left on my sleep meter and 10% left on my food meter. 

"The path to the Food Store is going to cost me 20% sleep and 5% food. The path to the Trough is going to take another 30% and 10%. I'll be zero on sleep and negative on food before this job bill is over. 

"So before I go on this trip, I'm going to eat the nearest available un-owned food (check my own inventory first), and then go to my own bed (and if my own bed is blocked or too far away, I'll choose the nearest safest unowned location to sleep.)

"THEN, I'll do my job, after I wake up full on sleep and food.

"And on top of that, if my job isn't being done, but someone with a full sleep and food meter is nearby and can do that job according to their available assignment priorities, just assign that character to the job and clear my job. 

"If I am already carrying food for the trough, I can drop it where I eat. Then pick it back up and sleep. Then drop it where I sleep. Then get out of bed and pick it back up, and go do my job. 

"Meanwhile, as I am sleeping or eating, this character with empty hands who is along my path, can just pick up the food and take it to the trough for me. Then, when I wake up, my job is done and I recalculate what to do next."

Finally:

"Does my job depend on having this item as I do it? IE: is my job to gather coal, and gather iron, to make steel at my forge?

"If that's true, then I need to bring coal and iron to my forge in my inventory. But I don't want to do that all day after every item is done. So I'm going to look at the number of coal and iron pieces that ALL my bills will cost. Then I'm gong to fill my inventory to max with loads of coal and iron, as much as I can carry in one trip, and bring that back to my Forge, where it will be marked as owned by me (so nobody comes and claims it.) Then after my stockpile is full with items I have claimed, I will start work on my Forge bill." 

There are a LOT of places for that to go wrong. Characters hauling items is actually a very computationally taxing process, on top of the complexity of unexpected "WHAT ARE YOU DOING!?" moments.

"WHY ARE YOU CROSSING THE MAP TO GET THAT ONE PIECE OF COAL IN FRONT OF A WHOLE SWARM OF VELORAX THAT WANT TO EAT YOU!?"

Additionally, if the food for the trough or the coal for the forge is on the top left corner of the map, and the tough is on the bottom right -- that could be 1000 tiles away! That's a full day of walking between mountains and valleys!

So to ensure we are actually getting a "smart" Character who wont just ignore their job and stay at home eating and sleeping, baffling players who need that new steel sword to fight off the next raid, we might opt to post some information for the player:

"Hey, my job path is too long. Should I do it or nah?"

There's always a risk the player meant to do this on purpose. 

Maybe their map got wiped out by a disease, and the only food is far away now. We can force the characters to harvest the last stand of corn, death march all the way across the map in sleep and food debt, to put it a specific container, and place the container near to the end goal at the trough. 

But that's about smart players, not smart AI. How you lay out your base matters in a game like this, and that's just part of the fun. The joy of making a perfectly humming base that's as efficient as you care for it to be, without driving your workforce insane or starving to death (if you care, which, well...) 

Ideally we don't want to compound the player's logistics planning with dumbass AI, who walk across the map past volcanoes and alien predators on their way to pick up that single grain of rice, when they had to walk past 30 full stacks of rice that were right next to them at the start of their journey, just to make rice to eat, before continuing to stack coal and iron at their forge. 

This will take another level in complicated as we introduce wagons and boats, which can often transport materials many times faster than characters hauling on foot. It is simplified by Backpacks and other Portable Containers, which can gather to them many items. And our Characters are smart enough to look for the nearest Wagon that way. It's still going to be a thing making sure the Wagon & animals all stay near their goals while loading/unloading, and finding the "barycenter" where all the paths to the wagon and the harvesting items is the shortest for all haulers.  

This is especially important to Miners and Harvesters. Four characters can be producing ores, while a fifth leads the wagon. 

Hopefully next month, possibly the month after, we can start to show some really early prototype gifs of this hauler in action (not the wagons, that's much later.)

 Won't be pretty yet but it'll be something.

If you're a fan of Civilization, Factorio, RimWorld, StarCraft -- basically any game that lets characters go harvest/mine resources and bring them back to a central location, this is going to be a treat.

Getting it to work is hard.

Getting it to work well is even harder. 

Finally, and this is kind of a Good News / Bad News situation, we decided to scrap the more performance positive User Interfaces we were making in Unity's new UI Toolkit and with C# Widgets, and go back to uGUI.

The simple reason is: I am a designer, and I excel at that layout in uGUI. UI Toolkit, is aimed at web developers and pure coders who need a simple interface for a simple game, and don't care of it looks good or not. 

There is nothing simple about Morningstar, it's one of the most complex games in the most complex genre. So UI Toolkit is like using silly putty to make a car engine. 

Is it better performance, especially in lists of inventory items like we have? Oh yeah. 

But can it actually make a beautiful UI free of visual glitches, tons of boxes, all at runtime, that I can see with my eyeballs as I make it, and have that match at every screen resolution? NOPE.

Unity introduced tons of glitches, bugs, and oversights -- so much so I want to know if they played their own game or not, and by that I mean, actually used the UI Toolkit to make real, usable interfaces for games.

As my friend cHill posted earlier.

This is a mild setback, because we've gone from a very janky, slightly uglier UI that actually has code working on an event bus and character renderer responding to inventory items being pulled on or off of their body... back to my old static placeholders.  

We will also be forced to struggle with making new item prefabs to bake as single sprites on their backgrounds, and spawn the item art solo when dragging/dropping, and totally redo the way lists of items are rendered for performance. 

But near and long term, with this hard coded UI that already exists, we can make our game.

We will have to come up with a new solution for modders to make their own complex UIs later on near release. Simple dialog popups are fine for 90% of mods, but it we want truly huge mods that fundamentally change the game, modders may just need to use Unity and attach dlls and their own code. 

Most modders I talk to don't like that, they want source code totally open, which we can't get away with. Second best is hooks to add their mods to, and the ability to change any menu through code, so they can add their own features (say, the Vampire mod wants to add Blood as a need on our needs meter window -- currently they'd need to make a new menu on another tab. Not the end of the world but it'd be "fine.")

That's going to be tricky and complicate our UI further. But that's just part of why May was such a month. 

And to end the month, from Allegra, we have begun to diversify our faces. 

These guys are greyscale, because their skin tone is applied in engine from a list of skin tone swatches. But they can be anything from albino to melanin rich with this system, these guys being right down the middle. 

Skin color, eye color, hair color -- all handled by color masks. 

Young men and old men, the super skinny near death and the hulking body builder, we got it. We can make fairly average looking people and staggeringly ugly or beautiful ones. 

We're also starting to work on diverse ethnicities, both from Earth and the natives to the Fragment. 

This is highly creative work. Inventing the faces of peoples who have never existed before is a whole speculative realm, and making it modular so a computer can spit out brand new appearances, that's a challenge too. Thankfully we've solved it artistically! 

It's also another technical challenge, as the faces are made of layers of sprites which we need to stick to Powers of 2 for technical reasons. Basically, if your image isn't a 512x512 or 256x256, then as the camera pulls away from your image, it can't shrink every pixel evenly. That causes ugly artifacts and also hurts performance.  

So we are in between a rock and a hard place:

Each eye, nose, mouth, hair, eyebrow, skin detail -- they need to overlap with what are called Alphas. 

Alphas tell an RGB image if it is transparent or not at each pixel. 

So these Powers of 2 mean we have images that are going to overlap all over.

One alpha on a solid background is no problem. 

But what if you have2? or 3? (in our case, up to 7 or 8!)

Suddenly the GPU needs to work overtime solving what layer is on top, and if it is or isn't transparent, with multiple samples every millisecond. 

You can avoid that by "baking" sprites offscreen once to a sheet that contains only the final result. But this is uncompressed on disk and it's HUGE in memory. It also takes time for the picture to render itself, and its smaller versions for different zoom levels.

Just like our terrain, these guys are 1024x1024. On screen, practically, we save on that cost by limiting zoom so they are 512x512, and only use the bigboy on the UI during conversations, etc. Heads are only 256x256 with eyes and noses only 128x128, or 64x64.

We've been kicking around the best solution for a while now. That is still being revised regularly as we try to optimize for our best case. 

It's actually working very fast right now, as is, even when several characters change outfits. There is a slight delay in the visual update as the sprites change, so we will definitely need to look into optimizing it. But it's not bad at all in context and hardly noticeable unless you changed 5-6 character outfits at the same time. 

Thankfully? No limb animations. :p

So we can get away with this style and optimize the hell out of it. 

---

Thanks for coming with me for yet another month! 

It's been a crazy one, but good stuff was done!

There's even more but I'll leave it for a week next month. :) 

Do you have any questions about what's going on up here? What suggestions do you have? I'd love to hear them.