Introducing Cogmind's History Log (Patreon)

Having introduced the mostly complete new scoresheet format last week, I then started working on its last remaining section: the history log. Now I'm already six days into the task and, oh my, this job is massive.

A "History log" here refers to a list of important events and actions that took place throughout a run, and when (turn number) and where (depth/map) they occurred. Even without any other supporting info from the scoresheet, by covering the highlights of a run in chronological order it can kinda read like a story of that run.

For some background research I went back to my old Morgue Files article and checked out the morgue file samples I'd collected there to see which roguelikes have such a log, what they record, and how it's all represented.

Here's DCSS, for example, recording notable enemy encounters, kills, leveling, skills, altar discoveries, god powers, artifacts, and more. It's pretty detailed.

Angband is similar, although a little simpler, with notable enemy encounters, kills, leveling, and artifact discoveries.

By comparison DoomRL is a little less informative (no turn counts) and doesn't show much in the way of character progression besides assembled items, but has some other content like instead reporting locations and related major events, and also an exclamation! point! after! almost! everything! Exciting!

Architecture

The basic history system architecture was already in place when I wrote about the scoresheet format last time, but there wasn't really any content yet, hence the extremely short history in the sample I shared. Well it turns out that supporting the wider range of messages needed a more expansive architecture as well...

On an individual message basis, the history log is actually quite similar to the regular in-game message log. Both require:

• Support for variables: Although a single message won't really be needed beyond it's own standard context, some of its content details may depend on the situation, for example having acquired a certain item would need to insert the item name as part of the message.
• Per-message knowledge settings: Not all messages are always known, even if some event did technically occur. There could be different general conditions determining whether to actually include the message.

Here are some of the messages I've defined so for, to show what I mean and how they're designated in the data:

GLOBAL indicates the message is always shown when triggered, not requiring any generic condition (other conditions can obviously be applied outside the message code itself, when determining whether or not to trigger it in the first place). ACTION indicates that the message will only be recorded if the actor who carried out said action is visible to the player at the time. POS means the location must be within the player's field of view. Within the message itself, %1 is a variable, and some messages may use additional variables, e.g. %2, or none at all.

Considering the various potential sources of noteworthy events, I also needed to add more than one way to trigger a history message to be added to the log. Aside from hard coding some triggers in the source, especially where they tie into mechanics (or simply hard-coded content), a lot of the history messages would naturally be triggered by other related content, so the architecture needed to support triggering by dialogue, terminal entries, and event scripts, all of which are stored outside the source code. So each of these systems needed to be updated to understand new bits of formatting that would enable them to add history messages to the log when applicable, and even inject variables where required.

As for the internal message data organization--everything exists in a single file! I debated this approach with myself for a while before actually starting on the system, but after working on it for a while it's quite clear there's no debate at all. With so many possible sources of history data, I needed a quick and easy way to see what does and does not already have a message associated with it, all in one place. that and an easy way to check everything for grammatical consistency.

Even now while much of the history message content I've been working on is relatively fresh in mind, I still need to go back and search for existing messages, or confirm certain aspects of them, and having them all in one place is saving a great amount of time there. This means both more efficient work and higher-quality results. And imagine how inefficient a decentralized approach would be down the line when all this stuff isn't fresh!

Here's the external file as it stands now (albeit not shared clearly because of course it's full of spoilers :P):

The largest individual source of history messages, event scripts, originally got syntax to automatically define new history messages in that file, creating them within the script itself without any additions to the code or by explicitly adding new history message definitions. This made it really easy to add new messages, but then they'd be "invisible" to the data and code, which could eventually be problematic for research and consistency analysis. At the start of this history log project I had originally defined about 60~70 messages that way before realizing this was a terrible idea. I'm glad I backtracked on it and moved all those definitions to the single file. Everything is together in one place now. Whew.

Message Types

The architecture is the easy part, after all it's just storing a bunch of message definitions and spitting them out when called upon. But having worked on Cogmind for close to six years now, yeah there's a ton of content to include in history messages, or at the very least considered for inclusion. This requires going through a whole bunch of stuff, hence this recent very not small detour xD

That said, I don't regret waiting until now to do it! Honestly this is better because although it's a lot of work, it also means the results can be even better because all of the needs of the system and various different message considerations can be taken into account at once, finding the best approach that works for everything, rather than making decisions without being sure of what might come in the future (and possibly having to redo old work).

The biggest question when starting out with content is the desired level of detail. There's a clear spectrum in our sample roguelike files above, from maybe one message per floor, to a few, to even many. What kinds of events are important or notable enough to log and serve as a reference for what happened during that playthrough? What's the threshold that will produce a fun and/or useful history log?

This of course also depends on the intent of the log. What's it for in the first place? In my mind, there are a lot of possibilities here:

• record notable achievements that might not appear elsewhere in the scoresheet
• remind the player of their feats, and challenges faced
• shed light on various otherwise "hidden" aspects of a run for other players reading the scoresheet, essentially those things that numbers and item lists won't fully reflect 
• for experienced players, knowing events the player did (or did not!) interact with can help give an idea of the play style at work
• showing the specific order in which everything played out within an individual map is also instructive with regard to style, or just trying to reconstruct the run for other purposes of understanding or analysis.

On this last note, aggregate scoresheet analysis will also become even more interesting, since we can get a much clearer picture of parts of the game players are interacting with that haven't ever been recorded before--lots of encounters and environmental factors that we know and love from player discussion and anecdotes but that previously required proactive discussion to bring out will now be accessible in every scoresheet!

The history log should effectively be kinda like watching a recording, only much (much) faster, and not nearly that detailed since we don't need to know all the common stuff that goes on--so more like a short story or concise summary.

At least I think it'll be decently concise? I haven't done a full test run yet, since the message list is incomplete anyway. I do intend to err on the side of more detail--basically anything that could be seen as unique or defining about a given segment of a run should be recorded (within reason, as in I don't have to add a ridiculous amount of code just to detect something :P).  In any case, it certainly won't be as lengthy or verbose as the message log, and it won't include anything insignificant, so it should be worth reading through regardless.

Note there will be some overlap between the history log and other parts of the scoresheet. Like both include ways to see how the player's build evolved over the course of the run, but each is a different way to examine the same information--in number form it's good for analysis, and extremely condensed, whereas in the history log it's spread out in chronological order among the other events.

But that's also the kind of information that gives other history messages more context, and the history log should also serve as a standalone representation of the run, not requiring any of the other data to supplement or be cross-referenced against it.

So what kinds of message types are we looking at here? Well, there are quite a few categories! Here's an overview:

• Exploration: Evolving and entering a new map
• Plot-related stuff: This is a category that other roguelikes don't really have, but is clearly a centerpiece of the history log, knowing what major events and NPCs the player interacted with (in the past this function has been partially served by the "bonus points" scoring breakdown, which was the impetus behind expanding that part of the scoresheet over the years, but a dedicated history log will do a much better job)
• Encounters: Non-plot special encounters are also an important part of the experience since most still have a mechanical/tactical/strategic impact on a run, so they should be included as well (and there are a lot of these...)
• Enemies: Dangerous foes and taking them down is worthy of the record
• Items: Finding and using special items definitely needs to be in there
• Environment interaction: Destroying special machines or otherwise doing something that affects the environment

The screenshot below was taken straight from my notes that I'm referencing as I do the work, listing all the game data I have to parse looking for content and mechanics worthy of inclusion. It also marks where I've progressed to as of writing this post, since I'm still in the middle of it all. (Note it includes some spoilers if you look closely enough, though most of the low-level details are probably pretty cryptic with all the abbreviations anyway.)

So far I've added 208 history messages, and yet there are more to come... Again, this is taking forever xD  

I look forward to the results, though, and am quite curious how it will read.

Another interesting idea related to History logs is allowing players to manually type notes to be added to the log as well--just enter a note and it'll be appended to the list at that time, recording the turn and location. I'm not sure about allowing players to insert data into scoresheets like that though :P