GMTK Game Jam 2020 Post Mortem (Patreon)

Hello everyone! 

This month, I hosted the fourth annual GMTK Game Jam. 

A jam, if you're unfamiliar, is a competition where people have to try and make a game in just 48 hours. And the game has to fit a theme, which is announced at the start of the jam.

If you're interested, you can now see my favourite games in this results video. Or go check out the full list of entries over on itch.

This year's jam was absolutely enormous. I knew the jam was going to be big when the sign-ups over on itch quickly surpassed the previous jams. By the time the event began, 18,326 people had joined: well over twice the number who joined up for 2019's jam.

Pretty much every year, about 30% of those people will actually submit a game (accounting for those in teams, people who forget, etc). And 2020 was no different. 29.8% of people submitted, leading to 5,477 games. Again, more than double last year's numbers.

These numbers actually put GMTK in a very nice place. Ludum Dare - a super famous game jam that's been running for 18 years - has always been bigger than GMTK. But this year, we snuck past them! Their latest jam in April received 10,000 sign-ups, and just shy of 5,000 games. (We're still far behind the in-person Global Game Jam, though, which saw 47,000 participants and 9,000 games in January!).

It's not a competition, of course. But it's still nice to be able to say that GMTK had - at the time of writing - the biggest ever online-only jam! 

In this post, I want to talk about a few things that happened in the jam, as a way of remembering what went right and what went wrong: all with the aim of improving things for next year!

The big win: the new rating system

One of the biggest challenges with the jam is deciding who can rate the games. We've always opened the jam to public voting (instead of restricting it to just people who submitted a game) because I firmly believe that the jam should be as open and inclusive to all as possible. It should never be an insular event that's only for the hallowed few who can make games.

This does come with problems, of course. Namely that it's open to abuse and can slip into being a popularity contest. I was determined to find a solution though, and so asked itch if they'd consider making a system where people couldn't vote on whatever games they liked, but would instead be presented with a random game. They'd have to vote on that - before they got given another random game to rate. 

Itch liked the idea so they ran with it, and gave it far more nuance. The final system (which itch finished making just a few hours before the jam ended!) presents public voters with 5 random games to rate. Each time they rate one, that slot is filled with another game. After rating 25 games in this fashion, the person is now able to vote on whatever games they like - giving them the same privileges as those who made a game (and never had to use the rating queue system).

I think it ended up working really well! It made it really hard to abuse the system because it discourages people from making fake accounts, or asking all their friends, fans, or followers to rate their game. It also distributed ratings to way more games: instead of everyone just playing the popular stuff, almost every single game got played and rated. 

So in the end, we saw 143,005 ratings. That's a whopping three times as many as 2019. 

99.7% of games received at least one rating, and the average number of ratings per game was 26.4: a huge increase from previous years. 

Also, while you might think that the rating queue would reduce the number of public voters - it actually went up! In 2019, 40% of votes came from the public. In 2020, that rose to 46%. There also wasn't much negative feedback that I could see (though, some people thought the 25 threshold was a bit high). So all in all, that was a huge win.

The big fail: communication and rule enforcement

This one didn't go so well. When the jam began in 2017, it was a scrappy, upstart community-based event where I could quite easily handle the majority of the communication myself, make up rules on the fly, and answer each individual question. By now, though, it's just too big for that sort of approach.

At this point, I've got an incredible moderation team on Discord (thanks Slam, Adam, Sandra, Hobusu, and Brioche!) but they couldn't do their job properly without really clear, thought-through, and easily enforceable rules from my end. And I didn't always provide them. Which made their job impossible at times. Entirely my fault, sorry! 

So, for example: while you can't change your game during the rating period, you can just upload a better version of the game to Dropbox and then write on your game page "The jam version is broken, go play and rate this one…" Is that allowed? Encouraged? Discouraged? Against the rules? Do we ask you to take it down, or immediately disqualify you? I don't know! 

I need to think more carefully about what things will happen in the jam (we've done four of these now, so there shouldn't be too many surprises) and construct and communicate easy-to-enforce rules. And if a surprise does happen, I need a better way of handling that. I'm looking into making a website for the jam rules, so there's a single, easily-updatable, one-stop source for information. 

Another thing that I need to fix is the grace period. After every jam, I've given people a certain amount of time to get late submissions to me. This year, I was utterly bombarded with over 300 emails from people. It took me and my fiancé until 1AM to sort through them all and reply, which just wasn't tenable. I've been working with itch on some solutions to make this easier to deal with. 

Other lessons

Art and audio - This year I tried to reward artists and composers by having a presentation rating criteria, and letting people say if they made the art and audio during the 48 hour window. 

Both were well received, but the latter needs more nuance (I'll change it from "all the art" to "the vast majority of the art"). Also, I'm working with itch to be able to set the weightings of each criteria, so presentation doesn't count as strongly towards the overall score as fun or originality.

(This year, people could personally decide on the overall score for each game. But it's also possible for itch to create an automatic overall score by averaging out the other results.

My influence - I do have to be mindful of my influence on the ratings. One of the winning games was "Cleaning the System", which I played on one of my streams, absolutely loved, and this caused a whole bunch of people to go rate the game highly and push it into the top 100. Good for me, but maybe not exactly fair.

This can also go the other way: on my streams, I was a bit dismissive of heavily narrative games (simply in relation to how they slow down the pace of a stream), which could bias the sort of games people make in future.

Obviously the best solution would be to not stream games at all. But because one of the main draws of this jam is that there's a chance that Mark Brown, king of video games, might play your game, I'm not going to stop doing that. I'll just be more mindful of my incredible powers.

Marketing week - I'm always a little concerned that the jam breaks down into 48 hours of making a game, and then… 160-odd hours of marketing it. And just in general, it's not ideal that people with a large fanbase (such as game devs with YouTube followings) will get more eyes on their game than anyone else.

The rating queue I discussed earlier definitely helped with that. I'm also not sure if itch should show the popular and top rated games during the voting week. Better to focus on random games and karma (which is where devs can increase their karma by voting on other people's games), perhaps? 

But at the same time, I do think that it's apt that marketing plays a role in the jam. If a game jam is a microcosm of game development in general, then it's suitable that developers need to spend some time making a pretty thumbnail, getting their game to YouTubers and streamers, tweeting about their game, and so on.

So, definitely something to tweak but not something I want to remove entirely. 

Some fun stats!

Okay, one final thing. Let's do some stats!

Pink River was definitely the most popular game in the jam. It received 866 ratings and 391 comments. Being on the front page of itch for a day probably helped with that… 

Over 130 games were called "Out of Control", or some slight variation. Over 600 games had the word control in the name.

I asked people if they made their own art during the 48 hours. The vast majority of people (77.6%) did, which was interesting.

As for audio? Only half. This one's a little less useful, as we don't distinguish between music and sound effects. I know lots of people made their own music but used preexisting sound effects, so had to pick "no". That's getting fixed for 2021. 

What about engines? 

The majority of games were made in Unity, which wasn't surprising. I'll need to provide slightly different options next year: Construct and Stencyl weren't as popular as I thought, and there's a lot of games in "Other / None". So I'll break that down further next time!

Anyway, I think that's everything. I'll be seeking out feedback in a formal capacity next week by way of a survey, so keep an eye out for that. But if you have any questions I might be able to answer, please drop them into the comments down below.