Researching and Writing Valve's "Secret Weapon" (Patreon)

Hello!

I just released a new GMTK episode all about Valve's development process. And I thought it might be fun to share my own process for making such a video!

The video started, aptly, with the story that opens the episode. During research for a different video (the MDA one, I think), I had stumbled upon an article about how Valve invented GLaDOS as a response to playtester feedback. That's such a cool story! I keep a huge archive of little game development stories and anecdotes on all sorts of topics in case I need them for future videos, so I stashed it away for a couple months.

I thought I could use this particular story to make a simple video about how Valve changed Portal in response to playtesting. I remembered that the developer commentary on that game featured endless anecdotes about responding to playtests, so perhaps I could make a video featuring all those stories? The idea for the title was "How Players Made Portal", which I still think is quite good.

But, as often happens with GMTK, my attempt to make a short, simple video fails and I end up working on something much bigger and more involved.

I started by going through the Portal commentary. To save me playing the game again, I read the transcripts on a fan wiki. I cut out all the ones that mentioned playtesting, and then grouped them into common themes like "teaching", "visual style", "pacing", "difficulty", etc. 

However, I realised that the all-important GLaDOS story wasn't actually in the developer's commentary. Maybe there were more stories out there? I went to my three main sources: Google, YouTube, and the GDC Vault. There I found lots of articles and a few lectures - two on Portal specifically and one on Valve in general.

For each article I find, I make a separate note in my notes app with the name, the link, maybe some thoughts or summary, and copy-pasted key quotes. The talks are a little more involved. I'll generally download the talk and import it into Premiere. From there I can automatically generate a transcript - this is useful for pulling out quotes without having to type them up. Then I'll watch the talk and take notes - I'll paste in the quotes from the transcript and use screenshots to add in the slides. 

One of those lectures, a GDC 2006 talk from Valve employees about their development process, really blew open the video. It was so full of interesting information, and so it couldn't just be a simple thing about Portal - it needed to be a much bigger video about playtesting and Valve in general. 

I could frame it around Portal, but also talk about the playtesting process more generally and include some stories from Half-Life and Left 4 Dead.

At this point I wrote a very scrappy outline of the video. I've learned, over time, to not think of "researching" and "writing" as two, separate steps. As soon as I start writing, even if it's just a bullet-point summary, I'll quickly realise that I have more questions, more things I don't know, more areas that need to be researched. And so I might as well do it simultaneously while researching to help guide the next steps.

On this one I realised that there were a few key questions that really needed answering: like, is Valve's approach to playtesting different to other developers? And where did it come from - how did they figure this out?

To answer the former, I looked for quotes where Valve talked about the importance of playtesting for them, and also did some research into the playtesting approaches at other companies. This made it clear that Valve is different, and the quote in the video about "I was not prepared for the amount of playtesting this company does" was perfect for expressing that to the audience. Also, Gabe's quote about playtesting being Valve's secret weapon felt like a perfect title.

To answer the latter I researched the development of Half-Life 1, which led to a brilliant Game Developer Magazine article, written by a Valve developer, which explained everything - how they scrapped and restarted the game, how they use cabals and playtesting, and so on. Magazines are a great resource for my videos because they often have stuff that isn't all over the internet already. I'll use archives to hunt down PDFs of these ancient issues.

One of the motion graphics I made for this video.

Throughout the research process I had read all the developer commentaries, listened to four podcasts interviews, watched seven talks, and read about 25 articles, interviews, and magazines. The final video cites 25 different sources. 

Like I say, I do have all of these as notes - but while working on the video I actually tend to have it all buzzing about in my head. As I'm writing I'll pop into my memory banks and think "oh, I could use that quote" or "ah, I can reference that story". Then I'll go back later and fill in the exact details from my notes. 

While writing, I always start with an outline that can explain the whole video in a few bullet points. For this video it was something like...

• GLaDOS was the result of playtesting
• Valve used playtesting extensively on Portal
• The student developers learned about this process when coming to Valve
• Valve started playtesting after the disastrous development of HL 1
• They've used it ever since, up to Half-Life Alyx
• Here are some tips they've learned along the way
• Conclusion

A strong skeleton like that helps make sure the video has a clear structure and flow, and isn't just a random collection of tangents and stories. Writing the video was then just a process of trying to flesh out that skeleton, and putting in as many strong quotes and stories as I can. I also try to preempt people's questions and comments - for this video it was important to add in notes that playtesting is not QA or focus testing (a common misconception online), that Valve has resources not available to all developers, and that when used poorly, playtesting can have a negative effect. 

Once the writing is done, I'll move on to the rest of the process - voice recording, making motion graphics, putting in all the relevant clips, adding music, and so on.

Though, I'll sometimes come up with more things I want to add, even after the script has been recorded and I've started editing. For this video I realised that I didn't really have a nice clear example of how Valve used playtesting on Portal (beyond the GLaDOS thing), so recorded an extra line about changing a level where players kept getting themselves killed. And I also thought it was important to state that this approach is best for linear, hand-crafted games, so recorded an extra line for that, too, and plopped it into the timeline.

The final part of the writing process came at the very end. During my research I had obviously came across the story of F-Stop, the cancelled photography-based Portal follow-up. My initial idea was to make a short bonus video on this and chuck it on Patreon and Nebula. But I changed my mind - for one, the story turned out to be quite short and there's not that much information out there on how F-Stop actually plays. And for two, I had been working on this Valve video for a whole month and was very much ready to move on.

So I decided to put the story in at the end, over the Patreon credits. This actually worked super well for improving the video's retention: instead of clicking off during the credits, people stuck around for the whole thing. That boosts the video's watch time by a few percentage points, which helps with the YouTube algorithm!

I'll probably do this more often in the future - there's almost always some little extra story that I couldn't fit into the main body of the video.

Overall, I'm happy with this video. It's funny... as I was making it I kept thinking "how have I not made this video before?!". It just feels like such a classic GMTK topic, and in hindsight it was such an obvious video to make. But I think the main reason is that I didn't really know a lot about playtesting, or value its importance, until I started working on my own game. Doing game dev really has helped me with other aspects of GMTK - so that's cool! I should probably finish that magnet game!

Anyway. This video featured a huge amount of research. I spent dozens of hours just reading, watching, and listening to Valve developers. Luckily for me... this is my absolute favourite part of the job! I just love learning about game design and development, and I enjoy the thrill of hunting down the perfect quote, or an interesting story that can tie the whole video together. Writing and editing the video is just a way for me to dump this stuff out of my head. But hopefully you enjoy what comes out!

Cheers

Mark