Friday, February 12, 2010

Level Design Primer: Notes on Playtesting

Friday, February 12, 2010
Playtesting is like working out--you can go a long time telling yourself you don't need it, and then when you finally get around to doing it it will hurt so much you'll probably stop again for too long.
But resist the temptation to quit. Understand that testing is an inextricable part of good design; what you're making is for the player after all.

You Are Not Your Work

I feel a little silly parroting this again, but it's worth it if gets through to just a few more people. Remember the lesson of every teary-eyed critique in art classes the world over--you are not your work.
It is of you, but it is not you. While you may be deeply invested in it, once you have made something it is out there in the world and separate from you. No matter how coarsely phrased a criticism may be, remember it is a reflection of the work and not you personally.
There are creative professionals that get by without learning this but boy does it cause them no end of grief. The pride and joy of creation I think must necessarily follow with detachment; a distanced, even clinical appraisal.
While I've found this kind of professional detachment also tends to shave the peaks of euphoric highs during development, it's more than worth it by pulling out of the gutting lows. You can be still be passionate and excited about your work without taking the emotional rollercoaster that comes from not being able to separate your work from yourself.

Conducting The Test

[This is assuming you're using what Will Wright calls a "Kleenex tester," a player without prior experience with the game who will likely not test again, rather than a professional or regular tester.]

Make sure your tester feels comfortable enough to talk freely, but do not get too friendly with them. Exerting the social pressures of a new acquaintance (or invoking your friendship with someone you know) means getting polite answers instead of useful ones. People will likely downplay or politely lie to your face to avoid an uncomfortable situation.

I usually start with a very short capsule summary of the game and it's premise (no more than a few sentences), along with any mission-critical info if they they're playing a level that's not the beginning of the game. I also make sure they are able to set any control preferences before they begin (though I don't know why we tolerate your kind, EDSFers. Go back to the Moon.)

Then it's time to prime the tester to think outloud. I'll say something like the following:

"What I'd like you to do is say anything that comes to mind while you're playing, things like "I'm frustrated," or "this part is cool."  Don't worry about offending me. If something is really bad you're doing me a favor by mentioning it, it'll help make the game better. Ask questions if they come up while you're playing, or if something isn't clear say so and I'll respond to everything--but after the test is over. While you're playing just enjoy yourself, pretend I'm not here."

Be very aware of how much physical positioning factors into the tester's comfort level. Grab your notebook and sit several feet back, as far back as you can off to the side and well outside their peripheral vision. Making sure they don't feel like you're hovering is important--nobody likes that feeling, and testing can already feel a little weird for most people already. Giving the player headphones can also help them feel less self-conscious. You can still come up and fix a show-stopping flaw or restart the level as necessary, but generally try to make yourself invisible for the duration of the test. You're there to observe.

Slow Motion Trainwreck

And here comes the hard part. Now you get to see all your brilliant plans laid to ruin as what you thought was simple and straightforward to the player is anything but. Try to take it in stride; now when I playtest I almost feel like a classic Freudian analyst, or a scientist regarding a lab trial. Clinical detachment is useful here.

Once the level is out of your hands there's nothing you can do to help it if something goes wrong. So if during the playtest something does happen (such as the surprisingly common scourge of "player error"), short of a crash or other complete show-stopper, you're just going to have to grin and bear it. This can be one of the most punishing aspects of a playtest but this is your lot. What's hardest to take often ends up the most useful information.

So pay close attention. Get impressions down immediately, but make as many notations about specific problems as you can. Missed cues, objectives that are ignored or unseen, key dialogue that doesn't seem to be heeded, horribly unfair firefight, collision problems, the player wandering off the map--all of the above might happen in a single test.
    This is your chance to see your world through someone else's perspective, so keep your eyes on the screen and write down any comments they make. Make a mental record of where they're looking--if there's an important element that never crossed their sightline, why or how did it happen? Did you misjudge the clarity of your layout?
As we talked about in the previous LDP article, no one tester's word is law, their experience necessarily represents just a single small data point. Yes, a single test can reveal a lot of issues that obviously should be fixed, but generally speaking you're looking for patterns from multiple sessions; reworking too much based on a single test can be very counter-productive. Balance your observations with instinct.


Regular testing will likely result in a greater knowledge of your own bias/hyperawareness of the level. What you feel like is "really overdoing it" might just be noticeable to the player, and the touches you consider "subtle" will probably escape notice entirely.
So test early, test often--it will get easier, and you'll have to redo so much less work if you get playtesters in at the earliest opportunity. Your sense of what works and what doesn't will naturally sharpen. You'll find yourself building level elements that anticipate typical player behaviors, rather than having to come and fix them after a test.


draco said...

quick read with a lot of thrills

welcome back to teh blogosphere mate!

Copperkat said...

Good read. I'd love to hear some of your personal experiences with play testing.

gauss said...

Hey thanks Copperkat. Most of the time, there's not much to tell. Well, maybe two quick stories.

One being that my awareness of physical positioning came from watching an overzealous designer physically crowd a tester--I mean literally at his elbow the entire playtest, staring him in the face.
Afterward he pressed him with leading questions about the game's unquestionable magnificence (which recalls my father's advice about not asking your family's opinions about a new girlfriend with "isn't she great?"). Had I been the tester I would have said anything positive about the game if it meant that I would have gotten out of that building unmolested.

The second story is one about instincts and confirming suspicions. I think I may have mentioned in a previous LDP article a personal design maxim: within conventional FPS design mores, you can't expect the player to make a sharp change in course, beyond about 45 degrees in so, if he's in open terrain (and the objective marker is pointing forward).
I knew this when I had built a certain sequence, and should have listened to my instincts. Not only did I end up correcting the offending portion of the level after numerous playtests, after release I've seen a few message board posts detailing exactly the same problem.
It's been both a blessing and a curse: a relief in the sense that it's confirmation that my instincts were right, but also a condemnation for not more significantly restructuring the sequence or removing it.

Anonymous said...

How much do you find the average playtester talking during the test? I ask because I'm infamous for doing that when I play any single-player game. My friends often come over just to watch me play through a game, and to enjoy the shouts of excitement/dread as I near the end.

Imagine the stereotypical movie-talker. "No, don't open the door! the killer is in there!!"

That's me.

gauss said...

It varied. When it was with friends and they were relaxed and knew that they could be honest and open about their experience while it was happening, and how helpful it could be to the process, they would tend to open up more. The talking--specific articulations about frustrations or blindspots ("man I am lost", etc.) were the most helpful.
Every once in a while a tester gives a running commentary and just one little nugget is a really important revelation for how something needs to get amended, changed or expanded.

The playing commentary thing can be interesting, in my experience it tends to change a lot of your own experience of the game when you have an audience of sorts for your jibes or heckles. Personally it tends to take me out of the action, but gaming with friends is generally a lot more fun.

Ben said...

On that note, do you ever get any unexpected *physical* (as opposed to verbal) responses from playtesters? When I think of my brother playing video games with a controller, he's usually moving his controller up and down in a rather exaggerated manner (as if it will somehow help him turn a corner in Gran Turismo, or reach a platform or hit an enemy in Sonic), sometimes even almost jumping up in his seat. I should note he's a perfectly well-rounded individual and doesn't have some hyperactive disorder. Is this something designers look out for, or indeed hope to achieve?

gauss said...

Hahah yes that is a desirable outcome. When you get physical response like that or my favorite, watching someone trying to lean around the corner in reality to get a peek at their target, it's an encouraging sign that they're deeply engaged in the game.

Reminds me of an engagement test I heard about but never got a chance to try--the so called "coke test".
What you do is ask the tester if they want a soft drink, but wait to deliver it until after they start playing. If they're fully engaged in the game (or other program), they generally forget about the coke.
The moment they stop, sit back and see the coke and take a sip from it comes when they're no longer fully engaged. So the more desirable outcome for testing is for that first sip of coke to come later and later.

dean said...

As a cartoonist, not a games designer, this is still fascinating stuff.

Anonymous said...

These articles nearly make me cry. I spent 30 grand going to Full Sail for game design, didn't finish because I can't code to save my life, yet I walked away feeling like I learned a lot about character and level design. I read like 4 articles from a professional on the same principles, for FREE, and I've learned more than I could have imagined in school.

Nothing against Full Sail I guess, it was a good school and the instructors did know what they were talking about...but we were taking classes for 7 days and moving on to something else. But still! This is college grade design curriculum here. Makes me want to become a designer again, though I know that day is long gone.

Keep it up gauss.

gauss said...

Sorry to hear about your experience Anonymous; the opportunism in higher education with respect to game development is deplorable.
It's a time when half the developers themselves are still figuring out best practices so it's more than a little disingenuous for some colleges to claim they can teach people how to make games. They're not universally bad, but they can't be easily recommended because of stories like yours.

Still though I'm glad that you enjoy this site and it's articles. I do wish you'd reconsider though; hope springs eternal.

Anonymous said...

Hah, gauss, don't doubt that the thought of giving it another whirl in some fashion isn't rattling around in my mind. Almost every other month I consider bugging my old high school pals that still live in the area to throwing a game together using XNA or something for the XBLA or some such. Know a couple artists, two computer engineers, among other talented people, and I'm the best writer among the bunch.

We wouldn't be the equivilant of BioWare, Valve, Bethesda or id by any stretch, but hey, I think we've got a decent core. If only we didn't all have jobs/women/children/still going to school/all the other piddling crap that comes up in life to deal with!

Post a Comment

gausswerks: design reboot. Design by Pocket