How we run a focus group

A focus group can only tell you what the people in the room think. Most of the work is making sure they are the right people, and that what they tell you is what they actually think.

1. Start from who buys the game

Every study starts with a spec: the players the studio sells to. We write it in terms we can screen for, such as the genres they play, the platforms they own, how much they play each week and the games they have finished recently.

“Fans of action games” is a hope. “Plays on PS5 and finished two third-person action games this year” is a spec.

2. Screen before anyone is invited

Players come from our panel. Everyone on it is 18 or over and has opted in twice: once when they signed up, and again when they confirmed their email address.

Before an invitation goes out, a short screener checks each player against the spec. We don’t invite people who work in games or games media, or anyone who has already seen the build.

Where a study needs a particular skill level, the screener includes an unpaid 15-minute trial, so a player’s profile is checked against how they actually play. A profile can say “plays on hard”; the trial shows whether it’s true.

3. Pay, then protect

Every session is paid, typically US$75⁠–⁠150 for 60 to 90 minutes, and the invitation says exactly how much and how before the player accepts. Every player signs an NDA before they see anything, and the studio’s name stays out of the invitation unless the studio wants it in.

4. Watch first, ask later

A moderator’s job is to find out what players think without telling them what to think. That shapes every question we ask.

• Open questions. “What’s going on here?” instead of “Was that confusing?”
• Behavior before opinion. We watch what players do before we ask what they think of it, because people explain their choices after the fact.
• Nobody defends the game. When a player is stuck, the moderator notes where and why, and only steps in if the session can’t go on.
• Alone before together. Players note their own reactions before the group discusses them, so the loudest voice doesn’t become everyone’s opinion.

5. Separate what they did from what they said

Every session is recorded. Afterwards we log each observation against its timestamp and group the observations into issues.

We keep two columns apart: what players did, and what they said about it. When the two disagree, what they did usually tells you more.

Each issue is then ranked by severity: how badly it affects the players it touches, and how many players it touches.

6. Report what to fix first

The report opens with a summary a producer can read before the next meeting. The findings follow, most severe first. Each one comes with the evidence behind it, the moments in the recordings where it shows, and a specific recommendation.

A debrief call then walks the team through the findings, so the questions they raise get answered by the people who ran the sessions.