Learning to Raid
How do teams learn something hard together with no teacher and no guide? I studied Final Fantasy XIV raid groups tackling a brand-new boss fight.
| Role | Methods | Tools |
|---|---|---|
| Lead Researcher | Case Study, Observations, Interviews, Content Analysis | Boris, R, Excel, Discord |
WHY RAIDS?
An Informal Class That Runs Itself
A raid is a difficult group encounter in an online game. Eight players have to figure out an unfamiliar fight by trying, failing, observing, and adjusting, often over many hours. Nobody is required to show up, and nobody is grading them. That makes raid groups a useful place to learn how collaboration and problem solving work when people choose to learn together, and what that can teach us about designing online learning elsewhere.
WHAT I ASKED?
Three Research Questions
How do experienced and less experienced groups differ in their team mental models: their shared goals, norms, and expectations?
How do those differences show up in how groups spend their time and communicate?
How does each type of group use tools inside and outside the game to collaborate?
Codebook Sample
| Code | Description | Theme |
|---|---|---|
| Hypothesis Generation | This refers to the act of strategizing and creating theories in ways to proceed through a mechanic successfully. When a member proposes a new idea, it can be supported with backup from observations. When an idea is presented to a group, individuals within the group can either agree or disagree with the idea and either provide reasoning or not. | Building Ideas |
| Agreement | Members show their approval vocally or in writing to a proposed hypothesis. This can have supporting arguments to approve |
Building Ideas, Evaluation |
| Observation | Members stating what they saw or are seeing during a raid. | Content Related |
| Question | Member asking a question to the group relating to how to proceed or asking for clarification. | Content Related, Elaboration |
| Prior Knowledge | Referencing or connecting their own or someone else’s experience and knowledge. | Content Related, Prior Knowledge |
| Callouts | Short communications meant to convey commands to the groups. | Group Management |
| Responsibility | Responsibility refers to when the topic of responsibility comes up. Either accepting, denying, or questioning responsibility for mistakes. | Psychological Safety |
HOW I STUDIED IT
OBSERVE, THEN DIVE
Four established groups attempted the first fight of a newly released raid tier without outside guides, while I observed as a non-participant through recorded livestreams. Afterwards, I interviewed three players of different roles of each group, totaling to 12. I coded the session footage for types of talk using a code book I developed. Types of talk included observations, hypotheses, callouts, and questions. I then paired those patterns with the interview themes.
WHAT I FOUND?
EXPERIENCE CHANGED THE PROCESS OF LEARNING, EVOLVING HOW INDIVIDUALS LEARN
Experienced Groups
Produced more meaning-making talk, like observations, hypotheses, and questions, and kept strategizing even during combat.
Spent additional time collecting data while planning strategies.
Turned quickly to tools not found in game when stuck.
Shared responsibility for mistakes more evenly across members.
Built strong social ties and practiced their roles well before the fight.
Produced a team mental model focused on promoting psychological safety and using data to make meaningful progress.
Less Experienced Groups
Needed about twice as much time between attempts to form and debate ideas.
Followed a teacher-centered model built around their most knowledgeable player.
Struggled to come up with ideas when that player stepped back.
Hesitated to use tools outside the game.
Valued social connection as much as, or more than, the learning itself.
Produced a team mental model focused on Guided Collaboration in a deeply social environment.
