The Open World genre has grown a lot over the past decade, with well-known franchises from multiple game studios. These range from the space opera Mass Effect to the criminal underworld of Grand Theft Auto. Yet this is still a relatively young and fluid genre, and that made us curious to see if we could identify interesting axes in this genre space using our gamer motivation data.
The Gamer Motivation Profile allows gamers to take a 5-minute survey to get a personalized report of their gaming motivations, and see how they compare with other gamers. Over 220,000 gamers worldwide have taken this survey. The 12 motivations that are measured in our model were identified via statistical analysis of how gaming motivations cluster together (based on data from 30,000 gamers).
Want to see how you compare with other gamers? Take a 5-minute survey and get your own Gamer Motivation Profile.
In the Gamer Motivation Profile, we also ask gamers to list games they’ve enjoyed playing. This allows us to generate motivation profiles not only for individual gamers, but also for game titles. We do this by aggregating the scores from the gamers who listed that game as a game they enjoy.
In this post, we’re going to map out the Open World genre, and we’re going to surface more of our analytic approach.
From Gamers to Game Titles
Our working assumption is that the core audience a game manages to attract provides the best proxy score for the game as a whole. A game with a good elaborate narrative will generally attract gamers who are interested in elaborate narratives. And a game might try to do many things at once, but its core audience will reveal whether only certain aspects of the game were playable and appealing. Certainly, the top 1% most hardcore gamers of any game (on any dimension) will play it in a dramatically different way from everyone else, but a game’s core audience (we would argue) is most representative of what a game is about.
The core audience a game manages to attract provides the best proxy score for the game as a whole.
A Quick Note on the Open World Games
For the data, we grabbed popular Open World games like Fallout and Assassin’s Creed. In the survey, gamers were allowed to list both game titles and game franchises, and we captured this information. In the charts below, games without specific title information refer to the franchise as a whole.
1st Attempt: Pick the 2 Motivations with the Most Variance
Given that we have 12 motivations in our model, and there are only 2 axes on a basic scatterplot, we need to figure out which motivations to use for our axes.
One approach is to just pick the 2 motivations on which the games vary the most on to try to maximize the differences in the scatterplot. In our case, these 2 motivations turn out to be Story (elaborate narrative, interesting characters) and Design (expression, customization).
The plot shows that overall, there’s a positive correlation between Story and Design. Open World games with elaborate narratives tend to provide more customization options. Games like Dragon Age maximize both, whereas games like Grand Theft Auto don’t really emphasize either narrative or customization. Uncharted is an interesting outlier–it has a moderately complex narrative, but has comparatively few customization options.
But because the two axes are so correlated, this particular graph isn’t really giving us a good spread of the genre.
2nd Attempt: Pick the 2 Motivations that are Most Conceptually Meaningful
Another way of going about this is to pick the 2 motivations that we think most differentiate gameplay in this genre. Last week, we had a blog post about the preference for exploration vs. campaign completion in Open World games. So we might pick Story (elaborate narrative, interesting characters) and Discovery (explore, tinker, experiment) as the two axes.
Now, we’ve got a much better spread across the map. And it’s clearer that the major game studios are clustered in different parts of the map. The Bioware games (Dragon Age and Mass Effect) emphasize narrative and characters while the Bethesda games (Fallout and The Elder Scrolls) emphasize open exploration and tinkering. Uncharted, with a more linear narrative structure, scores comparatively lower on Discovery.
The major game studios are clustered in different parts of the map.
This conceptually-driven method allowed us to pick better axes, but with 12 motivations, there are 66 unique combinations out there, so we need a more reliable way of figuring out the best visualization.
3rd Attempt: Combine Motivations to Maximize Coverage
Ideally, we want a repeatable process that maximizes the unique variance covered by each axis, and allows us to visualize more than 2 motivations at a time on a 2D map. Fortunately for us, there’s a statistical method to do just that.
And it’s the same method we used to identify the motivations in our model in the first place. We can use factor analysis to identify the 2 primary axes that maximize the variance across these game titles. These axes are created by aggregating highly related motivations, such that the resulting 2D map surfaces the relationships between a much larger number of motivations.
Let’s unpack these new axes one at a time, starting with the x-axis. The factor analysis found that, among Open World games, Story is strongly negatively correlated with Power (leveling up, getting powerful gear), Challenge (practice, high difficulty), and Competition (matches, high ranking). In fact, this relationship is so strong that it can be considered its own spectrum. On the left of the spectrum are the games that emphasize power accumulation and skill-based mastery (and don’t emphasize elaborate narratives), while on the right are the games that emphasize interesting stories and characters (and place less emphasis on accumulating power and difficult missions).
The factor analysis packs highly related motivations onto the same axis.
Now, let’s turn to the y-axis. The factor analysis found that Excitement (fast-paced action, thrills) and Destruction (guns, mayhem) were strongly negatively correlated with Design (expression, customization), Fantasy (being someone else, somewhere else), and Discovery (explore, tinker, experiment). Thus, games on the lower end of the axis emphasize fast-paced mayhem (rather than customization and exploration), while games at the upper end of the axis emphasize discovery and immersion (rather than guns and thrills).
The factor analysis packed a lot into our axes, but in surfacing these two underlying spectrums in the Open World genre, it is essentially letting us see how these games vary across 9 motivations all at the same time.
The Quadrants
The richness of the axes also hints at the four interesting quadrants in this map.
Open Mayhem: These games focus on fast-paced mayhem and explosions without a narrative boxing you in. Instead, there’s an emphasis on amassing power (stronger character/gear) and taking on challenging missions and tasks that take time to practice and master.
Action-Packed Narrative: These games feature thrilling action in the form of a structured narrative. While these are Open World games, the core gameplay revolves more around the linear “set pieces” rather than exploring the world.
Deep Story: These games have elaborate narratives and a wide cast of characters with different personalities and interesting backstories you can interact with. The gameplay pacing is slower, and there’s a strong emphasis on character customization, open exploration, and a rich alternate world.
Immersive Challenge: These games also give players a rich alternate world to explore with plenty of customization options, but instead of delving into a deep narrative and character relationships, the core gameplay emphasizes leveling up and accumulating power (think min-maxing) to take on challenging missions and tasks.
A Note on Overlap Between Game Audiences
Some of the dots from the same franchise are incredibly close to each other. So it’s a fair question to ask whether this is simply an artifact of the same gamers listing both games in the survey.
It turns out that there’s very little overlap. Take Mass Effect (series) and Mass Effect 3 for example. There are 10,713 gamers in the Mass Effect (series) dot, and 1,404 in the Mass Effect 3 dot, with an overlap of 135 (gamers who listed both games and counted in both dots). The same is true for Oblivion and Skyrim; there’s about a 10% overlap for the less populous dot. So these dots are based on mostly different groups of gamers.
Unpacking a Genre Space
Our attempts at mapping out the Open World genre led us to identify a 4 quadrant segmentation. Not only do the games cluster by game studio, these clusters also tend to fall into their own quadrants–the split between the Assassin’s Creed franchise and the recent AC titles is an interesting exception. In a way, the map is highlighting how different game studios have specialized within the genre space, and the game features and gameplay they prioritize. Each of the major studios has carved out their own section of the map to play in.
The map is highlighting how different game studios have specialized within the genre space.
Your Thoughts on the Level of Detail
We have a broad audience in mind for our blog posts, and with data like this, it can be difficult to gauge whether the explanation of the methods is interesting or if it makes the content too hard to follow. Let us know your thoughts in the comments below!
Hi Nick,
I found this post very interesting, and in general think it’s a pretty good approach to trying to classify open world games. I found it fairly readable, though I also do data analysis for a career so even concepts which I am not directly familiar with (e.g. factor analysis) still broadly make sense. I really appreciated the way you unpacked the axes in your final chart.
I had two quick points – I took your survey, and while I think you asking people what their favorite last few games are is a way to understand their preferences as a gamer, I think it may also be important to ask how much they played each of them, since someone who has played their top game for 300+ hours but their next favorite game for only 60 will be different than someone who has consistently played each of their favorite games for more than 100 hours, per se. I think this would be an important dimension to capture because someone who says elder scrolls 5 is one of their favorite games but only played it for 40 hours clearly didnt take the time to delver into the open world aspects of that game, whereas someone who spent 500 hours playing grand theft auto 5 may have treated that game as more competitively, or found ways to enjoy the gameplay experience that you wouldn’t otherwise expect of the average player. I know you sort of take this into account when you say the people who play any game the most are clearly playing it differently from other players of the game, but I think you could draw a clearer line there by asking for this information.
The second point is that I’d reconsider, at least a little bit, your interpretations of the quadrants in your final chart. While I don’t want to project certainty when discussing other people’s preferences, I’m pretty sure most gamers who played the games would agree that Mass Effect 3’s story was not deeper than Mass Effect 1’s, and definitely would not say either Dragon Age game’s plot was deeper than Skyrim or Oblivion, though your chart would suggest both of these to be the case. I wonder if your analysis may actually be picking up on “cinematic” narrative telling – for example, games that have extensive amounts of cut scenes or other pre-rendered narrative detail. Did you consider this?
On a side note, I’m really happy Quantic Foundry exists. I look forward to future posts.
Hi Jake – The “time played” issue is interesting and complicated, especially for people who burn through the campaign in open world games and don’t bother exploring. Including “time played” as a filter variable may bias the data towards open world play styles (or some other confound). And it raises the question of whether someone who simply played Fallout 4’s campaign (without exploring) has any less valid an opinion as someone who just explored post-apocalyptic Boston.
For the second point, note that “Mass Effect” refers to the franchise as a whole–i.e., people who put down “Mass Effect series” or “all Mass Effect” in the text entry. So the distinction is between the franchise vs. the last entry in the series.
In terms of “Story”, the factor also includes the breadth and depth of the characters you meet and interact with–the relationship drama part of the game, so I think that’s is pushing the score higher for the Bioware games, where there’s a wider cast of characters with drawn out backstories. Although what you’re suggesting is possible too–that the extensive “cinematic” scenes appeal to gamers who are interested in narrative.
You make a good point that a person who played straight through Fallout 4’s campaign may have just as valid an opinion as someone who took their time, but I guess what I was talking about was how this impacts your core assumption. If you’re using the type of people who constitute a game’s “core players” to proxy for the qualities of that game, you could be be led to an incorrect conclusion if your sample happens to include a bunch of run-and-gun players who value action over narrative and take a game like Fallout 4 and burst through the campaign only. Then, at least as I understand your methodology, you may classify a game like Fallout 4 as a run-and-gun, narrative-light game, when that surely isn’t how you’d want to classify it if you took a step back.
I guess what you’re saying is that even if Fallout 4 is designed to be an open world, “explore me forever” kind of game, if the majority of its core players aren’t playing it that way, what does that say about the type of game it is? Perhaps your survey is better at identify the “true” genre of a game, as opposed to its “intended” genre. At any rate, I wonder if you guys could make a deal with Valve to allow players to voluntarily submit their steam library statistics to you – what games they play, how much they played them, etc. That’d be a treasure trove of data for answering these kind of questions. Valve may closely guard that sort of information for its own marketing purposes, but it is the kind of information you can see publicly if you look at someone’s steam profile, so it’s not that guarded – it’d just be annoying to manually aggregate.
For Mass Effect – I see that now, and had read over this fact previously.
For story, I also understand what you mean, and I guess I’d just be quibbling about what makes a story deep which isn’t an especially important point.
Thanks for your response.
About the “time played” factor… most games I have only played for a small chunk or chunks of hours, but there are a few multiplayer games that I played for years and years for hundreds or thousands of hours that really stuck with me. The original Starsiege: Tribes, Quake 3/UT, Half-Life + mods/CS, EverQuest, PSO, Battlefield 1942, TF2, LoL, Global Agenda, HotS…
FFXI I played for 8 years and logged something like 10000 hours. Even if I was only actively pushing buttons for 7000 hours or so, it was a core part of my life for a really long time.
Some games I haven’t played as long, but have played over and over, like Super Mario World, Mirror’s Edge, Ape Escape, Diablo 1/2, Jumping Flash 1/2, Perfect Dark, the modern Castlevania GBA/DS games, Chrono Trigger, all the 2D and 3D Metroids…Bethesda open world games…
There may not be any difference in the way similar people view such games, but I do feel like certain games that I have invested significant chunks of my life to are statistically and personally more significant than others. FFXI, for a long while, fit the mold of the most appropriate game for me ever made, but is considered just as strongly by the system as something like Rocket League, which I have barely played in comparison.
Maybe time played per game is not important for the current scope of the surveys and analysis being done, but would think it merits some consideration.
Jake has some good points, but I disagree re: story depth with Dragon Age vs. Skyrim or Oblivion. With Dragon Age (all of them), story is basically all there is, and for the most part the core plotline is done in a certain order, with deviations for side quests. With Skyrim, once you get past the opening set of events (or if you have an alternate start mod), you can avoid the “main” story indefinitely, and play the game simply for resource gathering and treasure hunting. There are numerous side stories throughout Skyrim, and yes, some of them connect to the main story, but the entire game as a whole is much less story-driven than anything in the Dragon Age series.
My only issue with the statistics here is the placement of a couple of things in the graphs, primarily with where the first Assassin’s Creed game falls. I wouldn’t even consider the first AC an open world game, given that the plot goes in a particular order and there’s no way to deviate from it, as well as the fact that while you can pick and choose what to do and when while in a city, you do eventually have to do all of it to progress the story. The cities are closed regions, as are the roads connecting them, and they have specific problem spots with lots of guards, or people in your way, etc. But regardless of whether it’s open world or not, the fact that on these graphs it leans more toward power collection and fast-paced combat that isn’t narrative driven seems weird. There’s no leveling up in it, only the addition of more weapons to your arsenal and therefore more specialized attacks, and while skill does play a part in the gameplay, at a certain point you know how to deal with all enemy types easily, and combat becomes outrageously easy at times. The combat isn’t even particularly fast-paced; the enemy AI isn’t great, and they tend to come at you one at a time (and the rare times they don’t, it’s very easy to just sidestep the one you’re not actively fighting). I’m not sure where else a game with stealth mechanics would fall on these graphs, but the placement of AC just feels off to me. Initially I felt the same way about the placement of Borderlands, because it has a great and pretty huge story, but the bulk of the game IS just running around getting new guns and murdering enemies while blowing things up, so that one’s alright.
In the charts, games without specific title information refer to the franchise as a whole. Definitely my fault for not making this more clear. I went and bolded/italicized the line where I mention this earlier in the post. So “AC” refers to the franchise as a whole–i.e, the game audience that is a fan of the franchise (as opposed to gamers who specifically liked one of the titles).
You are missing world of warcraft and Final Fantasy 14: ARR on the charts above.
We’re saving all the MMOs for their own chart :)
Why is Uncharted mentioned in the first place?
We generated the game list using a triangulation approach. We started with games that we know are good Open World exemplars (Fallout, Assassin’s Creed, Far Cry, etc.). In our data set, we can use any game to look up its neighbors (games with the most audience overlap–corrected by baseline game popularity). As we iterated on this selection, Uncharted kept coming up. You’re right that it’s not technically an open world, but the devs describe it as “wide-linear”, and its outlier status is reflected in the charts.
In hindsight, Uncharted was helpful to include because it made it a lot easier to understand what the lower right quadrant was about. Uncharted made it easier to decipher why the two AC titles were in that quadrant and to label what that quadrant signifies.