Game Genre Map: The Cognitive Threshold in Strategy Games

By | 2016-10-17T20:02:33+00:00 January 20th, 2016|Analytics, Video Games|100 Comments

The Gamer Motivation Profile allows gamers to take a 5-minute survey to get a personalized report of their gaming motivations. Currently, we have data from over 140,000 gamers worldwide. In the survey, we also ask gamers to list their favorite game titles. This allows us to pivot between gamers and games–we can use the aggregated game audience profiles to compare games.

For example, is Civilization more strategically complex than SimCity? Well, we can compare their audience Strategy scores to find out. In this sense, the Gamer Motivation Profile isn’t just a benchmarking tool for gamers, it’s also a benchmarking tool for game titles.

The Gamer Motivation Profile isn’t just a benchmarking tool for gamers, it’s also a benchmarking tool for game titles.

Mapping Out the Strategy Genre

These audience scores allow us to explore game titles within a genre with incredible precision. Let’s take a closer look at the Strategy genre. First, we’ll pick 2 game franchises—Civilization and StarCraft. We can then use our data set to find the games most closely related to these two exemplars. This gives us a good mix of turn-based strategy, real-time strategy, management sims, and grand strategy games.

Game audience scores allow us to explore game titles within a genre with incredible precision.

Now, let’s plot all these games using just their Strategy and Excitement scores. In our Gamer Motivation Model, Strategy is the appeal of thinking, planning ahead, and making decisions; Excitement is the appeal of fast-paced action and gameplay that rewards rapid reactions. In the plot below, the games in the lower-right corner have high Strategy and low Excitement, while the games in the upper-left corner have low Strategy and high Excitement. And just to be clear, by “low”, I mean low relative to other games in the Strategy genre.

Strategy Games - Map

[Updated Explanation of Strategy & Excitement 01/24/2016]: Strategic complexity can come from different mechanics: having to decide which other players/AI to treat as allies or enemies and when to do so (vs. being assigned static allies and enemies), the variety of victory conditions and deciding which to pursue (vs. the same goal for all players), the variety of resources/factions that have to balanced and managed, and in general reflects the number and complexity of variables that have to be considered and the amount of planning that is necessary to reach a long-term goal.

Certainly, at the top 1% of competitive play, some games that are low on Strategy may be incredibly complex strategically, but we think the core engaged audience of each game (i.e., the average rather than the 1%) is more representative of what a game is about. This also helps explain why a game like Cities: Skylines has a relatively low Strategy score. While this city builder can be played in a painstakingly methodical way to optimize traffic, its positioning on the Strategy axis suggests that its core audience appreciates the ability to enjoy the game without being forced to play in a highly strategic way.

Excitement, in addition to speed and pacing, also captures the amount of thrill and surprises. This is why XCOM, with its combat visuals and possibility of critical hits, is more exciting to watch and play than Europa Universalis even though players in both can take as long as they want to make a move. The time pressure and possibility of game-changing card plays in Hearthstone is also what increases its Excitement even though it is a turn-based game.

A Proximity Map of Games

This visualization also functions as a proximity map. Games that are more similar will appear closer together than games that are less similar. So we see the 4X Strategy games cluster together in the lower-right section. In the middle of the map are the Real-Time Strategy games. We see Management Sim games in the lower-left section, and finally MOBAs in the upper-left area.

Games that are more similar will appear closer together on the map.

The map also shows how these sub-genres are related to each other, and in particular, the games that function almost as bridges or gateways between sub-genres. For example, do you enjoy slow-paced 4X games like Civilization, but want to try something more strategically complex? You should take a look at Europa Universalis.

The Empty Space is the Cognitive Threshold

There’s a good spread of games along both motivations, but the map reveals that the upper-right part of the map is entirely empty. In hindsight, this makes a lot of sense. The more strategically complex the decisions you have to make, the more time you need to process the information. There’s a cognitive threshold beyond which forcing you to make complex decisions under time pressure is simply not fun anymore. The games along the edge of this empty space are tracing out this cognitive boundary of fun.

Strategy Games - Cognitive Threshold

It’s interesting to see that DotA—a game noted for being hardcore—is either right on or, more likely, slightly beyond this cognitive threshold.

There’s a cognitive threshold beyond which the game is simply not fun anymore.

A Short Segue to the Theory of Flow

Psychology and game research geeks will likely intuit that this is a genre-level example of the theory of flow applied to gaming. Engagement occurs when a game can balance the gamer’s increasing skill by increasing the game’s difficulty. If the game’s difficulty outpaces the player’s skill, the game becomes frustrating. And if the player’s skill outpaces the game’s difficulty, the game becomes boring. Here in our strategy games map, we are seeing the upper limit of challenge across games within a genre.

The map of strategy games is a genre-level example of the theory of flow.

Easy Fun vs. Hard Fun

If the cognitive threshold is the line of maximum challenge, then the distance between a game and the threshold is a metric of easy vs. hard fun. Thus, we can trace out roughly 3 bands of games based on their distance from the threshold.
Strategy Games - Hard Easy Fun
Games in the Hard Fun band are demanding and unforgiving when mistakes are made–they are designed to tap as much of your available cognitive resources as possible. They tend to have highly specific goals and end-states. Games in the Balanced Fun band have a moderate amount of strategic complexity. They are designed to require thoughtfulness, without being demanding about cognitive resources. And it is possible to recover from minor mistakes. And then finally, games in the Easy Fun band tend to encourage more free-form creative play, where being strategic is an asset but far from being a requirement to enjoy the game. And where making mistakes is encouraged by design or is in fact part of the fun.

Games in the Hard Fun band are demanding and unforgiving when mistakes are made.

Genre Mapping Reveals Motivational Insights

The aggregated profiles from our Gamer Motivation model can be translated into game benchmarks. Not only does this allow us to visually map out the proximity of games within a genre, it also surfaces hidden psychological insights and motivational trade-offs underlying a game genre.

Whether you’re trying to understand the main motivational dimensions of a game genre or how your game is positioned among related games, we have the data and a repeatable methodology to powerfully visualize the hidden relationships between game titles and audience motivations.

We have the data and a repeatable methodology to powerfully visualize the hidden relationships between game titles and audience motivations.

Explore Game Audience Reports
Contact Us For Custom Analysis
By | 2016-10-17T20:02:33+00:00 January 20th, 2016|Analytics, Video Games|100 Comments

About the Author:

Nick is the co-founder and analytics lead of Quantic Foundry. He combines social science and data science to understand gamer behavior in large-scale game data.

100 Comments

  1. Survey: Hearthstone appreciated as more strategic, less exciting than League of Legends - GeekTechTalk January 20, 2016 at 1:56 pm - Reply

    […] synthesized data gathered from surveys filled out by over 140,000 game players to publish a blog post comparing the perceived challenge and thrill of popular strategy […]

    • Visrite January 26, 2016 at 5:07 am - Reply

      You’ve never played #EveOnline have you. Imagine being 1 unit in #Starcraft. Also when you’re not playing your unit other people can still attack your base. Your own freindly units can attack or steal resources from you. It’s also a 24/7 global struggle to convince volunteer units to fight for your cause.

      That cognitive threshold you speak about, that’s where #EveOnline is, way in the upper right corner.

      Why would someone play this game? Only chance I’ve had to use my college education has been helping #B-R happen. Which took months.

      When each unit is an actual person, shit gets extreme and fun.

      Visrite
      Former CEO of Someone Else’s Problem

  2. bee January 20, 2016 at 2:38 pm - Reply

    I feel like excitement is the wrong word to use here. Intensity is probably a better term.

    • Nick Yee January 20, 2016 at 3:04 pm - Reply

      Within this particular cross-section, perhaps “pacing” or “velocity” would have been better suited–“intensity” has a broader connotation; e.g., I find Europa Universalis to be incredibly intense when France declares war on me when I’m already at war with Austria. We primarily wanted to be consistent with the terminology we’re using, and since these are all meant to be gamer motivation labels (rather than game mechanic labels), we stuck with “excitement”, which I agree is not ideal in this particular case.

    • Sakkre January 21, 2016 at 10:46 am - Reply

      If it was intensity starcraft would be at the very top, no competition.

  3. Ysharros January 20, 2016 at 4:07 pm - Reply

    You guys always make statistics so sexy! ;)

  4. Andrew Liao January 21, 2016 at 1:12 am - Reply

    Um, but the cognitive threshold is just arbitrary?

    • Nick Yee January 21, 2016 at 1:23 am - Reply

      The cognitive threshold is something we can only estimate with the available data. We’re positing this boundary exists near the games that trace out the diagonal line. It’s certainly possible to create games that fall beyond the threshold (which may be what you mean by “arbitrary”?). We’re suggesting that games beyond this threshold are no longer fun because they’re too cognitively demanding.

      • Andreas January 21, 2016 at 4:30 am

        Be honest though, Nick. You guys wanted there to be games beyond the “cognitive threshold” to create some interest. If Dota and SC2 were beyond the threshold they wouldn’t be massively popular games and esports. What is more likely is that they are the closest games to your posited “cognitive threshold” and not truly beyond them. The reason you put them beyond is primarily buzz-value.

      • Nick Yee January 21, 2016 at 11:25 am

        If this were our intent, we would have been much more blatant about it in the blog title, section headers, and our social media framing of this post. We only mention DotA explicitly in the text once (we never mention SC), and only to make clear that it’s either exactly on the threshold or slightly beyond (the latter is our hunch). Note also that we have one chart that has DotA slightly outside the threshold, but we keep it in the Hard Fun band in the second chart.

        We would argue that what makes a game an attractive esport candidate is precisely the fact that not everyone can play it well, and for this to happen, it must have a high cognitive demand that makes it too stressful to be fun for a not insignificant number of potential gamers.

      • Andrew Liao January 21, 2016 at 2:21 pm

        Did you or did you not literally just draw a f–king line where you “think” there is a cognitive threshold or is the line based on something quantitative, like you know, actual social sciences / psychology / neuroscience uses?

      • Nick Yee January 21, 2016 at 4:45 pm

        Please refrain from the unnecessary language and tone.

        And it’s both. We’re estimating where the line is based on the quantitative data from over 140,000 gamers in an attempt to explain why the upper quadrant is empty.

        Do you feel that there is no boundary at all, or that it should be at a very different location? I hope we’re being transparent that we’re estimating where this boundary is based on the available data.

      • Michael Austin January 22, 2016 at 11:22 am

        I wonder if games at the cognitive load boundary make good esports because you can appreciate people that can support a higher cognitive load than yourself.

      • Nick Yee January 22, 2016 at 11:37 am

        I think so. In the same vein, the Spelling Bee only makes sense in languages where the phonetics are not immediately obvious. You can’t have an interesting Spelling Bee in Italian because the phoneme-to-spelling correspondence is so simple. But the Spelling Bee makes sense in English because the language is so irregular.

        A game must be difficult to master for the average person to make it an interesting competitive and spectator esport experience.

    • Marc Vaughan February 5, 2016 at 8:51 am - Reply

      I think the cognitive threshold will differ from person to person depending on their personality, education etc. – this probably explains why a lot of the games listed are passionately played by many people but aren’t strictly speaking ‘mainstream’ games (i.e. my title Football Manager does fairly well, but is a niche title in comparison to the likes of say Call of Duty which has a far wider general appeal).

  5. Game Genre Map: Strategy Games – GOING AAA January 21, 2016 at 1:37 am - Reply

    […] a company founded by two PhDs working on online game research, has published an interesting article on a game genre map for strategy […]

  6. Starcraft and DotA at the cognitive threshold of excitement/strategy - A Random Bunch of Interesting Things January 21, 2016 at 1:53 am - Reply

    […] by wolverinex2 [link] […]

  7. Mihaly January 21, 2016 at 2:18 am - Reply

    I found it very interesting, how Age of Empires or Command and Conqueror feels more “balanced” fun, only because when these were released, there wasn’t as much focus on competitive 1v1 matches with Ladders, videos and strategy guides as for Starcraft now days

    I believe, if AoE or RA2 were to be released now, with people would perceive them lot differently. All old strategy games are extremely demanding – as you are fighting somewhat against the UI and old AI – when it comes to high level 1 vs 1

    • Mark Terrano January 21, 2016 at 9:39 pm - Reply

      On the contraary (as an Age of Empires developer – Lead Designer Age of Empires II) there were international tournaments, how-to-play videos with commentary, and tens of thousands of multiplayer matches every night on the Microsoft (Gaming Zone) service. AOE2 tended to be more team focused than 1V1 play but competitive play isn’t anything new (and ladder, and ranked matches, and massive tournaments). World Cyber Games 2001 I think had 750,000 at the base level competing with a $300,000 US prize – so there was plenty of international competition. The focus on Esport has shortened the time to first combat, and the community has gotten substantially more confrontational.

      While the older strategy games are demanding – the level of intensity, speed of combat, and number of decisions made by players has increased quite a bit with the current eSports titles. The hour-long strategy game has been compressed down into a more intense and shorter experience to fit with that format.

      Age of Empires II was re-released (quite successfully) in 2013 enjoyed by long-time fans and players trying it for the first time.

  8. Using Data From Over 140,000 Gamers to Create a Map of the Strategy Game Genre – Internet and Tecnnology Answers for Geeks January 21, 2016 at 3:08 am - Reply

    […] by Nick_Yee [link] [2 […]

  9. Chris Keating January 21, 2016 at 6:35 am - Reply

    This is really interesting analysis but I do wonder about some of the details.

    I am not sure how much the “strategy” axis here means “strategy” as commonly understood. For instance, Civilization is really carefully crafted as a strategy game because it gives you a complex set of decisions based on the previous decisions you have made (and not much else) right through the duration of the game. By contrast, Total War games hardly have any strategic depth past the first few rounds once you know some basic principles and are in a position to exploit the AI’s inherent weakness. Similarly Europa Universalis, you need to find a “knack” of playing each country (and for some countries there is only really one “knack”) and after 200 years of doing it right you are so great a superpower that you are essentially now playing in a sandbox. (I say this as a big fan of EU:3, EU:4 and a bunch of other Paradox Interactive games!)

    I wonder if the more popular games tend to have scores close to the mean. If 100 million people play Civ5 and 10 million play Europa Universalis 4, it might be that the that the people who play EU:4 are more self-selecting, rather than the game being fundamentally more strategic.

    • Nick Yee January 21, 2016 at 9:09 am - Reply

      I don’t play Total War, but I do play EU. With EU, the number of parameters that can be manipulated for each country is much larger than Civ (and having to keep in mind how they impact each other and other attributes). The time it takes to make certain changes is also on a longer time horizon and with more random variables in play–such as preparing for and executing a Westernization. The much larger number of surrounding AI countries also makes the geopolitics more complex, as you have to consider a larger web of relationships.

      You’re right that once you get the “knack” of it, then it’s often easy to end up being the superpower, but whenever I mention EU to friends, I always add the warning that the learning curve is very steep, while I don’t feel like I have to mention this if I recommend Civ.

      With your second question, niche games that target more extreme motivations should by definition also have smaller audiences (since fewer gamers have extreme scores to start with), and more mainstream popular games ought to have more average motivations and thus larger potential audiences. But I think self-selection pushes the question further back in a sense: what is it about EUIV that signaled to high Strategy players to self-select and kept them enjoying the game if the game isn’t more appealing to them?

      • Chris Keating January 21, 2016 at 12:11 pm

        I agree completely about the difference between Civ and EU. I wonder if the “strategy” axis is really referring to “analytical intensity” as it encompasses things that count as management and tactics as well as strategy as a business-school or staff college graduate would understand it.

        Re the second point – yes – no doubt an analytically complex game will attract people who like analytically complex games. However, clearly there are other factors. To give a thought experiment example: Say Civilization 5 had never been published, but a small indie studio had created a mechanically identical game with isometric graphics and set in a setting based on an obscure medieval country from Eastern Europe – no brand, no marketing spend. Would it end up with the same fan base as the real Civ 5? No. As the strategy score is a function of the fan base – would it really have the same score?

      • Nick Yee January 21, 2016 at 12:30 pm

        Great point about the unbranded Civ 5. In a way, something like Endless Legend comes to mind. Perhaps only more dedicated Strategy genre fans would have heard of it, and this sampling bias increases the audience’s Strategy score. But your larger point is well-taken, there’s a combination of marketing/historical factors that may creep into the audience scores that is separate from the game’s design.

        And yeah, perhaps “decision complexity” or “analytic complexity” is the better phrasing for this.

      • Vohbo April 6, 2016 at 5:33 am

        I would postulate that EU 4 has far less strategical depth than Civ 5.
        When I first started playing EU 4 I was overwhelmed by the amount of mechanics in the game. The burden of knowledge is indeed very high and it definitely requires insight to be successful. However, once you have learned the game, what you end up doing is pretty much always the exact same thing. The start of the game will differ, but you will always strive for a situation in the mid game that is exactly the same.
        There are a few reasons for this. There are no victory conditions in EU 4 other than the ones you set for yourself. You can go for achievements as victory conditions, or you can go for world conquest, but these are goals you define for yourself and are rather arbitrary.
        Once you have chosen your goal, what you do is almost always the same: you try to get good allies. Then you follow a pretty much set pattern where you juggle peace treaties to conquer everything near you that you have eclipsed, while making sure your aggressive expansion stays as low as you can.
        Yes, you do a lot of things at the same time, and many things have a small influence on the strength of your empire, but very few decisions are actually impactful. Of those that are impactful, most are pretty much set in stone.
        There are very few things in the game that affect your playstyle, and for me it usually comes down to “finding a gap in the alliances of surrounding empires”. The wars themselves are uninteresting most of the time, the development of provinces is uninteresting and shallow. There is some depth to religion and some nations have interesting mechanics.

        Civ 5 on the other hand has enormous amounts of strategical depth. There are many social policy builds that can lead to very distinct playstyles, you can beeline critical techs and abuse mechanics in different ways to get ahead. The addition of Wonders that can combo with social policies or nation-specific modifiers adds even more replayability. Every game is different and requires a different approach to winning. The victory conditions are very clearly defined, which certainly helps. Quite often, your overall strategy (say, you are going for a culture victory), can fail, and require that you have a back up plan available. It offers a combination of long term strategical planning and tactical execution, as well as the precise management of your cities and empire.
        For example, one of my favourite strategies is to play a peaceful economic game with a science subtext until the end of the industrial era. This build requires commerce and autocracy, and allows you, when the build is complete, to instantly buy a fleet of Bombers that can then (with great tactical care) be used to conquer the world.

        Civ 5 is immensely popular because it is easy to learn and hard to master, the major problem it has is that the higher difficulties suffer from “weak AI’s that get huge bonuses”. This makes the gameplay on those difficulties very pigeonholed. The game suffers a lot from this, and it could be incredible if the AI were to be fixed.

        In short: I love EU4, but it really only has one way to success. If I want to outplay my opponents, I’ll go with Civ 5.
        I want to plan and build and then do something good (even broken sometimes), to edge out a close win by being tested. EU4’s endgame simply does not offer that.

      • Louis May 4, 2017 at 3:09 pm

        This is a reply to Vohbo’s — higher difficulties suffer from “weak AI’s that get huge bonuses”. This makes the gameplay on those difficulties very pigeonholed. The game suffers a lot from this, and it could be incredible if the AI were to be fixed.

        No I don’t think “AI fix” will change things for most people. On a standard map, standard settings, the good city spots are taken fairly quickly. AI gets bonuses per era, but cannot put that bonus to work by building more cities. Even if all the AI are all smarter, as long as they are all equally smart, it’s hard for one AI to simply over run the other — remember they all have the same cheats, and the same level of smartness. Inability to go wide means that there no tall versus wide dynamic — so Civilization V, on standard map, will always be the Tradition – Rationalism – Freedom build, with only minor modifications. I think no matter how smart you make the AI, as long as it’s 8 AIs on a standard map, it’s always tall beats wide. The only thing that breaks the monotony is when Zulu overwhelms his neighboring AIs — but Zulu cities are garbage. They only look good on the mini-map.

        Only on a huge map, with 4x more terrain, but just a few more luxuries, do you actually have the tall versus wide dynamic. That’s when you really have to make choices. Plus the Zulu’s will throw a spectacularly large army at you.

  10. Adrian January 21, 2016 at 8:21 am - Reply

    I’m asking this with a certain amount of facetiousness, but the definition of “easy fun” seems to fit very well on Dwarf Fortress, whose players frequently repeats the mantra “losing is fun”. Do you feel this is correct?

    • Chris Keating January 21, 2016 at 8:28 am - Reply

      I would guess that Dwarf Fortress ends up on the very “heavy” or “difficult” end of the spectrum, because you have to manage the game very closely to avoid total and imminent disaster. I.e. you are attending very closely to lots of small intellectual decisions and working out their consequences.

      Equally I would argue it isn’t really a “strategy game” as the game is all about micro-managing your Dwarves to make the most of your sandbox environment and not about defining and refining a strategy.

  11. Аналитики определили мотивации геймера-стратега | OnGames January 21, 2016 at 12:51 pm - Reply

    […] аналитики собрали данные тысяч геймеров, и составили нечто вроде сравнительного графика Ричарда Бартла, отметив множество популярных проектов на шкале […]

  12. Comparing How "Strategic" Strategy Games Actually Are | The Home of SciFiMusic January 21, 2016 at 3:36 pm - Reply

    […] You can check out the full findings here. […]

  13. พิสูจน์ทางวิทยาศาสตร์แล้ว DotA ต้องใช้กลยุทธ์มากกว่า LOL – Sakon's blog January 21, 2016 at 5:08 pm - Reply

    […] Quantic Foundry บริษัทวิจัยเกี่ยวกับเกม ใช้ผลจากแบบสอบถาม Gamer กว่า 140,000 คน วิเคราะห์เปรียบเทียบ ความตื่นเต้นเ… […]

  14. Peter January 21, 2016 at 5:21 pm - Reply

    I’ve been following you since early EQ days. Your work, and your publications are always interesting and informative, and I appreciate your engagement here – particularly in the face of our less articulate bretheren.

    Best wishes.

    P

    • Nick Yee January 21, 2016 at 7:50 pm - Reply

      Thanks, Peter! I really appreciate it. And boy–the EQ days. Isn’t it crazy how time flies?

  15. League Of Legends Is As Strategically Complex As World Of Tanks, Analyst Says | Kotaku Australia January 21, 2016 at 5:32 pm - Reply

    […] Quantic Foundry’s Nick Yee then took all of the data and then, as an experiment, decided to pl… “First, we’ll pick 2 game franchises—Civilization and StarCraft. We can then use our data set to find the games most closely related to these two exemplars. This gives us a good mix of turn-based strategy, real-time strategy, management sims, and grand strategy games,” the firm explained. […]

  16. Thomas Henderson January 21, 2016 at 5:34 pm - Reply

    So in your loexicon excitement clearly indicates a limited time decision loop that encompass the strategic decisions.
    This means Total war is a bit of a odd case since all the strategic decisions are turn based.

    I wonder if this makes sense for any strategy game where you can pause or slow time? Does this mean these are automatically not exciting? It seems by the positioning of all those games at the bottom of the excitement axis you are saying this.

    I’ve always seen time pressure strategy games like the traditional RTS or MOBA style games as their own animal for this very reason. But I don’t know if it’s truly as continuous as you describe it.

    • Nick Yee January 21, 2016 at 7:58 pm - Reply

      See this handy reference chart:
      http://quanticfoundry.com/2015/12/15/handy-reference/

      Excitement also encompasses surprises and thrills, which is highly correlated with the interest in fast-paced games and gameplay that requires rapid reaction times. So that’s part of what the scores are based on as well.

      So another way to think about this is: If someone were just watching you play the game, how entertaining would it be for them? And this “entertainment value” aspect is part of the Excitement score for the player as well. This is largely why XCOM scores higher than say Europa Universalis even though both let you take as long as you want for each turn. XCOM is more visually interesting and dynamic to watch because of the combat sequences, and I believe this same issue is at play with Total War.

  17. Comparing How "Strategic" Strategy Games Actually Are | WEBODILE - Your Personal Magazine January 21, 2016 at 5:49 pm - Reply

    […] You can check out the full findings here. […]

  18. Comparing How “Strategic” Strategy Games Actually Are | Feedlesticks January 21, 2016 at 5:49 pm - Reply

    […] You can check out the full findings here. […]

  19. Ed January 21, 2016 at 8:44 pm - Reply

    I think there’s a sampling bias issue here called “conditioning on a collider”.

    Assume that all games can be measured on “excitement” vs “strategy” axes. Although games might be randomly distributed over these two axes, successful games tend to maximize excitement + strategy. But because you’re only looking at relatively successful games, you will see a fictitious negative correlation between excitement and strategy.

    This is a good explanation of the phenomenon: http://www.theatlantic.com/business/archive/2012/05/when-correlation-is-not-causation-but-something-much-more-screwy/256918/ It also explains why good-looking shoes tend to be uncomfortable and ugly-looking shoes tend to be comfortable.

    • Nick Yee January 21, 2016 at 9:59 pm - Reply

      No. If we committed the fallacy of conditioning on a collider, we would have claimed a negative causal relationship between Excitement and Strategy due to the observed data–e.g., if you make a low Excitement game, then it must be high in Strategy in order to to be successful in the Strategy genre (or vice versa). We make no such claim. The data itself doesn’t show this pattern, and we do not even describe the correlation between Excitement and Strategy in the post.

      Instead, our core argument is precisely what your linked article is advocating–to examine possible causal truncation of the full population when analyzing the observed population. This is exactly why we proposed the cognitive threshold, and it’s what the article is saying is the RIGHT thing to do. We’re explicitly trying to explain the unobserved population.

      • Matthew Lewinton January 26, 2016 at 5:36 pm

        While I agree that ‘conditioning on a collider’ may not apply in this scenario I think there is a possibility that there is a bias in your sampling. When talking about cognitive thresholds the implication is that it is a population wide phenomenon that is being shown yet the sample provided, while incredibly representative of gamers, does’t account for non-gamers for which I believe the cognitive threshold shown above would be much lower. I guess what i’m trying to say is that is it not possible that the level of gaming expertise has a compounding effect on cognitive threshold you suggest?

      • Nick Yee January 26, 2016 at 5:55 pm

        Right. The graph is likely representative of the average core gamer, but everyone has their own cognitive threshold. And it may not even be determined by gaming history. I get motion sickness easily in FPS games, and that impacts both my thinking and reaction time.

        I think the cognitive threshold in the graph does represent an upper limit though for the core audience beyond which a game would stop being enjoyable by a vast majority of players–i.e, the threshold for non-gamers would start earlier, but here is where it definitely starts impacting the core gamer audience.

  20. Alex Chekholko January 21, 2016 at 11:37 pm - Reply

    I wonder where Eve Online falls on that chart!

  21. spineduke January 22, 2016 at 1:11 am - Reply

    Your cognitive threshold barrier seems to be based on pseudo science? How is this barrier defined? How are games quantified on this scale? What is the quantitative value of “not being fun anymore”. Please, details.

    • Nick Yee January 22, 2016 at 9:23 am - Reply

      Game quantification: See the first paragraph of blog post.
      Barrier definition: The estimated outermost edge of the game scores based on data from over 140,000 gamers.
      Not being fun anymore: The games with quantitative scores that fall significantly beyond this estimated threshold.

  22. John Kilbride January 22, 2016 at 5:42 am - Reply

    As an avid Counter-Strike player, I’m curious about the strategy metric that was used here. In high level game play, beyond the 98th percentile, the game has a fairly robust meta which isn’t well represented in the average run-and-gun of the low ranked games. Similarly, World of Tanks has a very complex system of interactions which rewards very methodical, strategic game play (good players can achieve a win rate above 65% with thousands of games).

    How was the strategy metric assessed? Is it based on the data of average, casual gamers?

    • John Kilbride January 22, 2016 at 5:50 am - Reply

      I should clarify, it seems other commenters are taking offense to the location of specific games. I wouldn’t expect the graph to represent the hardcore audience as that’s a very niche part of the total player base. If you’re interested, I believe there are some sights that pull data from Valve’s API which allows you to see the approximate bell-curve for CS:GO and DotA2.

    • Nick Yee January 22, 2016 at 9:15 am - Reply

      The Strategy metric (and similarly the Excitement metric) are based on gamers who listed these games as one of their favorites. Thus, it represents the core, engaged audience of each game, rather than gamers with extreme profiles in one way or another.

      As you noted, it is possible to play CS:GO or WoT in particularly strategic ways, and the same is true for “casual” games like SimCity or Cities:Skylines as well. It’s possible to painstakingly optimize traffic in Cities:Skylines in incredibly precise ways if you take the time to figure out the mechanics. But in all these cases, it would be misleading to score a game based on these extreme motivation profiles (which I think is part of what you were saying as well).

      • John Kilbride January 22, 2016 at 2:04 pm

        Thanks for the response! The data, in my opinion, makes a lot of sense.

  23. George January 22, 2016 at 10:18 am - Reply

    I’d very much like to see where Rainbow Six: Siege fits into this picture. Would it at all be possible to include it? I realize it’s a newer title, and that data might not be readily available.

  24. Game Genre Map: The Cognitive Threshold in Stra... January 22, 2016 at 11:55 am - Reply

    […] “Using data from over 140,000 gamers worldwide, we map out games in the Strategy genre to reveal the cognitive threshold between Strategy and Excitement.”  […]

  25. LordLoko January 22, 2016 at 12:28 pm - Reply

    Jesus, Imagine if aurora4x was in this chart.

    It would break this

  26. Keegan January 22, 2016 at 1:17 pm - Reply

    For the handful of games on this list that I am familiar with, I’d say they are right where they should be. Great article.

    Based on Nick’s comments and snippets of the article, it sounds like ‘Excitement’ constitutes a lot of things, from unpredictability to time given to make decisions, to ‘streamability’. Is there a place to get more in depth on the explanation of that metric?

    Also, SMH at the fanboys getting defensive about their games ‘placement’.

  27. Ineedme January 22, 2016 at 4:02 pm - Reply

    Warcraft is missing

  28. 500apm January 22, 2016 at 7:49 pm - Reply

    I think doing this study with professional gamers would have produced a variance in results. Counter Strike would definitely be sitting a lot further across strategic wise and Starcraft would probably have a category all its own etc.

    • hung January 23, 2016 at 12:02 am - Reply

      I would agree with your statement too. Even with the 140,000 sample size, what percentage of those 140,000 would are high ranking/caliber players? I think an average person would have a different definition of “excitement” versus the high caliber player.

      I have played League for over 4 years now and i definitely agreed that League is a Hard-Fun game, but i think both League/HoTS should have more “strategy” value to them. While i find excitement in playing an objective heavy game of League, i think the sample size in this study would probably find more excitement in subjective stuff, such as outplaying the enemy or just doing whatever they want.

    • Nick Yee January 23, 2016 at 12:13 am - Reply

      See this article for details on the gamer sample:
      http://quanticfoundry.com/v21-sample/

      Given that most players are not hardcore professional players, our perspective is that it’s more important for these scores to capture what the average player of these games looks like. And our gamer sample reflects a concentration of core (rather than hardcore) gamers.

      It also avoids the problem of designers inadvertently catering to an unrepresentative vocal minority of gamers.

      And focusing on any subgroup (e.g., only men, only people over 60, only people in Taiwan) would likely shift the map in some way. We felt the overall average would provide the most representative picture.

  29. Barry Staes January 23, 2016 at 2:56 am - Reply

    I wonder where Supreme Commander: Forged Alliance (SupCom:FA), OpenRA, and OpenTTD would be in this map..

  30. Studying gamers teaches us a lot about games too | Column Catcher January 23, 2016 at 4:01 am - Reply

    […] and quantify gamer motivations. To that end, it’s collected detailed information on more than 140,000 gamers, identifying their favorite games and what makes those titles most attractive. The output of that […]

  31. NODsArmy January 23, 2016 at 11:33 am - Reply

    just wondering where the homeworld games stand on this type of graph. was one of the hardest games i played back in the day, but it was still fun as all hell.

  32. r January 23, 2016 at 11:38 pm - Reply

    what about making perfectworld forsakenworld like it was befor ARC took over?

  33. VR teknolojisi oyun dünyasında nasıl bir yer edinebilir? | Etohum.com Blog January 24, 2016 at 3:07 am - Reply

    […] Örneğin PC’de hiç şans verilmeyecek oyunlar, mobil pazarı domine edebiliyor. Bu konuda Quantic Foundry‘nin yaptığı araştırma dikkat çekici olabilir. Şirket, 140 binden fazla oyuncu […]

  34. MichuV5 January 24, 2016 at 6:07 am - Reply

    This is very interesting subject. I wish my english would be a little better to discuse about that :)

  35. Game Genres and Eve | Serpentine's Eve January 24, 2016 at 7:17 pm - Reply

    […] found a web site that categorises games by their characteristics in order to create a recommendation […]

  36. Måns Olson January 25, 2016 at 5:13 am - Reply

    I think it’s interesting to note that the successful eSports titles are more or less all in the upper left quadrant. I’m thinking that the excitement/intensity of these are required to make them interesting to viewers, and thus suitable as a sport. You could definitely compete in Civilization or EU as well, but it may not be that much fun to watch. As an analogy, consider chess – not a great spectator sport, but popular as a competitive game.

    It’s not immediately obvious to me how Twitch streams and “let’s plays” come into this, and what their role is. Some of the non-eSports titles are obviously bringing viewers to Twitch without being particularly fast-paced or exciting. I’m sure creativity and expression are part of what lets a game attract viewers in this case. The chat and comments sections and direct interaction with the players probably pay a part too. Any thoughts on other factors I may have missed?

    • Nick Yee January 25, 2016 at 4:57 pm - Reply

      Duration and surprise are two other things that come to mind. I think there’s an optimal range for how long a match should last, which corresponds with our attention spans. A Civ/EU match is hard to watch because it could span 10-12 hours (or more), and it’s hard to spend that much time spectating just one match.

      Games that design for long-term gradual change (both Civ and EU for the most part) are also less interesting to watch because nothing really changes from a moment-to-moment basis. Compare this with Hearthstone or LoL where skillful or lucky plays can completely change the playing field in a few seconds. The anticipation of game changers like this gives the audience another reason to engage with the match. This is partly why Civ/EU gamers tend to post screenshots of a mid/end-game state, whereas it makes more sense to post video snippets of MOBA gameplay.

      • Måns Olson January 26, 2016 at 2:58 am

        Good point! I definitely think duration is key. I feel like the maximum time span seeming to be somewhere between one and two hours per match. Longer games also give an organizational challenge, in that tournaments become much harder to do.

        I’m still not entirely sure how people are interested in jumping in and out of longer games on Twitch, even though they wouldn’t if it was competitive. There’s something there that I can’t quite put my finger on.

  37. Miranda R January 25, 2016 at 10:24 am - Reply

    Given the sort of cost/benefit trade-off of optimizing the flow experience, I was wondering if it might be possible to “draw” that cognitive threshold line via receiver operating characteristic (ROC) analysis?

    I also think that this threshold could be personalized using cognitive measures, as the threshold is very likely to be highly variable (as you have stated).

    I am really enjoying this research! Thank you!

  38. Clippings: Strategic Assessments » Matchsticks for my Eyes January 25, 2016 at 9:08 pm - Reply

    […] week’s top link is Quantic Foundry’s map of the strategy genre, broken down along two dimensions: Excitement and Strategy. Europa Universalis is high strategy and […]

  39. Matt January 25, 2016 at 10:32 pm - Reply

    None of you knows Supremme Commander right?

  40. T Reyr January 26, 2016 at 1:11 am - Reply

    Very interesting and informative.
    Having played 80% of the games used, I can see why some games I just did not get along while others are still in my play list.

    There is however an important game missing from this, EVE Online.

  41. Jono January 26, 2016 at 8:53 am - Reply

    Nick Yee, where on this would you rate EVE Online?

    EVE is a very interesting game as it has strong elements of what you label as ‘strategy’ and ‘excitement’ which are alvaliable regardless of a users character age or skill level. Do you believe this game to be in the top right quadrant of this graph?

    In my humble opinion, I beleive it to be far into the cognitive threshhold. I understand that it isn’t massively popular (25k active players is still a lot though) and it has a very niche market in terms of player base. I can understand that Including a game that were to be far into the cognitive threshold contradictory to your findings as it would skew the results and where we would draw this line, but is EVE Online not worth a special mention due to its Cognitive Threshold?

    • Nick Yee January 26, 2016 at 1:43 pm - Reply

      We used Civ and StarCraft as the initial seeds to populate the map. This is why almost all the games are from the Strategy genre, with the exception of CS, which was dragged in by StarCraft.

      With EVE, there is certainly a lot of strategic complexity and planning involved, but would it be more fair to characterize its excitement level as long periods of tedium interspersed with moments of sheer panic? I.e., it has a lot of moments like Europa Universalis and a few moments like CS?

      In any case, we’re hoping to do more maps of other intersections in the future,and will keep EVE in mind.

  42. gemipat January 26, 2016 at 9:48 am - Reply

    This study is probably also dépendent on the cultural différences from one country to another and age as well, not to forget your kind of personnality (analytic vision of things vs global vs of things)

    And Yeah to the other posts about Eve online are right, I’m playing that MMO for the last 12 years (yes 12) and it is above everything, strategy and fun at its highest

  43. Andrew January 26, 2016 at 7:01 pm - Reply

    Have you considered hardcore games such as auroroa or dwarf fortress??

  44. Design - Understanding basics of Arts and Design | Pearltrees January 27, 2016 at 11:41 pm - Reply

    […] Spector: Go emergent – game design is not all about you. When Warren Spector goes shopping for games, he sometimes feels cynical, seeing shelves full of what he sees as "me-too" product — legions of licensed titles, sequels and derivatives. Design Principles FTW. A single game as a lifelong hobby. Do you finish one game and then move onto the next? This is the dominant pattern of play for gamers. How to be a Better Game Designer. Game Genre Map: The Cognitive Threshold in Strategy Games – Quantic Foundry. […]

  45. Mario January 28, 2016 at 6:43 am - Reply

    LOLLLL Football Manager > Easy??? > No excitment?? Go play a Skybet League 2 club and tell me it’s easy :P

  46. Marc Vaughan January 28, 2016 at 10:20 am - Reply

    This is a fascinating article and I’d love to hear more about how you went about the research and what exactly was asked in the surveys; especially in regards to determining what was a ‘low strategy’ and a ‘high strategy’ game.

    I’m conjecturing that low strategy games are those which users can pick up and play relatively easily, while high strategy are harder to comprehend initially? (my title (Football Manager) is shown as low strategy but has a huge amount of complexity to it, hence my inquiry).

    • Nick Yee January 28, 2016 at 10:36 pm - Reply

      See the first paragraph for where these scores come from. In the article, right after the first graph, there’s a section with updated detail on how we define Strategy and Excitement.

      • Marc Vaughan February 5, 2016 at 9:00 am

        That definition however is so vague as to be largely meaningless especially when comparing such disparate products as real-time war games, turn based simulations and sports simulations – I think that is probably one of the reasons its raised such discussion on this thread.

        Some products give a clear indication of the options available and consequences of them (ie. your units die, you lose) – others such as sports titles are somewhat unclear in this regard because your success is defined by objectives not death and the consequences don’t always prevent you from continuing in the game, plus failures can be in terms of an emotional influence upon your team which isn’t obvious unless you understand the genre and enjoy such products (its not obvious if you’ve ever played such a sim or not?).

        PS – I know the terms ‘vague/meaningless’ might have an emotional content, I’m not trying to insult your methodology and I realize that you are undertaking a hugely complex analysis and simply put you have to decide an approach in this regard – I was however wondering if instead of using mechanics you could instead use average playing duration or something similar as a biasing factor on top to increase its accuracy?

  47. Ben January 28, 2016 at 10:27 am - Reply

    LOL! Football Manager is “easy” and has “low strategy?” Have you tried taking a team from the Vanarama National League to winning the treble? Try and convince me that this is “easy fun” and has “low strategy.”

  48. KidEternal January 28, 2016 at 10:31 am - Reply

    Like Chess, World of Tanks might seem unexciting and less strategic to a layman than a faster-paced game with more complex mechanics like Monopoly, but anyone truly familiar with it knows that it’s one of the more cerebrally-challenging and exciting games in existence. It belongs just to the right of Civilization in Strategy, and just above LoL in excitement.

  49. LTL Did You Know… 29/1/16 | Library, Teaching and Learning ~ Te Wharepūrākau January 28, 2016 at 6:57 pm - Reply

    […] Strategy, Excitement and the Cognitive Threshold […]

  50. boo9 January 31, 2016 at 8:01 am - Reply

    In the shouting match that are the different Aeon of Strife Style Fortress Assault Games Going On Two Sides communities, you’d see arguments of which game was more strategic. It seems The audience for Dota values strategy slightly more than the league of legends audience does, while both score high in excitement. is there any suplementary data that is interesting here between the two games? is there larger overlap with dota and blizzard RTS games, given it’s history? overlap with other games? Do the audiences differ in age? How many days/hours they play per week?

  51. Triage | Pearltrees January 31, 2016 at 8:20 pm - Reply

    […] Game Genre Map: The Cognitive Threshold in Strategy Games – Quantic Foundry. The Gamer Motivation Profile allows gamers to take a 5-minute survey to get a personalized report of their gaming motivations. Currently, we have data from over 140,000 gamers worldwide. In the survey, we also ask gamers to list their favorite game titles. This allows us to pivot between gamers and games–we can use the aggregated game audience profiles to compare games. For example, is Civilization more strategically complex than SimCity? Well, we can compare their audience Strategy scores to find out. In this sense, the Gamer Motivation Profile isn’t just a benchmarking tool for gamers, it’s also a benchmarking tool for game titles. The Gamer Motivation Profile isn’t just a benchmarking tool for gamers, it’s also a benchmarking tool for game titles. Mapping Out the Strategy Genre These audience scores allow us to explore game titles within a genre with incredible precision. […]

  52. FPS Role Preference: Run-and-Gun vs. Long-Range Stealth - Quantic Foundry February 2, 2016 at 10:23 am - Reply

    […] Previous […]

  53. In FPS Games, Women Prefer Long-Range Stealth, Younger Gamers Prefer Run-and-Gun - Quantic Foundry February 2, 2016 at 1:43 pm - Reply

    […] Previous […]

  54. What is Social Graph Quantification? - Boston Commons High Tech February 3, 2016 at 10:18 am - Reply

    […] to represent the hardcore audience as that’s a very niche part of the total player base. [11] I think the cognitive threshold in the graph does represent an upper limit though for the core […]

  55. Новая шина February 17, 2016 at 9:27 am - Reply

    Fundamentally, the article is about cognitive load, and no matter how much of a master someone is at EU4, the game requires the player to take a lot of time to read the situation , digest enormous amounts of quantifiable information, and figure out the correct response.

  56. Revisiting the Strategy Genre Map: Age, Audience Homogeneity, and the Lasso Effect March 23, 2016 at 10:39 am - Reply

    […] first plotted out this Strategy genre map back in January. The data underlying the current map is different in 2 ways. First, we now have data from over […]

  57. Un studiu dovedeste ca DotA nu este distractiv April 11, 2016 at 1:59 am - Reply

    […] [Quantic Foundry] […]

  58. 8 ball pool cheats June 19, 2016 at 8:26 am - Reply

    Offering also utilized the instantly changing purpose.
    We have applied this program to carry on this cheat operating.

    You will always have the newest offering additional anty-ban script
    to the tool.

  59. Louis May 4, 2017 at 2:34 pm - Reply

    Total War is not that much strategy – lots of decisions don’t matter or are kind of obvious. Civ5 — the way most people play it, is not that strategic either. For example, on a standard map, Tradition-Rationalism-Freedom always works very well. Freedom is supposedly bad for domination, but on a standard map, Freedom gets faster artillery and bombers, and will absolutely destroy the AI militarily. On a standard map, always just turtle up in the beginning – because the AI cannot develop properly in the mid game, due to all the good city spots are taken by then. Remember the AIs are all equally dumb, so it’s not like one AI can just easily overrun another AI. The way to play Civ 5 is on a huge map, then much wider range of play is possible. Domination victory, using Freedom, on a huge map, is actually quite hard. Also AI is much stronger mid game because it can simply establish more cities. The basic four city strategy works perfectly on standard map often gets stopped by AI’s 15 cities. But looking at the so called Civ5 strategy guides, very few people play Civ5 on huge map.

  60. buy malware removal May 28, 2017 at 9:51 pm - Reply

    This app is sensible!

  61. http://fucoxanthin.net June 4, 2017 at 11:11 pm - Reply

    I have been exploring for a bit for any high quality articles or blog posts on this sort of area .
    Exploring in Yahoo I ultimately stumbled upon this site. Studying this
    info So i am satisfied to show that I’ve an incredibly excellent uncanny feeling I found out just what I needed.
    I such a lot without a doubt will make certain to don?t disregard this
    web site and provides it a glance on a relentless basis.

  62. Nicole June 5, 2017 at 11:15 am - Reply

    While I did not discover something spectacular about the
    antivirus portion of this program, having an antivirus and firewall tightly built-in has
    its advantages.

  63. geomineral.ru June 9, 2017 at 11:01 pm - Reply

    Fundamentally, the article is about cognitive load, and no matter how much of a master someone is at EU4, the game requires the player to take a lot of time to read the situation , digest enormous amounts of quantifiable information, and figure out the correct response.

  64. Orlando June 12, 2017 at 5:04 am - Reply

    Thanks for the sensible critique. Me and my neighbor were just preparing to do some research about this. We got a grab a book from our local library but I think I learned more clear from this post. I’m very glad to see such excellent info being shared freely out there.

  65. www.bahismerkezi.biz June 23, 2017 at 12:59 am - Reply

    What i do not understood is in truth how you are no longer
    really a lot more well-favored than you might be now.
    You’re so intelligent. You realize therefore significantly in relation to this topic, made me for my part consider it from so many various angles.

    Its like men and women don’t seem to be involved unless
    it’s one thing to do with Girl gaga! Your personal stuffs
    nice. At all times deal with it up!

Leave A Comment