Using factor analysis to cluster motivations is a fairly standard method in psychology, but figuring out how to best show the individual results back to people is relatively understudied. This is because, as most readers know, research psychologists tend to publish aggregate findings in academic journals rather than create customized reports for each of their participants.

Figuring out how to best show the individual results back to people is relatively understudied.

So when we started to put together the Gamer Motivations Profile Tool, we were entirely unsure about how gamers would react to a range of interrelated issues:

  • Will the results be perceived as being accurate to gamers?
  • How much detail do gamers want to see?
  • How in-depth should we get with the presentation of the math?
  • Should we show percentiles or use text labels for High / Moderate / Low?

A Hierarchical Approach

We decided to try out a hierarchical report structure.

  1. At the highest level of abstraction, we took a page from the MBTI: we created textual markers for the high (> 65%-tile) and low (< 35%-tile) scorers of each of the 5 motivation groups. For example, someone who scored high on Achievement was “Driven”, while someone who score low was “Relaxed”. No markers were given for the average range motivations. Thus, gamers were given at most 5 textual markers that provided a high-level non-mathematical overview of their gaming motivations.
  1. At the intermediate level, we used a radar graph to plot the percentile scores of the 5 motivation groups. Thus, gamers could see their gaming motivations represented visually based on 5 numbers.
  1. At the most detailed level, we used bar graphs to plot the percentile scores of each of the underlying motivations and provided descriptions of each motivation, along with examples from specific game titles.

But all this resulted in a fairly lengthy report. Our initial report clocked in at about ~1200 words. It contained the content we envisioned, but we weren’t sure how gamers would react to it. So we decided to run a beta test of the profile tool.

All this resulted in a fairly lengthy report.

The Beta Test

From our initial sample of 1,127 gamers, 625 were also subscribed to the mailing list. We sent customized reports to these 625 participants and invited them to fill out another survey after going through the report. 125 gamers participated in the evaluation survey. We learned 4 important lessons from the beta test.

Lesson 1: Gamers Found the Profile to be Very Accurate

82% of the sample rated the profile as either “Very Accurate” or “Extremely Accurate”. There was only 1 participant who rated the profile as “Slightly Accurate” or “Not At All Accurate”.

accuracy

Lesson 2: Gamers Found the Profile to Have Good Coverage

86% of the sample rated the profile as having “Good” or “Great” coverage of their primary gaming motivations (independent of how accurate those assessments were). It was good to know that gamers didn’t feel that important motivations were being left out.

coverage

Lesson 3: Gamers Found the Profile to Provide Right Amount of Detail

This finding was the most surprising to us. We thought that the data would skew the other way—that gamers would want to see less. But instead, we found that 73% of the sample thought the profile provided around the right amount of detail. 25% of the sample wanted to see even more detail than was provided. In the revised profile tool, we have included more gaming examples as many participants suggested in the comment box.

detail

Lesson 4: Many People Don’t Like Facebook

In the final section of the survey, we asked participants about their reactions to a hypothetical Facebook app for the Gamer Profile Tool. Here’s a representative selection of the responses we received:

I hate facebook! I would never ever use it!

Facebook is passé, all my gaming friends stopped using it in favor of Vox and Skype. Only so many teenagers out there. :)

Full disclosure, I avoid all Facebook apps.

ALL of this is very interesting to me and my guild has already shown interest; however, I dislike Facebook and it bothers me that it is such a popular medium to display content. It would be nicer to have the survey on your website with an optional FB widget to post your results. That way you get the best of both worlds.

Would prefer it be its own website or through a less invasive system than Facebook.

Based on these responses, we shifted our development efforts to providing the Gamer Motivation Profile on our own website and to be more agnostic about how the profile reports can be shared.

I hate facebook! I would never ever use it!

Read the companion article on how we identified the motivations in the Gamer Motivation Profile. Or try the Gamer Profile Tool and tell us what you think.