At the office today, I got into a discussion with two of my fellow graduate students about the distribution of scores you can get while playing Canabalt. Because (1) the layout of the levels in the game is fully randomized and (2) the difficulty of certain actions (specifically jumping through windows) is exceptionally high, we were intrigued by the possibility that a fully random model of scores, which completely ignores player-specific skill levels, could account for the distribution of scores you see in the real world.

While thinking about this on my way home from Rockville tonight, I decided that I should write a simple web spider to parse the notices on Twitter that Canabalt automatically generates. Thankfully, other people had already done just this, so I discovered quickly that you only need to search for on Twitter to get the relevant information. After spidering the results of this search query, I constructed the following histogram of scores posted recently to Twitter using Hadley Wickham’s ggplot2 package for R. Here’s the results:

Canabalt Score Distribution.png

While generating this plot, a question I had asked several other R users about a few months ago came up again: is there no way to get the Y axis label to be anything other than “count” when you generate a histogram? The qplot simply ignores any ylab argument you pass in, so I suspect that the answer is “yes, you simply cannot change this default without hacking the ggplot2 source code.”

Sometime this weekend I plan to follow this short piece up with a longer post containing more substantial statistical analyses of this data. If you’ve got interesting ideas for analyses, let me know.

8 responses to “Canabalt”

  1. danny B

    this is really interesting! i look forward to seeing your analysis!! :)

  2. Tigershungry

    Really interesting work. I wonder if given that the score tweets aren’t completely automatic (players being more inclined to tweet their ‘better scores’), I wonder if a curve based on data from all players run attempts would look different.

  3. coollavagames

    I think the nice detail here is, that the graph is itself also pixel art.

  4. axcho

    “I think the nice detail here is, that the graph is itself also pixel art.”

    Haha, good point. :) You could make a meta-Canabalt game where the levels are generated from the scores of other players… :p

  5. Hadley

    You currently need to use the full scale_y_continuous call. I have some ideas on how to ylab work for a future release.

    Neat use of ggplot2!

  6. Jeromy Anglim

    I’d be interested to see the relationship between practice and performance for particular individuals. I imagine that date stamps could be used to indicate time (i.e., 1 , 2, 3, etc. ) and twitter User IDs could be used to identify individuals. You could then summarise properties of the individual-level learning curves. You could answer questions like: what is the shape of the learning curve? Do people differ in their rate of learning?

    Of course, there are many questions about the way the data is sampled. It may be possible to find subsets of people in the data who seem to post every single game. These might be the most interesting individuals.

    There may also be ways of detecting fraudulent posts. For example, an unusual sequence of scores may indicate problematic posting.

    If scores do improve with practice, then it’s likely that skill is important.

  7. Nathan

    What’s up with the ggplot fad?