Giovanni Viglietta of the University of Pisa has posted up a paper called “Gaming is a hard job, but someone has to do it!”. 

In it, he not only analyzes a variety games to determine their complexity class, but he also arrives at a few metatheorems that are generically applicable for all game design. In other words, “include these features and your game gains fun.”

Remember, according to the Theory of Fun, pattern mastery and learning is why the brain plays games. And if you recall my presentation on Games Are Math, I made the case that entire classes of “tasty” problems can be described in mathematical terms (specifically, complexity class), because they are problems that always feel like they are on the margin of our ability.

So if you make use of these specific sorts of math problems — which are actually represented in the game as not looking like math at all, mind you — you are effectively inserting exactly the sort of problem that the brain finds most interesting.

These are not the only sort of problem the brain likes, of course — there ae psychological challenges, social challenges, physical challenges, emotional challenges, and so on. But an enormous amount of what we tend to call “gameplay” falls under the mathematical realm.

Among the metatheorems that Viglietti identifies:

Metatheorem 1. Any game exhibiting both location traversal and single-use paths is NP-hard.

Metatheorem 2. If a game features doors and pressure plates, and the avatar has to reach an exit location in order to win, then:
a) Even if no door can be closed by a pressure plate, and if the game is non-planar, then it is P-hard.
b) Even if no door is controlled by two pressure plates, the game is NP-hard.
c) If each door may be controlled by two pressure plates, then the game is PSPACE-hard.

Metatheorem 3. If a game features doors and k-switches, and the avatar has to reach an exit location in order to win, then:
a) If k > 1 and the game is non-planar, then it is P-hard.
b) If k > 2, then the game is NP-hard.
c) If k > 3, then the game is PSPACE-hard.

Despite the jargon, these are immediately applicable to your games right now, and phrased in game terms are fairly simple features.

The paper goes on to provide proofs and examples for games ranging from Boulder Dash to Doom.

When I said that narrative was not a game mechanic, but rather a form of feedback, I was getting at the core point that chunks of story are generally doled out as a reward for accomplishing a particular task. And games fundamentally, are about completing tasks — reaching for goals, be they self-imposed (as in all the forms of free-form play or paideia, as Caillois put it in Man, Play and Games) or authorially imposed (or ludus). They are about problem-solving in the sense that hey are about cognitively mastering models of varying complexity.

Some replies used the word “content” to describe the role that narrative plays. But I wouldn’t use the word content to describe varying feedback.

In other words, perverse as it may sound, I wouldn’t generally call chunks of story “game content.” But I would sometimes, and I’ll even offer up a game design here that does so.

The usual definition of “content” is “everything that isn’t code or rules,” meaning all the art and voiceovers and quests and whatnot. But that’s not what it means in this context, because we’re embarking on another one of thse annoyingly formalistic exercises here.

I have previously described the basic model I use for analyzing games formally as “a game grammar.” This was mostly a conceit for a presentation title, but in point of fact it fits the formal definition of “grammar” moderately well. You see, this model, which I have also termed an atomic model of game design, is concerned exactly with the morphology of games: the structure and form they take. It builds on the seminal work of Chris Crawford, who defined interaction as

a cyclic process in which two actors alternately listen, think, and speak.

 – Chris Crawford in The Art of Interactive Design

The game grammar model works the same way as all interaction does. The chief difference with game interaction is that one of those actors may actually be algorithmic: a computer, or a set of rules and processes. At core, a game is about figuring out the rules and processes that an opponent is using; said opponent might be a computer or a real person, or even the laws of physics and the physical constraints of your own body. Your job is to identify a goal (which might be handed to you by a designer, or might be one you set for yourself) and attempt to arrive at a way of interacting with this system that results in the outcome you want.

When we speak of a game system, that collection of rules is what we mean. Usually a system will be composed of multiple mechanics, each of which is made of up a variety of rules. A system like this has also been termed a “fun molecule,” an “atom” or a “ludeme” by various authors.

A system, though, is sort of like an algorithm, or a printing press. It repeatably performs a process, but given different stuff to work with, you can get a pretty different experience out of it. The term for the “stuff to work with” is content, and most of the time it is effectively “statistical variation.” An enemy with different stats, a level with different placement of platforms.

There is a class of games that focuses on user-generated narratives rather than on authorially imposed ones — you can read about the distinction in a very old talk called “Two Models for Narrative Worlds” I gave at the Annenberg Center at USC. In that talk I made the point that

These worlds can still tell stories. What we surrender is not narrative, but authorial control.

I coined the terms “impositional space” and “expressive space” to define the ends of this spectrum for myself.

Now, that talk long predates any of the game grammar sort of work. But effectively, my critique of quick-time-events and excess feedback used in narrative-driven games is primarily about impositional spaces, narrative imposed by the author(s) of the game; and it is essentially in a “ludic” context. And several folks took me to task for ignoring the expressive spaces and the spaces that are intended to serve as narrative generators in that critique.

Story, as it happens, has some rules too, largely based on how the brain works. For example, in The Art of Fiction John Gardner has a wonderful example of the ways in which repeated mention of physical objects causes them to become associated with emotions — in effect to become symbols. And then mention of objects associated with those objects does the same. In a sense, thematic freight becomes transitive.

That particular trick is used very very widely in all sorts of media. For example, Ravel’s Bolero has become thoroughly associated with sex thanks to the film 10, and now at this point you can conjure up that association by just playing that music.

Expressive spaces in games rely on this trick extensively. In fact, all forms of post facto storytelling by players do. They ascribe meaning to moments, and then the player builds a narrative arc through their selective memory of events. I often call this mythmaking, and we do it pretty much all the time, without even thinking about it.

In games designed to cause the player to put together stories, such as Sleep is Death, Facade, or Dear Esther, there is a system there, an algorithm — and then there is the statistical variation that is fed into it. And that statistical variation, the content, is actually little symbols and narrative moments, ones that are often impressionistic or disconnected. The “problem” the player faces is that of arranging them into a coherent whole.

The fact that symbols and moments and memories are profoundly intangible things does not mean that they can’t be manipulated in this way; fiction does so readily, as we have seen. From a mechanical point of view, though, they have much in common with the particular hand of cards you have been dealt, or the set of Scrabble tiles on your rack. You end your interaction with the system by making sense of them, which is different from finding a word in the tiles only by a matter of degree. Dear Esther‘s mechanics could be replicated with a different setting and group of symbols — to radically different emotional effect. When analyzed by the game grammar, we’d find two very different experiences to be the same game.

Let’s consider a thought experiment.

I was once in a discussion with some fellow designers and one of them was playing with the idea of a game about memories. I offered up a design idea whereby there was a map of a house, and there was a deck of cards, each card labelled things like “comfy armchair” and “deep closet” and “empty bookshelf.” The deck was shuffled, and some cards were laid in each room.

Players would then take turns tapping a card and telling a “memory” about that card and its place in that house. That this was the armchair where you remember curling up to read, a memory of safety and comfort; and another player says it was where they found great-grandmother when she finally passed away. All memories must be “true” — meaning, they cannot contradict anything anyone has said. After all stories were told, all the players decide which way they want to remember the armchair from among the stories told, by voting.

The person whose memory was selected keeps the card. At the end of the game, whoever has the most cards wins.

For greater emotional impact, you play this with real family, a real house layout, and real objects from your childhood.

Here we have both emergent consensus narrative and a game system. The memories are actually tokens in the game space — intangible ones, with a lot of emotional weight to them. You can approach the game mechanistically, and strategize. But you can also approach it experientially. It is mostly an expressive space. And ultimately, the real game lies in making sense of your family, its history. It is still pattern-matching, grokking each other and the complex web of relationships and half-truths and biased recollections that make up a family history.

In this game,

• narrative is input — the affordance given to a player, the “move they can make”
• narrative is a resource — accumulated and managed towards a victory condition
• narrative is actually content, user-generated even, providing statistical variation into the system
• narrative is feedback — its accumulation, in the form of individual symbols, is representing the gestalt “game state”

But it’s still not a mechanic. You could in fact replace the memories with differently colored poker chips, and everything would proceed in the same manner. The experience would be substantially different, and the emotional impact far less.

You could also de-game this. Don’t negotiate whose memories win out. Don’t have the rule about non-contradiction. You’d end up with the experience of looking through a photo scrapbook — and likely, you would not tackle the challenge of understanding that the rules push you towards.

This game has never been played. If anyone ever does, let me know what happens.

In the post title I said that narrative isn’t usually content. This game is an exception, as are the other ones I have cited. Ironically, games where narrative is content actually tend to have very very complex and robust rule systems. Chris Crawford’s Storytron has years of development in it, almost all in the systems design. Facade is an AI wonderment. And even this little non-digital game has as “imported” rules a host of psychology and past family history, rules that are deeply perilous to transgress. (The mere addition of other players always imports complex social rules into a game; in this case, the deeply personal nature of the interaction brings in yet more. “We never talk about her drinking problem” and the like).

Because of this, I have no issue reconciling formalism in examining the “ludology” of games with the “narratological” approach of examining games-as-stories. My issues with small-system-big-feedback games described in the other post have to do with the lack of substantive pattern-learning, the lack of player agency, and thus the lack of the fundamental qualities that games bring to the table. And in that, I include emergent-narrative games and expressive spaces, which I certainly consider games — more complex games, in point of fact, than most games are. So for those who felt I was bashing the entire genre of emergent narrative games, I apologize for the lack of clarity there; that was not at all where I was going with that post.

So where does this all leave authorially imposed story? Primarily in the realm of interactive experience design. Which is a different discipline from “game design” though they have tremendous overlap. I am biased towards our getting game design right, but that does not mean that interactive experience design isn’t a fascinating and deep area in its own right — or that it is unimportant to games. In fact, it’s incredibly important. But that’s a subject for another post someday.

From Twitter and the hilarious HULKGAMECRIT.

HULKGAMECRIT: @raphkoster @ibogost @larsiusprime HULK WONDER WHAT FFEDBACK WOULD LOOK LIKE IF HYBRID GAME DESIGN & WRITER WERE TAKE CONTROL OF NARRATIVE!

Me: @HULKGAMECRIT There are a lot of those hybrids working. I’m one, trained as a writer, have MFA, do systems design & direct games too.

Me: @HULKGAMECRIT I think that means feedback would likely look much like it does today

HULKGAMECRIT: @raphkoster HMMM HULK SHALL THINK ON THIS…. HULK ALSO REALLY NEEDS TO PUBLISH MOSTLY FINISHED HULK ARTICLE ON JRPG DIALOUGE….

Me: @HULKGAMECRIT Hulk shouldn’t think on it. Hulk should smash it.

HULKGAMECRIT: @raphkoster HULK DESIRE TO MAKE GAME INDUSTRY A STRONGER PLACE! THEREFORE HULK LAWS WERE ESTABLISHED TO SMASH ONLY WHAT NEEDS TO BE SMASHED!

@HULKGAMECRIT Feedback being the way it is may need smashed.

HULKGAMECRIT: @raphkoster EVEN THOUGH HULK IS GREEN AND SILLY…HULK SPENDS A LOT OF TIME THINKING! (HULK KNOW HULK IS WEIRD)

Me: @HULKGAMECRIT Never weird. Misunderstood

HULKGAMECRIT: @raphkoster <3

I guess it’s not easy being green. Or gray, though I admit I never think of him that way.

You have a system. Let’s say it’s a system where you can throw darts. And you have to open your bar in one week.

Throwing darts might have a bad interface. The dartboard might be too small or too big or poorly lit. Darts may be a perfectly nice idea, but the implementation of it needs tuning.

At this point, you have a feature, but not fun. It’s gonna take you four days to make it fun.

You can refine darts, get it fun. Make the UI good, have a great physics model and control, great graphics, and in general, you can get it to where you have a fun feature.

The exercise here is going to be threefold: making sure the inputs afforded to the user map well to their view of the “black box” that is the darts system; making sure the darts system itself offers interesting repeatable challenges; and making sure that the feedback from the black box Is both juicy and educational, so that the user can get better at darts. All this is hard.

Then you face a choice. Three days left, if you made darts fun. You can either go implement a pool table, or you can add content to darts. Content would be new kinds of darts, more kinds of dart games, etc. They don’t call for a new system, just other kinds of data. Not much new code (and new code runs the risk of introducing bugs). You can make the best darn darts game in the country if you spent the three days on that.

The pool table, you could get that in instead. But what if it’s not fun on the first try, just like darts weren’t? Then you’d have a decent darts game and a crappy pool table.

Which is better, having the best darts game available, or having a middling darts game and a bad game of pool?

Adding features offers the potential for fun, but fun comes from tuning and balancing. It isn’t magically there just because you got a pool table.

In order, I would have to tackle the black box, then the inputs, and finally the feedback. I can’t make a bad system much better with great affordances and while I can make it juicy, it usually won’t hold players.

I would choose to polish up darts, and promise to get a pool table in there as soon as I can. And when I do put in the pool table, it’ll be as good a game of pool as we can make rather than being rushed to fit in before the bar opens.

This isn’t the answer I would always have chosen in my career. I have done plenty of kitchen sink design. I have also settled for poor affordances and feedback far too often. I have even had perfectly wonderful systems be unusable because we could not figure out a way to make the feedback comprehensible.

I’ve always leaned towards elegant systems, meaning ones with few variables and few rules to them. When you populate a game with many of these, they very frequently end up leading to emergent behavior, which can be quite fun. But when you lean on the creation of simple systems, the temptation is even greater to have lots of them in your game. And that can and will lead to most or all of them feeling unpolished and unfinished.

I’ve gotten good enough at coming up with simple rule solutions that gosh, almost 10% of them work on the first try! (Yes, read that as sarcasm aimed at myself). But these days, I tend to assume that it will take me ten times longer to polish up and tune that rule system than it will to come up with the rules in the first place.

This is the original essay in which I worked out the basics of my game grammar approach. It later became a GDC talk. This essay was written in 2004, and the genesis of it was working through issues with the crafting system in Everquest II with Rod Humble. This essay no longer represents my current understanding of game grammar, but it’s a decent start.

This essay has never been publicly posted (it was originally posted only to a private game developer forum, on June 26th of 2004), but I thought I should make it available both for historical interest and also for the sake of clarifying some of the things that I now take for granted when I discuss game design here on the blog. I can’t expect everyone to have read everything I have ever written, of course, and in this case it’s even worse since some of the material was only delivered at conferences. So many of the responses to the article on narrative were clearly from folks unaware of some of this work that it felt like the right time to post it up.

Since this was written, I have met fellow travelers — boy, was I pissed when Dan Cook’s “Chemistry” article came out a few years later, and had such nicer diagrams! I also found Ben Cousins’ work on “ludemes” later, a term I gladly stole. And I think this served as some inspiration to folks like Stéphane Bura and Joris Dormans who have pushed this in fresh directions I would never have pursued. There have been grammarian get-togethers, Project Horseshoe whitepapers, and more. I even have a pile of blog posts that fit under the “game grammar” tag here on this site for those who are curious about more.

Lately, I’ve been spending a lot of time thinking about the nature of fun. I’ve reduced it down to a cognitive challenge, the notion that fun is the feeling you get when you are exercising your brain by solving a cognitive puzzle. Sometimes the puzzle is provided by a computer, sometimes by another player, but either way, your brain is basically trying to perceive a pattern; victory usually comes from identifying the pattern, then correctly executing on some action that the pattern does not account for. For further thoughts on this and its implications for game design as an art form, I refer you to my presentation “A Theory of Fun.”

This suggests that there are probably ways to break down or otherwise analyze what makes a given puzzle or challenge fun. Now, challenges or puzzles come in a very wide array of forms—spatio-temporal challenges like Tetris, and physical dexterity ones like soccer. What do all of these have in common?

Building a subgame atom
The following algorithm came about from attempting to map the basic features of MMOG combat systems onto MMOG tradeskills, which are usually regarded as not having met a sufficiently high bar of fun. Interestingly, MMO combat itself is often not regarded as having reached that bar either, and yet it succeeds in keeping players captivated for many months on end, when correctly executed.

A successful MMO probably needs to have many individual subgames (of which combat may be one) in order to be successful, and for maximum impact, each of them needs to fulfill all of the following requirements. In fact the atom needs to have certain known “system inputs” and “system outputs” so that it can be hooked together to build game “molecules” if you like.

We typically refer to each atom as being “a game system” in game design, but part of the point of this essay is to show that this definition is to a degree recursive. Once you have knitted together several atoms into a molecule, the molecule as a whole must also meet all the criteria for what makes an atom fun. When looking at a piece of interactive entertainment, it is made out of at least one atom, and possibly many molecules, as in the case of MMOGs. In the end, what we refer to as “scope of a game” is really measure of how many atoms it has.

Taking the MMOG combat example
I’ve selected hack ‘n’ slash MMO combat for my example, because it is a well-established mechanic that has undergone twenty years of refinement; it is a multiplayer mechanic, which brings in additional wrinkles for cooperative games that would otherwise be absent from the model; it can be performed either against computer opponents or other players; and it is typically not a full game in itself, but is one system among several in the MMO, thus demonstrating the recursive and connective qualities of game atoms.

When you break down what makes MMOG combat entertaining, there turn out to be a surprising number of required elements:

• Preparation is required. In most cases, this may be as simple as healing up before entering the encounter. At its best, there is an opportunity here for hooking the combat system into the larger network of subgames via the need for non-combat roles. At a minimum, different forms of preparation should be viable, in order to provide different tactical choices.
• Important point: preparation cannot be allowed to overwhelm skill. In fact, in a game that is a pure test of skill, you may not have variable preparation. It’s worth noting that when this is the case, advanced players will often come to the challenge less prepared on purpose, such as playing DDR backwards, handicapping yourself in chess, or playing Diablo in hardcore mode.
• A sense of place is required. Physical location affects tactics during the encounter, and affects the strategy of which locations to have encounters in. In other words, locations affect risk and reward.
• Important point: it’s easy to have locations that turn out not to matter, in which case this becomes merely tedious. When they do matter, however, they add immensely.
• A solid core mechanic. This is in the nature of a puzzle to solve—it’s a ruleset into which content can be poured, that is intrinsically interesting. Effectively, this core mechanic may be an entire atom in itself.
• Important point: Note that by itself, it is probably interesting only for a limited amount of time.
• A range of challenges. This is content; each enemy provides a unique puzzle. Combinations of enemies then provide additional puzzle types as well.
• Important point: whenever you reach the point where fighting the new enemy can be done using the same tactics as the previous enemy, you have not actually added a new challenge to the game. This is where players find repetition.
• A range of abilities required to solve the encounter. We design our encounters such that it takes teamwork to resolve them, via formally preventing any given character from having all the abilities required. Even in single-player games, however, the player who relies on a single ability usually ends up unable to compete at higher levels.
• Important point: Usually this starts out not being present at the lowest challenge encounters, and rises in necessity as you reach more complex encounters.
• Skill in using the abilities is required. Bad choices lead to failure in the encounter. This skill can be of any sort, really: resource management during the encounter, failures in timing, failures in physical dexterity, failures to monitor all the variables that are in motion.
• Important point: This says nothing about the level of preparation in advance of the fight, which leaves out the issue of skill entirely. It should be possible for a skilled and a non-skilled player to have radically different experiences, all other things being the same. Otherwise, players will rightfully say that they could automate the fight.
• A variable feedback system should be in place. The result of the encounter should not be completely predictable. Ideally, greater skill in completing the encounter should lead to better rewards, but some degree of variability in the reward is important to maintain interest and minimize “farming.”
• Important point: you’re playing with fire here, and if it makes you nervous, this is the only optional item on the list. Merely randomized drops is not sufficient, and it’s incredibly easy to alienate players with long waits for what they want. The feedback should be such that it is always worthwhile, but perhaps not to you—or there should be a system whereby the feedback you get (e.g., the loot) is perhaps tailored to the way that the encounter was resolved. The reward should always be contextual to what the challenge was.
• The Mastery Problem must be dealt with. The system cannot permit high level players to derive maximum benefit from less challenging encounters, or you get bottomfeeding. This has the detrimental effect of also closing lower level players out from access to the content.
• Important point: the only effective way to do this is to forcibly make the lower level content no longer appeal to (or even be usable by!) higher level players.
• Failure has a cost. Loss of the given encounter or challenge ejects you completely from the atom, and pops you out of the stack. Next time you attempt the challenge, you are assumed to come into it from scratch.
• Important point: next time you come back, you may be differently prepared. The challenge you faced does not necessarily need to be reset to its original state, as “wear it down” is a viable approach to a given challenge. This leaves open the question of penalties for failure that extend beyond the individual challenge.

Building an atom diagram
All of these things can be applied to any game system, but it requires approaching the issue theoretically. A simple list gives you this breakdown of required elements:

• Preparation is required.
• A sense of place is required.
• A solid core mechanic.
• A range of challenges.
• A range of abilities required to solve the challenges.
• Skill in using the abilities is required.
• A variable feedback system should be in place.
• The Mastery Problem must be dealt with.
• Failure has a cost.

You can, using this atomic method, consider games as directed graphs of game atoms. A given atom looks like this:

In this diagram, you will notice that the abilities and skills are balanced—that is because otherwise, you get the Mastery Problem. The core itself then becomes a loop of repeatedly testing these abilities and skills against those of the challenge. It is worth pointing out that this entire core mechanic may well be composed of multiple atoms itself.

In terms of ludological theory, since atoms “stack,” popping off the top of the stack effectively takes you out of the “magic circle” of privileged space.

You can then diagram games using this method, and see what the relationships are between game systems. Each input and output onto an atom can be linked to the input or output on another atom, and atoms can be nested within one another.

It’s beyond the scope of this little essay to try diagramming an MMO, but even checkers provides an illustrative example.

1. The preparation step is generally skipped, since this is a two-player game. However, a typical form of prep would be to handicap one side or the other.
2. The place is a fixed board, though interesting variant boards have been constructed using the exact same ruleset (CheckersFour being one such).
3.  The core mechanic of “checkers” is “capture all pieces.”
4.  The range of challenges lies in different possible opponents. In that sense, you can consider checkers to be a “simple game” in that it is not a multi-stage experience. Most board games are not, but something like a puzzle game, which swaps out the space and perhaps the preparation on every level, certainly is.
5.  The range of abilities to solve this is very limited. You have pieces on the board. You can move them. You can jump with them.
6.  The skill lies in choosing which piece to move with. This part is in fact another game atom, because each move is a problem to solve itself: move, or jump?
7.  The variable feedback of winning the overall game of checkers is entirely driven by your emotional response and that of your opponent.
8.  The overall game of checkers does not attempt to solve the mastery problem via formal mechanics. If you come up against a toddler, you will win. This will make for a dull game for both of you. Because of this, we customarily avoid this sort of match-up.
9.  Failure to capture all the pieces results in loss. Next game, you have to start over from scratch, without any of the captures you achieved last game. All record of achievement is lost. Note that layering on something like a rating system or rankings or win-loss records actually nests the entire game of checkers as an atom within the “climb the ranking ladder” game.

Now, nested within the game of checkers is another atom, that of moving a piece and attempting a capture.

1. The preparation step is actually that of the previous moves.
2.  The place is the specific tactical situation created by the previous moves.
3.  The core mechanic is “capture a piece.”
4.  The challenge lies in the risk of capture of multiple of your pieces.
5.  Your range of abilities involve moving one piece diagonally. Depending on the game situation, that may be a king capture or a regular checker capture, and it might include multiple jumps.
6.  There is no skill required in this atom. This is an indicator that you cannot nest another atom within this one. It is a fundamental atomic unit of checkers.
7. The variable feedback for success is interesting. There’s the direct success of capturing one piece, there’s extreme success of capturing multiple pieces via a chain of jumps, there’s the orthogonal success of failing to capture a piece but instead creating a king, and there’s the Pyrrhic victory of sacrificing a piece in order to create a better landscape in the next iteration of the atom. This variety is what makes checkers an interesting game. Removing one of these dynamics would make the game significantly poorer.
8.  Since there is no skill involved in the game of moving a single piece, there is no mastery problem.
9.  Failure ejects from the atom, and next turn you start on a new landscape that is worse off than the one you just left (because your opponent gets to move).

Practical applications
All the above sounds very theoretical and useless. What happens when you walk through a crafting system using this? Let’s look at what we do with tradeskills in most MMOs.

• Preparation is required. Ever since the days of Ultima Online, we’ve relied on “getting the right pieces” to provide the fun in crafting. So we’ve done well on the “preparation is required” aspect of things as far as crafting goes. BUT: Preparation in itself ideally follows all of these rules recursively (eg, we should regard the harvesting mechanic as needing to follow all of these rules in and of itself). By and large, we have not done that. We also have usually failed to take full advantage of the “preparation should be able to take multiple forms” part of this—we have by and large gone with static ingredients for static results.
• A sense of place is required. This is highly underexploited in crafting systems today. Different locales providing different advantages and disadvantages to crafting was explored a bit in Star Wars Galaxies, but has not been addressed much by and large.
• A solid core mechanic. By and large, we’ve relied on simple combination. That’s not an interesting ruleset in and of itself—it shifts all the burden of fun onto the preparation. We need a mechanic that “fights back” as AIs do in combat. This is the core of the issue with crafting. It is currently on the order of moving a single checker piece, as opposed to playing checkers.
• A range of challenges. This is, simply put, the range of possible craftables. However, it’s worth noting that since the solid core mechanic is missing, the range of challenges is not generally significantly interesting. The different items (as “opponents”) do not have different abilities or skill to use against you except for perhaps a variable failure rate.
• A range of abilities required to solve the encounter. This is usually true to a degree, but currently, most crafting systems do not require interdependence between players at the actual crafting step. Instead, they put all of that in the resource gathering step. The abilities needed are part of the preparatory step, since there’s no explicit use of ability during crafting itself. Requiring multiple individuals working together to create something has best been expressed by pizza-making in Sims Online, and has hardly been used anywhere else.
• Skill in using the abilities is required. This is missing altogether in most cases. There’s a minor amount of gambling in SWG’s crafting system, but that’s about it. Lacking a core, there’s no way to use abilities. A good core mechanic is going to provide scope for use of abilities, and for skillful application of those abilities.
• A variable feedback system should be in place.  In most cases, we do not do this. Currently, success almost always gives you exactly what you want. This means that crafting an item is actually simpler than moving a checker piece, which has multiple possible outcomes.
• The Mastery Problem must be dealt with. This problem crushes most online game crafting systems, as high level crafters completely block the market to lower levels players.
• Failure has a cost. This, we usually do.

In the end…
A game diagram can be regarded as fractal. When you diagram your game, each possible skill choice will be an atom, and systems will be built out of linked and nested atoms. From far away, the whole game looks like one atom, one where the failure and success cases are both “game over.”

If you have an atom that has a skill choice within it, and there’s no atom nested within it, you’re not done designing. If you have an atom that doesn’t hit all nine elements, that atom probably won’t be a fun game system, and needs to be redesigned.

Is this an algorithm for fun? No, but it’s a useful tool for checking on the absence of fun, in that you can identify systems that fail to meet all the criteria. As such, it may prove useful in terms of game critique. Simply check each system against this list:

• Do you have to prepare before taking on the challenge?
• Does the preparatory step pass this list as well?
• Can you prepare in different ways and still succeed?
• Does the environment in which the challenge takes place affect the challenge?
• Are there solid rules defined for the challenge you undertake?
• Can the one ruleset support multiple types of challenges?
• Can the player bring multiple abilities to bear on the challenge?
• At high levels of difficulty, does the player have to bring multiple abilities to bear on the challenge?
• Is there skill involved in using an ability? (If not, is this a fundamental “move” in the game?)
• Are there multiple success states to overcoming the challenge? (In other words, success should not have a single guaranteed result).
• Do advanced players not get a benefit from tackling easy challenges?
• Does failing at the challenge at the very least make you have to try again?

If any of the above answers is “no,” then the game system is probably worth re-addressing.

On page 155 of Theory of Fun for Game Design there is some sheet music. It looks like this.

Click for full size

This is that song, played on solo acoustic guitar. Download audio file (IdEst.mp3)
– download

The song is played in DADGAE tuning, one of my favorite “weird” tunings — basically DADGAD with an added 2nd. As usual, I miked up like crazy: two condenser mics aimed at the guitar (one at the soundhole, the other at the 12th fret) plus a bigger diaphragm mic sitting a couple of feet away. I also used a pickup on this one, a Dean Markley Promag Grand.

I have the sound space set up a little weird… the ambient mic is “in the back,” by applying a fair amount of reverb to it. It’s panned around 36% to the right. The fretboard mic and the pickup and panned hard left and right, with much lighter reverb. And the soundhole mic is dead center, with a dry signal.

This has been knocking around the house since 2003, but I just got around to recording it right before the holidays. Enjoy!

I love stories. My chief hobby is reading. I was formally trained as a writer, not as a game designer (there wasn’t really any formal training for game design I got started, but that’s another story). I think most game stories are not very good. And I quite enjoy games with narrative threads pulling me through them. When I find a game with a good story, I frequently prefer to the story to the actual game! So please keep that in mind as you read: I love story.

Narrative in a game is not a mechanic. It’s a form of a feedback.

This simple fact is frequently ignored, particularly in games aimed at the mass market.

Let’s start thinking about this by looking at what a game is. Games can and do exist without narrative. The core of a game is a problem to solve. As game grammar tells us, it’s actually typically a series of nested problems: I need to reach this location, which means I need to defeat enemies, which means I need to traverse space, which means I need to mash a button. Some of these, like “defeat enemies,” are complex problems in their own right. Some of them are trivial problems, such as “mash button.”

If you string these together, you’ll typically find that the problems will alternate between abstract problems and simpler interface problems. For example, most turn-based board games alternate between the complex strategy problem of “what move to make next” and the simple interface problem of “pick up piece and move it here.” Board games, of course, tend to be very forgiving regarding interface problems; if you drop the piece, nobody minds if you pick it up and put it where you meant.

If you take something like a racing videogame, you now have a fairly hard interface problem; the sensitivity of the steering wheel or the analog stick is now an actual physical motor challenge — often a bigger challenge than the cognitive problem of where to point your car. Just turn on the “ideal path” feature that most racing games provide and you’ll see that the challenge in the game as a whole tends to come from the dynamics of the controls and the black box of the performance characteristics of the car you have chosen.

In a game grammar model, you always have a black box model, and you select something to input into the system. The system is going to give you feedback as to what effect resulted from your action. The game is in figuring out what the rules are for the black box. Not in a rigorous way, mind you — you tend to arrive instead at a mental model that gives you a heuristic as to how to approach the black box. Your learning is generally more intuitive — or even motor memory. It tends to function less at the logical level and more at the gestalt level (cf fluid versus crystallized intelligence).

The brain therefore tends to eventually dismiss whole classes of interface problems as trivial. “Click mouse” is one of these. “Mash button.” Any black box which gets a heuristic of “guaranteed result” is going to make for a non-game very quickly.

Ah, but the feedback for even a trivial action is very important. It matters that we hear the sound when we click the mouse. And should the designer choose, they can make the feedback be hugely disproportionate to the problem solved. Feedback serves the purpose of cueing the user whether or not they are being successful in figuring out the black box. So we provide feedback each time an input is made, and the feedback is intended to help guide the user as to whether they are doing the right thing.

It is easy to see that if you remove any one of these things, you end up without a functioning game.

• Cut the input, and you have a screensaver.
• Cut the problem inside the black box, and you have a slideshow.
• Cut the feedback, and you have something ridiculously confusing that no one will tolerate.

That said, the brain happens to loooove feedback. It triggers reward mechanisms in the brain. It is remarkably easy to trick the brain into thinking that it has accomplished something when it really has not. This can result in the player getting hooked on the feedback for a black box system that is actually remarkably simple — or even designed to not teach the player anything at all, as in gambling. In design, we often terms designs “juicy” when they provide plenty of rich feedback, but we sometimes call them “exploitative” when they simply abuse feedback to keep someone going.

Games are a compound medium. They are made up of multiple other media, typically in the feedback. In other words, we rely on media such as film, writing, visual arts, music, and so on in order to provide the feedback. Games that do not rely on these other media much tend to get called “abstract” — a completely stripped bare game is actually a mathematical diagram or formula, not something easily seen or comprehended, so all games have to make use of other media at least a little.

But these other media are of course very powerful in their own right, and each have their strengths. You can use music to convey emotion using elementary musical techniques. For music it’s easy. Game systems can convey emotion as well, but for game systems it’s hard, and near as we can tell, game systems also have a very limited emotional palette. So we not only use these other media to supplement the overall experience, but these other media can convey information that exists in complete parallel to the game system.

The commonest use of a completely parallel medium that does not actually interact with the game system is narrative.

If we take the simplest form of “narrative game,” the choose-your-own-adventure type of books, what we see is that the overall problem presented to the player, “get to a good ending,”  is fundamentally a decision tree, or directed graph in mathematical parlance. (Check out this link for a great giant diagram of one). So the initial problem is actually a decently substantial one — how do you get to a good ending, given that you have  no picture whatsoever of the graph as a whole or even of what lies “behind the choice of doors”? Every page flip is a Lady or the Tiger sort of scenario.

What ends up influencing your choices is the feedback. Your first feedback comes from choosing to play at all: the first pages. This is exactly the same as giving you the starting layout of the pieces on a chessboard. From then on, each time you make a choice, narrative is used as the feedback mechanism to tell you whether a given choice makes sense or not in terms of leading you to your ultimate goal.

A bad example of these books would give you positive feedback all the way until it dumped you into a lava pit or other horrible doom. A good example would ensure that there was a rising action and feedback arc for each of the possible paths, leading to both a learning experience and a good story.

Side note: my favorite of these was actually one where you were trapped in a spaceship with nasty aliens, Inside UFO 54-40. There was no path to the exit. You had to actually traverse the entire graph to realize this, and then when you got suspicious, identify the page that had no inbound link to it — which then boldly celebrated your cleverness. The player’s lesson learned is therefore quite profound: sometimes situations are set up not to be winnable, don’t blindly obey the rules, and you should engage in lateral thinking when your survival is at stake.

Now I will pick on one of my absolute favorite games of the last year, Batman: Arkham City. There is a moment quite close to the beginning that has you climbing up a tower (spatial navigation puzzle with interface problems as well) only to reach the top of a cathedral where you are interrupted by a narrative moment: a video of the Joker playing on a television set, explaining that he has planted a bomb and you are about to die.

AFter this long bit of rich narrative, you are presented with a very very small game. You have to rotate your camera to point at a window, and press the A button. If you do not do it within the time allotted, you die. If you succeed, you are treated to a gorgeous cinematic moment of leaping through shards of shattering glass and spreading your cape to a rigid wing, as you float away from a savagely exploding edifice in the midst of the brooding panorama view of Gotham.

The diagram for this looks like a big pile of feedback followed by a tiny tiny problem, followed by another big pile of feedback.

This is a very common pattern in videogames these days. It even has a name: the quick time event. The black box is miniscule and simplistic. If we were to reduce the feedback, this “game system” would strike us as stupid. And it was stupidly simple in God of War, it was stupidly simple in Uncharted, etc. (Please note, I am naming some of my favorite games, made by colleagues and friends! Fortunately they, and Arkham City, offer plenty of rich systemic gameplay in between the movie bits. ) It is almost, but not quite, as stupidly simple as the page flip in a Choose Your Own Adventure book.

In fact, there was a quasi-parody series of games that basically took these sorts of super-simple game systems, and packaged them up with ridiculous feedback wrappers. It was called WarioWare. It derived all of its black box challenge from two factors: speed, and determining what the hell the confusing feedback meant. (The fun in the game was figuring out what stupidly simple action you were supposed to take).

There’s nothing wrong, to my mind, with using narrative as feedback. But we have to keep in mind that all that narrative and visual content is the expensive part of making the game. It is also consumable, whereas a systemically driven game system can provide many many problems to solve and heuristics to develop (and therefore fun to be had), with relatively few rules. Because of this, narrative content is destined to be expensive, short, and over.

If you have built a game where your graph looks like this

And at the end of it you leave the player with “replay” that is nothing but thisYou’re not going to get people to keep playing unless you keep releasing more content. This will matter quite a lot for any service-based game, be it MMO, F2P, social game, whatever.

I also feel fairly comfortable in labelling a game with that sort of structure as “a bad game design” even if it may be a great game experience. The bar that designers should strike for should include a rich set of systemic problems precisely because that is what the medium of games brings to the table. It’s what lies at the center of the art form.

If the systems of your game are outweighed by the feedback, you should grow suspicious. And if they are outweighed by feedback that takes the form of movies, you’re making interactive movies first and games second.

I gave a few books as gifts last year. These are the three non-fiction books I most often recommended to people. I ended up giving away multiple copies of these.

The thing they have in common: they all make you revise your view of the world.

1493: Uncovering the New World Columbus Created

It’s sort of like Connections but for world history since that date, with a huge emphasis on the way globalization changed the world. One of the best moments is when he talks about plants… as American as apple pie (apples are from Kazakhstan); Italian food without tomatoes (they are from the Americas); the Irish and potatoes; southeast Asian cooking without chilies (from Mexico); Switzerland without chocolate (also Mexico); and rubber having basically moved from Latin America (where there’s a native endemic pest that kills the trees) to Southeast Asia (where there isn’t… but 40% of the world’s rubber comes from those trees, we’re utterly dependent on it for airplane tires, electrical wiring, and medical use; and we’re just waiting for the moment when some idiot travels from Brazil to Indonesia with some mud on their shoe and kills the entire monoculture crop in Vietnam & South China.

I can’t resist, even though it will bloat the blog post…

There’s a chunk in there which explains how (if I can remember it correctly) the fact that Europeans were engaging in trade with Africa and got bitten by mosquitoes there meant that they became latent carriers of malaria… which then resulted in them getting bitten again by non-carrier mosquitoes in the tropics in Latin America, which resulted in the rapid spread of malaria to mosquitoes in the Americas, which meant that when silver was discovered in Potosi in Bolivia, or really riches anywhere, a malarial climate meant that the primary source of labor had to be imported from a population that carried a genetic resistance to malaria — namely West Africans; and this is why slavery never took strong hold in the territories outside of the anopheles mosquito’s climate range.

Of course, the malarial areas in the Amazon meant that the silver could not be exported via the relatively short Amazon River route, and therefore had to be carried by llama to Peru, where it was shipped to the Philippines and to Spain; this is why there is such a substantial Asian population in Peru today.

This also led to the introduction of potatoes to Europe, which led to a massive rise in population, but since potatoes were cultivated via “clones” from only ten genetic lines, there was a massive monoculture which later led to the Irish famine.  In Europe, it resulted in a massive financial crisis because of an excess of currency supply from all that silver in every nation in Europe, which led to dramatic inflation followed by the financial collapse of just about every monarchy on the Continent and the creation of a middle class.

Meanwhile, the reason there was an appetite for silver in the Philippines was because China had settled on silver (of which there are no significant local sources) as their currency, so they actually took something like half the silver from the Americas. In China, this ended up leading to a liquidity crisis when the mine in Potosi was exhausted which resulted in the Ming not being able to pay the troops for defense against the Manchu, so malaria indirectly caused the fall of the Ming dynasty. The Manchu then tried to exterminate the pirates who traded the silver with the Philippines, but couldn’t quite because they were hooked on tobacco at this point (thanks to the Amazonian variety having been smuggled to Virginia). But they did get a lot of the pirates fleeing into the hills with sweet potatoes and maize, which resulted in the deforestation of the hillsides thereby destroying the incredibly intricate and careful Confucian-era water management schemes, which led to the epidemics of flooding that China saw until they initiated their modern dam projects.

I’m just scratching the surface — it’s dense and intricate and in places nearly unbelievable. I loved it.

The Information: A History, a Theory, a Flood

It begins with African talking drums and the curious poetry they contain embedded in their beats, and from there embarks on a tour of all of information theory. We learn of entropy and Claude Shannon, of Babbage and Ada Lovelace, of the way that “a noisy line” is encoded into our very genome, of Maxwell’s Demon and the telegraph and even a small bit of number theory.  And it goes further, into quantum computation, tracing the way in which you can regard just about everything — the physical world itself — as information, data to be parsed and represented, in more and less accurate ways.

It does contain an excellent overview of information theory itself, but more importantly, it is a cultural history of information itself, how it spread and was conveyed. As this is the great cultural tide of our time since Gutenberg (and indeed, the book culminates with talk of Twitter), it resets yor entire view of what culture is and means.

I was struck by the chapters getting across the great shift we have undergone, from information’s scarcity to today’s utter overload — by the example given of Beethoven not knowing Bach’s music, while today we know it all. For me this rings true in the way in which all styles of music seem to be popular at once these days, with top hits featuring banjos and swing jazz and Motown and Delta blues all mashed together. With everything at your fingertips, even the context of why they might fit together is yet more information, and today’s child grows up in a world in which everything in the past is still the right now, because you can no longer even tell the dustier facts by their presence in dustier pages. In music, this leads to the Amy Winehouses and Adeles, who may as well have been born a few decades too late. In games, it is the 15 year old NES aficionado (my own son loves to talk about his collection of original Gameboy games… a platform that died while his mother and I were still dating).

The power of this book is also in drawing connections, connections between means of communication and the underlying math of communication itself. Through the course of the book we come to understand why computing is the way it is, and also, most critically, that information is not meaning. It ends on a note of awkward transcendence:

We are all patrons of the Library of Babel now, and we are the librarians…

The library will endure; it is the universe… We walk the corridors, searching the shelves and rearranging them, looking for lines of meaning amid leagues of cacophony and incoherence… every so often glimpsing mirrors, in which we may recognize creatures of the information.

 The Filter Bubble: What the Internet Is Hiding from You

The third book essentially takes the above two and applies them to the present moment. If virtual cultures are a new form of globalization that basically threaten the very notion of the nation-state; and if they, as virtualities, are premised entirely on the concepts of information and information theory, well, then, that means that we are not only at the verge of a Panopticon but also in the curious place where there are people who control those mirrors, and decide what we know about ourselves.

In short, this book is about the fact that we have set up our digital world to lie to ourselves. Repeatedly. We have built it, step by step, to confirm our biases and to never show us anything that troubles our placidity.

This is not a new thing; our buttons have always been pushed by media in whatever way earned media the most money. Well, these days our “libraries” — meaning, our search engines — and our “newspapers” — meaning the blogs we read — are tailored to us. The example I typically give of this phenomenon is what happens when you log out of Google (which I know, you never do!), and then search for “abortion.”

Google takes your IP address, the ZIP code it gets as a result, the sum of all the cookies it might have, and figures out whether you’d rather see Planned Parenthood or a right-to-life website as your top result. If you’re logged in and you have a Google+ profile, it’ll be the first result on a vanity search. The circulars you get, the identity files that companies maintain about you and sell to third parties… the ads that follow you from site to site to site… and yes, the news you get and the (certainly false) picture of the world you get, built to show you only what you would normally be shown.

Personalization services give you what they think you want, you see. And you are tracked and dissected and bucketed quite a lot. But there’s no filter for “I like variety” or “I like surprises” or “I like things I didn’t know I liked.” That’s why most citizens of the United States think that they know about the world, when in fact the world’s news almost never makes it past the borders. It’s how China maintains a great firewall, and how there could be thousands of North Koreans gathered to mourn the man who essentially built an alternate reality for an entire nation.

I love it that say, Amazon knows what books I have bought and liked and therefore offers up apropos selections for further reading. But the long-term result, which we see manifested clear as day, is that Republicans do not read books by Democrats, and vice versa. It is that millions of people who Tweeted yesterday saying “What is SOPA or PIPA and why is Wikipedia dark?” The information, it is out there. But it always reflects us right back to ourselves… well, that and a suitable, well-targeted banner ad.

We can go back to 1493 to see the long-term results of this sort of short-sighted thinking. “Recommended for You” is the cultural equivalent of China’s blindness with the Potosi silver mines.

Taken together, these three books give much to ponder. I can’t recommend them enough.

Feast your eyes on the book porn to the left.

Go ahead, click on it and get the larger picture.

Gorgeous, aren’t they? They’re the complete set of the D’Artagnan Romances by Alexandre Dumas: The Three Musketeers, Twenty Years After, and the three volumes of The Viscomte of Bragelonne,the final volume of which is generally better known as The Man in the Iron Mask.

They were published by Thomas Y. Crowell Co., no longer extant as such, in 1901. Not first editions — that would look like this – but glorious nonetheless. Gilt on the edging, inlaid on the relief covers, onionskin endpapers in front of every engraved illustration…

Nice enough that you can still buy an facsimile of this exact edition, alas without the rich red covers and with something fairly hideous on the cover instead.

They’re something to hold, to examine. Maybe not to read. Defintely something to have visible on a shelf where people can ooh and aah. They were given to me by my uncle for Christmas this year.

I have more than a few other books like that. I’ve got a hardcover American edition of the first Harry Potter, signed by Jo Rowling, made out to my daughter with a personalized message. A bunch of old books, a lot of autographed SF novels written by people I know, some of whom are pretty well known: Brin, Sterling, Doctorow.

I have a lot of the same books as epubs on my iPad. And it’s qualitatively different. The e-books are commodities, and if one get deleted, I won’t have any regrets. Whereas if my complete run of first printings of the Doonesbury compilations (even including the obscure one for the TV special!) were to get lost or damaged, I’d be quite upset.

There is a fetishistic quality to the physical object, a quality that means that I will probably never have a house without books. In fact, now that we have a larger house with bookshelf space, we have carefully placed 4×4 beams towards the back of every shelf so we can double-stack the books and still see every spine (a trick I recommend! Buys you two additional shelves of books per bookcase).

Oh, there’s a lot of books that probably I don’t need and will never read again. But I carefully gather up and keep in order every Ian Rankin mystery novel, every volume of Transmetropolitan, each of the slender paperbacks of Mafalda in the original Spanish. And I make sure they can be seen.

Signaling theory is a small branch of cognitive science which argues that quite a lot of the things we do are intended as signals to third parties — especially prospective mates — about our status and interests. I think of what we do at our house with books as being basically that: a sign of who we are. We’re just not the types of people who will ever live without physical books, because electronic copies aren’t visible, and aren’t valuable.

There is an implicit valuation of cultural objects based on physicality — indeed, implicit calculation of actual monetary value. Rarity matters — whether it’s the fact that one of my two copies of M.U.L.E. is still in its original shrink-wrap (the Commodore 64 one, not the Atari 8-bit one — I played the heck out of that one!) or the fact that my print of the goblin market from Stardust is signed by Charles Vess.

In fact, I am having great trouble keeping myself back from boasting about even more of my geek possessions as I write this. I am engaging in signaling to you about what matters to me.

It’s gotten a lot harder to signal music. It’s everywhere. The rarity and the scarcity is gone. Some music geeks are working hard to bring it back, by putting out vinyl limited editions, precisely because the physicality of the vinyl format, with its generous cover sizes, the patina of survival, the whiff of authenticity from when music seemed (I do say “seemed”) less commercial, all seem like the qualities that bring back that fetishistic element. People don’t collect the MP3s of Sgt. Pepper’s Lonely Hearts Club Band for the intact cut-outs in the album sleeve (that my vinyl copy from the 60s has, neener neener).

All culture, though, is becoming commodified. And cheapened — I mean that literally, cheapened on the open market. The benefits are enormous:

• Even as pro-level content has gotten dramatically more expensive to produce, we have seen the quality of amateur content explode. Now there are Beatles in living rooms everywhere. Far far more people have turned out to have talent than I think was ever evident in all of history.
• Creative output can be shared fairly trivially, enabling people who have never before had access to libraries’ worth of information to learn about darn near anything. Just tonight by daughter was complaining about her biology teacher, and I said that in this world, there’s no excuse… just go find another bio teacher at bioteachershangouthere.com. Which may not exist, but should. The point being that where information was scarce, it is now more like air. Polluted, but everywhere.

We live in an age where people are sagely telling us to move into tiny houses and get rid of the accoutrements of consumerism. An implicit message is that we should not be valuing many of the things we value nearly as much as we do. After all, is the value of The Three Musketeers in the binding or the text?

Bizarrely, I think for me, at least right now, it’s in the binding. Content is ever more ubiquitous. Containers are what has grown rare and precious.

I think of this now with our increasingly digital medium of games. I have a 60s edition of Risk, alongside a modern one. I have a handmade Nine Men’s Morris, a Lord of the Rings chess set. I have a MAME cabinet, and I can play Centipede with a trackball, dammit. And I still keep the cases of Playstation 1 games that I have not bothered to boot up in many years. But in not too distant a future, I’ll probably be paying quite a lot extra to have a physical object to fetishize and show off my devotion to my craft and hobby.

Someday, when everyone gets to carry around the complete works of humanity on a cufflink, it’s going to be interesting to see how we tell each other who we are.

WebPageTest waterfall of page loading here

I just upgraded my hosting plan, and we’ll see if that solves issues with the site performance. In addition to everything I mentioned in my last post, I also took some additional steps to improve performance, captured here for the sake of anyone else who has issues.

• I fired off an email to the maker of Category Icons, which seems to be one of the common slow queries. For a while today I had it disabled outright, but I seem to have more headroom on this new hosting plan, so the icons are back.
• I found that wp-cron.php, which is what the blog uses for scheduling posts and other cleanup tasks, was generating a lot of hits — as in, a quarter million this month alone, far outweighing most of the blog traffic! This appears to be a common problem. I followed the advice found here on how to disable WordPress’ built-in cron stuff and instead use a regular cron job. We’ll see how that does and whether everything works more or less like before.

• I found that the Exploit Scanner plugin, which I have mentioned on the blog here before, was a huge culprit in adding cruft to the wp_options table. I suspect that a few scans simply failed to complete — heck, I cancelled one the other day because it was taking too long — and left the entire state of the scan in the table. There were only two of these, but removing them reduced the table from 8.4MB to 99.2k. Yes, you’re reading that right. With that, my slow queries checking the wp_options table went away.
• While I was at it, I also went through the table and cleaned out obsolete options. Clean Options was helpful in this regard.
• Running tests with Webpagetest.org, I found that there were still some pesky files that were not getting compressed. So I added this to my .htaccess file to try to catch everything:

# BEGIN deflating
AddOutputFilterByType DEFLATE text/html text/plain text/javascript
# END deflating

After doing all that, the site was still dog slow, and Nick, the support guy at BlueHost who helped me out, spent 45 minutes with me trying to figure out what else it could be before we arrived at the conclusion that I had simply outgrown their basic hosting plan. In the last couple of weeks I have been averaging 4000-8000 hits an hour, thanks to the controversies around free-to-play and immersion and thanks to Hackner News, BoingBoing, and Massively… So I upgraded, and now getting the site — even non-cached, the way I see it as an admin — is ten times faster than just getting the first byte was earlier today. Almost all readers will be getting cached pages, so it should be even faster for them.

This post itself should serve as a decent test of how well all this worked!

Sometime soon I will also be taking advantage of the dedicated IP that comes with this hosting, which might help a tiny bit more… but it will result in the site being hard to reach for a while as DNS records update, so I’m going to let it ride for now.