Clark Aldrich On Simulations and Serious Games

Join me for a free webinar about HIVEs on Tuesday, July 14

clark.aldrich@gmail.com (Clark Aldrich) — Tue, 07 Jul 2009 12:47:00 +0000

If you want a preview of some of the material that will be covered in my upcoming two books, and you are just beginning the journey (i.e. this is a beginner level program), join me for a Training Magazine webinar, called The Unifying View of Highly Interactive Virtual Environment (HIVE) Learning.

All the information about the webinar and registration link are here:

http://www.trainingmagnetwork.com/topics/show/326

(If you’re not already a member of TrainingMagNetwork.com, the free registration only takes about 30 seconds.)

Click here to join the discussion group:

http://www.trainingmagnetwork.com/groups/show/261

Here's the description:

Many practitioners have been struck by a paradox. They have sensed an overlap between virtual worlds, games, and educational simulations on one hand, and yet they know that one does not equal the other.

This emerging, unifying view of HIVE learning is the future of education. It represents, finally, the practical convergence of best practices and technologies, leveraging and building upon what we already know for better results for all involved. The critical trick, however, is knowing when to look at virtual worlds, games, and educational simulations as part of a greater whole, and when not to let this holistic view obscure the critical differences between them.

You will learn:

What is the relationship between virtual world, games, and simulations, and how and when each should be used.
How to coach students in "learning to do" rather than just "learning to know."
How to measure HIVE effectiveness.

Obviously, feel free to preorder either book in the meantime. Both the publisher and Amazon track pre-orders carefully in determining how much to promote the books, so ordering early helps spread the word about the importance of Simulations and Serious Games even more than later orders, as well as earns my eternal gratitude!

Great free branching story authoring tool called Twine now available

clark.aldrich@gmail.com (Clark Aldrich) — Thu, 02 Jul 2009 22:08:00 +0000

The point of this blog is to get more people creating, deploying, and using sims. One of the most intuitive place to start with non-linear content is a branching story format.

So it is really great that Chris Klimas has created and released for free and in open-source a marvelous, visual, and easy to use branching story authoring environment called Twine. Get it at:

http://gimcrackd.com/etc/src/

He has also put up a lot of help, in the form of both videos and text, to get started. This is a helpful modeling tool for developers such as myself to test out content, even content that will eventually be more dynamic than pure branching. More importantly, I can imagine instructors at all levels, from middle school to college, asking students to use Twine to create branching stories for homework or deeper reports.

Authoring tools are the most obvious bottleneck for simulation development. Chris Klimas has made a powerful and intuitive tool that allows anyone who wants to create branching stories to be able to do it now.

The first thing to do is fire the old training guard

clark.aldrich@gmail.com (Clark Aldrich) — Wed, 24 Jun 2009 15:31:00 +0000

I have now been involved in many situations that have played out almost identically.

There is an innovative, typically "older gen-x" sponsor who either develops a custom simulation or brings in some off-the-shelf simulation. When the simulation is finished, the sponsor sends it out to some training power brokers for their input.

To be clear, at this point, the simulation either represents months of work, or is an award winning program with documented case studies of success.

In more cases than I care to count, the response from these old guards is "no, it will never work. Scrap the project." This is, of course, disheartening. So I often dig deeper. 10 times out of 10, when probed, I find out the same two things from the old guard.

First, they engaged in less than 10% of the whole program. So for a two hour program, they may have looked at 10 minutes.

Second, their "reasons" for dismissal were staggeringly capricious. "The font size is too small" or "the program both reads the text and shows the text at the same time which is counter to instructional methodology" or "I had to click the 'next' button too often."

To me, such counter-innovation smugness should be grounds for immediate dismissal. No graceful exit. Do not pass Go. Do not collect $200.

It is one thing to say, "Sorry I don't have the time to review your program," or "I am so deep into my own experiment that my view is skewed" or "What happened in the focus groups" or even "I wish I could comment, but I am so far removed from the target audience as to not be valuable." It is another altogether to craftily do the least amount of work possible to "validate" a hastily-conceived intellectual position.

These people are clearly not serving the organization that pays them each month. What I don't understand is why they take the position that they do.

Here are some theories:

Are they more scared that the new program will be a public failure or a public success? Failure might mean that all formal learning budgets are cut, while success might switch the balance away from their budget to someone else's.
Do they truly believe in the validity of their own pettiness? Have they become so superstitions and intellectually brittle that they believe in a set of little rules as the key to success in formal learning programs? Is their circle of colleagues so narrow that they have feedbacked-looped their own theories into fact?
Have they just learned to treat everyone as badly as they feel they have been treated? Are they mean because they can be mean?
Are they, just by being in the profession of lecturing, far too comfortable giving out poorly thought out and researched positions as absolute fact?
Do their minds get overwhelmed when they see media on a computer screen that is interactive and "game-like," causing them to thrash about in panic looking for something familiar and safe to intellectually judge and hit?
Do they just have a rote process that they have developed for reviewing material that they are blindly applying?
Do they resent being put in the place of a lowly student? Do they hate to learn?
Are they simply insane?

Now, I admit that these comments haven't derailed any programs that I have tracked. But they sure take the wind out of a lot of great sails. (What is then amusing is seeing the smiling portraits of these old guard members in conference brochures next to blurbs about how innovative they are. I have heard some brag in public about programs they have tried to suck the support from in private. And honestly, either they are fabulous liars or they are truly unaware of their own hypocrisy.)

I get periodic bulk emails, letting me know that people who have been nothing but obstacles to the entire formal learning industry have retired, and each one fills me with rejoice. (Sometimes I even get personal emails from these old guard members asking for recommendations when they get appropriately sacked, which fills me with something else.)

That is why I often say at the beginning of conference presentations that everyone who has been in the industry for more than three years should leave, because they won't act upon anything I say. I pretend to joke but I am serious.

Sadly, if World War II created "the greatest generation," the baby boomers now in power in the formal learning industries have got to be the worst generation. Many have hoarded power by pandering to critics and cynics to ultimately lower the vision of the entire industry to using e-learning slide shows to teach people how to do simple processes and other test prep.

Thankfully, the wheels of time are relentlessly pushing the old guard out of the way. I just hope there is a profession left when they are done with it.

Four Intellectual Traps for Understanding Learning

clark.aldrich@gmail.com (Clark Aldrich) — Sun, 21 Jun 2009 14:31:00 +0000

In trying to rebuild our capability to capture and develop knowledge and wisdom, we have to back away from some of our sacred constructs. Here are four of my own observations:

1. School is not a useful model for learning. But learning to ride a bike or a foreign language is. Schools are only good for teaching people how to be students, and maybe teachers.

2. Books, magazines, and movies are not a sufficiently useful model for capturing wisdom. Would you learn leadership or innovation that way?

3. Professional (or other highly structured) sports are not a useful ideal for play. But pick-up games are. Professional sports are a better model for work.

4. Computer games are not a useful ideal for play, any more than white bread and candy are good models for food.

I will be delving into these in more detail in the weeks to come.

Using an interface to motivate players to wait

clark.aldrich@gmail.com (Clark Aldrich) — Sun, 07 Jun 2009 16:54:00 +0000

One of the trickiest interface challenges in sims is to motivate a player to wait. How can a simulation convince the player to hold off on doing something they want to do immediately? This is critical, as so many Big Skills (such as security and relationship management) require, to apply them successfully in real life, plenty of appropriate holding back.

Recharging Shield. In Halo: Combat Evolved, the player may best want to take cover and wait for the shield to recharge before jumping into full battle.

Here are some ideas, including both that can be learned through experience and visualized in the interface:

Saving up: This is the most straightforward. Players have to earn a certain threshold of some resource before they can move ahead. For example, they may need to earn $35 before they could buy a bus ticket. There can also be variations, where the longer one waits and earns, the better solution can be bought. For example, one can earn $35 for a bus ticket but if one waits and earns $200 and one can buy a plane ticket.
Charging up: In a charging up situation, the longer one waits (over the course of a limited time), the geometrically more powerful the effect of an action will be. So with a certain type of weapon, charging it for two seconds and will result in a blast of power 10, but charging it for four seconds will result in a blast of power 100. Likewise, at a staff meeting, not talking for a while may earn the group's attention when you do have something important to say.
Limited inventory: In some cases, the player only has a non-renewable amount of some resource, and spending it badly leads to missed opportunities.
When things are going well: Here, everything is going well, and the player has to realize that getting involved may hurt, not help.
Earn interest: Here, any unused resource grows. The higher the interest rate, the more motivation for a player to wait. In other words, doing the same thing one turn later results in having more.
Window of opportunity: Here, there is simply a time when an action is effective, and a time when the action is either ineffective or counterproductive. A level-ending boss may only be injure-able after firing his or her own weapon. This can be absolute or more subtle. For example, Company A may best buy Company B when Company A's stock price is at its highest and Company B's stock price is at its lowest.
Probing/ identifying: In some cases, there may be a question as to even do an action or not. A person coming out of the fog may be an enemy or a colleague. Waiting can result in more information to make that decision.
Dry powder: In some cases, there may be better options later, or an environment where surprises come up that need resources to either make things much better or keep them from being much worse.

We are all so used to twitch games. But some of the most interesting sims have us as tense holding back as striking out.

Learn more about HIVEs in the June/July special issue of Innovate Magazine on Online Simulations, Role Playing, and Virtual Worlds

clark.aldrich@gmail.com (Clark Aldrich) — Mon, 01 Jun 2009 13:00:00 +0000

I have been presenting my model of Highly Interactive Virtual Environments around the world in the last few months. I have been grateful with the amount of excitement it has garnered (and frankly relieved, as successfully implementing HIVEs is the centerpiece of Learning Online with Games, Simulations, and Virtual Worlds, the more traditional of my two upcoming books).

Given that, if you are interested in this subject, I am happy to announce that the June/July special issue of Innovate, on Online Simulations, Role Playing, and Virtual Worlds is out today. Moreover, I have been able to write a "keynote" piece for the issue, called Virtual Worlds, Simulations, and Games for Education: A Unifying View (keynote not in the sense of the most important piece, but as a level setter). My piece introduces the HIVE model, and importantly to many, in a language and tone that is more formal and familiar to many academics than my usual, ah, overly-jaunty prose.

I hope you like the whole issue, and find my piece a worthy addition.

A Proposed Simulation Assessment Methodology

clark.aldrich@gmail.com (Clark Aldrich) — Sat, 23 May 2009 13:12:00 +0000

I have been wrestling with implementing an online educational simulation assessment methodology to prove that learning has happened during the course of a simulation to third parties. Rather than being theoretically perfect, however, to meet the needs of my clients, I needed something that works well in the field. I think I have come up with a methodology, let's call it "The Bristol Method", that might work quite well to assess a one to two hour long course.

The core assessment methodology is a simple model: A student will get sets of screens, with the task of quickly connecting boxed items from the left with the correct corresponding items on the right by drawing a line with a mouse.

The directions to the students will be as follows:

You will get five sets of screens; in each you will connect an item from the left column with an item on the right column. You will do this by using the mouse to click on the item on the left, which will highlight, and then click on the most corresponding item on the right. A line will be drawn showing that they are connected. If you make a mistake, reclick on an item in the left column, and do the process again. There is a 90 second timer for each screen. If you are done before the time allowed, you can press the “done” button. If you finish early (before the timer runs out), you will get some bonus points.
After each screen, you will see which are the right answers and a score. Your score is based on both the number correct, as well as any bonus points for hitting “done” before the timer goes off. The first of the five screens will be a practice round, which is not scored.

To launch the assessment in the first place, the student have to demonstrate a basic understanding of the connecting action, as such:

Then, again per directions, the first screen is a practice screen. For example:

This will be followed by a review screen, showing which answers are right or wrong (this is important feedback to reward students), the time left if any, and a score comprised of the two. Specifically, students get 60 points as a base, and then seven points for every right answer, plus one point for ever ten seconds left over on the clock, up to a maximum of 100 points (these points may be adjusted).

Students then have the ability to go back and replay the test screen if they want. When they are ready, the students can go through the four "real" assessment screens and sets. Each of the four real assessment screens would show five scrambled questions and answers, drawn randomly from a pool of about fifteen possible pairs. The tone of the assessment should be like a computer arcade game, with “fun” animations. At the end of the assessment they also get a final score with an average of all four sets.

The Questions

Obviously, the Bristol Method requires good questions. They can vary, although be organized by screens. So screens could include:

Define the terms;
Given brief overviews of situations, identify the best strategy;
Given quotes, identify the meaning.

The test questions must line up with the learning objectives of the course.

Process

This same test (although with randomized questions each time) could be applied before and after a student went through a simulation, and can also be applied before and after a student went through a non-simulation class, as well as no class at all for baselines.

Result

If the simulation was effective, the result should show that students answered more question correctly, and in less time, and in a way that is directly comparable (and favorable) to a shift in other methods or no methods. An aggregated graph may look like:

Rational

I believe this Bristol Method would solve a handful of traditional problems.

First, it would move quickly, and be enjoyable (or less miserable than a traditional test). It could have some great bells and whistles. This is critical because any student taking a pre-test in a subject in which they don't know much can necessarily be miserable, and a harsh way to start any educational program. (Starting a two hour long simulation course having to answer questions to which a student doesn't know the answer can create defensiveness that puts a damper on the entire experience.) By matching (which rewards some knowledge by reducing the number of possibilities with each correct answer), by having a short timer (win, loose, or draw, the experience is over quickly), and by learning what they got right and wrong in the review, the experience can be as painless as possible. Further, the lower the student resistance to an assessment, the easier it is to create base cases.

Second, by measuring speed, a test can ask questions that in an untimed methodology, the student could "figure out" given more time (which are often the types of questions one wants to ask anyway.). It also minimizes the cheating options of an open-book tests. And given that an advantage of a simulation is instinctive, intrinsic knowledge, looking at speed is relevant.

Third, the content is fair and objective, and can show improvements (or lack thereof) in a way that is convincing to the outside world. And the long term time frame and low completion rates of 360 assessments, while more fair, are avoided.

Conclusion

There are a range of theoretical evaluation strategies, none of which are perfect. But I hope the Bristol Method provides authentic, reliable results, that fairly and efficiently evaluates simulation deployments. I will be testing it in months to come and let you know how well the results work.

The top five reasons why computer game designers should care about serious games

clark.aldrich@gmail.com (Clark Aldrich) — Thu, 21 May 2009 12:49:00 +0000

For those people getting excited about Harrisburg University of Science and Technology's Learning and Entertainment Evolution Forum (LEEF), here are the top five reasons why game designers and publishers should care about serious games:

Many of the most successful computer games ever, such as Roller Coaster Tycoon and SimCity, have been serious games. (See Big Skills as a game design challenge, and even Middle Skills.)
Serious games are making significant progress around artificial personalities, including dialogue, body language, and belief systems, that traditional computer games need. (See Creating Artificial Personalities, not necessarily Artificial Intelligence.)
Much as current movies are borrowing heavily from documentaries (shaky-cam, anyone?), so can computer games borrow from serious games (including virtual products) new interfaces and game-play models to add realism to experiences.
Serious games are developing new genres of interfaces, goals, and gameplay that can be evolved into either completely new computer games or additions to existing genres (See Genres). Because serious games designers are not trapped by the conservative design required of huge budget productions, they can explore faster.
Adopted and supported games used in classrooms is maturing into a long-term, stable source of revenue that circumvent the three-month hit-or-failure current computer game model. (See Top Ten Serious Games and Educational Simulations used in College Classrooms.)

Alan Kay and human universals vs. non-universals

clark.aldrich@gmail.com (Clark Aldrich) — Wed, 20 May 2009 19:43:00 +0000

When I was talking to Alan Kay about educational simulations a few weeks ago, he shared a model that I found helpful. Kay spoke about human universals vs. non-universals.

The universals are a list of characteristics of virtually all cultures, and certainly all children, share. These universals include:

Social
Language
Culture
Fantasies
Stories
Tools, Art, Technologies
Goals, Plans ...
Play & Games
Fixed Rules, Flexible Strategies
Case based learning
Case based reasoning
Superstition
Religion/Magic
Theater
Simple, Short term fixes
Quick Reactions To Patterns
"The Other"
Supernormal Responses
Vendetta

He compared these to non-universals, which I would describe as non-intuitive perspectives but, once hard-earned, are seen as self-evident. There are examples of systems (the often invisible stuff connecting actions and results) written about here. These represent a "cultural technology," and include:

Writing & Reading
Deductive Abstract Math
Model Based Science
Thought, Thought, Thought
Equal Rights
Democracy
Similarities over Differences
Slow Deep Thinking
Legal System over Vendetta
Perspective Drawing
Theory of Harmony
Agriculture

His point was that movies and advertisements and other pop-culture tend to invoke (and pander to) the universals. Those are the easy things, the hardwired things. I am guessing all upcoming summer movies will borrow much from the first list. But in Alan Kay's perspective, education must develop conviction in the non-universals.

I think all of us designers have dipped into (sometimes heavily) the list of universals, and even included some as acceptable learning outcomes. And fairly or not, a lot of people associate games (even serious games) with the reinforcing this list of universals.

The trick may be to "pace than lead," to use the universals as a pathway to the non-universals. Students praise our design in the short term for the universals we reinforce. But they praise our content in the long term for the non-universals.

A lot of "revolutionary" thinkers about twenty years ago, like John Seely Brown, asked more of us to summon our hidden child, to challenge assumptions and unlearn our baggage. Given how many of the people in the workplace have defaulted to "winging it" (invariably with huge amount of fake/unearned confidence and even underlying threats) I may now implore more of us to nurture our hidden adult.

Podcasts for the Learning and Entertainment Evolution Forum (LEEF) on June 18-19

clark.aldrich@gmail.com (Clark Aldrich) — Tue, 19 May 2009 18:08:00 +0000

For those of you who like to hear an author's voice rather than read his or her words, here are some short podcasts, at http://www.harrisburgu.net/leef2009/podcast/podcast.xml.

Or:

Clark_1.mp3

Clark_2.mp3

Clark_3.mp3

These lead up to Harrisburg University of Science and Technology's Learning and Entertainment Evolution Forum (LEEF) on June 18-19, where I will be a keynoter. For more information, check out: http://www.leef2009.net/.

I hope to see you there!

What is the difference between a game and a simulation?

clark.aldrich@gmail.com (Clark Aldrich) — Sun, 17 May 2009 16:53:00 +0000

I am often asked "what is the difference between a game and a simulation?" I introduced a HIVE framework to suggest that Virtual Worlds, Games, and Simulations were nested concepts, better understood as discrete parts of a continuum than either as synonymous or totally separate.

The difference between game and sim is in both the media itself and the attitude and goal of the player engaging it.

Allow me to go a bit deeper. One useful analogy for a virtual world is the synthetic world of a swimming pool. So I created this chart of how the various activities one can do in a pool line up with both games and simulations.

As always, I welcome your feedback.

Clip of Japanese Version of Virtual Leader

clark.aldrich@gmail.com (Clark Aldrich) — Thu, 14 May 2009 12:52:00 +0000

Virtual Leader and (VLeader 2009) has been translated and deployed all over the world, proving that there is a unvirsality to many of the Big Skills written about here. Take a look at a clip of one version, created by SimuLearn's wonderful Japan partner, I-Think:

The Competition between 21st Century Skills vs. Retraining vs. Science and Math in Obama Education Priorities

clark.aldrich@gmail.com (Clark Aldrich) — Mon, 11 May 2009 00:12:00 +0000

I have been spending a lot of time in DC over the last month, participating in planning sessions at increasingly high levels. And I am being drawn into the interesting competition in the Obama administration between three different education objectives.

The first is to develop what is being popularly called 21st Century skills (that I have called Big Skills). These are skills that have not showed up on traditional curricula, and are around topics like leadership, project management, innovation, and stewardship. The excitement in this area is that it could challenge the traditional K-12 curricula in areas that would both help students immediately in their day-to-day life, give them more power and control of their entire lives, and also align schools with business, whose absence of such critical skills have largely resulted in the current economic crisis and US decline in global competitiveness. The problem is that plenty people believe that all of the skills are unteachable. Thus, in swinging for a home run, Obama could spend precious time and resources and whiff completely. And of course plenty of academics believe leadership (and other "learning to do" skills) is vocational.

A second contingency is focused on how to retrain American workers for what is thought to be new jobs in the new economy. They wonder, for example, what will it take to train fired car manufacturer employees into people who can install and maintain new wind turbines.

A third contingency believes that education should be increasingly focused in the traditional but underfunded and underdeveloped areas of pure science and engineering. These advocates cite recent declines in patents relative to other countries and innovation based manufacturing as proof that we need to double down, or even triple down, our efforts in these areas. The critics however might suggest that the current emphasis on science and engineering is too limited, too exclusive, and just not the right fit for too many people.

The good news for us simulation designers is that we will play a critical role in any of these three areas. We may uniquely be able to create media to support the 21st-century skill goals. We could drastically cut the costs and increase the efficiency (including scale) of a retraining focus. And we could lead the revolution in re-thinking and re-engaging a new generation in science and engineering. The only bad scenario however may be the most probable - when all is said and done, nothing new really happens.

The Need for Sleep to Process Information in a Simulation-Centric Class

clark.aldrich@gmail.com (Clark Aldrich) — Sat, 02 May 2009 13:23:00 +0000

Should we add "bed," alongside whiteboard and lab, in our list of great educational tools?

I have found with any experiential and complex-systems based learning program (such as around a Big Skill like project management), it is paramount to have the participants first get an exposure to the task, and then "sleep on it" before continuing. When students were not able to sleep on it, they were anxious and dissatisfied and learned less. In contrast, when the students did break up learning with sleep, their subconscious processed and assimilated the information, and they returned to the program the next morning without the trepidation they had shown the night before and in the control groups.

Said simply, the same program that took the same number of hours, if broken up with a good night's sleep, resulted in significantly better student enjoyment and, more importantly, organization and retention of the material.

Possible problems of ignoring the role of sleep

The existance of this simple rule can hurt simulation deployments in at least three different ways. First, this can confound a training group's insistence on a "one-day" or "half-day" program, especially where students are unreliable in doing any prework (universities, thankfully, don't have this problem). This also can hurt some attempts to measure the effectiveness of simulations, as researchers often try to control all variables and shoe-horn in an entire simulation experience in a single (often long) session. Finally, this can hurt the widespread adoption of a simulation if an evaluator tries to skim a simulation in a half-hour, and then "doesn't get it" so doesn't support it.

Chunking well

As with a fine wine, authentic learning has to breathe a bit. A simple chunking process, where students experience at least 30 minutes to an hour of the interface in its entirety and at least some of the mechanics, even ideally getting a little stuck (which can be done as homework if the students are responsible and the deployers of the class have credibility), sleep on it, and then dive in to harder levels can be the difference between success and failure, between meaningful experience and frustration and confusion.

The Change In The Seven C's of Formal Learning: (Content * Curricula * Coaching * Certification * Community * day Care) / Cost

clark.aldrich@gmail.com (Clark Aldrich) — Wed, 15 Apr 2009 10:47:00 +0000

One could argue that at the end of this Age of Linear Content (roughly spanning from Gutenberg to Google), the value of traditional content has plummeted. Almost anything, in theory, I could learn at Brown University (or more specifically, on which I could be tested at Brown), I could "pick up" on my own, probably on the web or maybe a book.

On almost any subject, the collection of videos, podcasts, and blogs provide access to a wealth of content richer than the content in any single formal learning experience. And that trend will only continue. So as the value of linear content declines, where does that leave schools and other formal learning programs?

Of course, content is only one part of the value proposition of formal learning programs. The full equation looks something like: (Content * Curricula * Coaching * Certification * Community * day Care) / Cost, where:

Content: The material supporting any learning objective.
Curricula: How the content is chosen, validated, organized, and presented.
Coaching: The individual attention helping each student overcome their individual weaknesses, answer specific questions, and leverage their individual strengths, as well as provide motivation.
Certification: Proof and documentation that a level of competency has been reached (which also provides motivation).
Community: A group of peers that both make learning more effective and engaging.
day Care: The ability to house students for a specific time.
Cost: The amount of resources, including student time, a program requires.

Today, in the short run, schools will have to either lower their own costs dramatically or increase the value of the other components to maintain the same value proposition. But that is only a short term step, as more services such as social networking sites eat away at other C's.

In the long run, schools will have to re-invent content. Schools will have to stop their addiction to linear "learning to know" content, and think more of "learning to do." This dynamic content is not only more powerful and relevant, but it also requires and benefits from the other C's more than linear.

Grappling with an Assessment Framework for HIVEs

clark.aldrich@gmail.com (Clark Aldrich) — Tue, 14 Apr 2009 13:47:00 +0000

This blog puts forth the premise that content designed to change behavior focuses on three connected pieces: actions, results, and the often invisible system that connects them. I have probably spent more time that I should on the entries and thoughts around Actions and Systems. The notion in a sim of Results is much tougher, and I have correspondingly underwritten about them.

Results in computer games are often simplistic, if not simple. Beat the boss or opponent. Find the key and get to the door. Build a big enough army. Finish stacking the blocks before the timer runs out.
Some victory may be table-based. Achieve a park of size X with an income higher than Y and an average customer satisfaction of Z by time A.
Other results may be based on a balanced scorecard methodology. Here, there may be three or four metrics that may compete for resources in the short term but all necessary in the long term.
Students often want a single score. What is their "grade"? If they play a sim twice, how do they calibrate their relative performances between the two plays?

As is often helpful, I like to like to look at real-life as a guide. Here are some questions. How would you assess the following:

A potential spouse?
A job opportunity?
A walk in the woods?
Dinner?
A new boss?
A new customer?
A diet?
An advertising campaign?
This morning's commute to work?
A child's third grade experience?
A child's college experience?
A new chair?
An old chair?
A car?
An investment?

What makes for success with each of these? When is success measured? What is the implication of relative success?

The more one thinks about it, the more confusing it can get. Does this lack of common constructs around what makes for success necessarily stunt our ability to create any formal learning program?

Measuring HIVEs

Let me up the ante just a little bit. According to my own research and others, it is become clear that we need a (better) methodology for assessing the effectiveness of Highly Interactive Virtual Environment (HIVE) Learning. For example, here are some questions:

1. Should the assessment of interactive content be interchangeable with the assessment of traditional content?

2. How do we acceptably measure Learning to Do and Learning to Be?

3. Whose data do we trust? Corporations? Schools? Vendors?

Learning Strands

I believe the answer, at least for industry metrics, will follow the framework of learning strands. Specifically, one has to prepare a program (and then assess it) much as one prepares a meal - deliberately mixing a variety of ingredients and conditions. Further, meals fit in a context. They meet goals that fall into strands: a need for calcium; a need for convenience, even portability; a need for protein; a need for flavor; a need for recognition of the chef. These strands have to be individually calibrated for the specific context.

The future of metrics and evaluations of programs may rely on their ability to meet a collection of learning strands, rather than a big, all or nothing score.

My interview with Harvard Business School Publishing's Denis Saulnier

clark.aldrich@gmail.com (Clark Aldrich) — Sun, 12 Apr 2009 22:07:00 +0000

I was able to do a somewhat in-depth interview with Harvard Business School Publishing's Denis Saulnier.

Here is my first line: "It has become clear to me that we are at the end of the era of linear content, which let's define here as beginning with the Gutenberg Press and ending with its manifest destiny of Google and Wikipedia."

Read it here.

Creating Artificial Personalities, not necessarily Artificial Intelligence

clark.aldrich@gmail.com (Clark Aldrich) — Fri, 27 Mar 2009 13:53:00 +0000

As I have often written about here, there is a difference between a simulation and an educational simulation. A simulation is a model of something, often predictive and diagnostic. An educational simulation is an experience that includes some simulations, balanced with other things, all for the sake of creating transferable behaviors or perspectives. Thus a simulation of a cockpit may have every knob, while an educational simulation during an early level may just have the most critical ones.

Likewise, in educational simulations, there is an increasingly need to differentiate between creating and using an artificial intelligence and an artificial personality.

Artificial Intelligence

It seems that AI's generally fall into four overlapping categories:

Expert system: A system used to solve a task that provides judgement or other help that takes the place of or augments an experienced professional. A doctor may use an expert system to suggest reasons what might be wrong with a patient.
Model of human brain: A computational model that strives to both use the same techniques as a human brain and provide similar output. For example, using an artificial stroke on a neural network may not destroy any data, but simply make it take longer for the program to access it.
Adaptive problem solving: An algorithm that has a goal, sensors, and parameters, but otherwise may grow and evolve in response to an unpredictable environment.
Skynet.

However, few educational simulations or serious games will delve into any of these AI areas. They are computationally difficult and intensive, and as with the cockpit example, not always the best to meet learning objectives.

Having said that, we still need to feature avatars in any program with a learning objective of helping people "deal" with other people. And these avatars cannot simply be driven by broad branching algorithms. They have to be dynamic and responsive.

Artificial Personalities

Literally, as crass as this sounds, we have to have avatars that can serve as backboards to enable players to repeatedly practice actions and calibrate their actions against the response. These avatars need to be consistent, and often have visualizeable inner workings. Let me give two broad examples from two projects that I have done.

In Virtual Leader, the model of leadership frames that a leader has to

surface relevant ideas,
ensure a power base,
moderate the tension and working environment,
before driving hard to complete the right work.

The artificial personalities that are represented by the avatars in Virtual Leader, then, all represent distortions of that model. Some characters just wanted to complete any work and were intolerant of side conversation. Others wanted to moderate or lower tension no matter what. They would switch off tough ideas and praise people a lot. Still others wanted to amass power, sucking up to power figures, building alliances using common assets for personal gain, and taking ownership no matter what. (Now admittedly, this model was fairly computationally heavy. )

In a new simulation, I am modeling the personality of single individuals through the three distinct attributes of (a) what the person is doing, (b) what the person is intellectually thinking, and (c) what the person is emotionally feeling, all on a single conceptual map. As a result, the avatar may be asked to do something, and then do it once, but then abruptly stop doing it (showing compliance not commitment), because his/her hands are not aligned with either head or heart (i.e. it is one thing to wear your seat belt once, it is another thing to see yourself as someone who wears seat-belts, and to understand the risks of not wearing one). Or as the avatar gets stressed, the influence on his/her behavior begins to be influenced more by where his/her heart is than where his/her mind is. This may seem complicated conceptually, but the algorithms and modelling are no tougher than making a sim of a bunch of cars driving around a parking lot. Further, the artificial personality is totally transparent and can be visualized to a student of the sim.

Key Attributes of Artifical Personalitites

Given this, the three key attributes of an artifical personality are:

Computationally light-weight (runs in Flash), and more often a form of fuzzy logic;
Provide behavior models that are instructional to engage, aligned with the learning goals;
Are dynamic enough to both respond to a student in an open-ended way and also allow for a programmer/level designer to create (at least) three or four distinctive varieties (a good physical analogy would be how shifting just five variables can change the behavior of a vehicle from dynamically handling like a sports car to handling like a garbage truck).

Leading, not Following, Academics and Computer Games

This work into creating satisfying Artificial Personalities typify the challenge of our industry. On one hand, we don't get much help from either academic research (my degree in Cognitive Science from Brown University is not overly useful here) or computer games. And we will never create characters of indisputable accuracy and predictiveness to real world situations (a primary success criterion for academics). But on the other hand, we can create experiences that radically change and improve the behavior of students (a primary success criterion for educational simulations), are really engaging, will work their way over to games in some form, and we can still create small algorithms that turn out to be more powerful than dissertations in illuminating behaviors of real people.

My fourth book, Learning Online with Games, Simulations, and Virtual Worlds, available for pre-order

clark.aldrich@gmail.com (Clark Aldrich) — Wed, 25 Mar 2009 15:27:00 +0000

I am pleased to announce that my fourth book, Learning Online with Games, Simulations, and Virtual Worlds: Strategies for Online Instruction , is now available for pre-order on Amazon. This book goes into selection and implementation detail around Highly Interactive Virtual Environments. Along the way, it prepares instructors for the new challenges of developing a true culture of interactivity and helping students "learn to do" not just "learn to know."

This book is aligned with, but not part of, my Simulation Trilogy, which will be complete this summer, and is made up of (and can be read in any order):

I realize there may be some confusion between them (especially because so far, no new book replaces an existing book before it), so feel free to ask any questions about what subjects each will or will not cover, and I (and hopefully others) will try to answer.

As anyone who does this type of research knows, kind words go a long way, so thank you to so many who have encouraged this continued effort.

Need Help in Collecting Examples and Methodologies of Evaluating Simulation Effectiveness

clark.aldrich@gmail.com (Clark Aldrich) — Fri, 13 Mar 2009 14:56:00 +0000

I have asked for, and received, examples of rigorous evaluations of sims from many people in the past. Over the next few months, I am now going to assemble all of those along with new reports. I would like to be able to better both frame and answer the questions, "What can we expect Sims to do?"

This is not a report for a specific client (although a few have kicked in - thanks!). Thus I can share everything, and will make my journey as transparent as I can. If you have any work that you feel has helped you down this path, I would be grateful to learn about it.

Best,

Clark

Virtual Worlds, Games, and Simulations: The Challenges of the Next Five Years

clark.aldrich@gmail.com (Clark Aldrich) — Sat, 07 Mar 2009 14:49:00 +0000

Across my own client base, there are increasing questions about how the next five or so years will play out. Here are some thoughts on the peeling away of the "post-literacy" onion.

2009: Understanding and Procuring forms of Highly Interactive Virtual Environments

Organizations will continue to explore their understanding of Highly Interactive Virtual Environments. They will seek to resolve the differences in structure, use, best practices, and metrics between virtual worlds, serious game, and educational simulations.

More organizations will purchase access to virtual worlds. In corporations, they will use them primarily for building communities and bridging distances, although about 80% will be greatly underused.

Meanwhile, organizations will commission their own full, self-contained sims from scratch for foundational skills sets (mostly using external vendors, which will continue to have 8 months to 1.5 years development time), and slowly buy (and often modify) off-the-shelf sims. We will see a proliferation of stand-alone mini-games (although often connected to online communities) as the dominant model

Both socially focused virtual worlds (as opposed to the still under-developed area of virtual worlds as platforms for sims) and self-contained sims, when done well, will work better than people realize, creating a rethinking of the multitude of flawed assessment methodologies such as Kirkpatrick. However, organizations will still pursue the Sisyphean task of "managing through metrics," trying to come up with an ROI for either an active community or the development of save-the-company skills.

Within universities using virtual worlds, the activity of students building basic interactive content will become a critical and increasingly dominant rationale for the continued use of the environments. Schools that do not focus on the students' role of building interactive content will wind down their use of virtual worlds for easier tools, such as enhanced virtual classrooms with basic 3D emotiveness. At the same time, the military will continue to push the leading edge of using sims to develop soft-power through the application of critical interpersonal skills.

Newspaper publishers and book publishers, as well as schools and internal corporate training functions, will find themselves in increasingly dire shape. And the sale of interactive applications, such as via iTunes and for Android, will continue to flourish. IBM will launch a new initiative into this space.

2011: Authoring in HIVE environments and presenting increased complexity

In about three years, we will see the widespread availability of robust and easy to use authoring tools and environments, mostly with the functionality described in the upcoming The Complete Guide to Simulations and Serious Games (and foreshadowed in http://www.pivote.info/). While small vendors will be launched to initially meet these authoring needs, these tools and capabilities will be increasingly be aggregated by the biggest software vendors. And the availability of these tools will enable large organizations to bring in-house sophisticated authoring capabilities (as students who grew up authoring in Second Life enter the workforce). The time it takes to build a useful sim will be reduced asymptotically to about four weeks, but larger budgets will be available for more sims that take multiple years to build (the short and the long of sims development reflect the maturity of the tools and the market value of the potential respectively).

New genres will continue to be created around "learning to do" content, embracing the simplifications or pieces of the Big Skills (aka Twenty-first century skills). Linear content will be viewed with increased suspicion as thin and ineffective,

Distinct parts of Highly Interactive Virtual Environments will merge and be seen as a continuous whole. Students and teachers will expect the smooth ramp up and ramp down between the real world, the open virtual world, the fun game, and the relevant simulation.

The communication gap between students and teachers will be a national epidemic. Institutions supporting schools will try, and fail, to build simulations around traditional content, such as biology and literature. Second Life will go out of business as the last corporate customers follow the younger generations to better looking and more dynamic, but also more splintered, environments. Ironically, as the virtual world market fragments, the plaftorm for sims will converge. Adobe Flash will run everywhere (including hacked future versions of Xbox's and Playstations) and be the common authoring environment of choice, enabling schools to assign sims without babysitting hardware. IBM will launch a new initiative into this space.

2013: Rethinking Knowledge and Subject Matter Experts

By 2013, we as a culture will finally be rethinking the possibilities and necessities of captured wisdom. Research organizations and consulting groups will reluctantly reject the easy lens of linear content, and due to competition and client requests follow a Research and Analysis Process for simulation based content, even when not building a simulation. Reports will talk about actions, systems, and results, not just processes and tips. Search engines will be significantly challenged, with huge investments and infrastructure trapping them in old content, as people realize, "You can't learn leadership from Google."

Increasingly, everyone from The MacArthur Foundation to Accenture will default to producing interactive content over passive. Reports produced will not be binders but experiences; not bullet points and inspirational quotes but equations, interfaces, and dynamic relationships. This new research will cycle back into increasingly detailed simulations themselves. Journalism will go away as a distinct college major and career.

Open source simulation design will flourish, and be compatible with the professionally created content. With the popularity of the $49 laptop, China and India will both announce that a majority of their school curricula across all ages will be simulation based. Computer game makers will enter the educational simulation and serious game space for real here, as they see there is a market for finished goods, but they will be too late to create a real brand. They will still manage to wipe out large tracts of smaller companies. IBM will launch a new initiative into this space.

2015: Applying the New View of Knowledge and Wisdom

Moving forward, school curriculum in the US will then be really retooled around teaching innovation and stewardship (and other Big Skills or "21st Century Skills"), as the seminal philosophy of Tim Ellis are rediscovered and embraced. The first Pulitzer prize to a simulation will be announced in 2015, as well as the last use of multiple-choice standardized tests (after years of decline). The last textbook publisher will fold. Pure linear content will be looked at the way we listen to scratchy phonographs. Finally, and truly, the most valuable content in the world will be educational simulations and serious games. IBM will launch a new initiative into this space.

Commentary: What if math is really sloppy and inaccurate?

clark.aldrich@gmail.com (Clark Aldrich) — Thu, 05 Mar 2009 15:31:00 +0000

The computer game Civilization IV quotes a compelling thought: "If in other sciences we should arrive at certainty without doubt and truth without error, it behooves us to place the foundations of knowledge in mathematics." -Roger Bacon (Opus Majus, bk.1, ch4.).

I was thinking about this when a foundation head asked me the other day, how would I use sims to teach math?

Play Cisco's [Binary Numbers] game.

To answer that question, one first has to ask oneself, what is math, anyway? (And by the way, I use math everyday, quite a bit, and it is essential to create most simulations.)

And maybe the second part of the question is, as the Bacon quote suggests, is math perfect? After all, 1 + 1 always equals 2. And 5! always equals 120. Isn't that perfection? It seems like it.

Except, what if the symbols and numbers of math is a form of pedagogy (including taxonomies, graphs, and abstractions)? What if math is best seen as a layer of content on top of, and to augment, real experiences? (If someone throws a ball, using math I can figure out where and when it is going to land. If someone is spending X dollars, I can tell when they will run out.) In that case, the question of the "perfection" of math rests not just on the self-referential math-to-math manipulations (where math becomes a bubble-world), but also the real life-to-math, or math-to-real life transitions.

Here's a simple example: if I drive 60 miles per hour for 3 hours, I will have travelled 180 miles. That is a perfect statement. But does that perfectly translate to real life? Probably not, because no one drives exactly 60 miles per hour, and, perhaps, few people drive for exactly three hours. The math is sloppy and inaccurate, but good enough to be pretty helpful.

Or a simpler example: If I combine two piles of hay, what do I get? One pile of hay!

So, beneath a faux, self-defined perfection, in fact, math is sloppy and inaccurate, if asked to on- and off-ramp to the real world. Likewise, in an academic setting, the learning about math requires the systematic stepping back from, even refudiation, of reality.

What's the Point?

Is there a point to the math observation? Maybe.

Only if math is better defined as a tool for improving our relationship with the real world, not just as the rules of an insular, perfect little pocket-world, then we can create simulations to make people great at using math, instead of creating simulations that helps people become great at knowing math.

Furthermore, multiple levels of self-referential systems ("the point of second grade is to prepare a student for third grade," or "the stock market will go up because it has gone up," or "if you beat the simulation, you know how to use the skill in the real world") are the signs of a fall.

But breaking the perfect pocket world requires a view of math that is at odds with current schools, text books, standardized testing, and in fact entire philosophy of being. The odds of changing all of that are not so good. Especially because, quoting Roger Bacon, math is perfect.

As Yogi Berra would have said, "It's the Big Skills redux all over again."

trigger: the question is, what changes everything?

clark.aldrich@gmail.com (Clark Aldrich) — Wed, 25 Feb 2009 22:15:00 +0000

A mechanism or condition in a System that, in reaction to a specific condition, brings about some discrete and significant change, or at least presentation of information.

There are big triggers and little triggers. While any commitment to a discrete Action is technically a trigger, this entry will focus on the big triggers.

From a design perspective, triggers are the flip-side of primary variables. While primary variables tick up or tick down, always ready to be calibrated, triggers are typically all-or-nothing.

For example, in a game world, your character, after losing health (a primary variable), might die (a trigger). In the real world:

After working hard to improve organizational productivity (a primary variable), you might get a promotion (a trigger).
After working hard to figure out a solution with a perspective client (a primary variable), you might get the contract (a trigger).
After building support for your bill (a primary variable), you might get a favorable vote (a trigger).
Of course there are failure triggers as well: losing a big client, or having a factory break down.

Accomplishing any tasks/ objectives/ goal/ mission typically is a trigger. Any definition of triggers needs to include both the condition that sets it off (including the necessary sensor), and also the consequences (often multiple consequences) of being set off. When studying what a subject matter expert knows/does, the question is, "what are events that if happen are (at least temporarily) irreversible and that change the dynamics?"

With a unit, a trigger can change an internal AI state or launches a script.

Feedback loops, the rich gets richer, and chain reactions all represent trigger-like behavior, even if there are no pre-defined triggers at play.

Author's note: A person pulls a trigger in a shotgun and kills a quail. One can only imagine how hard it would have been to kill that quail without a shotgun. Likewise, the notion of a trigger in practice is, "what is the least amount of work I can do to have the biggest possible reaction?" I throw a rock and a window shatters. I swipe a credit card and walk away with six new shirts and a great pair of shoes. I buy a stock and make a lot of money. Language and love are both triggered in our own bodies. Is learning as well?

Spend some time engaging some of the best sims in the world

clark.aldrich@gmail.com (Clark Aldrich) — Wed, 25 Feb 2009 14:44:00 +0000

If you want to see some of the best examples of educational simulations and serious games, go here: http://www.actonsims.com/.

You do have to register, but it only takes a second and there is no downside to doing so.

A bit of background: Acton Business School for Entrepreneurs in one of the newly accredited (within the last decade) universities that give me faith, even excitement, in the future of formal education.

These sims are great for the following reasons:

Despite their cartoonish looks, they are deep and real. The content is absolutely top-tier. You will learn real, advanced business school content.
They take about an hour or so to play.
They are completely flash based.
They represent a level of depth and production value that is deep enough to really help the students, but not so deep as to not be a valid challenge for all other academic (and corporate) institutions.

There is no question but that these represent the future of content.

I would challenge any educational institution, from MIT and Harvard to Microsoft and General Motors, to use these as their benchmark. There is not excuse for any organization with a desire for a competency in formal learning to produce a lot of these.

I am not sure for how long these sims will be available to use for free, so take advantage while they still are.

Venn diagram

clark.aldrich@gmail.com (Clark Aldrich) — Mon, 23 Feb 2009 23:00:00 +0000

A pedagogical technique of organizing elements into three, overlapping categories by using overlapping circles and labels. Venn diagrams avoid some of the artificial edge enhancement found in pure taxonomies such as book chapters or static definitions.

Venn diagram on basic differences and similarities between game, simulation, and pedagogical elements. (Click on to enlarge) Sims combine Simulation, Game, and Pedagogical elements, although many elements fall in two or even all three categories. This may be more useful than answering the question, "What is the difference between a game and a simulation?"

See Simulation Elements.

See Pedagogy and Coaching.

See Game Elements.