How They Got Game

NIST, Stanford Collaborate to Catalog Early Microcomputing Software Data

Henry Lowood — Wed, 21 Sep 2016 17:27:23 +0000

Chad Boutin at the National Institute for Standards and Technology (NIST) has just written about the multi-year project with the Stanford University Libraries, recently concluded, to preserve software in Stanford's Cabrinety Collection. This project also added significantly to the catalog maintained by NIST's National Software Reference Library team. You can read about it here.

Games and Interactive Media lecture series @ Stanford, 2016-17 schedule

Henry Lowood — Tue, 20 Sep 2016 17:56:49 +0000

Games and Interactive Media offers an intensively multi-disciplinary lecture series during the Stanford academic year. We are now beginning our third annual series. Please see the attached schedule and plan to attend!
Henry Lowood

Attachment	Size
Games and Interactive Media_Poster_Fall_2016.pdf	86.21 KB

Current Game Preservation is Not Enough

Eric Kaltman — Mon, 06 Jun 2016 20:59:19 +0000

(Portuguese translation, courtesy of Artur Weber)
This post is a distillation of some current thoughts on game preservation (extending to software preservation) that arose from a presentation I gave at Stanford two weeks ago. Video of that talk is here. The discussion in this post is a little more advanced and focuses mainly on the last 10-15 minutes of the talk. I have also posted a link to another presentation I gave at the Netherlands Institute for Sound and Vision in February. This earlier one is exclusively about the issues with standard game preservation. If you are unfamiliar with this whole topic, definitely check it out.

TLDR; The current preservation practices we use for games and software need to be significantly reconsidered when taking into account the current conditions of modern computer games. Below I elaborate on the standard model of game preservation, and what I’m referring to as “network-contingent” experiences. These network-contingent games are now the predominant form of the medium and add significant complexity to the task of preserving the “playable” historical record. Unless there is a general awareness of this problem with the future of history, we might lose a lot more than anyone is expecting. Furthermore, we are already in the midst of this issue, and I think we need to stop pushing off a larger discussion of it.

The standard model of game preservation

Any preservation activity must first decide on its scope, on the boundaries of what it considers to be worthy of saving, and on the basic definitions for the objects involved. In game preservation, the standard model for most (not all, as mentioned below) of the work I’ve been involved in resembles the image below.
Standard preservation model
There are essentially three major areas of interest: the physical extent of the game, the data stored on it, and the hardware necessary to run it. Briefly, the focus of a good deal of preservation effort is on the issues associated with accurately extracting data from physical media and maintaining both the physical media and hardware. For the former, there is an assumption that the data extracted from physical media can be run through the use of an emulator. This is a piece of software designed to take in old game data and make it playable on a modern machine. We are in pretty good shape with the emulation of the “major” computer systems and consoles released pre-2005 or so. (I have “major” in quotes because most emulation effort is directed towards the more popular platforms, with commercial success being the predominant criteria for selection as opposed to technical innovation, gameplay advances, or any other suitable historical metric.) The MAME project, with its new open source licensing and the inclusion of MESS, provides support for hundreds of systems, some swimmingly so, and others just barely. I’m aware that many other emulation projects exist, that they are highly fragile since many of them are passion projects, and that they suffer a lack of proper institutional backing. But at least there is a chance that if you have the proper game data, you can put it in an emulator and get back some playable experience.

The long term physical maintenance prospects, on the other hand, are not particularly good. Physical media and hardware degrade over time, and without upkeep will simple cease functioning. Data stored on magnetic disks, optical discs, etc. will stop being readable, and so again, there is considerable effort in getting the data off of them. Some of the more popular systems may always be physically playable due to the shear number of units available internationally, and the in-depth community knowledge of their technical specifications. There are numerous new consoles that are designed to play Nintendo Entertainment System games, for example, and maybe future production processes will allow for atomically perfect copies of older hardware. Regardless, both of these scenarios, either wholesale emulation or continuous physical maintenance will not hold up going forward.

This is because the standard model of preservation, and the solutions devised for it do not reflect the current state of game technology or our networked world. The historical object has now changed to a point where is does not reflect the methodologies developed for legacy games and game hardware. When I think of a video game, as an object, I think of the NES in the picture above. It is a singular experience, designed for a single piece of hardware and without any awareness of or dependence on the outside world. The problem is that for people not born in the 1980s, say people born around 2000, the video game, as an object, is a networked application on a mobile phone. Now, I’m not saying that if you were / weren’t born in either of those times then you must have these preconceptions. I’m sure some people born in 2000 totally love the NES. But that the preservation effort that I see in games is for the preservation of the past as seen from the standpoint of those active in game preservation. Those people want to make sure that their past isn’t lost, that you can still find out about and play M.U.L.E. or Escape from Mt. Drash, for the foreseeable future.

This fear of loss is totally understandable; there are other mediums whose early histories are replete with stories of historical devastation. I routinely hear, and think about, the fate of silent film, where a significant percentage of its early productions are permanently lost. Much of the loss is due to a lack of public acceptance of early film as something worth preserving. The modern idea that film is a medium in need of a preservation record took nearly 50 years to fully develop, in which time much was lost. Early games may have been in that same state, but I’m beginning to think that that’s no longer the case. There are extensive historical listings of games available, certain early consoles are nearly totally preserved (in the sense that there is available data from most of their games), and there is constant, sustained community effort to reinforce those records. I’ve come to believe that games (and software) are going to deal with an inverse of the issues confronting early film. Namely,

We are producing objects that are getting more technologically complex, more interdependent, and less accessible. And we are producing them at a rate that dwarfs their previous historical outputs, and that will terminally outpace future preservation efforts.

Future Issues

(As a quick aside, this thinking is based, in part, on the recent survey work of David S. Rosenthal in Emulation and Virtualization as Preservation Strategies and on some ideas from James Newman’s book Best Before. If you would like more insight on the issues with networked emulation see the former. For gameplay preservation and historical considerations of play, check out Newman.)

I don’t think I need to support that games are getting more technologically complex. Major studios are much, much larger then they used to be, and teams for AAA titles are incredibly large. Over 1000 people made GTA V. Around 10 people were responsible for DOOM. I know that modern, independent games have moved development back to smaller teams, but their work is also now dependent on toolsets and technologies that are significantly more complex and sprawling then those of even a decade ago.

That games are more interdependent is also not a controversial position. Most games are now distributed over a network, do not have physical dissemination of any kind, and require some form of network connection for play or updating. This is one of the major problems with future preservation activity. When a system is dependent on multiple parts that are asymmetrically disseminated, reconstructing the object and its played experience become much more difficult. By “asymmetrical dissemination” I mean that the player of a mobile game, like Candy Crush, receives a game client through an app store. That client requires a connection to a foreign game server to deliver the experience, and handle monetary transactions and record keeping. The older model of software preservation cannot deal with this asymmetry of access. If the foreign server turns off at some point in the future, that part of the game system is now, most likely, irrecoverable. If that server is gone, then in all likelihood, the game client will no longer function. This risk is inherent for a majority of current games. Even when the client is not as dependent on a server connection for play, there may be other parts of the system, like updates, automatic patching or mobile platform APIs, that do still require network access.

One way out of this problem is to emulate everything involved in the entire network. Get all the server and client software functioning again through emulation or modification and POW! back comes the game play. While this might be feasible on a technical level, you have the issue that any significantly networked game or activity probably also had a contemporaneous community associated with it, and a social space within the game program itself that cannot be recovered. This is most striking in MMOs, where the game is essentially an operationalization of a social world. If you recover the whole system, you really don’t get back much of the experience. (For more on this, see The Preserving Virtual Worlds Report, and Henry Lowood’s discussion of “perfect capture” (paywalled here).

The major problem with full network emulation is access to the information necessary to recreate a system or network of systems. Most game servers are proprietary, considered corporate secrets, and will never be distributed in a manner that allows for their emulation. That is not to say that with significant, dedicated reverse engineering effort you can’t create something that imitates a private server, you can. As the work on Nostralius shows, you can even get all the way to potential litigation. But the effort required to emulate networked APIs adds another level of complexity to the preservation task. You can no longer just save a game’s data and expect it to function; you need to save its whole network. From a preservation perspective this is an untenable nightmare. The technical burden is now significantly increased along with storage and reproduction requirements.

Another issue with access is the online distribution systems that supply most games. Whether through Steam, Google Play, PSN Network or iOS, we’ve offloaded management of our game data to cloud services and wall gardens. This is problem since most of these services reserve the right to strip access to content at any time, and to update that content without providing access to previous versions. Without dedicated preservation efforts, and an implicit dependence on tools created for piracy, once an application is gone from these services it will be hard (or impossible) to get back. Remember back in 2007, when everyone was using iPhones as light-sabers? Well if you don’t, I can’t show you it since the original application was removed for IP issues and replaced with a different one. Remember when the Undertaker in Hearthstone was like totally broken? Well I really can’t show you that one (as a playable experience) because it was patched out ages ago. There is now an assumption that older versions of things should not be recoverable in a playable way, since the older version is now obviously deficient and should be forgotten. Like how once they colorized Casablanca, we threw out that crummy black and white version.

The most pressing and least mentioned issue is the scale of the problem as it exists today, and how it will continue into the future. I’ve presented two different preservation objects, the first, according to the standard preservation model, is a non-networked program made for a dedicated, non-networked platform. The second is a highly network contingent object that we don’t really have a viable preservation model for. Obviously there are cases in between these two extremes (and have been for most of the history of games), but overly complex straw men risk becoming real and complicating the argument.

I made a series of graphs to illustrate the problem. (They are highly empirical, as you will see.) The first two are plots of the development of traditional, standard model games vs network contingent ones over the history of games. Until now, or the recent past, the perception has been that things have developed apace. There are a lot of networked games, but also a lot that do not appear to require network access to function (this is probably now untrue, even for singular experiences, but again strawmen!). Both developments are probably following some form of exponential curve, as more of the world gains access to the technical knowledge and tools needed to make games.
Development Until Now

Development to Future
The issue appears when projecting into the future, since the two curves are definitely not the same. In fact, the vast majority of the games that will be made by humanity going forward will be network contingent. This is a simple reflection of our networked society, and the expectations of networked connections that are now built into human experience from childhood. This issue also pans out when you look at the available (albeit slightly problematic) data. Mobygames, a website that tries to account for all historical games, lists around 55,000 titles in their database. Of those titles, around 20,000 alone are for some form of the Windows operating system. Steam, the popular PC game distribution service has a total of 9,175 games available for play on the current version of the Windows operating system. So roughly half of all the games created for Windows since its inception as an operating system are currently available for Windows 10, network contingent, and distributed through Steam.

The situation gets a bit more ridiculous when you consider mobile distribution platforms. A majority of the applications available through the iOS App Store are games. According to recent numbers from Pocket Gamer there are over 500,000 games available for download to your iPhone. That is, based on the MobyGames numbers, there is an order of magnitude more games on the App Store, right now, then have been created for the entire history of video games. The scale of production is immense, and all of those games do not fit into the previous considerations of game preservation, they are all network contingent. One response is that most of those games are probably crap, and while that might be true, who’s deciding what’s crap and what’s a “legitimate” game? If some young, future world renowned game designer made a bunch of unpopular apps before hitting the big time, we’d probably want those to be preserved, right? The second two graphs highlight the other important implication, which is that most future games will be disseminated without physical form. I still don’t know of preservation models for network distributed content, and it will soon be the only type of content that is created.
Distribution Until Now

Distribution Into Future
The incredible production rates of current games, and the inability to currently preserve them all will lead to a situation where predominantly single player, non-networked games are overrepresented (or in many cases the only representatives) in the playable record. I don’t see a way to actually avoid this situation except to be aware of it. If developers and designers disseminate more information about their development practices, or take on a standardized framework for the free dissemination of historical source code and assets, then those breadcrumbs could help prevent complete loss. You can’t reconstruct an entire loaf from the crumbs, but you can gain a sense of its texture, ingredients, and baking process.

Solutions?

Okay, so the situation is looking pretty dire, what are some ways to combat this looming preservation quagmire? (It’s also not looming, since it’s already here.)

First is to consider what we are trying to save when we preserve video games. What I thought we were trying to save is the ability to play a historical game at some point in the future. What I’m thinking we need to do is to record the act of play itself. This is the position of James Newman (and probably others). To "record the act of play" is to develop dedicated methodologies to save how people today interact with their games. These methods would involve video capture, textual description, and some means to describe that data for future contextualization and recovery. We probably can’t save all the games on the app store in a playable state, but we could probably record a video of some part of them, along with a player describing the experience. This is a way to hedge against total loss, since there will be at least some historical record of the object aside from its name on a list somewhere (if even that). The good news here is that YouTube and Twitch are effectively archiving gameplay everyday. The bad news is that we need means to organize and back up those records, since they will only last as long as Google and Amazon consider them profitable or useful.

Second, maybe get the people creating these things to dedicate a little more time to basic preservation activities. Releasing records of development, production processes, and legacy source code would allow other insights to be made about games, and in some cases allow for their recreation or recovery. I know this is a tall order, since the current corporate culture around all software development is to play close to the chest. It would be nice however, if the best advice archivists could offer game developers wasn’t to just steal things from their workplaces.

Third, there needs to be more societal pressure and motivation to legitimate games as cultural production worth saving, a la film, and to create larger institutional structures that fight for preservation activity. I spend some time at film preservation conferences, and it is fairly common that high-level executives from major Hollywood studios attend, speak and participate. This is partly due to the commercial viability of film preservation, since old films can still stock streaming services. But it’s also due to a culture in the film industry itself that understands and appreciates its history, the importance of maintaining that history, and the societal benefits that come from preserving cultural heritage. It would be great for the games industry to realize that as well.

Lastly, everything I’ve discussed regarding game software extends to software in general. And in that area, I think things may be even worse. Games at least form a semi-coherent class of software that can be framed as a cultural production worth saving. I don’t see the same consideration for office software, embedded systems or mobile applications and that’s a shame.

Cabrinety Cassette Capture

Charlotte Thai — Sat, 26 Sep 2015 00:31:28 +0000

This article was originally posted on Special Collections Unbound on September 14, 2015. Special Collections Unbound is a blog showcasing the work done by the Stanford University Libraries, Special Collections Department.

In the ongoing Cabrinety-NIST Project, NIST normally performs all disk imaging, but there is an exception to the rule. When the Stanford fall session begins in late September, a subset of the Cabrinety collection will be used as teaching materials in the Rhetoric of Gaming class. Rather than send the Cabrinety boxes containing these materials to NIST (which is located in Gaithersburg, MD), and risk them not returning in time for the class, I decided to do all disk imaging at Stanford.

One of the more challenging media formats found in the Cabrinety collection are computer cassettes, also known as datasettes. These media carriers can form nightmarish tangles, so audiovisual archivists working with them must maintain constant vigilance to forestall potential chaos.

The Cabrinety “teaching set” consists of 16 boxes containing 666(!) software packages, 60 of which are computer cassettes. The Stanford Media Preservation Lab (SMPL) provided invaluable assistance with capturing data from this obsolete format.

In a dedicated space located in the Stanford University Libraries branch location in Redwood City, CA, there is a room equipped with 8 Nakamichi CR-7A cassette decks. The outputs from these 8 cassette decks are recorded simultaneously while connected to 2 Mytek 8x192 ADDA converters, with Cubase 7 software being used to capture the PCM audio stream as WAV files. [Note: This is an oversimplification of a complex process that also involves several layers of quality control.] I stayed in that room for several days and created a lot of noise pollution – computer cassettes make a very unpleasant sound, akin to that of a dial-up modem. Here’s a 7-second sample!

Computer Cassette WAV file (7 seconds)

There were also normal audio cassettes mixed in with the computer cassettes to provide relaxing pockets of intelligible sound in the midst of the data noise.

Audio Cassette MP3 file (12 seconds)

One of the problems I encountered while working with the very last set of computer cassettes was that the Coleco ADAM format did not fit into the cassette decks. The cassettes seemed like they should fit, but the cassette deck player doors would not close. The reason for this was that the Coleco ADAM cassettes were missing holes in the bottom that would allow the doors to close. There were two solutions to this: a) Drill the holes or b) Temporarily rehouse the cassettes. I went with option b in order to maintain the integrity of the original media carrier.

This was the basic workflow for rehousing the Coleco ADAM computer cassettes:

1. Prepare an empty cassette shell by removing the screws and opening the top.

2. Open the Coleco ADAM cassette by removing the screws and opening the top to expose the tape.

3. Carefully(!) take hold of the tape reels, lift them out of the Coleco ADAM cassette, and transfer them to the empty cassette shell casing. Make sure the tape spins properly.

4. Replace the top of the temporary cassette. Check again that the tape is spinning properly.

5. LABEL the temporary cassette so you know which tape reels are inside, and also which side of the cassette is side A or side B.

6. Replace the top of the Coleco ADAM cassette so you don’t lose the tiny screws for it and set aside.

7. Perform data capture of both sides of the tape reel using the temporary cassette that was just created.

8. When the data capture is complete, reverse the earlier process to put the tape reels back into their original Coleco ADAM cassette.

There were 20 Coleco ADAM format computer cassettes, some of which contained 30-40 minutes of content on each side, so this was extremely time consuming, even with 8 cassette decks going at once. This was quite a learning experience and one that gave me a stronger appreciation for the challenges faced by those tasked with preserving analog materials. Also, I learned that whenever you are starting the long process of capturing data from time-based media, treat it like a road trip – make sure to take a bathroom break first!

How to Give Cartridge-Based Video Game Data an Extra Life (Part 2)

Charlotte Thai — Tue, 28 Apr 2015 23:02:53 +0000

The following is a guest post by Christopher Fox, a term employee at the National Institute of Standards and Technology (NIST). For over six years, Christopher has provided NIST's National Software Reference Library project with data entry services and the application development demands that support those services. He just completed his senior year at Hood College in Frederick, Maryland, where he earned a Bachelor's in Computer Science. He started working on the video game software part of the Cabrinety Project when the group hit a dead end trying to collect data from gaming cartridges. Since Chris enjoys gaming and found a way to collect save data from older games for his own purposes, he offered to explore and share methods of extracting data from these obsolete formats.

*Disclaimer: Trade names and company products are mentioned in the text or identified. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products are necessarily the best available for the purpose.

This is a follow up piece to my previous post on how the National Software Reference Library at NIST is collecting data from several different types of gaming cartridges.

Since my last post I extended my research into Nintendo Entertainment System (NES) and Game Boy cartridge data collection. The earlier problem was difficulty finding a device capable of collecting the ROM from these platforms for a wide variety of games.

NES:

I researched the CopyNES previously, but vendors for them were hard to find. Luckily, they recently came back into production at RetroUSB. RetroUSB is a website that specializes in development tools for older console generations such as the NES, SNES (Super Nintendo Entertainment System), and the Nintendo 64.

The CopyNES is a hardware NES tool that allows the user to collect or even write data to a NES cartridge. We were only interested in collecting data, so the ability to write data to cartridges was disabled to avoid any data being written to the cartridges in the collection. In order to install the CopyNES, the CPU of the NES gets taken out and then plugged into the CopyNES. The CopyNES is then placed directly on top of the NES motherboard so that it can interact directly with all the data that the NES reads while still being able to play cartridges. The CopyNES also has a USB socket that allows the NES to be connected to a computer so that the ROM from the cartridges can be collected directly.

Whenever you want to collect a ROM from a cartridge you plug in the game as if you were going to play it normally, turn on the NES with the USB attached, and run the program that comes with the CopyNES hardware to download the ROM in a .nes file. This procedure is great for collecting ROMs from individual cartridges but comes with a few problems since we have such a large amount of cartridges in the Cabrinety collection. To explain these problems I will first have to elaborate on exactly how the ROM collection process works with NES cartridges.

The first thing to understand is that every NES cartridge has what is referred to as a Memory Management Controller (MMC) or what is commonly called a mapper. This is a chip that is soldered directly onto the cartridge that performs all the address decoding and bank switching - in other words, supplies information to the NES on how the data should be read off the cartridge. Without knowing which mapper is used for the cartridge, the NES cannot read and play the game and thus the ROM cannot be collected. There are dozens of different mappers for different cartridges made by different companies. Luckily a list of almost all NES cartridges and their respective mappers has been made and released to the public. Once you look up the mapper for a specific cartridge - let’s say, “Gauntlet” - you will see that the mapper used for that game is MMC3. Now you can run the software that came with the CopyNES and enter in the associated mapper to allow the NES to read and collect data from the cartridge.

The problem with this method was the need to look up the mapper for every cartridge before dumping the ROM, which takes extra time and makes the whole process very tedious. Also since the software that came with the CopyNES is a Graphical User Interface (GUI) it is not easy to use the command line to make an automated script of the ROM dumping process. However, since there is a surprisingly large community of people interested in collecting NES ROMs, someone has developed a Linux/Mac OS X based command line program for the CopyNES called Copynesl. The developer goes by the name Crade and the forum in which he writes about his software is located here. Copynesl also requires a library called LibCopyNES which was developed by Wookie who also writes on the same forum. This software allows the user to specify the mapper while running the software, which makes the ROM collecting process fully automated.

However, another problem still exists. Unlike the SNES and Sega Genesis, which have built in anti-piracy checks, the NES has none. The SNES and Sega Genesis calculate the checksum of the ROM using a console specific algorithm and compare that checksum to a value stored in the ROM itself to confirm that all the data is untampered. The NES did not use this method and therefore causes issues when we want confirm if the ROM we have collected is a valid read. Thanks to the large community of fans collecting NES ROMs, someone has already solved this problem. Bootgod has a website that hosts a large database of NES cartridge information that is uploaded and then confirmed to be accurate by others on the site. On this site, you can look up specific cartridges and a lot of detailed information about each cartridge. The information I focused on using was under “ROM Details.”

In the “ROMs Combined” row, the value under CRC32 is the output of a cyclic redundancy check (CRC). In laymen’s terms this is an algorithm that will output a string 32 bits in length based on every single bit in the file. If a single bit is changed in the data then the CRC32 will change completely. This value can be used in a very similar way to the checksums I mentioned above. Now that we have a confirmation of what the CRC32 value of the ROM should be, we can calculate the CRC32 of the ROM we have collected and compare it against the database to prove whether or not the ROM is valid. However, the value that is in the database is not the CRC32 of the entire ROM file, but just the ROM data itself. The ROM file that is created with the CopyNES includes the ROM data and header data from the cartridge. The header data contains information that is used when loading and playing the ROM, similar to the NES mapper and is 16 bytes in length. Therefore I needed to create a temporary copy of the file but with the header removed and perform the CRC32 on that file and use that value for comparison.

Once I was able to run the cartridge data collection software in a way that could be scripted, and had a way to confirm that the data collected was accurate, I was able to write an automated script for the entire data collection process.

I’ve adapted the same script I used for collecting data for Sega Genesis and SNES to include all the functionality for NES cartridges.

The CopyNES has a feature to put the NES into a state called “Play Testing.” This allows the user to use the NES as a functional gaming system to see if the cartridge is connected properly, and if it will even run in the first place.

Once the cartridge is confirmed to be working properly, the user hits “Enter” and the ROM is collected.

I mentioned before that every NES cartridge needs to have the mapper specified ahead of time. However, having the user look up the mapper for every cartridge and enter it in by hand becomes tedious and leaves room for user error. To fix this issue I found a file that comes with the Copynesl software called “Mappers.dat” which has a list of every mapper that the CopyNES currently supports. I then parsed this file to store every individual mapper so they can be added as an argument to the command line tool. The command line tool is run as follows:

sudo ./copynesl –d $plugin –m $mapperNum –o $filename

The argument after “-d” specifies the plugin/mapper. The argument after “-m” specifies the mapper number which is a value that is stored in the header data of the ROM file (that specifies what the mapper type is for that cartridge.) The mapper number is also listed in the mappers.dat file next to its respective mapper. The argument after “-o” specifies what the ROM file will be named after the dump.

Now that I had these values stored, I tried to collect the ROM using the first plugin and mapper number on the list. After that ROM was collected I compared it against the database for a CRC32 match. If it didn’t match I moved on to the next plugin and mapper number on the list. This continued until I got a match. Once I found a match I renamed the file to the correct game title and the program exited.

Now that the ROM has been collected successfully it can be opened in an emulator and played just to make absolutely sure that the data is correct.

Game Boy ROMS:

I talked about my current progress on collecting Game Boy in my last post and some of the problems I was facing at the time. The first device I tried to use to collect the Game Boy cartridge ROM was a Retrode plugin. However, this device could only collect ROMs from a select few games, and I wasn’t able to collect a single ROM from the cartridges that were currently being processed. The Smart Boy was something I started to look into while working on my last post but upon further research discovered that it produced the same problems as the Windows version of the CopyNES software. It is GUI-based so scripting the whole process wouldn’t be possible. Also, the Smart Boy has several options for overwriting the data in the cartridge, and data could be accidentally deleted.

The newest method that I discovered involves using an Arduino and a Game Boy cartridge adapter. An Arduino is a popular microcontroller that uses several pins to send and receive data. I used the Game Boy Cart Shield which attaches to several pins on the Arduino at once and sits on top. This allows all the pins of the Game Boy cartridge to be accessed by the Arduino. The instructions for this are written out in detail on Inside Gadgets’ website in an article by Alex, one of the moderators on the site. I have had great success with using this method to collect both the ROM and the RAM of every Game Boy cartridge I have come across.

The ROM contains all of the data on the cartridge that cannot be edited and was uploaded to the cartridge by the manufacturer. The RAM contains all the data written to the cartridge by the user. This is the “save” data. The ROM is saved with a “.gb” extension while the RAM is saved with a “.sav” extension. The Arduino collects the ROM from the connected Game boy cartridge and sends the data via USB to the host computer. A program then waits for this data and writes it to a file. The process is repeated for the RAM on the cartridge. However, some cartridges do not contain any save data so dumping the RAM can be skipped.

Game Boy cartridges can also be tested for accuracy by using an anti-piracy system similar to the ones used for the Sega Genesis and SNES games in my previous post. However, Game Boy cartridges have multiple checks and other means of verification as opposed to just one like the other cartridges. The first thing that gets verified is the “Header checksum.” The algorithm for calculating the header checksum in PERL is as follows:

my x=0; # x is the calculated checksum
for (my $i=0134h;$i<=014Ch;$i+=0001h){ # 0xxxh is a hexadecimal address
x = x - $bytes[$i] - 1; #$bytes is an array of all the bytes in the dumped ROM’s
}

This algorithm goes through the entire header of the ROM and generates a value in decimal. If you convert this value to hexadecimal and then take the lower 8 bits they should be the same as the value stored at 0x014D in the ROM. The next checksum is referred to as the “Global checksum.” The algorithm for this one is relatively simple:

my x=0; #x is the calculated checksum
for (my $i=0;$i<=$file_length;$i++){
next if ($i == 014Eh || $i == 014Fh);
x = x + $bytes[$i];
}

You take every byte from the beginning to the end of the ROM and add it together. However you skip the bytes at addresses 0x014E and 0x014F where the global checksum value is stored. Once you have the sum you compare the answer to the global checksum.

The last form of verification is different from the rest. The Game Boy checks that the ROM of every cartridge contains the “Nintendo Bitmap.” The bitmap is a series of bytes that make up the Nintendo logo that shows up when you turn on your Game Boy with the cartridge plugged in. Once this series of bytes is confirmed accurate the game will continue to be loaded. The Game Boy will cease running if any of these bytes are out of place.

Once I had all of the checks in place I implemented the entire Game Boy ROM/RAM collection process into my existing PERL script.

The first thing the PERL script does is call a C program that tells the Arduino to send some data located in the header of the ROM through the USB to be used by the C program to determine how much data should be received and therefore how much data to write to the ROM/RAM files. The metadata for the cartridge is also printed out for the user. With this information, it then begins the actual process of saving to POKEMON RED.gb/POKEMON.sav. In this case, the metadata states that the ROM size is 1MB so it will dump until 1024K. It also states that there is RAM in this cartridge and it is 32K in length so that will begin next. Once the ROM/RAM has been saved to their respective files, all of the verification methods are used.

After the process is complete the ROM can be loaded into an emulator. The emulator will also automatically load the POKEMON RED.sav file as long as it is in the same directory as the ROM file. The save data below is actually from my own cartridge that I played as a kid.

Works in progress:

Commodore 64 ROMs:

I haven’t performed extensive research on how Commodore 64 ROMs work yet but I have found a method of collecting the ROM from Commodore 64 that may potentially be of use. It involves using an actual Commodore 64 similar to how we used a NES console for collecting NES ROMs. I would need to take apart a Commodore 64 and re-wire a few connections on motherboard in order to allow the ROM data to be collected through the disk drive using specific programs for each type of Commodore 64 cartridge. Using an Arduino is always an option as well and would involve a similar setup to the Game Boy. I will continue to research possible solutions.

Interactive Media & Games Seminar Series

Charlotte Thai — Fri, 27 Feb 2015 01:42:04 +0000

A few weeks ago I attended one of the lectures being given at Stanford University as part of the Interactive Media & Games Seminar Series. Every Friday at noon, this seminar presents open lectures for students, staff, and faculty who want a window into the sometimes esoteric world where games and academia collide. I was initially going to summarize the lecture I attended (given by HTGG’s own Henry Lowood!), but as all of these lectures are made available online, it is just easier to copy and paste the links….I mean it makes more sense to provide access to the actual lectures.

Also, if you’re a Stanford student, this seminar is listed as one-unit course BIOE196. So next time you get any grief for playing games instead of studying, you can casually point to this course listing as proof that those two activities are sometimes the same thing.

It's on: Game Histories from MIT Press

Henry Lowood — Wed, 04 Feb 2015 22:57:40 +0000

Raiford Guins and I just received some great news: MIT Press has given the proverbial greenlight to the "Game Histories" book series that we proposed to them. We are absolutely delighted to be the co-editors of this series, which will kick off with two collections of essays, to be followed by a series of monographic studies, probably 2-3 annually.

The first two titles in the series will be Debugging Game History: A Critical Lexicon, which Ray and I have been co-editing, and Zones of Control, co-edited by Matt Kirschenbaum and Pat Harrigan. We expect that these books will serve as foundational collections for game history and the history of military simulation, respectively.

Finally, we expect to begin soon the process of reviewing full book proposals for "Game Histories." The goal is to publish single-authored books in the series starting in 2017, after the two above-mentioned titles appear next year. Of course, the exciting bit in a project like this is seeing what prospective authors come up with. We can’t wait to start!

Mass Media

Charlotte Thai — Thu, 02 Oct 2014 00:18:39 +0000

This month I decided to try and make a giant poster that showed all the different format types that are present in the Cabrinety collection. I've gone through 6,000+ titles so far and found 21 unique kinds of software media. Since I keep discovering more it seemed like the poster would never get done. So instead of waiting for it to be complete and perfect, and possibly too big to ever upload to this site, here it is in all its glory:

I expect the caption text might be too small too read on this version of the poster, so this is what it says:

First Row (L-R): Atari 400 cartridge, Sega Game Gear cartridge, Intellivision cartridge, Commodore 64 cartridge, Vectrex cartridge, Atari 2600 cartridge

Second Row (L-R): ColecoVision cartridge, Atari 7800 cartridge, computer disc (3.5"), Atari 400 cassette, Sega Card

Third Row (L-R): TI-99/4A cartridge, Tandy Color Computer cartridge

Fourth Row (L-R): Atari 5200 cartridge, Sega Genesis cartridge, Nintendo Entertainment System cartridge

Fifth Row (L-R): Computer disc (5.25"), Super Nintendo Entertainment System cartridge, IBM data cartridge

Sixth Row: Sega CD

ASSORTED SOFTWARE MEDIA TYPES FOUND IN THE STEPHEN M. CABRINETY COLLECTION IN THE HISTORY OF MICROCOMPUTING, CA. 1975-1995.

THIS REPRESENTS ONLY A FRACTION OF THE COLLECTION.

Past Lives

Charlotte Thai — Fri, 08 Aug 2014 23:19:22 +0000

Many lifetimes ago I worked for a video game enthusiast magazine called Tips & Tricks. It featured several strategy guides every month and also printed thousands of cheat codes. When the magazine was still alive and kicking, there was a short Wikipedia entry describing it and all of the employees. Upon a recent check I saw that this entry had gone the way of the dinosaur. This was a bit of a letdown, but also a wake-up call that for something to be preserved, someone needs to advocate for its inclusion in the historical record.

Enter Roger Burton, the Editor-In-Chief of Game Losers, a website devoted to covering underreported sports and video game stories. A former reader of Tips & Tricks, he was curious about the history of the magazine, and thanks to the aforementioned lack of information on Wikipedia, he decided to track down many of the old writers and interview them, including yours truly.

So if you are interested in reading about my sordid history as a video game journalist way, way, waaaay back in the day, Burton’s “A Comprehensive Oral History of Tips & Tricks – The #1 Video-Game Tips Magazine” has got you covered.

California Extreme 2014

Charlotte Thai — Tue, 15 Jul 2014 16:12:12 +0000

California Extreme: The 18th Annual Arcade & Pinball Show was held from July 12-13, 2014 at the Hyatt Regency in Santa Clara, CA. This is a show where collectors who own classic arcade cabinets and pinball machines bring out their wares for the public to play firsthand, giving a new generation of gamers the opportunity to experience very old titles like Space War or Computer Space. It’s an excellent value with all of the machines set to free play, and the cost of entry only $30-40 a day for one adult. The Saturday show ran from 11am until 2am the next morning, so any gamer with the stamina could play for 15 hours (more if s/he paid for early entry.)

The last time I attended California Extreme was about a decade ago, and what I remember most vividly about that experience (aside from the fact that someone had to sleep on the hotel room radiator) was that I played a lot of Slick Shot, and that it wasn’t nearly as crowded as I was expecting. There was space to roam around, and a lot of the machines I wanted to play were open.

This year was completely different – the crowd had grown exponentially, there were a lot more kids, and jumping on a machine you wanted was a matter of luck and timing. Playing Slick Shot was much more awkward because you had to swing around a full-sized cue stick in a packed space. There were also a lot of kids running around holding stools so they could reach high enough to operate the controls on the upright arcade cabinets. That was adorable. What was not so adorable was that some of them were also hogging the machines. Some of us want to play Tapper, too, kid!

My memories of how to play some of these games was quite rusty. One of the few that I remembered playing as a kid was BurgerTime. For some reason, now I was under the impression I had to kill and trap the roaming enemies inside the hamburgers, instead of just building the hamburgers with the already laid out ingredients like a normal person. I also played a bit of Paperboy, and had to reeducate myself that in the arcade universe, riding a bicycle over a sewer grate means instant death.

One of the newest games on the floor (and possibly one of the most valuable) was a Fix-It Felix, Jr. cabinet that was modeled after the fictional game featured in the Disney movie Wreck-It Ralph. Disney created a few of these arcade cabinets as promotional items, and gave them a scuffed-up look so they’d seem like authentic machines from the 1980s.

The first time I waited in line for a chance to play it, there was one person in front of me so I figured I wouldn’t have to wait that long for my turn. Instead, she proceeded to destroy the game, level after level, like she’d been playing it for years.

When I came back and tried it out myself, it was much harder than it looked. Anyone who wants a chance to play the game can check out the online version, which has a much easier default setting. If you want to own the arcade cabinet, it will cost you about 20K (although the one at the show was quite clearly labeled as “NOT FOR SALE.”)

While wandering the floor in search of open games, I saw a sign for something called the Digital Game Museum. They had Puppy Pong set up behind them, a game I had only seen previously during GDC as part of a Videogame History Museum display. Intrigued, I stopped by to introduce myself. The DGM is a nonprofit organization dedicated to the preservation of digital games, and they are local to the Bay Area (Santa Clara). I told them about Stanford’s recent work with UCSC on GAMECIP, an IMLS-funded project to create videogame metadata standards and controlled vocabularies for libraries, archives, and museums. This is something they were interested in as well – so far the available standards at places like the Library of Congress or the Getty are woefully inadequate to deal with videogames.

Now that I am an old person I like to sit down at least once every 10 hours, so we headed to the Console Games room. This room was located in a tiny room that seemed like it was several miles away from the main floor.

It featured cushioned chairs, Super Nintendo and Sega Genesis games, and a small sales area. I played Sonic the Hedgehog for the first time in years, and only passed the first three levels before dying (sad considering how many times I beat this game during my wasted youth.) There was an Atari E.T. cartridge for sale that was prominently labeled as “Not from the landfill” as well as a container filled with Genesis controllers for $5.00. Those poor Genesis controllers…I almost bought one just because I felt sorry for it. Damn you, Toy Story, for making me anthropomorphize inanimate objects!

The rest of the show was a bit of a blur, and we all tapped out at dinner time. I only took a few other pictures (which are shown below) of diminishing quality. I apologize profusely to pinball enthusiasts everywhere for my clear arcade machine bias in this post.