• Find a web page that has a complete list of games: http://en.wikipedia.org/wiki/List_of_wii_games (Wikipedia has a lot of pages like this).
• Use Perl regular expression matching in PHP to scrape this list from the page and insert into my database.
• Done.

Let me dive a bit deeper. All the names on this web page are marked with the same HTML pattern:

It’s <i><a href="[some link]" title="[THE NAME]". With a bit of knowledge about regular expressions, and after I have used htmlentities() to encode the markup, the expression I need is:

preg_match_all("/<i><a href=(.+?)title=(\")(.+?)(\"| \()/", $r, $matches);

What this does is capture all occurrences of the regular expression I’ve used in the string $r, and place those in the array $matches. Furthermore, each (...) has its own set of matches, so that the $matches variable is actually an array of arrays. So, the names of these Wii games that I am looking for are all in $matches[3]. All I have to do is iterate through that array and put each name in my database.

But wait, there’s more…

Something I should explain further: The most important part of my regular expression match is this: title=(\")(.+?)(\"| \(). I’m grabbing the content of the title attribute because the actual text sometimes contains more than just the game’s name, but sometimes the title also contains something in parentheses, which is a Wikipedia disambiguation. So, there are two things that might come after the title attribute: a quotation mark, or a space & open-paren. With the pipe, I capture both, and I always get nothing more than the name of each game, without even having to trim().

Scraping is fun, eh?

Wii Fit, which goes for a retail price of $90, is reaching prices of $132 hours before the auctions even close. To make things worse, these prices indicate that people are actually offering to pay this much more for something that retailed at $90. Clearly, the Ebay problem is a scalping problem. Ebay was always meant as a way for people to resell things rather than just throwing them away. For the buyer, Ebay ought to be a place where you can find deals as well as products that aren’t sold in stores. I really wish it wasn’t the prime place for scalping.

Unfortunately, you can’t blame Ebay for allowing this practice. They are simply providing a service, and allowing users the freedom to take advantage of that service as they please. The real responsibility is on the buyers; people should refuse to pay any more than $90 for Wii Fit, ever. If people did that, no one would try buying them just to resell them, and it would be much easier for a guy like me to buy a Wii Fit at retail price direct from a store.

Maybe I’m just dreaming. It’s hard to get everyone to think in the interest of others. I’m just writing this to make the statement that I refuse to pay more than $90 for the Wii Fit I’ll eventually have. In the meantime, I’ll stick to jogging and whatever else I can do to lose weight.

Note: I have no qualms with paying less than $90 for Wii Fit, ha ha

If I gave you a bucket and told you to sit on a beach and count grains of sand, one by one, all day long, you would do it. You would do it because it’s fun. You might insist that I’m wrong, but I know I’m right because millions of people do this every day. Not only are people willing to do this, but many businesses and entrepreneurs are driving their revenue models on making people count sand. It’s happening so much that I needed to coin a term to describe it, especially since I would like an easy way to describe to people what I do for a living. I came up with “Crowdherding.”

Crowdherding is a lot like Crowdsourcing, which Wikipedia, in its infinite wisdom, describes as:

“a neologism for the act of taking a task traditionally performed by an employee or contractor, and outsourcing it to an undefined, generally large group of people, in the form of an open call. For example, the public may be invited to develop a new technology, carry out a design task, refine an algorithm or help capture, systematize or analyze large amounts of data”

Wikipedia’s wisdom doesn’t stop there. It also says:

“The term has become popular with business authors and journalists as shorthand for the trend of leveraging the mass collaboration enabled by Web 2.0 technologies to achieve business goals. However, both the term and its underlying business models have attracted controversy and criticism.”

I think this is simple enough. Crowdsourcing is cool when you think about all the ways in which it takes place:

• Wikipedia never has to buy any content. All the information within is generated by users, and the system works in such a way so as to ensure a manageable level of accuracy.
• Kongregate’s library of games is made up of Flash games that have been submitted by outside developers, running on a platform that allows those games to tie into the site. Kongregate splits revenue with these developers, and in return Kongregate gets one of the most engaging and feature-packed online gaming sites.
• Amazon doesn’t pay people to write reviews. It encourages users to submit reviews and aggregates them, and then encourages users to rate those reviews to ensure a reasonable level of quality. That’s crowdsourcing twice.

The way I see it, all forms of Crowdsourcing have one thing in common: the members of the crowd are building something valuable. Crowdherding, which I believe happens much more often, does not have this quality. I would describe Crowdherding to be:

“the act of convincing a group of people to perform a rote task, akin to sitting on a beach all day and counting sand, usually through various incentives, whether real or virtual”

Some examples:

• In RPGs, players often refer to “grinding,” which is the task of continuously fighting monsters to raise their avatar’s level, or continuously searching for items to complete quests. They call it grinding because it’s a lot like grinding an axe on a wheel; one simple task has to be done continuously to make the axe sharp, and until it has been done for long enough, the person grinding the axe is really bored. A Ragnarok player once told me that it would take him ~250 hours to get his character from level 98 to 99, where the process involves fighting tons of monsters and nothing else. He did it.
• FreeRice has users play a vocabulary game; for each word a user gets correct, 20 grains of rice are donated to the UN World Food Program. In the month of May, over 4 billion grains of rice were donated. The rice is paid for by the ads that appear on the bottom of the screen. The website operator claims to not take any profit.
• The Facebook game Friends For Sale! awards you $10,000 of in-game currency for visiting the application at least once in a 4 hour period. This is essential to the developers because it keeps the daily numbers up and it is enticing to the users because money is necessary to buy friends in the game. In the past month, Friends For Sale! was the #2 most popular game on Facebook, after OWNED!, another game that follows the exact same principles.

As mentioned before, in all examples of Crowdherding, the members of the crowd are not building anything. They are not generating content. They are simply gaining incentives, whether that be points for their avatar, virtual currency, or a sense of well-doing for helping the needy. In all cases, the actual mechanics of what the user is doing can be described as nothing more than clicking (on a link, or a button, or an on-screen enemy). If you think about it, Crowdherding has been around for a long time. Think of any case where users are being driven like sheep to do a simple task that generates revenue for someone else, and you have an example of Crowdherding. While it sounds bad, I wouldn’t think of it as good or bad. It’s using psychology to generate revenue from user activity. It’s getting people to look at your webpage instead of someone else’s. It’s a concept you have to understand if you want to be successful in this ad-driven industry. And in the case of Free Rice, that herding leads to purely humanitarian goals.

So let me describe what makes successful Crowdherding:

• Offering valuable incentives. This value can be virtual, in which case the actual cost might be much lower. A virtual incentive might be a cartoon depiction of a hat which your avatar can wear inside a game. This hat only costs as much as a one-time fee charged by the artist who made it, so once it has convinced a few certain number of people to generate a certain amount of ad revenue through their activity, it has paid itself off. A service that uses both real and virtual incentives is Live Search Club.
• Making the process of counting sand engaging. Even though the user maybe be performing some rote task, they are more likely to do it if it feels fun. This may involve watching a funny video or playing an engaging game.
• Making the process of counting sand & gaining incentives somewhat difficult. People who play RPGs may complain about grind when they encounter it, but the fact is that the best online RPGs all have it. This is because grind is hard, demanding a serious time commitment from the user, and when users make it through that grind and gain their reward, they feel a sense of accomplishment that they wouldn’t have if they had received the reward quickly. This, in turn, inflates the value of the incentives, making them even more desirable to users (and building a vicious cycle). Also, the more a user invests in a game, the more they feel attached to that game and are reluctant to give up playing it.

That’s Crowdherding as best as I can explain it. Hopefully now you have some insight into my line of work. Food for thought: is Crowdsourcing a subset of Crowdherding, or are they distinct practices, or what?

In a startup, it’s typical for ideas to be plentiful and resources to be limited. It’s also typical to have a lot of problems, as a company moves from scratch to (hopeful) success. This means that there will always be reasons to argue, and there will always be a need for solutions.

I have often seen someone in the middle of a discussion do nothing more than state existing problems, as if saying them out loud will somehow fix them. Stating problems doesn’t make them go away, nor does it encourage others to fix it. An example of doing so would be, “Our projects take too long to complete because we don’t have any task monitoring or follow-up in place.” It’s way more useful to propose solutions to existing problems. While people may be annoyed when they are reminded about the same problems they already deal with everyday, the same people are excited to hear about solutions that might put those problems to rest; after all, solutions are what people get paid for. An example would be, “Our projects take too long to complete because we don’t have any task monitoring or follow-up in place, but we can fix this by using project management software A and assigning someone to be product manager.”

Now this is already a huge improvement, but in the limited teams that are typical of startups, if you want to see problems actually get eliminated, you have to be willing to take this one step further and actually implement your own solutions, or at least find the right people to do so. Another example: “Our projects take too long to complete because we don’t have any task monitoring or follow-up in place, but we can fix this by using project management software A and assigning someone to be product manager. I’ll purchase an X-user license for this software and have CD’s for us by tomorrow so everyone can install it, and I propose that we start hiring for a new position of product manager; in the meantime, I can fill the position to get things rolling.” This is what separates constructive contribution from complaints, and I can say for certain that this attitude is what every startup needs.

When you send a gift from the Mozilla Firefox collection to a friend, they will see this link encouraging them to try Firefox:

So how about showing your support for Firefox and sending these gifts to all your friends? They’ll thank you later.

• Preview mode: click “Preview” in the header to see how the layout will actually look, then click “Edit” to return to editing mode.
• <HR> tag: Press Z to toggle having an <HR> after the currently selected container.
• Lorem Ipsum: Press C to fill the current column with Lorem Ipsum text. Press V to empty it.

It also has some small fixes and improvements, most importantly:

• The bug where old styles or IDs would continue to show up in the generated CSS is gone.
• The HTML and CSS output are nicely boxed and separated.
• The “Hide this” link for slide-down dialogs is more standardized and sits in the top right.

These changes required some major updates to the Construct code. For example:

• Floated columns only take up space if they have content in them, so they all have a non-breaking space (&nbsp;) even when they are empty.
• Having <HR> tags between containers means there are siblings that are not containers, but the up/down arrows are still supposed to scroll through containers. What used to be:

$('#construct .container.selected').prev();
// ... 
$('#construct .container.selected').next();

Had to become:

$('#construct .container.selected').prevAll('.container').slice($(this).length - 1, 1);
// ... 
$('#construct .container.selected').nextAll('.container').slice(0,1);

And I’m not sure if that’s because there’s a bug in jQuery or that’s just the intended way.

• Adding <HR>’s and paragraphs to the mix meant my parser had to account for them, since they are both different from DIVs. For now I have these two checks thrown into the recursive function that parses each element:

var tagName = el.nodeName.toLowerCase(); // tag name!
 
if( tagName == 'hr' ) { 
  markup += tabToDepth() + "&lt;" + tagName + "&gt; <br>";
  return markup;
}
else if( tagName == 'p' ) { 
  parseDepth += 2;
  markup += tabToDepth() + "&lt;" + tagName + "&gt;Lorem Ipsum...&lt;/" + tagName + "&gt; <br>";
  parseDepth -= 2;
  return markup;
}

I’ll have some screenshots and maybe a fresh tutorial up soon, but make sure to check version 0.5 out. Also, I think it’s worth mentioning that with what I did on Sunday, it’s apparent that I can achieve a lot more in terms of allowing users to add their own sample content to the layout as well as allowing them to customize more things, such as colors and background images, as well as content styles. More than ever before, I can see the potential for a completely semantic, clean, standards-based layout editor built on top of Blueprint and jQuery, with a very minimal amount of code. It’s cool stuff.

If anyone has any ideas for what I can do to reach more people with this project and make it better as well, do share.

/**
 * XCache
 *
 * @package XCache
 * @version $Id$
 * @copyright 2007
 * @author Cristian Rodriguez <judas.iscariote@flyspray.org>
 * @license BSD {@link http://www.opensource.org/licenses/bsd-license.php}
 */
 
class XCache {
 
    private static $xcobj;
 
    private function __construct()
    {
    }
 
    public final function __clone()
    {
        throw new BadMethodCallException("Clone is not allowed");
    } 
 
    /**
     * getInstance 
     * 
     * @static
     * @access public
     * @return object XCache instance
     */
    public static function getInstance() 
    {
        if( !(self::$xcobj instanceof XCache) ) {
          self::$xcobj = new XCache();
        }
        return self::$xcobj;
    }
 
    /**
     * __set 
     * 
     * @param mixed $name 
     * @param mixed $value 
     * @access public
     * @return void
     */
    public function __set($name, $value)
    {
        xcache_set($name, $value);
    }
 
    /**
     * __get 
     * 
     * @param mixed $name 
     * @access public
     * @return void
     */
    public function __get($name)
    {
        return xcache_get($name);
    }
 
    /**
     * __isset 
     * 
     * @param mixed $name 
     * @access public
     * @return bool
     */
    public function __isset($name)
    {
        return xcache_isset($name);
    }
 
    /**
     * __unset 
     * 
     * @param mixed $name 
     * @access public
     * @return void
     */
    public function __unset($name)
    {
        xcache_unset($name);
    }
}
 
//example use...
 
$cache = XCache::getInstance();
$cache->foo = PHP_INT_MAX;
$cache->bar = range('a', 'z');
var_dump($cache->foo);
var_dump($cache->bar);
unset($cache->bar);
var_dump(isset($cache->bar));

If all this looks daunting to you, let’s consider what developers usually use: global variables. One might do:

$cache = new XCache;

in a file like cache.setup.php and include that in the rest of their files. Then, in a function, one might do:

function get_from_cache($key) {
  global $cache;
 
  return $cache->$key;
}

This might seem simple enough, but there are a couple problems with this approach:

• Even in scripts that don’t need to connect to the cache / create an instance of the connection, the connection still occurs. In a high level application with a lot of scripts being run, it’s important to avoid unused connection overhead.
• Every function or method has to call the global variable with “global $var,” though one could argue this isn’t much different from calling a singleton object.

The important point is the first one; I’m always developing for speed in my day-to-day work, and I’d like to put all my PHP scripts together into a coherent application without having to think about which pages of an application should be connecting to the database/memcache/xcache and which ones shouldn’t. I would like to have the connection interfaces available without having to worry about the connection overhead unless I absolutely need to make those connections. And I never, ever, want to connect to any of these services more than once in a page. I want to use the same connection from right when I first need to open it to the end.

Singletons make all of that easy. That’s why I wrote my own for mysqli and Memcache:

/**
 * XDB
 *
 * @package XDB
 * @version $Id$
 * @copyright 2008
 * @author Christian Montoya <montoya.christian@gmail.com>
 * @license BSD {@link http://www.opensource.org/licenses/bsd-license.php}
 */
 
class XDB {
 
    private static $xdbobj;
 
    private function __construct()
    {
    }
 
    public final function __clone()
    {
        throw new BadMethodCallException("Clone is not allowed");
    } 
 
    /**
     * getInstance 
     * 
     * @static
     * @access public
     * @return object mysqli instance
     */
    public static function getInstance() 
    {
        // this is broken up for easy reading
        if( !( self::$xdbobj instanceof mysqli) ) {
          self::$xdbobj = new mysqli(DB_IP, DB_USER, DB_PASS, DB_NAME);
          // here I am using defined constants for all the DB creds
        }
        return self::$xdbobj;
    }
 
}
 
// Examples
$DB = XDB::getInstance();
$select = $DB->prepare("SELECT * FROM blog");
// ...
$DB = XDB::getInstance(); 
// still the same, single mysqli connection

/**
 * XMem
 *
 * @package XMem
 * @version $Id$
 * @copyright 2008
 * @author Christian Montoya <montoya.christian@gmail.com>
 * @license BSD {@link http://www.opensource.org/licenses/bsd-license.php}
 */
 
class XMem {
 
    private static $xmemobj;
 
    private function __construct()
    {
    }
 
    public final function __clone()
    {
        throw new BadMethodCallException("Clone is not allowed");
    } 
 
    /**
     * getInstance 
     * 
     * @static
     * @access public
     * @return object Memcache instance
     */
    public static function getInstance() 
    {
        if( !(self::$xmemobj instanceof Memcache) ) {
          self::$xmemobj = new Memcache();
          self::$xmemobj->pconnect(MEMCACHE_IP, MEMCACHE_PORT);
        }
        return self::$xmemobj;
    }
 
}
 
// Examples
$Mem = XMem::getInstance();
$Mem->get('key1234');
$Mem->set('key5678', 'saved', 0, 3600);

Remember, these singletons will return a new instance (connection) the first time they are called in a script execution, and everytime thereafter they will return the same instance that was already opened. If they are never called, a connection will never be opened, so there are no wasted connections.

In my opinion, this is way better than depending on global variables. If you are interested in using this, feel free to take the code here. Just remember to remove the examples

Updated 4-15-2008 with fixed code, thanks Cristian Rodriguez