Day 0 began roughly as follows:

2:00pm CS50x goes live
2:02pm 500 users online
3:00pm 10,000 users online

Tommy MacWilliam '13, CS50’s sole head TF one of CS50’s head TFs, shares even more detail in his blog.

Day n-1, meanwhile, concluded with

150,349 students registered
100,953 students engaged
10,905 pset0 submissions
1,482 project submissions
1,388 certificates awarded

whereby students who “engaged” watched content, asked questions, and used apps, even if they didn’t submit work. Receipt of a certificate required submission of all work with scores of 60% or higher.

Submissions of problem sets and quizzes trended as follows:

Daily engagement in the course, as measured by Google Analytics in terms of unique visitors, trended similarly, peaking early, then reaching some form of equilibrium (that appears to trail off as the course’s end loomed!):

If we zoom in on the course’s final weeks, unique visits hovered around 2,500 per day:

Visits by country, meanwhile, were ordered as follows, with Google Analytics reporting at least one visitor from nearly country in the world:

As for the students themselves, upon submitting Problem Set 0, students were asked to submit a form that inquired as to their background before CS50x and motivation for taking CS50x. Based on 10,905 submissions (which may or may not be representative of CS50x’s 150,349 registrants), CS50x’s demographics were 20% female and 80% male (whereas CS50 on campus was 36% female in Fall 2012). The average age was 29, with a median of 27 and a mode of 21. CS50x’s eldest student was 80 and youngest student was 10. Even younger were Louis and his brother, who said “hello, world” as well. Though filming those hellos apparently required one take too many.

Per the below, 56% of CS50x students had no prior background in CS, versus 75% in CS50 on campus:

Most students had at least a degree from high school, if not beyond:

Out of the 10,905 students who submitted pset0, 10,137 (93%) intended to do all of the course’s work:

To be fair, out of the 150,349 students who registered, only 10,137 (7%) intended to do all of the work (else they’d presumably have submitted at least pset0).

Meanwhile, out of the 10,905 students who submitted pset0, 3,381 (31%) took CS50x because of the prospect of a certificate:

To be fair, out of the 150,349 students who registered, only 3,381 (2%) took CS50x because of the prospect of a certificate (else they’d presumably have submitted at least pset0).

For most students, then, “success” didn’t necessarily mean a certificate. And “completion” wasn’t necessarily the goal. Among students' motivations for registering were:

Ultimately, at least 1,388 students will receive a certificate like John Harvard’s here:

To put all these numbers into perspective:

CS50x’s “completion rate” (whereby completion is defined as submission of all work with scores of at least 60%) was thus:

By contrast, 703 out of 706 students (99.6%) “completed” CS50 on campus this past fall. But, to be fair, for most CS50x students, “completion” wasn’t necessarily the goal. Indeed, tens of thousands “engaged” in some form.

For the curious, version 2 of CS50x will debut in late 2013 or early 2014 once we’ve had a chance to re-tool and improve! In the meantime, all of version 1’s content will remain freely available at cs50.tv, at youtube.com/cs50, and at x.cs50.net as well as in iTunes and iTunes U. And even before version 2 of CS50x debuts, content from Fall 2013 of CS50 itself will become available at cs50.net in September 2013.

Truth be told, I thought I’d set up a "cronjob" on our server whose purpose in life was to execute that geocoding script nightly. Unfortunately, it seems I never did. (Or, in my defense, I did but set it up under a username that I later deleted. Though that probably still makes it my fault.) But no matter, what a perfect opportunity to talk about cron!

cron is a utility generally found on Linux systems (and others) that allows you to schedule the execution of commands. For instance, if you’ve written a command-line program called foo, you can schedule foo to be executed every 5 minutes without having to log into the server yourself every 5 minutes and run it manually. Alternatively, you could schedule foo to be executed once nightly, or once weekly, or, really, anytime you’d like, however often you’d like. cron is often used by the system itself to, say, rotate logs (i.e., rename them so a file doesn’t get too big). In CS50, we use cron (via rsnapshot) in order to back up students' source code every 5 minutes to save them the inevitable anguish of accidentally deleting some file. (The disk space it’s cost us to maintain so many backups has been well worth the tears saved!)

Anyhow, using cron is wonderfully simple. You essentially run a command like

crontab -e

that allows you to edit (per the -e) a text file (otherwise known as a "cron table"), each of whose lines represents a scheduled command. In order to enable daily geocoding of our Google Spreadsheet, I added the one-liner below to the webserver’s own crontab, where /path/to/geocode is the full path to the command-line geocoding script I wrote:

@daily /path/to/geocode

It turns out that my one-liner is actually short-hand notation for

0 0 * * *

which means, right to left, that this command (/path/to/geocode) should be executed every day of the week, every month, every day of the month, when the clock strikes 0 hours and 0 seconds, where * denotes "every." In normal-people terms, it means "run this command every night at midnight." To be honest, I almost always forget which number means which, but Wikipedia has a wonderful cheatsheet. Allow me to excerpt:

.---------------- minute (0 - 59)
|   .------------- hour (0 - 23)
|   |   .---------- day of month (1 - 31)
|   |   |   .------- month (1 - 12) OR jan,feb,mar,apr ...
|   |   |   |  .----- day of week (0 - 6) (Sunday=0 or 7)  OR sun,mon,tue,wed,thu,fri,sat
|   |   |   |  |
*   *   *   *  *  command to be executed

I should confess that I’ve actually told a little white lie in the interests of simplicity. Our cronjob is actually

@daily /bin/nice /path/to/geocode

which means that the command should be "nice'd" before it’s run. Like cron itself, nice is a utility found on most Linux systems (and others) that allows you to raise and lower commands' priorities (i.e., how much CPU time they get). By default, nice lowers a command’s priority, so our cronjob really tells the system to "run this command every night at midnight but run this command at a lower priority so it doesn’t slow other, more important things (like file downloads) down."

Odds are, if you have a shell account on a Linux system (that you can access via SSH), you have the ability to schedule cronjobs, at least if the system administrator allows you. If you instead have an account with a third-party Web host that only provides you with a Web-based panel, you may be able to schedule cronjobs via that panel’s GUI in a slightly more user-friendly way.

Anyhow, thanks to cron (and my actually using it), cs50.tv's guestmap is indeed now updated nightly, and it currently looks like this!

When we decided (spontaneously one night) to implement this guestmap, we weren’t sure if anyone would notice (or care!) if we did, so we didn’t want to spend too much time on its implementation. In particular, we didn’t want to bother setting up an entire database (e.g., MySQL) just for this one application. Simpler would have been to store folks' submissions in a local file (e.g., CSV, SQLite, or XML). But the plan was to start replicating cs50.tv's content across multiple servers, in which case we’d then have to keep copies of that file synchronized across multiple servers. Doable, but somewhat annoying. If we used a local file as our database, we’d also have to worry about concurrent writes (as when two visitors happen to submit their names and hometowns simultaneously, the result of which might be to lose one of the visitor’s data). Also doable, and not very hard (thanks to functions like flock). But we’d still have to implement the form via which we ask for visitors' names and hometowns. (And at this point in the story, it was probably already midnight or so.)

If only there were someone to whom we could outsource both the form and the database…

Turns out there was! Odds are you’re familiar with Google Spreadsheets, which is essentially Google’s web-based answer to Microsoft Excel. It’s actually wonderfully useful, in large part because it makes it so easy to share (and edit simultaneously) spreadsheets with other people. But with Google Spreadsheets can you also create web-based forms whose submissions end up in a Google Spreadsheet: each row represents a submission, and each column represents a visitor’s answer to a particular question. And you can then share that spreadsheet with yourself (or others) in a variety of formats, among them CSV.

Nice! Even without writing any code, I was almost finished implementing the guestmap! (Okay, admittedly, it took a few minutes to create the form and share it with myself as CSV, but way fewer minutes than it’d have taken to implement all that myself.)

So, at this point in the story, I had two URLs: one for the form and one for the CSV file. My plan was (1) to embed the former in an iframe on cs50.tv so that folks could add themselves to the guestmap without leaving cs50.tv itself and (2) query the latter anytime someone wanted to look at the map. All that remained was to write the code that queries the CSV and generates the map.

Thanks to the Google Maps API, this part, too, was amazingly easy. To embed a Google Map in one of your own pages, you first need to include the API in your page’s head, as with:

<script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>

You then need to provide a container for it in your page’s body, as with:

<div id="map" style="height: 100%; width: 100%;"></div>

Lastly, you need to tell Google to transform that container into a map, as with:

new google.maps.Map(document.getElementById("map"), {
 center: new google.maps.LatLng(42.37411257777324, -71.11905097961426),
 disableDefaultUI: true,
 mapTypeControl: false,
 mapTypeId: google.maps.MapTypeId.HYBRID,
 navigationControl: true,
 zoom: 2
});

Of course, that map won’t have any markers on it until you plant them yourself. Planting a marker, though, requires knowing the GPS coordinates of each visitor’s hometown. Probably not something most people know. So, how can we geocode (i.e., convert) each visitor’s city, state, and country (that he or she inputted via the Guestmap’s web-based form) into a latitude and longitude?

Well, the Google Maps API does provide a JavaScript geocoding service, but using that service would mean that every visitor’s hometown would need to be geocoded every time someone pulls up the guestmap by that someone’s own browser. Certainly not the most efficient approach. Plus, Google appears to limit the number of geocoding requests a browser can make in a short period of time, the result of which is that some visitors wouldn’t end up marked on the map. (We, um, learned that the hard way.) Better to do the geocoding server-side and remember each visitor’s coordinates so that we needn’t query for them again.

As luck would have it, Google also provides a web service with which you can do exactly that. Via Google’s Geocoding API can you ask for the GPS coordinates of some location (e.g., 33 Oxford Street, Cambridge, Massachusetts) with code like the below:

$o = json_decode(file_get_contents("http://maps.googleapis.com/maps/api/geocode/json?address=33+Oxford+Street,+Cambridge,+Massachusetts+02138&sensor=false"));

I called json_encode in the above because what’s returned is a JSON object like the below:

{
  "status": "OK",
  "results": [ {
    "types": [ "street_address" ],
    "formatted_address": "33 Oxford St, Cambridge, MA 02138, USA",
    "address_components": [ {
      "long_name": "33",
      "short_name": "33",
      "types": [ "street_number" ]
    }, {
      "long_name": "Oxford St",
      "short_name": "Oxford St",
      "types": [ "route" ]
    }, {
      "long_name": "Cambridge",
      "short_name": "Cambridge",
      "types": [ "locality", "political" ]
    }, {
      "long_name": "Cambridge",
      "short_name": "Cambridge",
      "types": [ "administrative_area_level_3", "political" ]
    }, {
      "long_name": "Middlesex",
      "short_name": "Middlesex",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "Massachusetts",
      "short_name": "MA",
      "types": [ "administrative_area_level_1", "political" ]
    }, {
      "long_name": "United States",
      "short_name": "US",
      "types": [ "country", "political" ]
    }, {
      "long_name": "02138",
      "short_name": "02138",
      "types": [ "postal_code" ]
    } ],
    "geometry": {
      "location": {
        "lat": 42.3787770,
        "lng": -71.1168170
      },
      "location_type": "ROOFTOP",
      "viewport": {
        "southwest": {
          "lat": 42.3756294,
          "lng": -71.1199646
        },
        "northeast": {
          "lat": 42.3819246,
          "lng": -71.1136694
        }
      }
    }
  } ]
}

And so we can get at the GPS coordinates we want with code like the below:

$lat = $o->results[0]->geometry->location->lat;
$lng = $o->results[0]->geometry->location->lng;

And we can then store those coordinates somewhere. (We happen to use the Google Data Protocol in order to write them back to our own Google Spreadsheet in a column that the web-based form itself doesn’t touch.)

With all those pieces now assembled, we render the map itself with code like the below, wherein CSV is the URL of our Google Spreadsheet:

<!DOCTYPE html "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" style="height: 100%;">
  <head>

    <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.8.1/build/reset-fonts/reset-fonts.css" />

    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>

    <script type="text/javascript" src="http://yui.yahooapis.com/2.8.1/build/yahoo-dom-event/yahoo-dom-event.js"></script>
    <script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>
    <script type="text/javascript">
    // <![CDATA[

        YAHOO.util.Event.onDOMReady(function() {

            var gmap = new google.maps.Map(document.getElementById("map"), {
             center: new google.maps.LatLng(42.37411257777324, -71.11905097961426),
             disableDefaultUI: true,
             mapTypeControl: false,
             mapTypeId: google.maps.MapTypeId.HYBRID,
             navigationControl: true,
             zoom: 2
            });

            // resize as needed
            YAHOO.util.Event.addListener(window, "resize", function() {
                if (typeof(gmap) !== "undefined")
                {
                    google.maps.event.trigger(gmap, "resize");
                }
            });

            // prepare infowindow
            var infowindow = new google.maps.InfoWindow();

            <?

                // get subscribers
                $rows = array();
                $handle = fopen(CSV, "r");
                fgetcsv($handle);
                while ($row = fgetcsv($handle))
                    $rows[] = $row;
                fclose($handle);

            ?>

            <? foreach ($rows as $row): ?>

                <? if (!@$row[9] || $row[12]) continue; ?>

                var marker = new google.maps.Marker({
                 map: gmap,
                 position: new google.maps.LatLng(<?= $row[10] ?>, <?= $row[11] ?>)
                });
                google.maps.event.addListener(marker, "click", function(e) {
                    infowindow.setContent("<div style='width: 240px;'><b>" + <?= json_encode(CapitzeLastName($row[1])) ?> + "</b>" + "<br />" + <?= json_encode($row[9]) ?> + "</div>");
                    infowindow.open(gmap, this);
                });

            <? endforeach ?>

        });

    // ]]>
    </script>

    <title>Map</title>

  </head>

  <body class="yui-skin-sam" style="height: 100%;">
    <div id="map" style="height: 100%; width: 100%;"></div>
  </body>
</html>

The irony is that at least once per day I think to myself, "Oh that would make for an interesting post." Clearly my problem is with execution! Unfortunately, sleep usually trumps execution. Suffice it to say I don’t have a diary.

Those who follow cs50.tv (and its sister sites cs75.tv, cs259.tv, and computerscience1.tv) may have noticed some downtime over the past several days. The server on which cs50.tv lives (nay, lived) is actually a pretty decent machine: a Dell PowerEdge 2950 III with two quad-core Xeon CPUs, 32GB of RAM, and 6TB of disk space. Most everything in the machine that can be redundant is redundant: it has two PSUs so that if one of them (or the electrical circuit to which it’s connected) dies, the other can take over the load; 6 HDDs in a RAID6 configuration, which means that as many as 2 HDDs can die without downtime or loss of data, and 2 NICs so that if one of them (or the switch to which it’s connected) dies, we can move the other to our second switch. Because they have moving parts, PSUs and HDDs are among the few components in modern computers that tend to fail these days, so these redundancies help mitigate that risk.

Unfortunately, a server tends to have just one backplane, a circuit board into which important stuff connects (namely, in our case, HDDs). And that’s what keeled over this past Wednesday, for no particular reason. The first symptoms were that the VMs that lived on this box became suddenly unresponsive (i.e., cs50.tv "went down"); on at least one of their consoles was mention of "I/O errors," which generally mean RAM or disk problems. We thus proceeded to look at the server’s own console, which boasted, let’s call it, a red screen of death. (The server was running VMware ESX 4.0, whose screen apparently turns red when really bad things happen.) We thus rebooted the whole server and booted into the RAID controller’s configuration utility (a tool with which you configure RAID6 and such), which informed us that disks 1 and 2 were "Failed" and, get this, disk 0 was "Missing." I’ll admit I wondered for a moment who had stolen my hard drive.

Unfortunately, reseating the hard drives achieved nothing, nor did rebooting several times. It was time to call Dell. We were fortunate to reach an excellent technician (thanks, Fadi!) who concurred that the problem was most likely the server’s backplane, the "Failed" disks, and/or the RAID controller (in descending order of probability).

Within 24 hours, Dell was on site. (Thanks, Tom!) Sure enough, we were in need of a new backplane. Once installed, all 6 of our HDDs started blinking green. (Green is good.) Drives 1 and 2 now appeared as "Foreign," which meant they were functional but unrecognized, presumably the result of the failed backplane. Thanks to RAID6, no matter: we imported the drives back into the array, and we were back in business.

Well, almost. Unfortunately, on its way out the door, the backplane had taken some of our data with it: an assortment of files and directories were either corrupted or lost. And so, despite all our redundancies, we had to restore from backups anyway. (Thank you, Mr. Murphy.)

If there’s a silver lining in this particular episode, it’s that we used it as an opportunity to upgrade (almost) all of CS50’s servers to ESXi 4.1 in one fell swoop (which involved wiping some drives anyway), and we also transitioned cs50.tv from lighttpd to nginx, the latter of which, we’re hoping, will perform a bit better under load.

This is CS50, behind the scenes.

Spot it yet?

djm

CS50 has a suite of apps that (with a few exceptions) share a common navigation header. Note the common drop-down in the top-right corner of HarvardEnergy, HarvardMaps, HarvardNews, HarvardTweets, and Shuttleboy. (Hm. Which of those names is not like the others?) Up until recently, that common header was the result of a whole lot of copy-paste, which was getting increasingly annoying to maintain, and so I decided to factor it out (like I probably should have from the get-go). Within the body element of each app’s index.php now is, essentially, a line like:

require("/local/path/to/common.php");

For purely aesthetic reasons, though, I wanted the links to the left of the drop-down to be crimson (#600) and the links to the right of the drop-down to be black (except for the "50"). All of those stylizations are easy to do with inline CSS, as with the below:

<a style="color: #600;">API</a>

However, I also wanted the links to be underlined only when hovered over. Half of that effect is easily achieved with similarly inline CSS:

<a style="color: #600; text-decoration: none;">API</a>

Unfortunately, a pseudo-selector like :hover can’t be implemented in this same fashion. (See Stack Overflow for a nice discussion.) Of course, it’s easy to include in an internal (or external) stylesheet:

<style type="text/css">
<!--

a:hover
{
    text-decoration: underline;
}

-->
</style>

But, for a page to be considered valid XHTML, any style elements must be children of a page’s head, not body, element. The implication is that I’d need to put that CSS inside a second file (e.g., style.php) and then add, within the head element of each app’s index.php file, a line like:

require("/local/path/to/style.php");

Or I could let HTTP grab that CSS by adding, within the head element of each app’s index.php file, a line like

<link href="/remote/path/to/style.css" rel="stylesheet" type="text/css" />

whereby style.css contains only the CSS, not the XHTML tags. I was annoyed by both solutions, though, since my goal was to simplify, ideally by centralizing all common code in just one file.

It occurred to me that a lot of JavaScript libraries support "lazy-loading" of CSS rules, whereby you can incorporate additional CSS rules into a page via JavaScript (cf. YUI 2’s Get Utility), but I didn’t want to have to include yet another JavaScript library in every app just to underline some links. Fortunately, you can re-create that same functionality yourself quite easily! Inside of common.php, I essentially included the JavaScript below:

<script type="text/javascript">
// <![CDATA[

// wrap code in an anonymous function so that
// variable (e) doesn't conflict with variables defined elsewhere
(function() {

    // create style element
    var e = document.createElement("style");
    // add attribute to element
    e.type = "text/css";

    // define CSS rules in a text node
    var t = document.createTextNode("div#apps a { text-decoration: none; } div#apps a:hover { text-decoration: underline; }");

    // make text node a child of element
    e.appendChild(t);

    // make element a child of page's head element
    document.getElementsByTagName("head")[0].appendChild(e);

})();

// ]]>
</script>

As you may have gathered from its comments, that code instructs a browser to insert this node inside a page’s head:

<style type="text/css">
<!--

div#apps a
{
    text-decoration: none;
}

div#apps a:hover
{
    text-decoration: underline;
}

-->
</style>

(I added some whitespace to the above for clarity.) Incidentally, the XHTML in common.php is wrapped with

<div id="apps">

and

</div>

to ensure that my CSS rules are applied only to the elements within. Ironically, it took me longer to write this post than it did to actually code that solution! But a neat application of JavaScript DOM functions that offers some insight into how libraries like YUI achieve their own magic!

djm

Perhaps needless to say, someone forgot to interpolate his or her variables.

djm

Sorry, no account number or PIN in the screenshot!

See the bug? It’s pretty subtle at first. But it boils down to a difference in opinion with BoA as to what "today" means.

We’ll leave it to the reader (that’s you) to noodle over why this bug happened around the time it did.

djm

See it yet?

Indeed, the price of those Kickers should probably be $6.30, not $6.3. Of course, as floating-point values, the two are arguably the same (imprecision of floats aside), but most people prefer their money formatted to two decimal places.

How to fix? Without seeing the code, it’s hard to say for sure, but odds are the bug relates to improper use of a function like printf, which allows programmers to format numbers as they see fit. For instance, if f is a float, it can be formatted like money with printf in C as follows:

printf("%.2f", f);

The .2 tells printf to format the float to two decimal places. Of course, the site’s probably not implemented in C (in which much of CS50 is taught). In fact, a quick glance at the site’s URLs suggests it’s implemented in Java (c.f. http://www.dominos.com/home/index.jsp) using JSP, which has its own version of printf as well as alternatives, including java.text.NumberFormat, java.text.DecimalFormat, and java.util.Currency. Of course, because of the imprecision of floats, they really shouldn’t be used for money at all, unless Domino’s is willing to lose (lots of) pennies over time. Better to store prices as cents or use some fixed-point data type (e.g., MySQL offers a DECIMAL data type that might help in this case).

Whatever the language or database in use, though, someone screwed up. And it shouldn’t have been hard to fix! I suppose replacing Kickers with Chocolate Lava Crunch Cakes was, as they say, a workaround.

djm

But that ends tonight. We’re finally crossing the first post off the to-do list. Rather than aspire to be special, though, why don’t we aim a bit lower and just explain the intent of this blog.

I happen to teach a course (CS50) at Harvard College as well as, like Dan, a few at Harvard Extension School, and, most every day of the week, it seems, some interesting technical question or topic comes up that’s fun (for us, at least) to discuss. Given that both of us learned most everything we know from others, we thought it’d be fun (and only fair) to have some of those discussions a bit more publicly in hopes that others might enjoy learning what we’ve learned from others.

Officially, this blog is "on the intellectual enterprises of computer science and the art of programming." In the interests of full disclosure, we stole that from CS50’s own catalogue description. But it would appear to give us license to discuss most anything of technical interest, so that’s why we chose it. Admittedly, it’s probably misleadingly lofty, since I wouldn’t be surprised if our next post has a lolcat. But, overall, we do aspire for this blog to be educational (though occasionally silly) in nature. So rather than just comment on technical things, we’ll try to explain them along the way.

In the interests of self-defense, I should say that both Dan and I used to cringe anytime we heard the word "blog," since we didn’t quite understand why we’d want to read on a website what other people, oh, had for breakfast. But these things seem here to stay, and a blog seems an apt forum to share technical thoughts publicly. So thus was born the CS50 Blog.

Actually, originally, this blog was to be about funny bugs we’ve encountered in real life, so the next few posts may be a bit screenshot-heavy as we clear out our backlog. Rest assured they have nothing to do with the intellectual enterprises of computer science.

Anyhow, Dan tells me he had yogurt + granola + nuts + berries for breakfast. I had lunch.

This is the CS50 Blog.

djm