"Example of routed travel distances used on walk score."

If Paul Butler can create pieces of art out of playing with data, then I can… aspire to try. Having gotten access to a firehose of data, I set out to play with it over the weekend from a local coffee shop.

With some analysis, exploration, and photoshop, I ended up telling a story about the city of Seattle, the two tech giants established in the city, and the local public transit system. Below is the resulting map of how far one can get via public transit and walking from the centers of Amazon (left) and Microsoft (right) tech headquarters within 25 minutes. It takes a minute to get from one outline to the next. The visualization is as if the the entire area was a pond, and one was to drop rocks at the locations of transit stops, at the time one can arrive there. The resulting ripples fill in the space at walking speed.

"Amazon vs. Microsoft transit time map: 25 minutes"

It’s pretty clear that for a given company, having a reasonable commute will dictate exactly what side of the Lake Washington one would have to reside on. It’s also interesting to see how parts of Kirkland are outright inaccessable, while sections of Bellevue and Queen Anne are underserved. Of course this map doesn’t tell the complete story, as both Amazon and Microsoft also offer private shuttle services, and Microsoft has many sattellite offices throughout the city. This has been brought up in the comments on Reddit; through by far the most upvoted contribution was this derivative piece of art.

"Flaming guy catching a kicked ball."

All of the underlying data is available via Travel Time API and can be used to make the same kind of travel maps. Walk Score already uses this technology for finding hotels closest to multiple meetings, apartments closest to a particular location (school or work), and travel maps. Some other interesting ideas that I would like to see someone do in the future would be around mobile locations that quit making assumptions that I can jump across river/highway/barriers to get to their local deals. Programmable Web is likely the place to track API mashups for just such applications.

Update: there has been an addition to drive time api examples to include a rush-hour drive time estimates mode. Neat!

Drive Time API example

No related posts.

To be fair, U Waterloo CS students also install Doom-like shooters on plenty of “foreign computer architectures”.

Jokes aside, Computer Science vs. Computer Engineering is often a tough choice to make, but here are some ideas for figuring out the subtle differences between those programs.

Related posts:

1. Choosing between Computer Science and Computer Engineering
2. Chemistry for Software Engineers

New bell ad: stop the Belligerrence; from @ubbtor

UBB (Usage Based Billing / “Metered Internet”) is a hot topic right now, with many Canadians angry over the failures of CRTC and the subsequent forced price hike of the internet, even when using small competing ISPs.

There were arguments about online games (Steam):

Thanks to UBB, I can no longer use Steam in Canada thanks to your ridiculously low caps provided by my ISP. This is total bullshit as I am now forced to use a brick and mortar store to purchase my pc titles rather than committing a little under half of my bandwidth to downloading one new game title.

There were arguments about keeping in touch with one’s family:

I’m in the US military and my wife’s Canadian. During my last deployment, one of the easiest ways for her and I to keep in contact while she finished up her University degree in Ontario was webcam. She had Roger’s internet and they’ve had this bullshit implemented for some time. So, she had some crap 25 Gb max rate and we hit that most every goddamn month just trying to stay in contact. (Yes, in hindsight we should have switched and whatever. That was then, this is now)

And of course many arguments over economic impact on consumers:

I’m about to lose an extra $200 a year just because some large corporate giant says so: Recently, Bell Canada has had the Canadian Radio-television Telecommunications Commission (CRTC) make a decision that aims to take more money out of the pockets of Canadian consumers and ruin online services that are competitors to Bell Canada.

A quick video intro to what’s happening: Strombo Talks About The Impending Metered Internet

And many many more arguments about why metered internet is a bad idea. Here, I’ll talk about how Usage Based Billing will disrupt (in a bad way) Computer Science education, innovation, and as a result, the software industry.

– Coffee shops (traditionally hubs for exchange of ideas and birth place of innovations) will no longer be able to offer free wifi, in a fear of punitive per-GB charges. The stereotypical image of the next Silicon Valley entrepreneur starting a company out of a coffee shop will vanish, and with that a pool of innovation.

– Students will be limited in exposure to new technology. Proposed 25 GB download caps are just too small. Software downloads can take up a large chunk of the allowed quota: iPhone SDK is 3.5 GB, the latest version of Linux (Ubuntu) is 0.7 GB (plus all of the updates!). Tinkering with new types of operating systems and making applications for one’s phone is the inspiration that’s necessary to get students interested in the pursuit of technological education, but such can easily hit the allowed bandwidth cap, and will likely be discouraged in low-income households.

– Online learning resources will take a hit. TED.com is an amazing resource that delivers ideas that are persuasive, ingenious, inspiring, and informative. Of course video streams quickly add up to the bandwidth caps. Schools have just started moving towards the internet to better teach the material, but now rich media (videos, interactive applications, etc) will likely be reconsidered, in other to keep the resources equally available to all students.

– The Cloud is the bleeding edge of technological innovation. As companies like Amazon, Google, and Microsoft innovate new ways of thinking about communications, data, and computer resources; Canadians are getting locked just to their local machines. If we can’t backup our data (Dropbox) and stream music (Pandora), then we are not in an environment where we can create such companies.

– Limit of research with large datasets. The Netflix Prize is the first that comes to mind where open experimenting lead to development of better technology. Twitter offers even larger data streams, offering insights into everything from geopolitics to the stock market. A Canadian example would be the the Goldcorp Challenge, when in 2000 Goldcorp has released its geological data to the world in an attempt to save a gold mine in Red Lake, Ontario from closing. Many submissions came in, including those from students. The mine ended up finding over $6 billion (that’s billion with a B) worth of gold. Open access to large datasets has clear impact on innovation that results in new technology, insights, and creation of resources. There’s more data than ever available on the internet now, but exploration of such will be discouraged with metered bandwidth fees.

– Similar to research on large data sets, contributing to distributed scientific research will become discouraged as well. Folding@home is a distributed computing platform, researching diseases “such as Alzheimer’s, ALS, Huntington’s, Parkinson’s disease, and many Cancers”. Bandwidth caps will discourage contributions to this and similar research models.

The initial counter argument might be: “just pay more to get as much access as before”, but this is not that simple. Price-conscious University students will start second-guessing if they should explore their innovative ideas, when data consumption is met with punitive fees. High schools students often have limited say over the connections in their household, and low-income families might get into a conflict between providing an open learning environment and paying extra costs.

It’s clear that telecommunications companies such as Bell and Rogers fear technological innovations that was brought by Skype, YouTube, and Netflix; but pricing such innovations out of the market will have a far greater impact on the environment that allowed for such innovations (and many many more) to be developed in the first place.

What can you do? http://www.reddit.com/r/canada/ is our battleground; a lot of information and ideas are posted and discussed there. Excellent list of actions one I take. Sign the petition. Attend a rally in downtown Toronto. Let everybody else know how this change will affect them. Join the discussions: @OpenMedia_ca, @ubbtor.

Related posts:

1. Wishing for cake through fibre optic cables

Cartoon by Jeffrey Koterba

There still seems to be a lot of misinformation about the outsourcing of technical jobs. Strictly speaking, yes, some jobs are either moved or created in India, China, or a number of other places with people and relevant skills. At this point a distinction should be made between “moved” and “created” — both India and China are huge and rapid growing markets, and a lot of international companies want to expand and set up shop in those locations. Such activities create _new_ jobs, but I feel that some people lump it all together with the dreaded “moved” (true outsource) jobs when they see sensationalist headlines in newspapers.

So what kind of technical jobs move offshore? The general trend is “cost saving”, so it’s typically low-value/low-skill positions; call centres are popular, but also other levels of “support” positions. While support also falls into an incredibly broad definition of IT, lets assume that a pursuit of a degree leaves us with a higher level of skills. To be fair, when the management doesn’t have the understanding or the budget for a high quality development team, the entire department could be let go. If such was a software company, that company would have already been dead.

I don’t feel that intellectual outsourcing is about exploiting developing countries for cheap labour (as oppose to this being the case for physical labour; stereotypically for shoe factories). It’s cheap pay for either low-skills at market rate, or cheap pay for poor quality “high-skills”. There are high quality high-skill software developers in all of those countries — they migrate to where the high paying jobs are (or start their own companies in their home countries). It doesn’t seem like anyone is selling themselves short to get the jobs. It’s just that in today’s global economy, a failing company/department/project could make that final leap for the cheap options rather than folding right away. As I see it, those jobs would have been lost regardless of the outsource option being available or not.

Things might change in the future, but for now there are a number of software jobs that are guaranteed to stay local (for some definition of “local”) and high-paying:

Startup jobs — it’s you and 4 other guys building the next Facebook (at $50 billion valuation, we are partying like it’s 1999 again). Since just over a year ago, my favourite startup job posting site Startuply has more than doubled the number of companies that are _currently_ looking to fill positions. There is simply no room to cheap out on innovation. The same is actually true for any technology company that is trying to develop new products and lead its industry. Besides, it’s now easier than ever to start your own software company.

Just about any technology other than Java — offshore shops that aim to attract outsourcing business specialize in the lowest common denominator of the software industry, which happens to be Java. Ability to pick up on other languages opens up many new opportunities that might be immune to any outsourcing for years to come. This is especially true of emerging technologies, more so in the mobile space. Acquiring the skills, knowledge, and hardware to develop for the newest release of, for example, an iPad simply has very different economics than doing generic projects in the same technology, year after year.

High reliability jobs — there are just a few jobs that require one to carry a pager; they include doctors and software development engineers that make 6 figures per year. What if one of Amazon’s servers crashes and some pages become inaccessible at 4am? There’s absolutely no way that the responsibility to fix things _right now_ will fall on anyone who’s not getting paid high enough to get out of bed and start coding in the middle of the night. When the company’s online store sells up to 158 items per second (yeah, second), _any_ downtime is incredibly more expensive than $100,000+ salaries.

Along the same high-reliability/high-skill line of thought, Google has recently given out a 10%+ raise to the entire company (and some Engineers still leave to work for Facebook instead).

Having worked at both startups and major software corporations, it has always been the case that _everybody_ is always looking to hire. Many will even pay employee referral bonuses for simply pointing towards a candidate that accepts a job. An important note, of course, is that there’s a certain skill baseline that needs to be crossed to get hired. This is a matter of having the education, experience, and drive for high quality results. Above this quality requirement, there are more job openings than people able to fill them.

So sure, there’s some outsourcing happening in the industry, but the jobs that we really want — they have a pretty good outlook. In fact, the salaries and the hiring outlooks are so good, that CareerCast has named Software Engineer to be the best career for 2011 (up from 2nd place last year).

P.S. “Software Engineer” is typically used in an American definition, where the term “Engineer” is unregulated. Meaning that any of Computer Science / Software Engineering / Computer Engineering degrees could lead to the same job. Feel free to pick a major that suits you best.

Related posts:

1. More on the future of Computer Science Careers – outlook still promising
2. Computer Science jobs for University Students

I think I have a tip for making this choice. Unfortunately I came to understand it only a few years after entering the University myself. Looking back, this would likely have helped me make a better choice to begin with (and saved me the trouble of figuring out the differences years later, and going through a complicate process of changing majors). The approach is simple, yet time consuming; though I guarantee that this will be faster than spending a year (or two!) switching majors.

Sample checklist for Computer Science vs. Computer Engineering courses

1. Go to the website for your top University of choice. (That decision is a topic for another discussion.)
2. Find a course calendar for two programs you want to compare.
3. Ignore the courses that overlap in content, and write out all other notable courses. Depending on the level of detail you want to go into, you can pull in all of the course combinations, including minors and options, though a less detailed bird-view might be enough to steer one towards their preference.
4. Assign positive points to courses of interest and negative points to required courses that you would rather avoid.
5. Sum up the results.

Originally, I was thinking of comparing Computer Science to Software Engineering, but there was too much overlap to make a good example. Though this previous article could be of interest — The spirit of Software Engineering.

I think it should also be noted that some of the Computer Engineering courses are accessible to Computer Science students via the Digital Hardware Option, at least at the University of Waterloo, which was taken as basis for this comparison (on the grounds of me attending that University).

Here I’ve discovered that ECE courses put emphasis on energies inside the systems — power management, thermodynamics, magnetic fields. Also, I’m not a fan of required Chemistry course. Personally, I’m not that interested in such level of detail, especially when the area of application leaves computers and gets into power-plant scales of energy transfers. The better circuit design courses are available via an option, so the key question becomes: how fascinating are those 4th year courses?

Computer Science is interesting in a way that it has much more flexibility. This allows to both avoid dull subjects and to pick up courses in the areas of interest. On the flip side, there are certain elective requirements that need to be fulfilled, such as the courses from the so-called “arts groups”.

Personally, I’m really interested in getting to Compilers via the CS route; even if that doesn’t earn me an Iron Ring.

Related posts:

1. Computer Science at Waterloo: the new Scheme of things
2. Computer Science at Ryerson University
3. Online introduction to Computer Science from Stanford Engineering (for free!)

xkcd's Interesting Life

There is this perception that a typical programming job involves being locked away in a grey cubicle at some LargeCorp Inc., buried in mediocre tasks, and rarely seeing the light of day. While this grim illustration is not exactly the case (or I was simply fortunate enough to avoid such fate), there is some motive to such stereotype. So just as a classic xkcd suggests:

“Quick, fashion a climbing harness out of cat-6 cable and follow me down.”

We’ll be going on an adventure through on-site programming positions that are anything but typical.

Inside a shipping container, in the middle of an ocean…

Original photo by cfarivar, reused from another boat post

The first location comes from the experiences of Reto Meier, who’s job took him out to sea — When Offshoring Your Development Team Means Buying a Boat.

By the time you realise their office isn’t exactly in Portland, you’re sitting in a 10′ sea container floating in the middle of the Indian Ocean pressing a button every 108 minutes.

It’s a pretty intense story, and is worth a read.

South pole

Original photo by µµ

How about the South Pole? The IceCube Neutrino Observatory has a job posting — just some requirements: Linux, security, Perl/Python/Java, and to “deploy to the South Pole in early October for 12-13 months with no possibility of leaving during the winter months from mid-February to mid-October”.

To be fair, the science is pretty amazing, and some people would jump at an opportunity to get close to this kind of research.

Race around the world

Audi preparing for Pikes Peak Hill International Climb

What about debugging C# “memory leaks” (or as close as one can get to such in a managed memory language) in Mojave desert, during DARPA’s Grand Challenge?

Or racing to the clouds with an autonomous Audi in a Pikes Peak International Hill Climb?

As Universities and research groups push autonomous cars to their limits, various races and challenges will take the technical teams throughout the world.

Out of this world

STS-127 mission, via TopTechWriter.US

The sky isn’t even the limit. Julie Payette, a Canadian Computer Engineer, has “conducted research in computer systems, natural language processing and automatic speech recognition”. She has then gone to space. Twice.

In 1999 Julie has gone on STS-096 as a crewmember to operate the Canadarm robotic arm. In 2009 she has gone on STS-127 as the flight engineer, operating robotic technology.

Scott Maxwell has gone from writing software to driving the Mars rover. Not really on-site, but “it’s the greatest job on two planets.”

No related posts.

Martin Kess, one of the founding members of CompSci.ca, has been busy working on a fantastic looking game for the BlackBerry platform: Addictive Tower Defense. This isn’t the first of the quality games to come from the CompSci.ca community members, but it certainly raises the bar once again.

I’ve been playing the beta version of the game for a while, and I am blown away by the level of detail put into this game, just by a couple of indie developers. So I caught up with Martin to talk about the game.

What inspired you to make a commercial game, while still being a student?

Well, my main goal in life is to have a wikipedia page that doesn’t get deleted within 30 minutes of its creation.

I was working at Sybase, and Mike, my boss at the time, decided that in his spare time he wanted to do a project on the BlackBerry, and so we started throwing game ideas back and forth. The real inspiration thought were stories of John Carmack and John Romero (co-founders of id Software and yes, those are links to their Wikipedia pages) putting everything aside to make awesome video games.

And why commercial? Well, I thought that there was an off chance that I could retire a little bit of this quickly growing mountain of student debt.

Currently attending the same school with Martin, I had to ask how that played in with making games.

How do you feel that your 5 years of doing CS at Waterloo impacted the development of the game?

Well, co-op especially has turned me into a very productive programmer, so I think that I am pretty good at cranking out a lot of code. To be clear though — I mostly worked on gameplay and tools, Mike did the engine.

CS241 (Foundations of Sequential Programs, also known as “baby compilers”) was pretty relevant — I wrote a little scripting language to define the levels.

CS341 (Algorithms) and CS466 (Advanced algorithms) taught lots about algorithms , which was probably the most useful.

Math, being a touchy subject when it comes to video game development, also came into question:

What about the Math courses, did any of them come in useful in either development of the game or balancing out the gameplay?

The math was actually pretty straightforward — it wasn’t much beyond Calculus 1. Though MATH 239 (Combinatorial Analysis and Graph Theory) also came in handy. We used just the Dijkstra’s algorithm to find the shortest path for the enemies to take, but we also made a nifty flood-fill algorithm to update the shortest paths quickly when a tower got built. It was too expensive to rebuild the shortest paths every time a tower was placed.

Table of upgrades

Anything else?

English classes were very important! I did most of the writing on the site and in the game.

So to wrap this up… What’s next?

Next? *laughs* Next I have to rock this CS488 (Introduction to Computer Graphics) project.

We’re going to keep releasing maps for the game, and likely we’ll start working on Game #2, although we aren’t sure what that will be, and will probably take a break until Christmas at least to get caught up on sleep and other such things.

Tower Defense gameplay

So there it is. A story about indie game development, a Computer Science student in debt, and one of the most polished mobile games I had a chance to play. Check out the game’s website for more details — AddictiveTowerDefense.com.

Related posts:

1. Sample requirements to break into the game industry
2. Treating video games as software

Software jobs posted to U Waterloo

I’m often asked about available jobs for Computer Science students. More so in light of the recent economic downturn. An interesting point to reflect against is a benchmark from 2007 when “950 Computer Science related jobs” were advertised to the University of Waterloo students. This Fall’s 782 seems lower than pre-economic-meltdown numbers, but a lot of large corporations now group multiple positions into single postings. RIM alone has 100+ student positions advertised over just 3 posts. So it’s difficult to compare those numbers accurately.

Other technology power-corporations (Google, Microsoft, Amazon) continue employing students as well.

A new trend I’m observing, since 2 years ago, is that there is a substantial increase of job postings from much smaller start-up companies, as well as not-so-small ex-startups (Facebook in particular had a few different positions advertised).

Economic hardship breeds innovation. It puts large corporations into a compromised position, and that opens up an opportunity for small, agile, smart, inexpensive, but super-productive startups to come into play and compete with established corporations. Unconvinced? Today Startuply is listing “1375 jobs from 1293 startups”, and I’ve noticed some of them cross-listed on Waterloo’s job website, specifically looking for students. Quality software always needs to be build, and Computer Science students who do their craft well are “pretty much guaranteed an interesting job, that actually does something useful to society”.

Alternatively, there is always grad school.

Which I might or might not be considering. Hint: I am. Though with so much opportunity for interesting work, it will be a difficult decision to make.

Related posts:

1. Computer Science enrollment trends
2. (Important) Computer Science jobs are not outsourced
3. More on the future of Computer Science Careers – outlook still promising

"New CD browsers" on Flickr, by San Mateo County Library

Recently, a lot has been happening in the peer-2-peer space, at least from the legal perspective. The Pirate Bay trial, plus various torrent websites shutting down; TorrentFreak is an excellent news source for all that goodness. While a lot of people scream that RIAA/MPAA (and their international equivalents) are not keeping up with the times and technology, we are not really any closer to having an entertainment distribution system that takes advantage of available technology to make it convenient for consumers and one which fairly compensates the industry that creates the said content. So I propose starting a discussion, by presenting a scheme that pulls p2p technology and current copyright laws closer together.

There are two basic premises:

1. Music is purchased as a license for an individual to indefinitely enjoy a particular piece of entertainment.
2. It is permissible to lend someone a physical music CD, provided that it is original (such as store bought), and not a copy.

The latter point is what allows one to sell their used CDs (imagine that, there’s an actual business that does that, with 11 locations in Ontario — The Beat Goes On), borrow CDs from a public library (the Toronto Public Library has 59 838 titles (in multiple copies!) available), or to lend a CD to a friend (just try to outlaw that, and the general public will actually start taking interest in the application of copyright laws, and the next election).

The former point stipulates that a physical CD is simply a token, showing that the current holder has a license to enjoy the media; the recordings on the disk are simply a convenience factor.

So the natural step, at least as it seems to me, is to digitalize this license tokens, and let them be shared over p2p technology, as you would have shared a physical CD with a friend. Here’s what I have in mind:

Scheme illustration, put together with free icons from IconsPedia

All of the music is readily available for download (similar to current .torrent approach) and copies could be cached on your device (to save bandwidth), but it is not playable unless a license token is also available on the system. The license tokens will act as unique digital keys, which could be borrowed and released back into the cloud of p2p.

If one had purchased a set of license keys for the latest music album, but isn’t listening to this particular music items at the moment (school, work, sleep.. there are many reasons why ones entire music collection isn’t played 24/7), those keys are available to be given to someone else. Similarly, one could temporary take possession of someone else’s license, while such is available.

To prevent leechers, and make for a fairer sharing experience, a model similar of current private torrent trackers could be applied — those who contribute more keys will get priority status. Bonus points for contributing keys to high-demand or rare media. Maybe preference for own social graphs.

The net effect of automating request-play-release cycle is that a) sharing music will be completely legal, b) ease of use would be on par with current technology, and c) popular content (ones with more simultaneous key usages) will see a proportionally larger share of revenue. Just think back to days of mixtapes, and trading CDs with friends; except that it’s now done every 3 minute, through a wire, and you are friends with the world.

The caveat is that, yes, one would actually have to pay for some of the licenses to take part in this sharing community process. Though ones choice of which artists to support and generalizing the use of granted licenses should make it a fair deal. iTunes and Amazon’s MP3-Downloads show that a lot of people are still willing to pay for very specific music purchases online.

But now, given the fact that “pirates [are the] biggest music buyers”, via Ars, do we really have to implement such an explicit model?

Those who download “free” music from P2P networks are more likely to spend money on legit downloads than those who are squeaky clean, according to a new report out of Norway.

No related posts.

Byte Club TV is a webcast show profiling tech companies, typically around Toronto area. One of the shows producers, Kristan “Krispy” Uccello is fairly involved in Toronto’s tech community. I often see him at Rails Nite events; he’s a pretty cool guy.

It looks like a while ago they’ve visited and filmed at the Computer Science department of University of Toronto. This isn’t an opt-out to avoid writing up an actual overview of UofT’s Computer Science, but until then it’s still an interesting video to watch. Pretty shots of the glass CS building & Greg Wilson talking about the department, industry, and pink hippos.

Greg: “One of the things that I think the department has to do, is a better job at reaching out to people to get that message across; is that almost anybody who can program well is pretty much guaranteed an interesting job, that actually does something useful to society.”

Related posts:

1. Computer Science at Ryerson University
2. Computer Science jobs for University Students
3. Academic – X06: GameCamp (Toronto)