Rob Coup - the rest is missing

Fame and Fortune from Open Source Development

2008-06-29T22:21:00.009+12:00

In May I presented to students and staff from the University of Auckland as part of the Electrical and Computer Engineering seminar series. What was I talking about? Getting involved in open source software - why would you want to? how to get started? and what to expect once you're there.

| View | Upload your own

As usual, the slides would be a lot more helpful with some words to go along with it :) Basically I talked about my experiences with a number of open source projects, and particularly my work with the Dojo Toolkit.

Google's Summer of Code and Highly Open Participation programmes have been a fantastic source of new contributors to open source, and at Dojo we've been lucky to be involved for the last three years with Summer of Code. GHOP last year showed what a huge range of relatively easy things can be done to make an open source project better and just how many ways there are to get involved.

A few of us have been gently pushing for ECE to get software students to work on open source for their final-year undergraduate projects where it makes sense. This year there are two students adding new features to the Player/Stage robotics project, and it was interesting to talk to them after the seminar and find out how they'd been getting along and what issues they'd run into.

Key things out of all this:

projects want new contributors
the best way to get started is to do something
there are a pile of ways to get involved, of which coding is only one
what you get out is proportional to what you put in
anyone can do it, especially you :)

Web 2? Web 1? What does it mean?

2008-06-29T20:57:00.009+12:00

Back in March (yeah, I know) I got invited to do a presentation to a group of high school technology students at Avondale College. So, I decided to talk to about Web 2.0 and why it matters, and what the big deal is anyway. Specifically what the differences between 1.0 and 2.0, and what it practically means for students, as well a company like Koordinates.

| View | Upload your own

The slides are best when they go along with the talk, but hey :) I thought it went really well, and I got some really good questions.

I spend a few minutes afterwards talking more closely to one of the senior classes. The students were working on the process of technology development projects, and we talked about how we work at Koordinates. I was a little disappointed (but not surprised) that the waterfall model is the standard still being taught, and nobody knew what agile was. At Koordinates we do cycles of 1-2 weeks for decent sized work, and we're releasing code every day or two for bug fixes and minor tweaks. It is much easier for us to manage, design, code, test, and deploy many small changes than massive changes that affect everything. In fact, the more changes that stack up the itchier we get to push it out to production. To support all this we've built and tied together a bunch of tools so it is damn simple to push changes between the different environments in controlled ways.

The talk was part of FutureInTech, a programme run by the Institute of Professional Engineers, aiming to get students of all ages interested in science and technology as career options by involving young engineers. The regional facilitators do a really great job getting all sorts of people into schools to talk and work on projects. If you're interested in talking or getting someone into your school, get in touch.

Dojo from the Air

2008-04-21T20:08:00.006+12:00

While I'm sitting here 35,000 feet above the Tasman I thought it'd be a good time to update you all on the goings on in the world of Dojo from my point of view.

1.1!

Dojo released 1.1 after a lot of hard work. The team worked really hard to get this release out, and it adds a lot of polish to 1.0 and a number of cool new features. Some quick snippets from me:

dojo.attr()

We now have a smooth & nice way to access attributes on DOM elements, normalising access much in the way dojo.style() does. And, just like dojo.style() can now, if you pass in an object as the name, it'll set a bunch of attributes for you in one go:


// get the current value of the "foo" attribute on a node
dojo.attr(dojo.byId("nodeId"), "foo");

// we can just pass the id:
dojo.attr("nodeId", "foo");

// use attr() to set the tab index
dojo.attr("nodeId", "tabindex", 3);

// set multiple values at once, including event handlers:
dojo.attr("buttonId", {
 "name": "myButton",
 "value": "123",
 "onclick": function(e){
     // an onclick handler connected via dojo.connect()
 }
});

api.dojotoolkit.org

Our API browser is back, newer and better. You can search and browse all the code in Dojo, see method arguments and examples, and dig into the code if necessary. And when you figure out a tip you can comment in the appropriate place to help others out. We think this is a fantastic addition, and I've been using it on a daily basis for a while now.

dojox.help

Not really part of 1.1, but happened about the same time. The talented Neil Roberts whipped up an awesome demo based on some suggestions from a dojo cookie. Basically it accesses the API site live and allows you to get method arguments and descriptions directly in Firebug. I often get some code semi-working and test and experiment using the Firebug console while I finalise it, so this is a real boon.

Let's have a go:

load this page in Firefox with Firebug enabled
in the Firebug console, type the following:
- dojo.require("dojox.help.console") // loads the help infrastructure
- dojo.attr // prints the method arguments for dojo.attr()
- dojo.attr.help() // prints the method description, examples, etc
revel in its coolness, dig in if you're using trunk, or wait patiently for it to appear in 1.2

Summer of Code

On Monday the Summer of Code students for 2008 will be announced. Dojo received a huge pile of great applications, and we had a really hard time evaluating and ranking them. Unfortunately the funding isn't infinite and neither is our mentoring capacity, so we've chosen what we think is a pretty exciting mix of projects. More to follow next week after the announcements!

The coding exercise we set went down pretty well. I suspect many of the students spent a lot more time on it than we asked, but there are some damn cool demos in there, which I'll try to show off over the next few weeks. Next year we'll try something a little more focussed, but its actually fairly tricky to think up the right balance.

Anyway, time to land now. Better be off. :)

Prepping for the Summer of Code

2008-03-18T18:58:00.004+13:00

So, it was announced today that Dojo will be part of the Google Summer of Code in 2008. w00t! For students, you get a modest amount of fortune (US$4500) and some degree of fame (TBA), and Dojo gets your nice shiny code. This is our third year and the second time I've been involved. Looking at the quality of some of the applications that didn't make it this year, I'm a little scared that we only just scraped in.

Since the programme was announced on 25 February, we've achieved quite a bit, so I thought I'd jot it down:

Figured out who's organising it: me, since I volunteered again ;)
Thought about what was good and bad from 2007 and 2006 and what we'd change
Write up a pile of documentation:

An overview of SoC and Dojo for mentors and students
A FAQ for students about what we and they should expect

Created our project ideas list - basically we need to list some projects we think are both interesting and achievable by a new student in a couple of months. This involved a huge brain dump from me, followed by trying to herd our contributors into reviewing it and adding their own ideas, notes, and corrections. We've got some pretty good stuff down, but I know there's more great ideas out there still.
Posted to the blog and our mailing lists looking for mentors and more ideas. We're trying to get 2 mentors per student this year, with one from the "core" Dojo team and someone else from the Dojo community who wants to get more involved - we have a lot of smart & talented people and we want to pull them closer.
Answered a pile of emails, forum posts, IRC, and IM conversations with students and potential mentors.
Wrote our application to Google. Only 175 out of 505 organisations have been accepted, so it needed to be good.
Wrote a coding test - shamelessly ripping off the idea from my good friends at SilverStripe we decided that our students should do something practical as part of their application too. Still some more tweaking to do there too.

That took until this morning. Phew. What's next?

Start matching enquiries to mentors
Hassle some more of the core committers to get involved
Figure out how we're going to deal with version control & ticketing for the SoC projects
Sketch out our mentor 'teams' and fill in any gaps
Decide how many projects we can handle
Keep on top of the emails and other enquiries

Student applications start flowing in on the 24th, then they need to be reviewed and voted on by the 11th. Then I can relax for a bit (unless I end up mentoring a project as well) and just make sure everyone is communicating and making progress. Until evaluation-time.

February in NZ Tech - KiwiFoo, Webstock, Amazon...

2008-02-22T01:16:00.005+13:00

So I'm both late and lazy getting around to my writeups!

KiwiFoo was awesome! So many interesting people, and everyone was open for sharing and discussing - the corporate walls were down for a couple of days. I was involved in some great sessions - both issues I'm passionate about - open government data, student internships, IE8, developing OSS communities; and stuff that was just plain interesting - Meraki, getting Internet to Puhoi library, blog ads. There was a bit of linux-ness (Toby trying to use my Macbook for his 3D augmented reality demo didn't go as smoothly as we'd hoped), and hundreds of chats with like-minded and interesting folk. The weather was great (even a bit of cricket on Saturday night), and the venue and food was top notch. Nat may have gotten us there, but Jenine made everything work, and we appreciate it!

Last week was Webstock, and 3 of us took the trip to Wellington. Simon Willison's session on OpenID was great - particularly how Sun has OpenID identities for each of its 34K employees. For SaaS applications, especially cross-company collaborative ones, this is fantastic. The authentication stays inside each person's 'home' company, but any service anywhere can easily utilise it, and thats just the first benefit. Microsoft: add an OpenID provider to Windows Server!

Lots of other memorable stuff, but Nigel Parker's 5 minutes on online privacy in social networks had lots of peoples eyebrows climbing up their foreheads. And as he said, for his daughter he's making decisions about whether and where she'll be online before she knows what online is. It's standard for me to drop names of people into Google to see what comes up and what I can find out about their backgrounds and involvements - and not just job applicants, but any business or personal contact.

Heading down to the Brewery Bar on the waterfront on Thursday evening for a few beers outside was great, especially with the background sounds of Phoenix Foundation's gig up the road. Thanks again to Mike, Tash, Siggy and the others for your hard work! And to the ProjectX and PlanHQ folks for hanging with us visitors :)

This morning I was at Mike Culver's presentation on Amazon Web Services put on by the UoA Centre for Software Innovation. A good mix of people who quickly grasped the issues. AWS makes it so easy for US businesses to get off the ground without investing huge amounts in infrastructure and the ops to make it work. The problem from this (and other) ends of the world is international bandwidth costs. As far as I'm concerned, the NZ government should be bending over backwards to get the likes of Amazon to set up here to help our software startups get up and running and growing (as should every other government, but they're not mine). Mike's talk in Wellington went well too, so at least he knows we're all enthusiastic.

Only downside: another mention from an international guest of how NZs broadband is sucky and expensive. As Rod said, this is our national reputation to global influencers. Not good.

Children, Lego, OLPC, and IP

2008-02-22T00:25:00.006+13:00

So, I stumbled across an absolutely fascinating article care of Nat's Radar roundup: Why We Banned Legos

Now, I grew up in a Lego-mad house. It's probably one reason why both my brother Mikey and I are working in engineering-related fields. For years and years we had Lego structures and contraptions erected around the house, and one of my very first memories is playing with Lego with my cousin Hamish. Mikey and I had wars about lots of stuff as we grew up, but and I'm sure more than one was about Lego - I'll have to ask Mum.

I read this article just after reading Tom Coates' review of OLPC, and that sparked some thoughts. The Legotown issues are about access to physical property - Lego bricks. I'd say western society in general feels that physical property is commonly understood with similar attitudes. Intellectual property (patents, trademarks, copyright, ownership of ideas & discoveries, the whole shebang...) is a whole different kettle of fish, and I'd say society as a whole barely grasps the concepts, is only starting to get to grips with it, let alone has common understanding and agreement.

OLPC is all about giving knowledge and the capability to learn to kids around the world. Great. But this is physical property (a cool green high-tech laptop) combined with intellectual property (information and access). If I was a kid I'd be pretty disenfranchised if the kid down the road knew everything about everything because she had an OLPC and I didn't. Much more so than if she had a cool toy and I didn't (hey, everyone grows up like that right?).

we were struck by the ways the children had come face-to-face with the frustration, anger, and hopelessness that come with being on the outside of power and privilege.

Francis Bacon once said "Knowledge is power", and he was right. Magnitudes more powerful than controlling physical property. Normally kids don't need to deal with IP issues, but they learn about physical property early. I think the OLPC will change that, especially if only a handful of kids in a community have one. And I suspect the frustration and anger will be bigger.

Now, don't get me wrong, I think OLPC is a fantastic scheme, I'm a supporter, and I've had plenty of lively discussions around it over the past couple of years. And was lucky enough to have a play with one in San Francisco in October!

But just something to ponder :)

Automated Dojo testing - DOH & Selenium-RC

2008-01-03T13:02:00.000+13:00

Before the 0.9 release, Dojo replaced its old unit-testing framework with a new one named D.O.H. that is miles nicer and cleaner. It'll run in both the browser & under Rhino, supports defferrables and is relatively easy to customise.

At One Track Mind we've been using Selenium-RC to drive unit-testing of JS code since Dojo 0.4 (of our stuff mostly, not Dojo itself) and it wasn't ever particularly clean. The basic idea is to run a suite of unit tests across a series of browser/version/OS combinations and collate the results. So you can find that some irrelevant little change b0rked Opera but only on Windows.

Selenium's API for writing tests (for JS stuff anyway) is pretty awful compared with DOH and we'd done ours in Python, which was then launched from Ant after the Dojo build was done. But what Selenium-RC does do well is fire up browsers on multiple operating systems, do magic proxying, and allow you to interrogate the pages.

A thread on dojo-interest this week sparked me back into looking at it again. I noticed that Selenium-RC now supports running from Javascript. (to be honest, the java bindings would have always worked but someone's actually spelt it out in small words now). Since the Dojo build system is now Javascript-based, why not make the Selenium test runner as well?

Plan:

Have a config file defining which browsers to launch, which machines they're on, and what tests to run.
Launch each browser via Selenium-RC
Run the tests via the normal DOH browser runner.
Use Selenium to extract the results from DOH.
Collate the results from the various browsers and produce something useful.

So, after a bit of toying I came up with this:

Drop seleniumRunner.js, seleniumRunner.config.js, seleniumRunner.sh (or the .bat if you're on Windows), and selenium-java-client-driver.jar into util/doh/ in your Dojo install.
Put selenium-server.jar on each test machine, then run java -jar selenium-server.jar -multiWindow so it listens for the browser-control messages.
Edit seleniumRunner.config.js and change browsers and rootUrl to match your setup. The rootUrl needs to be reachable from each test machine.
run ./seleniumRunner.sh seleniumRunner.config.js from util/doh/ on your workstation
It'll load the config, fire up the browsers on each machine, run the unit tests from Dojo core, and print the pass/fail/error stats for each.
Each browser is kicked off and monitored in a separate thread (not strictly necessary but too cool to resist doing).

Some issues I ran into:

unless I ran the selenium server in multiWindow mode Safari and Firefox would pop up Print dialogs (!?!) whenever the test page was loaded. But Safari never initialised the test page if it was in multiWindow mode. On OSX and Windows. gah.
Opera on OSX didn't set up the Selenium proxy properly (localhost:4444 for reference).
IE didn't like doing a dojo.connect() via the selenium javascript commands for some reason.

The next step is to improve the reporting. This will probably necessitate writing another browser runner that produces a JSON report (or changing the existing one). Modifying DOH to have an isComplete flag would make life simpler as well. I'd want to know which tests were run on each browser (since not all tests run everywhere), which ones passed and failed and had errors. A hunk of JSON which could be loaded into a Dojo Grid for exploration might make an interesting way of viewing the results.

Cool data in the browser with Dojo

2007-12-16T22:28:00.001+13:00

This is basically what I talked about on Saturday at Barcamp (which was heaps of fun and big props to Ludwig for organising it all!).

Cool data in the browser with Dojo

We're now at a place with Javascript development where we can handle structured data really well in the browser. And then we can start doing some cool stuff - grids, charts, maps, and so on. The Dojo toolkit calls it dojo.data.

Basically dojo.data is an abstraction - sitting between where the data comes from (tables, lists, xml, csv, json, Flickr...) and what you want to do with it (querying, widgets, graphs, grids, maps, ...).

Stores

A data store describes where the data comes from. This could be from in the HTML page (from inline lists or tables), from static sources (XML, CSV, JSON), dynamic sources (database-backed querying) or external APIs (like Flickr). Different stores can implement different sections of the dojo.data API depending on what the source supports - reading, writing, identities, and so on. Its pretty easy to create your own store implementation if none of the existing ones suit.

Data Widgets

Data widgets present data from a store. The idea is that we can create kick-ass widgets once that work with data from any source. The widgets interface to the data via the dojo.data APIs and then do cool things to present it. Data-driven widgets include form components like ComboBox, the tree, the grid, charting, image galleries, and so on. We'll just talk about a couple of them.

The Dojo Grid was released in September by SitePen, Mozilla, and others. It is the Javascript table to end all tables, with a truly impressive feature list - paging, sorting, filtering, subgrids, events, formatting, internationalisation, and the list goes on. Of course, the grid can consume data from a store, so we can have all these impressive features for CSV data, XML data, or anything else. SitePen has a two part tutorial on a few of the grid features, or you can check out the documentation (still in-progress) and the tests.

Dojo Charting provides vector-based charting in 2D, and now 3D thanks to the work of Kun Xi and Neil Joshi during this years Google Summer of Code. Underneath it uses dojox.gfx which leverages SVG, VML, Silverlight, or Canvas depending on the browser to draw the vector graphics. Charting is pretty new but already has an impressive set of features: lines, columns, areas, bars, pies, grids, axes, themes, ... see the tests for some examples. The dojo.data implementation in charting isn't fully complete yet but its still easy enough to get it working.

Demo Time

Enough talk - lets see all this in action. I've grabbed some 10-minute weather readings in CSV format from 18 Auckland stations for a 24 hour period in November from NIWA's climate database. Then built a simple data explorer where you can select a station in the table and see a line chart of the temperature and wind speed over the 24 hour period for that station. And there's a map, because that's what I do.

Play with the demo

You'll notice that there is no page-loading involved (AJAX goodness), you can sort the grid by clicking on the column headings, and that it's all very smooth and responsive. View the source and read the comments to see what is happening underneath.

Enjoy!