Your keynote at Edge of the Web is titled “The Wisdom of Community”. How do you define a community on the web?

I like to take the broadest possible definition, so I’d say that a community is any group of people brought together by some common denominator. Lots of things have invisible communities around them – people who care about that thing – even if they haven’t ever met each other. The web gives us the power to find each other, which is why it’s such a powerful, and terrible, place for social interactions.

It seems that, given enough people, a comment thread can quickly devolve into rapid fire insults and grandstanding. How can we foster and encourage thoughtful commentary and participation?

There are two parts to this, and you can only control one of them.

The part you can’t control is human nature. People are weird, messy, and complicated. They react strangely to things. They behave irrationally in groups. And you can’t change that.

The part you can control is how your website interfaces with that strange mass of humanity. Certain interfaces stand up better to the weirdness of humanity than others. It takes a lot of trial and error, and the line is always moving because we’re evolving.

We’re all still learning how to behave online. It’s the first time in human history that you can be both alone and in a group at the same time. How weird is that? It’s no wonder we haven’t figured it out yet.

One of the benefits of investing your time and efforts in building a community is the the personal rewards that come from interacting with the members of the community. One of the perils of that can be when things start to go wrong and it can lead to heartbreak. You had a fairly public parting of the ways with JPG Magazine. Not wishing to get into specifics with you, but how hard was that to deal with, and did the community that you and Heather helped foster help you “heal”?

JPG was an enormous learning experience for me, both for what I learned about fostering creative communities, as well as learning some hard truths about founding a company. There are a million stories out there about founding partners in a startup falling apart. In fact, it’s probably what happens more than anything else. The only difference in this case was that I chose to talk about it publicly. I did that because I had to – the community that built JPG deserved to know what happened. But one side-effect was that many other founders and entrepreneurs came out of the woodwork to tell me their stories. It helped me immensely to know that I wasn’t alone.

The web never ceases to amaze me. When you approach it with honesty, it comes back a thousand fold.

With seriously large amounts of money going after the “Social Web”, do you see people having multiple social locations (eg facebook, twitter, flickr, Goole Reader etc) as a hinderance, or does this fragmentation actually help, by allowing us to compartmentalise our social interactions into different areas, much like we do in our offline lives.

Good question! To a geek, this repetition of signup, login, create profile, over and over, just seems like an inefficiency. But people are inefficient. We all maintain multiple identities already. Who we are with our parents isn’t the same as who we are with our friends. So I’m not as bothered by all these disparate digital identities as some. In fact, I think trying to meld them is going against human nature.

“Strange Light” was an intensely quick project, born from an idea and fully realised in less than two days, and seems to have had an incredible response both from photographers and from the web at large. Do you see these magazines as having their own community? Or one that augments existing online ones (such as flickr)?

I think that magazines are, essentially, communities. At least, the successful ones are.

In the case of the first issue of Strange Light, there were two groups out there: the people who experienced it were a community of experience, and the rest of us who saw the astounding photographs coming out of the storm formed a loose-knit community of interest.

I was hoping that a few of those two groups would want a beautiful printed edition of photos from the storm, and they did! What I like about this is it allows the web to do what it’s good at (connect people) and print to do what it’s good at (create a coveted object).

And finally, when the hell is MagCloud going to deliver to Australia? It’s not like we’re on the other side of the world or something.

The short answer is, I don’t know. But I can tell you why it’s complicated.

Most of the printed matter around you right now is printed in China, put on a boat, and then a truck, maybe a train, and then another truck, and finally makes its way to you. It’s horribly inefficient and disastrous for the environment.

MagCloud is changing all that, because we’re going to have print partners all over the world. So when you buy a magazine on MagCloud, it’ll always be printed as near to you as possible. That way the buyer pays less, the publisher makes more, and we expend less carbon shipping paper all over the world.

But that means shipping is much more complicated than just licking a stamp. We’re setting up printing partners as quickly as possible, but it takes time. We’ll get shipping here as soon as possible. Promise.

Derek will also be running a workshop on “Creative Community Building. Tickets available now!

Then you need to create a “.autotest” file with the following:

Then it’s just a case of typing “autotest” in RAILS_ROOT.

Simple.

Around a year or so ago, someone posted a link to this photo on Flickr:

duty

Uploaded by Godiex { … } on 20 Dec 06, 1.20PM WST.

Certainly an interesting photo, and one that deserved further investigation. So, I started looking at Godiex’s photostream, and noticed there were a few photos of this girl, who turned out to be his girlfriend. I found her photostream, and then saw this photo

home town

Uploaded by randompanda on 6 Jun 05, 10.41AM WST.

which is a shot of the hill I grew up on (our old house is just around the corner). I thought that was pretty intriguing, and so I added both randompanda and Godiex as contacts.

It turns out that Godiex is an artist from Argentina, and that he and randompanda began a long distance relationship in the comments on a photo on flickr. So, all of a sudden, I’m watching the relationship of two people I don’t know, who met through a photo sharing site. Their photos showed them being apart, randompanda going to live in Buenos Aires, and then the pain of separation when randompanda had to return back to Australia (I presume caused by visa-expiry). I’m guessing I felt an affinity for their situation, as Caren and I spent 10 months of our first 2 years together in different countries.

Through my virtual stalking, it appeared to me that they were waiting for the opportunity to be back together and that all would be well again. Then Godiex posted this photo:

this moon is over

Uploaded by Godiex { … } on 28 May 09, 5.10PM WST.

In the description for the photo he basically said that their relationship was over, effectively being ended by the distance apart. It was weird. Here were two people that I had never met, and only shared the random fact that I grew up in the same town as one of them, and I followed their relationship through the photos they posted. And I was sad that they had made the decision to break up. And it’s clear from the comments on the last photo that there’s a large number of other people feeling the same way.

Well, I hope that these people are still together at least (read the comments on that photo for an amazing story).

DropBox

DropBox is simply file synchronisation done right. You install it, drag the files you want to have synced between two computers into the Dropbox folder, and BOOM, it copies everything up to their server, and when you set it up on the second computer, copies everything back down again. After the initial sync, which can take quite a while if you’ve got lots of stuff, it’s pretty much seamless. Create, or make a change to, a file on one computer, and before you know it, it’s available on the other computer.

Need access to those files from another computer, or even from your phone? No problem, they have a web interface that give you full access to all of your “synced” files. Simple.

To add a lovely cherry to the ice cream sundae that is DropBox, they also have versioning. That means if you make a series of dunderhead changes to a document, you can just rollback to the last good copy. Awesome.

Git and GitHub

For the longest time (mostly while I was still working on Windows) “Source Control” was some mystical mumbo jumbo that only people in large companies needed to worry about. Of course, that was mainly because at the time the options for Visual Studio only included Microsoft SourceSafe, which is perhaps some of the crappiest, most unusable software ever foisted on the unsuspecting public by a company with a long history of foisting crap on the unsuspecting public.

Once I switched to MacOSX and started working with Rails, it became clear that all source control systems weren’t created equally. Subversion seemed to be the choice of the community, so I loaded everything up onto my own SVN server. A world was opened up to me, and I could code with wild abandon, safe in the knowledge that I could rollback with no danger. Of course, that was the ideal world, but the basic approach worked. I created branches, tagged releases and deployed sites from SVN.

As with most things, people started grumbling about SVN and some of it’s shortcomings: difficulties in merging different code branches, no distributed repositories and some general discontent. Git was the solution, and the rails community just jumped ships in the middle of the night. Before you knew it, everyone was using Git, and also GitHub. I won’t go into the specifics of why this combo is so good, except to say that if you’re not using git, well, good day to you sir.

git and config_files

In addition to using git to store your development work, you can also use it to manage system configuration files. I found the wonderful config_files. It’s a collection of shell, git, vim and irb configuration files. So? How does that help?

A great feature of git, and github in particular, is that you can “fork” a repository. You effectively copy someone else’s repository, and you can then make all the changes to it you want, without affecting the original. If you want to let other people know about your changes, you can send the original developer a “pull request”, and they can grab your changes, and incorporate them, or not.

So, fork the config_files repository, and then clone it to your own system. Run the “install.sh” script, which basically just sets up a series of symlinks in your user directory. These symlinks point to the files in the git repository. You make any changes you want to the repository and commit them up to github. Then do the same on your other computer, and bingo, you’re synchronising your configuration between machines.

And why would you want config_files? Well, you get some great all round system setup tidbits, but my favourite bit is how it prints the current git branch as part of your prompt, saving you the need to try and work out what you’re in the middle of doing.

Tasty.

Backup Overview

Using a combination of Super Duper, a couple of backup external HDD’s, Dropbox, Jungle Disk and Amazon S3 I have built up what I think is a relatively comprehensive and reliable backup system.

Local Backups

SuperDuper is a wonderful program that creates bootable mirrors of a hard drive. Effectively what this means is that if a hard drive fails, you can just replace it with the backup copy. You can schedule the backups to run as often as you want, which ensures that your backup copy is fresh and useable in case of a disaster.

And what am I backing up? I have two main drives that I use: The system disk (the built in drive in my iMac which contains all of my work files, personal documents, applications and the Operating System), and an external FireWire800 drive that stores photographs, videos, movies and TV shows.

Amazon S3

Amazon S3 is basically an online storage system that can be accessed via an API to upload, manage and retrieve data. Amazon charges for both the uploading of data (US$0.10 per Gb), and then for the actual storage of that data. The uploading charge is a one off cost, and the storage costs are charged monthly (US$0.15 per Gb per month).

Jungle Disk

One downfall of S3 is that it’s not actually setup to be used without additional software to manage the mechanics of the backups. Step up Jungle Disk. It’s pretty simple software: you give it your S3 account details, tell it what to backup, when to back it up and it will go ahead and take care of it for you. Simple. And then you get your monthly bill from S3. Nothing else to do.

One big downside of the whole online backup setup is the time it takes to actually backup any large amount of data, and that’s a limitation of my internet connection more than anything else. When you’re uploading 100+Gb, be prepared for a bit of a wait

The cost of Amazon S3

While the monthly cost of the Amazon S3 service makes it a perfect online backup solution for data that you would class as priceless (photos, videos of the kids, work files, personal documents), when it comes to backing up stuff that can be easily replaced at a relatively low cost, such as iTunes TV shows and Movies, there soon comes a point where it’s not actually economically feasible to use online storage.

iTunes TV Shows

The average 1 hour (42 minutes of network TV) HD episode of a TV show on iTunes is about 1.4Gb. In addition to this, you also get an iPod/iPhone compatible SD version which is generally 600Mb. So, a single TV show is effectively 2Gb of data that needs to be backed up.

What is the cost of backing this file(s) up? Well, there’s a US$0.20 charge for uploading it to Amazon S3 initially, and then a US$0.30 charge per month. At that rate, it only takes 10 months of storing the data online until you’ve actually paid for the file twice. This means that if you lost the original file 12 months after you first bought it, you’d actually be better off buying the file again.

This effectively renders online backups for iTunes TV Shows pointless, considering how often you’re actually likely to watch a TV show, and also how cheap the cost of having a local HDD mirror is.

iTunes Movies

Currently iTunes Movies are only available in SD (boo to the movie studios and Apple for this one) so the files aren’t as large as they could be, but they’re still pretty sizable, weighing in at 1.67Gb for the recent “The Dark Knight”, which cost US$14.99. And how long until it’s not feasible to store this on Amazon S3?

At the purchase price, it would take 4.99 years until you’ve paid twice. However, Apple drops the price of new release films to $9.99 after about 4-6 months, so the replacement cost is greatly reduced. At this price the cut-off becomes 3.33 years. Obviously this timeframe requires a judgement call as to whether or not it’s worth it. Personally, I’d rather just trust my local HDD backups.

iTunes/Amazon Music

Music files are obviously considerably smaller than video files, and as such are going to incur a greatly reduced monthly fee for storage. Your average iTunes album costs approximately US$9.99 and is generally around 110Mb. This small size means that it will actually take about 49.95 years until it’s been paid for twice, buy which point you’ll either be A) dead, or B) listening to music on your personal bone implant that plays whatever music you want that’s being broadcast by SkyNet.

So, music is one area where it’s probably economical to maintain an online backup of your files, particularly considering how annoying it would be to go and re-purchase the 800+ albums you’ve got in your library in the first place.

Other possible solutions and problems.

Whenever you’re talking about local HDD backups, it’s always worth considering a Drobo, which is a redundant array of HDD’s that theoretically keeps your data happy and safe. I don’t have a Drobo, but I know that users who have one swear by them.

The one problem with local backups is that, of course, they are susceptible to local threats, e.g. fire and theft. There’s no point in having duplicate hard drives that slavishly mirror each other if some reprobate comes along and pilfers them both. Which means you need a third mirror, that you store off-site. Which in turn means you need a fourth drive that you store off-site in rotation with the third one. An endless cycle.

However, we did spend a bit of time in Singapore, and one of the best places we went was to Little India. There’s a large Indian population in Singapore, and they seem to be largely centered around the Little India area.

One of the things you notice about Singapore is the massive amount of construction that’s been going on, and the fact that almost none of the “old Singapore” remains, having been bulldozed and rebuilt with modern buildings. Little India, by contrast, is almost in original condition. It doesn’t have any of the sanitised vibe that exists elsewhere in Singapore. As a result, it’s certainly the most vibrant and lively area that we went to.

Another great thing about Little India is the food. Eating out in Singapore can be quite expensive if you head to the “western” restaurants, but if you want to eat some of the “local” foods, then it’s quite remarkable how cheaply you can eat. We ate (twice it turns out) in a wonderful tandoori restaurant, stunned at how little everything cost. A chicken tikka, aloo methi, naan, raita, rice and samosas for $30. Gold.

Little India from Jordan Brock on Vimeo.

And here are some photos I took that are up on Flickr.

I’ve been working on an updated version of the Soil Quality website for a little while now, and recently needed to deploy the site to a staging server for testing. I ordered up a 1/4Gb Accelerator from Joyent, configured it, and deployed the app, just like I’ve done with 20+ other sites, and BOOM, straight into a brick wall.

I opened up the log files and saw this error:

** Daemonized, any open files are closed.  Look at /tmp/soilquality-mongrel.8200.pid and log/mongrel.8200.log for info.
** Starting Mongrel listening at 127.0.0.1:8200
** Define INLINEDIR or HOME in your environment and try again

Wonderfully descriptive I know, but something I’d never run into before. So, fire the the google and it turns out it’s a common error, with a common fix: just put

ENV['INLINEDIR'] = RAILS_ROOT + "/tmp"

into your config/environment.rb file, make sure the directory exists and that it’s writeable by the mongrel, restart, and away you go.

But of course, that didn’t fix it, did it.

Much hair pulling ensued. I finally enlisted the help of Darcy Laycock and together we managed to track the problem down to the attachment_fu plugin, which was causing the problem as the mongrel process booted. OK, so now we knew where the problem lay, but what was causing it.

It turned out to be the ImageScience image processor, or more specifically the way the attachment_fu plugin and ImageScience work together ON A 2ND GENERATION JOYENT ACCELERATOR – eg the ones that use pkgsrc. Didn’t seem to cause the problem on an ubuntu machine, nor on one of the older BlastWave based Accelerators. I’m not sure why as of yet, as I was more worried about getting some sleep last night when we managed to fix the problem.

And the fix? Basically, strip ImageScience out of attachment_fu. Remove

vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/processors/image_science_processor.rb

and remove “ImageScience” from this line

@@default_processors = %w(ImageScience Rmagick MiniMagick Gd2 CoreImage)

in this file

vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/attachment_fu.rb

Like I said, I have no real idea why this is happening at the moment, but I’ll try and work it out and update this post if I get anywhere.

Once again, thanks to Darcy for A) being a sounding board, B) helping fix the problem and C) being up and available when I needed him

There is a huge list of international and national speakers on the program, one of whom is Ben Buchanan. I recently conducted a short email interview with Ben, talking about some of the subjects that he’ll touch upon in his talk Hacking Humans: Advocating for a better web

Ben is just one of a range of awesome speakers who are going to be presenting at Edge of the Web on the 6th November. On the following day, there are 4 half-day workshops on a diverse range of topics.

And then, to cap the two days off in style, Friday night sees the WA Web Awards, where, in a burst of glorious self congratulation, the best of the WA Web industry are recognised for their efforts. Get your tickets here.