as days pass by: a weblog by Stuart Langridge

Not a lawyer any more

sil — Thu, 11 Mar 2010 13:36:11 -0800

So according to Naked Law, the technology blog for the law firm I used to work for, Centrica contracted Accenture to write some software, and that software turned out not to work like they wanted. The court found Accenture liable for around u00a329 million. The lawyer response?

[This] demonstrates how widely the Courts are willing to interpret 'direct loss'. This will comfort the customer, but IT suppliers may want to consider strengthening their liability clauses.

Or, y'know, writing software that doesn't suck.

This is why I no longer work for lawyers.

Writing a Rhythmbox plugin

sil — Wed, 03 Mar 2010 13:29:36 -0800

Yesterday, as part of Ubuntu Opportunistic Developer Week, I did an IRC talk about writing a (basic) Rhythmbox plugin in Python.

You can see the talk, the slides, and the plugin at http://www.kryogenix.org/code/RBMicroBlog/.

More desktopcouch tips and tricks

sil — Wed, 03 Mar 2010 15:51:17 -0800

Manuel is at it again, with a second round of desktopcouch tips and tricks. He's building up quite a library of stuff over there. Now hopefully I can persuade him to link them from the desktopcouch documentation...

Browsing my Android phone over wifi at home

sil — Wed, 03 Mar 2010 15:19:53 -0800

Having to plug my Android phone into my computer via USB so that I can copy new files onto it is a pain. So I thought to myself: it's got wifi, right? Can't I just run an ftp server on it?

And, lo, I discovered On Air. Cool Android app; it's got One Massive Button and that's it. The One Massive Button just turns on a file server and exports it over the network.

(Actually, I lied. There's also one tiny button which toggles the type of server between FTP, WebDav, and AppleTalk. So I set it to FTP, since that's easy to connect to from Nautilus -- Places > Connect to Server > FTP (with login). On Air displays the IP address of the phone, and a randomly generated 4-digit password.)

So, that's all I need; I can now browse my phone without plugging it into the computer.

Then I got to thinking. It'd be dead handy if that automatically got started up when I was (a) at home and (b) the phone was plugged into the mains (because I don't want it being a server while I'm wandering around my house, and the mains plug is near the computer). A bit more poking around led me to Locale, an Android app which lets you watch for certain conditions (like geographical location, charge state, and so on) and then take certain actions when the conditions are triggered. Seemed ideal. But Locale is $10, and the "am I in a certain place" trigger was another dollar, and "am I plugged in" was another dollar, and I thought: yeah, $12 isn't much, but I don't really need this, so maybe I'll do without.

And then @michaelwells pointedout on Twitter that the "Setting Profiles" app might help. So I took a look, and basically, it's Locale, but free. Nice.

So, install Setting Profiles, set up a "rule" with conditions "connected to my home wifi" and "charger is plugged in", and an "action" of "start On Air". Done.

I'm really quite pleased with tying those things together.

Minor complaints, because nothing is perfect (perfection is finality. finality is death. nothing is perfect. there are lumps in it)

I wish On Air had a "start up the server automatically" setting, thus obviating the need to press the One Big Button.
I wish On Air advertised its FTP server with Zeroconf. (Probably it doesn't need to for the author's Mac use case because AppleTalk servers already do that without Bonjour?)
Settings Profiles doesn't show up in my apps list! That's really quite weird. To start it I have to go back into the Market, choose Downloads, and then start it from there. I'd file a bug, but there doesn't seem to be any useful way of doing that. How am I meant to do this? Leave a comment on the app in the Market?

Settings Profiles (website)

On Air

DesktopCouch talk at FOSDEM

sil — Mon, 01 Feb 2010 08:50:30 -0800

Nice! Manuel, heroic desktopcouch community contributor, is speaking about Making your users happy, and "cloudify"ing your app with desktopcouch at FOSDEM. I can't make it there this year, but that talk will be excellent :-)

Adding Ubuntu One support to your applications, an IRC lecture

sil — Tue, 26 Jan 2010 10:11:08 -0800

This year, as last year, I did an IRC talk as part of Ubuntu Developer Week. This time, it was about adding Ubuntu One to your applications, and what that will bring to your users. (Once again my xchat plugin for IRC talks was massively helpful!

Introduction

Hi! I'm Stuart Langridge, from the Ubuntu One team, and this talk is called "Adding Ubuntu One support to your applications".
You can ask questions at any time in #ubuntu-classroom-chat. Please write QUESTION at the beginning of your question so I can see it more easily. I'll stop at the end of a section and answer some questions.
During this talk I'm going to explain the different ways your applications can take advantage of Ubuntu One...
...talk about why you'd want to do that...
...and give a couple of sneak previews of things that you can't *quite* do yet but will be able to do for Lucid 10.04, so you can be ready!
Firstly, though, remember that "integrating with Ubuntu One" isn't really the point; the point is to give features to your users that make their lives easier.
Now, obviously, using Ubuntu One will make you taller, it'll make you more attractive to the appropriate sex, and it'll make the sun shine more brightly, but remember that it's all about providing things that your users want...
...and Ubuntu One gives you the infrastructure you need to do that. If it doesn't, come talk to us!

How to do cool things with file synchronisation

OK, so, the first thing is: files.
As you know, Ubuntu One automatically synchronises files in your Ubuntu One folder to Ubuntu One itself in the cloud and to all your other machines.
This means that you can take advantage of automatic file backup just by having your application save things to the filesystem!
Imagine that you want attachments that you receive by email to be stored online and backed up so you can get at them later.
Most mail clients will already let you do this in some way. So, change the folder that these attachments are saved in to be an Ubuntu One folder and suddenly all your attachments are backed up online and available on every machine!
For example, there is a Thunderbird attachment called "Attachment Extractor" (https://addons.mozilla.org/en-US/thunderbird/addon/556)
Install it, select "Automatically extract attachments on email receipt" in Tools > AttachmentExtractor settings > Auto-Extract, and set the "Save Path for Attachments" to /home/YOU/Ubuntu One/Mail Attachments, and that's all you need!
Similarly, if you want off-site backups of your computer, then you can install one of the many simple backup programs, like backintime (http://backintime.le-web.org/) and choose the backup folder to be inside your Ubuntu One folder.
With no extra effort you've got off-site online backups which you can get at from any of your machines or from the web.
As another example, imagine you've built yourself a digital picture frame out of an old laptop, which is a pretty cool project for your weekends. How do you get the photos on it?
Well, when I did this, I created a new Ubuntu One user for the picture frame, and then I shared a folder with that picture frame user through Ubuntu One.
The picture frame just did this: eog --slideshow "/home/pictureframe/Ubuntu One/Shared With Me/pictures from Stuart Langridge"
and then to add new pictures to the frame, I just copy them into the folder that I shared.
I could share that folder with anyone else, too, and allow them to write to it, and then they'd be able to add pictures to the frame too!
The way I'll do this from 10.04 Lucid onwards is to use the new User Defined Folders feature, where any folder, anywhere, can be designated as an Ubuntu One synced folder
and you can elect to have some folders but not others synced to particular machines
So I'd create my "digital picture frame" folder in my home folder (or wherever), and then mark it as a UDF.
Then on the picture frame itself, I'd sign into Ubuntu One, and say that I only want to subscribe to that particular UDF and not any of the others.
so then files that I put into "digital picture frame" on my laptop will be synced to the picture frame
and the picture frame just displays all the files in that folder
ta daaah, it Just Works.

Publishing files publically

A much-requested feature which will be available in Lucid is "public files", the ability to publish an Ubuntu One-synchronised file to a public URL so anyone can get at it.
This is useful for the user directly; they can easily make a file available to the world.
It's also really useful for applications, because you can now build one-click publish-to-the-world directly into your apps.
Imagine that your app is Shutter, which is like the standard Gnome Screenshot tool on steroids.
If you install Shutter and then hit the Print Screen button, it'll take a picture of your screen.
One of the main reasons people take screenshots of their screen is to show them to other people, which makes this a perfect case for publishing the screenshot to a URL so anyone can see it.
So, in Shutter (or your similar app), provide a "Publish with Ubuntu One" button.
That button should do the following:
1. Save the file into the user's Ubuntu One folder (just save, as normal)
2. Call the publishing API
The publishing API will be D-Bus; I can't confirm exactly how it will work, yet, because the team are still constructing it, but keep your eyes open for more details!
This shows a basic principle of Ubuntu One's design; pretty much anything you can do explicitly, like publish a file, or share a file, can also be done using the D-Bus API from applications.
I'll answer some questions here about using Ubuntu One file sync in your apps.

Questions

kklimonda|lernid QUESTION: Is there a full GObject U1 API for storing data, settings etc. or do we have to make use of ~/Ubuntu One/ for now?
Data and settings are best stored in desktopcouch if you want to take advantage of all that lovely synchronization goodness; there are benefits to doing it that way, and I'll talk about desktopcouch in a bit.
But you can certainly store settings files in ~/Ubuntu One if you want to
and obviously actual data *content* that your apps work on, your documents and so on, are stored in ~/Ubuntu One
mhall119|work QUESTION: in 10.04 you will be allowed to have multiple sync folders outside of ~/Ubuntu One/ ?
mhall119|work, yep, you sure will. UDFs (user defined folders) are a big feature that'll land in 10.04.
strycore89 QUESTION : will we be able to share single files and not whole directories ?
No. Sharing is done folder-by-folder, and we're not planning on changing that.
mhall119|work QUESTION: right now the local folder names match the online folder names. With the ability to have multiple sync folders, how are naming conflicts resolved?
I'm not sure what you mean here by "naming conflicts"; maybe we can pick that point up after the session, or later on?
foobob QUESTION: are you thinking of giving some GB more to the free plan?
I don't believe we are at the moment, no.
yltsrc QUESTION: will be ready symlincs in U1?
yltsrc, UDFs, being able to mark any folder as Ubuntu One synchronized, takes the place of symlinks. If you use, er, some alternative to Ubuntu One and you're used to putting a symlink in, say, ~/SyncedFolder which points at ~/SomeOtherFolder to get SomeOtherFolder synced, you don't need to do that in Ubuntu One; you just mark SomeOtherFolder as a UDF and it will be synced for you.
lantash49 QUESTION: Will there be more detailed specification w.r.t storing settings in ~/Ubuntu One? Storing them there directly seems to conflict with the XDG directory specs.
lantash49, one obvious approach to this is to mark XDG_CONFIG_DIR and XDG_DATA_DIR as UDFs, so they're synchronized across your computers.
yltsrc QUESTION: i want participate in testing U1, how can I do it?
Drop into #ubuntuone and talk to the team about what's going on and how you can help; we've had some really helpful community members already, like rtgz, who's done tons of stuff
taj QUESTION: Are there any plans to make U1 cross-platform?
There's a sprint at PyCon US to port Ubuntu One file sharing to Windows:

and mandel, another incredible community member, is currently porting desktopcouch to Windows: http://www.themacaque.com/?p=372
both of those projects are great, and if you can help that'd be wonderful
mhall119|work QUESTION: If I put a symlink inside of a UDF, that points elsewhere, does U1 sync that as a link, or as a copy of what is linked to?
it's synced as a symlink.

Desktop Couch

The second leg of Ubuntu One for developers is desktopcouch, the CouchDB on your desktop.
If you missed my last Ubuntu Developer Week talk about the basics of desktopcouch and what it's for, you can read it at http://www.kryogenix.org/days/2009/09/03/desktop-couch-irc-talk
In fact, you can read all the desktopcouch documentation at http://www.freedesktop.org/wiki/Specifications/desktopcouch/Documentation/, and you should do so :-)
The bit that's most likely to be relevant here is the "I am a developer, and I want the applications I write to store data in and work with desktopcouch" section. It contains much that is relevant to this talk!
http://arstechnica.com/open-source/guides/2009/12/code-tutorial-make-your-application-sync-with-ubuntu-one.ars/1 is a really great guide, written by segphault, of how to support cloud synchronization of data in your applications.
Essentially, if you use desktopcouch to store your application's data, instead of using sqlite or flat files, then the data will be synchronised between all your machines *without you having to do anything*.
So your app will work the same on all your machines with no effort.
To store a record in desktopcouch, simply do:

>>> from desktopcouch.records.server import CouchDatabase
>>> from desktopcouch.records.record import Record
>>> db = CouchDatabase('testing', create=True)
>>> r = Record({'key':'value'}, record_type='http://example.com/testrecord')
>>> record_id = db.put_record(r)

and to get it back, by record ID:

>>> fetched = db.get_record(record_id)
>>> print fetched['key']
 'value'

The Quickly project, for writing Ubuntu apps, have created "widgets" that use desktopcouch without you having to think about it. CouchGrid, for example:

>>> from desktopcouch.records.couchgrid import CouchGrid
>>> keys=["key","another key"] # to label the columns
>>> couchgrid = CouchGrid(db_name, record_type=record_type, keys=keys)
>>> mywindow.pack_start(couchgrid)

and then you have a CouchGrid, which is a data table where all the data is automatically retrieved from desktopcouch and stored back into desktopcouch when changed
http://www.youtube.com/user/calorielookup#p/a/u/0/Vwr5Xw5ZrIE has a video demonstrating couchgrids
You don't have to know anything about desktopcouch at all for this! Just use the widget and your data will be saved to the database and synced to your different machines with no effort or code on your part at all.
https://wiki.ubuntu.com/Quickly/Snippets has a few more examples of using desktopcouch from Python and from Quickly apps.
So, cloud-enable your apps! Let's see my song ratings in Rhythmbox or Banshee or Amarok or Exaile or Quod Libet appear on all of my machines. Let's see my Chrome bookmarks appear everywhere, like my Firefox bookmarks already do.
Remember, too, that apps can share data. If you want to store song ratings for Amarok in desktopcouch, don't make that storage Amarok-specific; that way, if I decide to switch to Quod Libet I don't have to re-rate all my songs!
I'm happy to answer questions about sharing of data between apps if you have them.
Also, I'll stop now to take some questions about desktopcouch generally.

Questions

andypiper QUESTION: what is Quickly....?!
Quickly is an app that helps you make Ubuntu apps really...quickly
it takes care of putting together the framework of an application for you, and it makes a deb package for you and uploads the code to launchpad, etc. All the boring stuff that you don't care about when you just want to get your app done
the next session, by didrocks, is about quickly, so I'd stick around for that :-)
https://wiki.ubuntu.com/Quickly has more
yltsrc QUESTION: is available any standard for database name? And how resolve conflicts with different application?
yltsrc, there isn't a standard, as such, for database names, because what the desktopcouch project is trying to avoid is laying down lots of standards that you Must Comply With
we're a "rough consensus and running code" sort of group :-)
the best place to have discussions about that sort of thing is at http://groups.google.com/group/desktop-couchdb which is the desktopcouch mailing list
desktopcouch records are deliberately set up to have two sections, a "main section" and an "application specific" section
so the idea is that apps co-operate on the main section and store app-specific data in the app-specific section
for example, Evolution contacts sync in Ubuntu One is done by Evolution storing your contact data in desktopcouch
but the format for a contact has been defined
and the contacts are stored in a database in desktopcouch called "contacts"
so Thunderbird, for example, could *also* sync its contacts to that same database, in the same format
and Evolution stores Evo-specific data (like, say, an internal ID) for a contact record in the app-specific part of the contact record
and Thunderbird would do the same
what this means is that if you decide to switch from Evolution to Thunderbird, you've still got your addressbook! You don't have to export it and re-import it every time you move between applications
yltsrc QUESTION: U1 require python (2.6). Is any plans to run it on python3?
I don't believe there is, at the moment, no.
b1ackcr0w QUESTION: SpiderOak is said to be going Open Source soon, are there oppertunities to integrate their services with U1?
There could be, certainly. I haven't looked into that in detail myself, but I'll be sure and bring it up!
mhall119|work QUESTION: I want to hack on the server-part of U1, short of becoming a Canonical employee, what can I do?
At the moment, you can't, I'm afraid. The source code for the server part of file synchronization is closed.
The source code for the server part of desktopcouch is CouchDB, though, and work on that is perfectly doable.
We worked closely with the CouchDB upstream project to make sure that all the changes we needed to use CouchDB in Ubuntu One were part of the main CouchDB trunk.
yltsrc Is avalably any FEATURE ROADMAP for U1?
that's...sorta what this talk is :) I can see how it might be useful to have a more formal statement of "what's coming up", though. I'll bring that up with the team.
yltsrc QUESTION: is U1 uses dbus? and how i can extend my application to use it?
Yes. The syncdaemon, which is the bit that handles file synchronization, exposes D-Bus commands for almost everything it does.

Web apps and desktop apps talking via Ubuntu One

Since desktopcouch is a CouchDB, and since your data is synchronised not only to your other machines but up to the cloud as well, this opens up the intriguing possibility that you can build a web app that can share data with your desktop app.
Ubuntu One's CouchDB uses OAuth for authentication.
This means that your web app can ask permission from the user to read your desktop app's CouchDB data directly from Ubuntu One.
So the data that appears in your desktop app can also be available to that user on the web.
This is a pretty new area, so nobody's using it yet, but it's possible.
You use OpenID to log the user in to your web app using Ubuntu One -- you may have seen "log in with Twitter" buttons, or "Facebook Connect", which work on the same principle.
Once that's done, your web app can then request permission to read the user's CouchDB database to get at the data that your *desktop* app stored there.
So, if we think about the song ratings example above, you can build a web application that can show all the song ratings that the user has entered and allow them to be edited.
Your users can then manage their song ratings or playlists or whatever else you've stored directly from a web interface if they're not at their actual computer.
Sharing data between web apps and desktop apps is quite hard, and not many people are doing it, so this is a chance to offer a feature that no-one else has!
DesktopCouch plus web apps give you the ability to share that data without you having to do any extra work.
This also gives you the ability to offer mobile apps as a web interface, so people can work with their data from their smartphone and see their work reflected on their desktop when they get back to it.
Are there any questions about how linking up desktop apps and web apps would work?

Questions

mhall119|work QUESTION: how does it handle when a file is edited on2 different computers before syncing?
If that happens, you'll get a .u1-conflict file showing you that there was a conflict, for file synchronization.
In desktopcouch, there will be a conflict record for the data.
One of the things that we have on our Big List Of Stuff To Do is some sort of graphical conflict management to make dealing with these situations even easier, but that won't be done for 10.04.
So at the moment, handling data conflicts is done by the applications that use the data; the applications have a better idea about how to merge/overwrite/handle conflicts than Ubuntu One does most of the time anyway, because Ubuntu One just moves the data around, it doesn't always understand it.
qense (late) QUESTION: When you define a UDF outside the Ubuntu One directory, where will it be synced to when you enable/install Ubuntu One on a different computer?
It'll be synced to the same path, so if you mark ~/myfolder as a UDF on computer 1, it'll appear as ~/myfolder on computer 2.
duanedesign QUESTION: Where can I find documentation on the U1 API
the desktopcouch API is defined (mostly) in the desktopcouch documentation at http://www.freedesktop.org/wiki/Specifications/desktopcouch/Documentation
the file sync API isn't defined anywhere yet, I don't believe, because we haven't had a chance to do it, so it's reading the source, I'm afraid. https://wiki.ubuntu.com/UbuntuOne has some details, though.
emoreno QUESTION: Is there any way to use a Private Cloud rather than U1 with the same API
Not at the moment. The protocol that the syncdaemon uses to talk to the cloud isn't secret, though, so it would be possible to implement a private cloud.
mhall119|work QUESTION: does U1 offer any kind of versioning on it's data, like the ability to retrieve an older version?
Not at the moment, no. We've talked about that, but it isn't available now.
OK, that's a brief tour of what services Ubuntu One can bring to your apps. You can make the files your app uses be synchronised, you can publish those files publically, you can store data from your apps that will be available everywhere, and you can share that data with web applications.
nealmcb QUESTION: Re the question on working on the U1 server end - Elliot Murphy recently talked about your open file storage protocol, noted his "build no silos" policy for U1, and talked about e.g. Tomboy integration. Can you expand on the places that people can work on servers that integrate with U1 protocols?
desktopcouch is a CouchDB, so it can replicate data with any CouchDB, including other desktopcouches on your LAN -- see the desktopcouch-pair tool for this, in the desktopcouch-tools package -- and so doesn't even need a server at all
Tomboy notes sync was done by us implementing the Snowy API -- Snowy is a Django server to which you can sync your notes, designed by the Tomboy team, and they're looking at doing their own deployment of it
qense QUESTION: Any change there will be a small collection of snippets showcasing webintegration of Ubuntu One?
if someone does that I'll love them forever.
I'd certainly like to, it's just finding the time :-)
OK, if you have other questions, feel free to ping me, or chat on #ubuntuone!

PyCon sprint to get Ubuntu One on Windows

sil — Fri, 22 Jan 2010 08:57:49 -0800

More than one person has said: Ubuntu One seems good but I need it to work on Windows too. Well, y'know, the roar of the people is not to be denied. Lucio Torre and some of the Ubuntu One team are going to be running a sprint at PyCon to port Ubuntu One file sharing to Windows. If you'd like that and you could help out, add your name to the sprint page!

Workus interruptus

sil — Sun, 17 Jan 2010 10:22:40 -0800

Yesterday I had to go to hospital (no worries, I'm recovering). But, between operation and antibiotics and fever, I've been bizarrely ill and not ill, this weekend. I feel perfectly normal for two hours, and then I have to go and sleep or shiver in bed despite being boiling hot or whatever. So, since all knowledge is useful, in my lucid* moments, I've been conducting an experiment, starting with this hypothesis:

If you're going to work eight hours in a day, it's better to work in lots of small blocks with lots of long breaks between those blocks, rather than working a solid eight hours and then having the evening off.

The intuitive answer is obviously that that won't work, at least not in my job -- there have been reams and reams written about how getting into "the zone" is the most productive time for hacking, and taking breaks (or phone calls) knocks you out of "the zone" and stops you getting as much done. This seemed like an ideal time to test that theory. Progress is made by challenging the status quo*, after all.

Well, I can report that the prevailing theory is right, and I am no Galileo breaking down the paradigm and ushering in a new theory of work. Every time I get into something, I have to stop and go and pathetically lie down and recover. Now, it's possible that I just can't concentrate because I'm not well, I admit it (so don't go basing any psychology Ph.Ds on this observation), but it's become reasonably clear to me that this interruption-based approach to work...doesn't work. So, as you were.

A thousand elephants, er, fans

sil — Wed, 13 Jan 2010 15:49:34 -0800

Blimey. Credit where credit's due, Jono set a target of Shot of Jaq getting a thousand Facebook fans in a week, and in the privacy of my head I sneered at him and figured it wouldn't happen.

It's great to see that another thousand people have joined the conversation. Who's next? :)

Sanitising HTML

sil — Sun, 10 Jan 2010 14:42:51 -0800

It was pointed out to me that comments on my old posts were showing as raw HTML (you know, a sort of <p>this is a comment</p> sort of thing). I knew this. However, the reason it was like that is because it occurred to me about five minutes after releasing thort, the engine that now runs this place, that comment HTML was just displayed. Unsanitised.

Cross-site scripting, anyone? Oops.

So I just threw an "escape" filter into my comment template (which uses the great Trimpath JavaScript templating engine) so that I couldn't be brutally pwnt by anyone posting a comment.

Finally this evening I thought: I'd better do something about that. Two minutes of Googling brought me to Caja's HTML sanitizer, written in JavaScript. It was the work of but a moment to throw that into the CouchDB view that generates comments so that outputted comment HTML was sanitized. It was the work of but one more moment to also throw that into the client-side JavaScript that displays a posted comment. It's really nice being able to use exactly the same code on client and server.