As we come to the final month of the JISCPress project, we had some great news over on WriteToReply last week where we were able to announce that Eduserv would be covering our hosting costs for the immediate future (Eduserv funds hosting for WriteToReply, eFoundations: Write To Reply).

So what exactly does the platform we’ve been working on have to offer? Here’s one of the ways I think of it…

A document publishing platform that automatically atomises documents to the paragraph level, allows aggregated commenting at the paragraph and ‘user’ level, and supports the republication and re-presentation of documents in a variety of standard formats at the document level.

The first part of the process is the (manual assisted) ingress stage, in which documents are imported into the WordPress environment such that each substantive document section ideally maps onto a single WordPress “blog post”:

An RSS for the document as a whole, with one item per section, is generated automatically by the WordPress platform. A single item RSS feed is also generated for each page (so the content of each page can be easily transported around the web).

The second part of the process is the atomisation of each post, carried out automatically by the Digress.It theme, in which each paragraph in the document is given its own unique URI, derived from the URI of the web page (“blog post”) the paragraph appears on:

Potentially, an RSS feed can also be produced for each page in which each paragraph is a separate feed item, thus allowing a page/section to be transported around the web via a single feed, but in atomised form.

The paragraph level chunks produced by the atomistation process can be transcluded as independent elements in independent web documents in other documents by a variety of means (as an embeddable object, via XML, txt, JSON, etc):

The default nature of the WordPress platform allows comments to be made at the level of each web page, with an RSS feed of comments for each page being published ‘for free’. JISCPress extends this functionality by allowing comments to be associated with discrete paragraphs. Views over the comments are also available at the user level, (that is, grouped according to the user who made the comments, wheresoever they are made in the document). An additional RSS fed of comments by user is also available, which means that a document on the platform can actually be used as a scaffold for a critical response to the document by a particular user.

A further level of innovation is based on the automated generation of ’semantic tags’ at the page level. Once generated, tag based collections of posts can be syndicated in the normal way via WordPress generated tag based RSS feeds:

JISCPress also benefits from the Trackback mechanism implemented by WordPress. When a page or paragraph URI is linked to from a third party web page, a trackback to the originating page may be captured, which we interpret as the automated capture of links remote annotations or comments about the document.

When considered in these terms, the JISCPress/WriteToReply platform is seen to provide a powerful means of publishing documents in which individual sections may carry their own unique URI, and individual paragraphs within a section also contain their own unique URI (which in many situations may be rooted on the section URI).

The platform can also be regarded as republishing – or re-presenting – each section (i.e. page) and each paragraph as an independent entity. That is, whenever a document is published via the platform, each separate paragraph may also be thought of as being independently published “for free”, in the sense that:

- each paragraph is independently addressable,
- each paragraph is independently commentable, and
- each paragraph is independently republishable/syndicatable.

So, given that, can you think of any ways in which the JISCPress/WriteToReply platform can support your document publishing and comment gathering strategy?

So it feels as if I haven’t been posting that much on this blog over the last few weeks, but I have been blogging elsewhere, 2-3 times a week, in fact, on:

- the Arcadia Project Blog;

- the Arcadia Mashups Blog.

Here’s a quick round up of some of the more notable posts that you can find over there that I would, in the normal course of events, have probably posted here on OUseful.info:

• Do Libraries Cater for Today’s Researchers and Research Students?, a quick response to a new interim project report from the British Library reporting on a longitudinal study about the information related behaviour of research students; [Arcadia Project]
• Open Library Training Materials and Custom Search Engines, in which I describe a simple Google Custom Search Engine that searches over all the UK HEI Library websites. If you’re looking for infoskills/Library training resources, this might be a handy way of finding them… [Arcadia Mashups]
• Getting Started With Yahoo Pipes: Merging RSS Feeds, an quick guide to creating your first Yahoo Pipe [Arcadia Mashups]
• Stack Request Delivery Slots: the Cambridge University Library has many of its borrowable books on closed stacks, which means you have to request the books and then they are fetched for you. Might it be an idea to allow bookable collection slots, so you can request an item for a future date? [Arcadia Project]
• BookServer – Like URIPlay, but for Books…? and LibraryDNS, cf. RadioDNS: Books that Can Phone Home, where I pinch a couple of ideas from online video (content negotiation) and digital radio (linking radioplay back to the web) and start to apply them to books; see also What If? Livescribe Book Support for one possible way of relating written notes to (e)books; [Arcadia Project]
• The Library’s Role in Organising “Course Knowledge”, or how the Library might be the natural place for all manner of content recommendation services to learners; also relates to Looking Out for “Linked Course Data, where I have a poke around various bits of the Cambridge university website looking for unique identifers that can be used to pivot around; [Arcadia Project]
• JISC MOSAIC Competition Entries – Imaginings Around the Use of Library Loans Data, a brief review of the JISC MOSAIC (Library loans data) competition entries;
• ‘Get Current URL’ Bookmarklet Pattern and ‘Get Selection’ Bookmarklet Pattern, in which I describe a couple of bookmarkelt patterns that have generic reuse value; each post includes a bookmarklet generator to help you create your own bookmarklets [Arcadia Mashups]
• Visual Links – Sharing Links With QR Codes, or what might a library catalogue look like if we added a QR code to it; also includes a demo of Cambridge Library short link service that could provide the first step in making it easier to scribble down a book reference [Arcadia Mashups]

They’re all Library related, so if that’s your thang, maybe worth a read…?

Although I’m now half way through my Arcadia Fellowship (sigh….:-(, it wasn’t until last weekend that I spent my first weekend in Cambridge, and finally got around to doing some culture stuff (a couple of galleries, a recital, an excellent lunch in Michaelhouse (thanks for the tip, Huw:-), and so on…

It also made me realise how I haven’t really got into the swing of making the most of my time here, so over the next few weeks I intend to check out the various Cambridge events calendars (of which there are several – more about that in an Arcadia post somewhen…) and start getting some events in…

In fact, I’ve already started, writing this as I am having just got back from a talk tonight by science communicator (and presenter of Material World, Thursdays, 4.30 pm, BBC Radio 4, also on podcast ;-), Quentin Cooper.

This (public) talk, on public perceptions of scientists, was one in a series arranged by CSAR, the Cambridge Society for the Application of Research (events listing), and just one of many dozens of public talks listed on the talks.cam website (again, I’ll write more about that in a forthcoming Arcadia post).

Ever an entertaining speaker, Quentin described the various stereotyped views of “scientists” (lab coat, mad hair, glasses, a crazy smile, and bubbling test tubes and bunsen burners everywhere), as well as suggesting a little experiment for us all to try at home: search for the word scientist in Google image search…

(Turning Safe Search seems to have very little effect (on the front page of results, at least…). Trying the same thing in locale specific versions of Google image search using the local word for scientist is apparently also illunimating…!)

You can also try it with “face search” switched on:

(Just by the by, here are image searches for engineer (face search), technologist (face search).)

Another interesting observation came from a BA web survey that had asked people to name their favourite on-screen scientists. The ambiguity in the question, unsuspected when it was first posted, lead to the majority of answers relating to fictional scientists rather than science/scientist presenters (Why Dr Who beats Einstein these days).

How scientists portray their own work was also on the agenda – and as I’ve long believed, sometimes a little help from the arts can help. One particular set of examples came from the Cape Farewell project, a “cultural response to climate change”, in which various cohorts of (notable) scientists, artists and musicians went off to see the effects glacial melting for themselves. Sometimes it’s the most obvious things that catch you completely by surpise – like the observation that as glaciers retreat, they might uncover islands that have been previously unmapped, an idea picked up by artist Alex Hartley in his piece Nowehere Island.

Anyway, here are a couple of random thoughts I came away from the event with…

When’s someone going to write a drama like This Life or The Office (or pushing it, No Angels, Teachers, Party Animals, A Very Peculiar Practice etc etc) based in a lab/hi-tech factory, where a bunch of “scientists” (as in sci/tech/eng/maths) folk just get on with the everydayness of their working life in a home and work context? Or has there been one and I’ve missed it?

Folk attending the talk were given the option of taking away an attendance certificate for CPD purposes. If I was an informal learner, could I use such an attendance certificate in partial fulfilment of a more formal academic something?

All in all, a good night out; and another one upcoming tomorrow [i.e. on Tues Nov 3rd 2009]: Thinking Like a Dandelion: Cory Doctorow on copyright, Creative Commons and creativity.

So over the last few days, there’s been so much chat around the roll out of official Twitter lists that some people have probably even blogged about them. Ad hoc lists (aka ‘groups’) have been available for some time on a variety of twitter clients, of course, but now there’s ‘central support’ so it seems like everyone is hyping around them (OMG, OMFG, Twitter has lists. Lists! etc.)

I’m still waiting on the API (though a draft spec has been posted) so I can have a go at creating lists automagically from hashtag groups (various posts), but in the meantime, here are a couple of little toys that try to spot what’s going on within the context of a particular list.

First up, a pipe that will look to see what hashtags are being used by folk on a particular list. The pipe feeds off a list of tweets by list members (using the URI pattern http://twitter.com/USERNAME/lists/LISTNAME/statuses.xml ) and then reuses pipework from Searching for Twitter Hashtags and Finding Hashtag Communities, replacing the search elements with a list feed:

So what? So you can enter the URI of a list and see what hashtags folk on that list have been using recently – Twitter list hashtags pipe:

A second, related pipe (this time reusing the Twitter name search pipe (that looks for people who’ve been tweeting particular search terms) and the Twitter names atomiser (i.e. tokeniser) pipe) will look for the names of folk who are part of a recent conversation on the list by virtue of being @d at… Twitter list conversants pipe:

One thing to note about each pipe – make sure you enter the URI in the correct form – http://twitter.com/USERNAME/lists/LISTNAME

If I get a chance, I’ll harden the pipe with a regexp to defend against the missing lists path element; but at the moment, if you donlt use the correct the URI pattern, the pipe will break…

That’s all…

PS the pipe should be hardened now to accept URIs of the form http://twitter.com/USERNAME/LISTNAME

Picking up briefly on Peter Murray Rust’s exhortation to the keynote attendees at ILI2009 that libraries must rediscover Ranganathan’s Five Laws of Library Science to their heart if they are to survive:

I thought I post some free association thoughts on what the five laws say to me. Note that I’m not a librarian, have never studied library science and don’t normally work for the library, though I currently am on an Arcadia Fellowship with the Cambridge University Library. Which is to say, my interpretation may not be the conventional, or accepted one…

So here we go:

Books are for use.
Hmmm… Books are for use… they are they to be used… they exist to be read… they exist to impart knowledge, information, emotion. They exist to communicate. As such, maybe they are social objects? But maybe also, they contain information or knowledge that enables things to be done, ideas to be understood? Maybe they are the next step in helping us do something, achieve something?

In a 2003 blog post outlining ideas for what was to become The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture, John Battelle describes Google’ssearch operation as a database of intentions:

The Database of Intentions is simply this: The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result. It lives in many places, but three or four places in particular hold a massive amount of this data (ie MSN, Google, and Yahoo). This information represents, in aggregate form, a place holder for the intentions of humankind – a massive database of desires, needs, wants, and likes that can be discovered, supoenaed, archived, tracked, and exploited to all sorts of ends. Such a beast has never before existed in the history of culture, but is almost guaranteed to grow exponentially from this day forward. This artifact can tell us extraordinary things about who we are and what we want as a culture.

That is, every search we make is an expression of some sort of intention. There is a point to every search.

So maybe in the same way, a book might be able to satisfy some intention? Or maybe I’m getting ahead of myself, because second up we have:

Every reader his [or her] book.
So at any point in time, there is a book that I need, that will somehow “help”? This ties back to a book that can satisfy an intention I have, perhaps? My current problem, or situation, is unlikely to be one that has never been met before, never been addressed by someone, somewhere, in some particular book?

Every book its reader.
And conversely, at any point in time, for every book there is someone who would benefit from reading that book? The book is a satisfaction of some intention? There is someone who would benefit from being recommended that book, maybe? (The ideal search engine would be an answer engine, would return only the single answer you need for a particular query, maybe?)

Save the time of the User.
Which means what? Give them the book that they need, in a timely fashion? Make it easy for them to discover the right book, or the right part of the book, that they need, with the minimum of fuss, or noise in the recommendations? Give them full text search, extended indexes in the form of semantic tags and on-demand access, maybe?!;-)

The library is a growing organism.
The library is a living thing. As a living thing, it must adapt to survive. As a living thing, it inhabits an ecosystem, a network, a network characterised by the making and breaking of new and old connections, by the flow of resources across those connections.

Hmmm… so how are these laws actually interpreted by the Library Science community, I wonder? And to what extent do they apply in the context of search engine queries, results and the resources pointed to by those results? Would it be fair to say that it is Google, rather than Library, that has taken these laws to its heart? Would it be fair to say that several of the laws at least hint at making effective recommendations to users, as Lorcan Dempsey suggests in Recommendation and Ranganathan?

Some time ago, I started exploring how treemap representations might be used to provide a quick overview of the make-up of the membership of local council committees (Glanceable Committee Memberships with Treemaps).

Following a lazyweb request to @countculture about expressing complete council committee membership data from Openly Local (Topical Versions of Local Council Websites… Give Us the Data and We’ll Work It For You), and the rapid fulfilment of that request ;-), here’s a proof of concept about how to use that data to power a treemap from the Javascript InfoViz Toolkit (JIT) to provide a glanceable display of the make-up of Isle of WIght Council committees, colour coded by party:

Each committee is represented as follows:

{children: [
 {children: [],
  data: {
   $area: "3", $pcolor: "purple",members:"Jonathan Francis Bacon, Paul Fuller, Heather Humby, "}, id :"ccl12_0", name: "Independent"
 },
 {
  children: [],
  data: {
   $area: "5", $pcolor: "blue",members:"Ivan Bulwer, Susan Jane Scoccia, Albert Taylor, Jerry White, David G Williams, "},
   id :"ccl12_1",
   name: "Conservative"},
{
  children: [],
  data: {
   $area: "1", $pcolor: "darkorange",members:"Colin Richards, "},
   id :"ccl12_2",
   name: "Liberal-Democrat Group"}
], data: {$area: 9}, id: "ccl12", name: "Licensing and General Purposes Committee"}

With another quick bash at the code, I should be able generate these views on the fly from the JSON data feeds provided on the OpenlyLocal site. (At the moment, the route I take to generate the Javscript object that powers the treemap is a really clunky one:-(

What never ceases to amaze me, though, is how a tweak from one representation of a dataset (that is, the JSON data published by OpenlyLocal), to another (the JIT treemap representation) allows the creation of interactive visuliastions as if by magic :-)

If you want to play with your own treemaps in the meantime, this bit of Javascript will produce a simple representation of committee. member and party data that can be visualised within Many Eyes WIkified:

for (var i=0;i< c.committees.length;i++){
  for (var j=0;j< c.committees[i].members.length; j++){
  	document.write("\""+c.committees[i].title.replace("&", "and")+"\""+","+"\""+c.committees[i].members[j]["first_name"]+" "+c.committees[i].members[j]["last_name"]+"\""+","+"\""+c.committees[i].members[j]["party"]+"\""+"<br/>");
  }
}

(where c is the javascript object that is published as the JSON feed from a committee page on OpenlyLocal, such as this one for the Isle of Wight).

Here’s an example:

Whilst this treemap doesn’t allow you to fix the colours in the way that the JIT component does:

TM.Squarified.implement({
   'setColor': function(json) {
     return json.data.$pcolor;
   }
 });

($pcolor is a variable I set for each committee member saying what colour should be displayed for them…), the Many Eyes Wikified does allow you to manipulate the tree representation that powers the treemap, e.g. by reordering the way in which the different elements are displayed:

What I really need now is a way of creating the hierarchical JIT objects on the fly from a table based representation… Hmmm….

Just after I put together the pipework for Getting Started with data.gov.uk, Triplr SPARYQL and Yahoo Pipes, I also cut and pasted some of the code from a previous map based mashup to demo how to make a SPARQL call via a pipe that calls on the UK Gov education Linked Data datastore from within a web page, and then display the geocoded results on a map.

Here’s the demo – School openings and closures in the UK, 1/1/08-1/10/09

If you View Source, you’ll see the code boils down to:

//schools closed between 1/1/08 and 1/10/09
q="SELECT ?school ?name ?opendate ?closedate ?easting ?northing WHERE {?school a sch-ont:School;  sch-ont:establishmentName ?name;sch-ont:easting ?easting; sch-ont:northing ?northing; sch-ont:establishmentStatus sch-ont:EstablishmentStatus_Closed ; sch-ont:closeDate ?closedate ; sch-ont:openDate ?opendate . FILTER (?closedate > '2008-01-01'^^xsd:date && ?closedate < '2009-10-01'^^xsd:date)}"

u=ur+encodeURIComponent(q);
getPipeGeoData(u, 'parseJSON_purple');

In all I make three calls to a pipe that calls on the data.gov.uk education datastore, one for schools opened between 1/1/08 and 1/10/09:
SELECT ?school ?name ?opendate ?easting ?northing WHERE {?school a sch-ont:School; sch-ont:establishmentName ?name;sch-ont:easting ?easting; sch-ont:northing ?northing; sch-ont:openDate ?opendate . FILTER (?opendate > '2008-01-01'^^xsd:date && ?opendate < '2009-10-01'^^xsd:date)}

one for schools closed between 1/1/08 and 1/10/09:
SELECT ?school ?name ?opendate ?closedate ?easting ?northing WHERE {?school a sch-ont:School; sch-ont:establishmentName ?name;sch-ont:easting ?easting; sch-ont:northing ?northing; sch-ont:establishmentStatus sch-ont:EstablishmentStatus_Closed ; sch-ont:closeDate ?closedate ; sch-ont:openDate ?opendate . FILTER (?closedate > '2008-01-01'^^xsd:date && ?closedate < '2009-10-01'^^xsd:date)}

and one for schools proposed to close:
SELECT ?school ?name ?easting ?northing ?opendate WHERE {?school a sch-ont:School; sch-ont:establishmentName ?name;sch-ont:easting ?easting; sch-ont:northing ?northing ; sch-ont:establishmentStatus sch-ont:EstablishmentStatus_Open__but_proposed_to_close; sch-ont:openDate ?opendate . }

(I cribbed how to write these queries from a Talis blog: SPARQLing data.gov.uk: Edubase Data;-)

The results of each call are displayed using the different coloured markers.

(The rest of the code is really horrible. Note to self: get round to learning JQuery.)

Whilst mulling over the extent to which I could use the delicious social bookmarking service to provide an ad hoc tagging tool for web pages*, I wondered whether or not it was possible to do a site limited search in delicious.

And apparently, it is possible, using an obvious (and guessed at) notation: “open access” site:ac.uk

The search appears to return results of bookmarked pages limited according to the site: search limit.

So has the OUseful.info blog been bookmarked at all over the last month?

Hmmmm…. :-)

* so for example, I can look up the delicious bookmark details for a page given its URI in the following way: http://feeds.delicious.com/v2/{format}/url/{url md5}
Here are details of a pipe that handles the MD5 encoding for you and gives you a feed relating to who has recently bookmarked a given URL: DeliURL pipe

PS the url: search limit also works, so it’s easy enough to see who’s bookmarked a particular page with a limited search on a specific URI, rather than having to go the MD5 lookup route:

Unfortunately, the delicious search results pages don’t natively provide an RSS or JSON feed output… but I wonder: can we get those via YQL? Arghh, of course they don’t – the search results pages disallow robots, even Yahoo’s own robots:-(

So as one of the things on my Arcadia project to do list, I’ve started looking for consistent identifiers that might act as useful pivot points between various bits of the Cambridge’s online offerings (the public stuff on www.cam.ac.uk, as well as the Raven authenticated, password protected stuff on the de facto VLE, Camtools.

Ideally, I’d like to find to some Crown Jewels, something like OU course codes, for example, but I fear that is not going to be possible…

Anyway, it’s still early days yet, so as w have a meting with the Management Services Information Division, MISD tomorrow, to see whether or not they have data that we might use to generate affiinity strings for users of the Newton Library Catalogue, et al., I thought I’d have a look at whether different bits of their Student Administration and Records: CamSIS Coding Manual link together at all:

The diagram was created by cut’n'pasting data from the coding scheme web pages, then using Graphviz to chart the links.

For what it’s worth, here’s the dot file:
graph G {

"A01" -- "Cambridge Colleges";
"B01" -- "County Codes";
"B03" -- "Country Codes";
"D01" -- "Current UCAS Courses";
"D03" -- "Current/Archived UCAS Courses";
"D05" -- "Academic Careers";
"D06" -- "Academic Programs";
"D07" -- "Academic Plan Types";
"D08" -- "Academic Plans";
"F01" -- "Awarding Bodies";
"F02" -- "GCSE Subject Codes";
"H01" -- "Subject (Tripos) Codes";
"H03" -- "Examination Paper Codes";
"J01" -- "Grading Codes";
"J02" -- "Further to Class Codes";
"K01" -- "Degrees";
"M01" -- "Faculties and Departments";
"S01" -- "Source of Fees";
"Z02" -- "Ethnicity Indicators";
"Z04" -- "Disability Indicators";
"Z05" -- "Program Status Codes";
"Z06" -- "Program Action Codes";
"Z07" -- "Program Reason Codes";
"A01" -- "College Code";
"A01" -- "College Description";
"B01" -- "County Code";
"B01" -- "County Description";
"B03" -- "Country Code";
"B03" -- "Country Description";
"D01" -- "Course Code";
"D01" -- "Course Description";
"D03" -- "Course Code";
"D03" -- "Course Description";
"D05" -- "Academic Careers Code";
"D05" -- "Academic Careers Description";
"D06" -- "Academic Program";
"D06" -- "Academic Program Description";
"D06" -- "Academic Careers Code";
"D07" -- "Academic Plan Type";
"D07" -- "Academic Plan Type Description";
"D08" -- "Academic Plan";
"D08" -- "Academic Plan Description";
"D08" -- "Academic Plan Type";
"F01" -- "Awarding Body Year";
"F01" -- "Awarding Body Sitting";
"F01" -- "Awarding Body";
"F01" -- "Awarding Body Description";
"F02" -- "GCSE Subject Code";
"F02" -- "GCSE Subject Code Description";
"F02" -- "EBL Subject Code";
"H01" -- "Subject Code";
"H01" -- "Department Name";
"H01" -- "Department Code";
"H03" -- "Subject Code";
"H03" -- "Exam Catalogue Number";
"H03" -- "Exam Title";
"J01" -- "Grading Scheme";
"J01" -- "Grading Basis";
"J01" -- "Grading Code";
"J01" -- "Grading Description";
"J02" -- "Subject Code";
"J02" -- "Further to Class Code";
"J02" -- "Further Class Description";
"K01" -- "Degree Code";
"K01" -- "Degree Code Description";
"K01" -- "Degree Short Description";
"M01" -- "Department Code";
"M01" -- "Department name";
"S01" -- "Fees Source Code";
"S01" -- "Fees Source Description";
"Z02" -- "Ethnicity Code";
"Z02" -- "Ethnicity Description";
"Z04" -- "Disability Code";
"Z04" -- "Disability Description";
"Z05" -- "Status Code";
"Z05" -- "Status Description";
"Z06" -- "Programme Action Code";
"Z06" -- "Programme Action Description";
"Z07" -- "Programme Action Code";
"Z07" -- "Programme Action Reason";
"Z07" -- "Programme Action Reason Description";
}

The next step is to see what else we can link into this, and maybe also draw boundaries around various clumps according to which unit owns those particular sets of data (MISD, the Computing Service, the Library, Caret/Camtools, the Departments, Cambridge University Press etc etc.). After all, even if we can find one data set that does manage to key into another, political or data protection boundaries may make it…. difficult to link those data sets and get the data flowing…