OpenRepositories 2010 Presentation Proposal (Long Version)

Any repository solution provides facilities for Creation, Management, & Editing of Content as well as facilities for Searching & Browsing through that content. Experience has shown that when a solution binds these two areas of functionality together too tightly, the system becomes brittle and unworkable, discouraging innovation. Our work on the Hydra project has produced a flexible and intuitive solution that combines these two areas in an almost entirely decoupled fashion. This solution, which is already working in multiple Hydra applications, is built on a three-part pattern where Blacklight handles Search & Discovery, ActiveFedora handles Creation, Management and Editing of Content, and a small application called Shelver supplies the crossover point by indexing the content into Solr so that it will show up in Blacklight. This three-part approach reflects a strong pattern for designing and/or improving repository solutions. The main pivot of this approach is to treat indexing as its own separate part of the application and to allow that indexing processes to evolve constantly as part of the application development cycle.

This work is the product of combining established best practices, best of breed software, and lessons learned from an iterative approach to application development. While our implementation is focused on Fedora Repositories, the software could be used in multiple contexts and the pattern is certainly applicable to any content-oriented application.

The anatomy of a Hydra Application

Note: This is a working model of the functional structure of a Hydra application. The complete designs for the final features and functionality of Hydra applications reach far beyond what is presented here. For more information on the greater vision around the Hydra project, please refer to the Hydra Project pages on the Fedora Commons wiki.

• The portion of a Hydra application that handles Creation, Management, & Editing of content is provided by the Hydra Core, which consists of ActiveFedora along with a few Hydra “helpers” which integrate ActiveFedora into Ruby on Rails.
• The Search & Discovery portion of a Hydra application is a Blacklight installation - nothing more, nothing less. As with any Blacklight installation, its behavior and appearance are likely to be customized but otherwise there is nothing Hydra-specific about it.
• Shelver (which can be run either from within the application, from the command line, or as a JMS listener) indexes content and its salient metadata in Solr, usually pulling that content from Fedora.

These three components — Blacklight, and Hydra Core and Shelver — work in concert to present a consolidated repository solution to the end user. Meanwhile, the three components are sufficiently decoupled that each could be run as a freestanding application. They interoperate based on a minimal contract that revolves around decisions about what information should be in Solr and how it should be represented in the Solr index in order to achieve the ideal search experience.

In the process of customizing or extending a Hydra application, some changes require modifications to all three components, but most changes impact only one or two of the components at a time. This makes it very easy to iteratively improve the application and adapt to real world needs.

This structure grew naturally out of a process of exploration. In early 2009 developers at UVA and Stanford discovered that it was relatively easy to put Blacklight on top of a Solr index that had in turn been been populated by ActiveFedora — effectively turning Blacklight into a search & discovery interface for that Fedora repository. Based on that, we tried dropping ActiveFedora-driven views & controllers for editing Fedora content into the same Ruby on Rails application as an existing copy of Blacklight. It worked like a charm. The two systems happily coexist. What we found was that as long as we could change and refine how the metadata percolates from Fedora into Solr, getting Blacklight to operate together with the ActiveFedora management component was completely straightforward.

With most Hydra applications, all content is stored in a Fedora Repository. However, there is nothing to prevent you from adding non-Fedora content to solr and having it show up in the (Blacklight) search & discovery views. Of course, that content will not be editable unless you implement the code to integrate with that content’s host system.

Best of Breed: Blacklight & ActiveFedora

Blacklight is a next generation Search & Discovery tool. It was intentionally designed to serve a single purpose - Search & Discovery - without having any knowledge of indexing, cataloging, or even the location of the content it’s searching through. Whatever information you have in your Solr index, Blacklight will help you expose a rich, faceted search interface for exploring through that information and displaying detail views of individual records. This open-ended design made it very easy for us to integrate Blacklight directly into our Hydra applications as-is. The ease with which we achieved this seamless integration is a testament to the quality of Blacklight’s design.

ActiveFedora is a Ruby library that encapsulates the details of interacting with a Fedora repository and provides high-level tools for defining data models, creating Fedora objects, and modifying the data associated with those objects. While opening the door to rapid, iterative application development, ActiveFedora also attempts to expose and accentuate many of the strong design patterns inherent in Fedora. ActiveFedora’s emphasis on flexibility and design patterns provided us with many opportunities to make our Hydra applications robust and re-usable. In particular, ActiveFedora makes it possible for multiple Hydra (and non-Hydra) applications to operate on top of a single Fedora repository, thus achieving the goal of providing many lightweight views onto complex, heterogeneous repository content.

Hydra Core: the building blocks of an interface for creating & editing Fedora content

Hydra Core provides the Ruby on Rails code that handles Creation, Management, & Editing of Fedora content. This primarily consists of Rails helpers for generating edit interfaces and Rails controllers to handle the submissions from those interfaces.

Fedora allows a great amount of freedom with respect to data models and metadata. As a result, we could not simply create a single generic content management interface in Hydra. Instead, we created a number of “helpers” that allow you to deal with your Fedora content and its metadata at a high level of abstraction. For example, the editable_metadata_field helper generates the HTML for displaying an editable version of whichever metadata field you specify. All you have to know is what field you want to display and where it is stored within the object. Everything else is handled for you.

The forms generated by the Hydra helpers need somewhere to submit their data to. This is handled by the Rails controllers provided by Hydra core.

Underneath the helpers and controllers, Hydra Core relies on ActiveFedora to handle connecting with Fedora, modeling Fedora objects & metadata, and performing the basic operations of creation, retrieval, updating and deletion.

Shelver: a script that brought unexpected freedom

When we wrote Shelver, we didn’t anticipate how integral it would become to the application development process. Shelver started out as something extremely simple. A developer at Stanford initially wrote it in order to populate a Solr index with some working data from a Fedora Repository. Over time, as needs arose, we built out the script to be more robust. It soon became apparent how crucial it is to be able to modify and/or augment the behavior of your indexing tool. In most other systems, the indexing tool is either implicit (relational databases) or external to your application and difficult to re-configure (ie. Fedora GSearch). As a result, when working with other systems, discussion of (and changes to) the indexing strategy are kept to a minimum. In contrast, since we had Shelver at our disposal, we found ourselves constantly tweaking it to satisfy new functionality. This ability to tweak our indexing routines gives us radical freedom to explore new features, improve the search experience, and increase the quality of search results.

Eventually, we pulled shelver into the Hydra app itself so that we could trigger it as part of the save/update process, though we retained the hooks for running it as a command line tool as well. We also did this because we found that changes we made to shelver often corresponded to changes in the search interface. Shelver was continually evolving in conjunction with the application, so it made sense to track the code with a single versioning system.

Approaches to Indexing: from RDBMS to Fedora + Shelver

RDBMS (data model & search index combined)

If you rely solely on a Relational Database to drive your application, your data model (the database schema) is also your indexing model — any search oriented changes necessitate changes to your data model. This makes it difficult to refine and extend the search & discovery portion of the application without impacting other areas of functionality.

RDBMS + Solr (separate search index from data without much thought to the conceptual differences)

A number of tools exist for pulling content from a relational database into Solr. This achieves the goal of separating the search index from the data itself, allowing you to have an indexing model separate from your data model. However, often with these systems the indexing methodology remains tightly bound to the data model. This is more of a conceptual stumbling block than a technical one. It’s easy to underestimate the complexity and distinctiveness of indexing for search & discovery. It is not enough to index your data in Solr; you must think differently about how and why you put it there. This “thinking” must be manifest somewhere in the application’s code, ideally separated from the rest of the application.

Fedora + GSearch + Solr (freestanding tool specifically handles indexing)

Fedora is explicitly designed with the idea that you should separate your data model from your indexing solution(s). This allows us to use any variety of content models and metadata schemas to represent our content in Fedora while pulling that information into any number of indexes to suit specific searching needs. The most common indexing approach with Fedora repositories is to use Fedora GSearch to pull Fedora content into a Lucene, Solr or Zebra index. This approach has the benefits of completely separating the data from the index while also providing a freestanding, configurable tool to handle the process of indexing.

GSearch was designed with the goal of enabling 1) full-text searching of Fedora content and 2) indexing of arbitrary XML metadata from Fedora objects. It runs as a web application alongside Fedora, listening for JMS messages or REST API commands telling it to (re)index Fedora objects. The process by which GSearch indexes the content is implemented as a mix of XSLT and Java code.

GSearch establishes the strong best practice of decoupling both the search index and the indexing process from the data itself. This pattern was part of Fedora’s design all along, but thus far GSearch has been the clearest manifestation of it.

Because it was designed specifically to enable full-text indexing using XSL Transformations (XSLT), GSearch operates on the premise that you are transforming the content in order to put it in the index. In a basic system, transformations are sufficient. However, most repository solutions eventually need to actively process the data when indexing it, performing complex actions in order to decide how to populate the search index. Because XSLT does not lend itself to performing such complex processes, you must modify Java code if you want to implement this type of processing in GSearch. Modifying that code has proven daunting for most. Very few projects have taken on the challenge of modifying the GSearch code itself. Those that have modified the code have only done so in minimal and relatively stable ways.

Fedora + Shelver + Solr (allowing the indexing methodology to constantly evolve)

If you want to provide a great search & discovery experience in your application, you must make it easy to iteratively “massage” the indexes. Anyone who manages a Blacklight or VuFind installation on top of their ILS (or anyone who participates in Code4Lib) can attest to the fact that in order to achieve a truly successful search & discovery experience you must continually refine the way you index your metadata. Little changes in your indexing methodology can bear tangible results for end-users.

In building SALT, the first Hydra application to combine Blacklight with ActiveFedora, we created Shelver as an alternative to GSearch because we wanted to be able to specify our indexing process in Ruby code and, where possible, we wanted to use simple mapping files rather than being forced to use XSLT and Java to perform those actions. We assumed that Shelver would be a relatively simple application whose code rarely changed. After all, when using GSearch we rarely changed the XSLT and basically never changed the Java code. We expected that the same would be true with Shelver. We were wrong. Shelver is constantly changing because we are constantly coming up with new things that we want to do to improve the search & discovery utilities in our Hydra applications. As time passes, the code of Shelver itself has stabilized, but the instructions for how to index specific data from Fedora continually morphs as a regular part of the application development process. In fact, touching the Shelver code has become such an integral part of our work that we can’t imagine building a repository solution without this kind of freedom.

Conclusion, Observations and Best Practices

To review, some of the recommendations coming out of this work are to

• use indexing as the crossover point between decoupled solutions for searching through and managing your content
• make the indexer an explicit, evolving part of your application
• use flexible components that were designed with iterative development in mind
• re-use established best practices where possible
• combine best of breed solutions for astounding results

We were pleasantly surprised to discover how easy it was to combine Blacklight and ActiveFedora into a single Fedora solution. The three-part pattern that emerged out of this effort, which now constitutes a basic Hydra application, builds on well established practices and serendipitously combines them in a stable, intuitive way. This in turn provides a strong base for us all to carry out a great amount of innovative work in the coming years.

I convinced Thursday’s emcee to let me conclude the morning announcements by showing Dev8D’s video saying hello to us and recording a quick video of the whole conference saying “Hi Dev8D. How’s the weather in London?”.  Lesson learned: when recording 200 people saying hello, keep the microphone away from your mouth.

A snowstorm was threatening to disrupt everyone’s travel plans so we spread to the winds pretty swiftly once the figurative gong rang at noon, but I managed to grab lunch at Asheville’s infamous 12 Bones Smokehouse with some friends before departing.

Code4Lib 2010 Last Day from Matt Zumwalt on Vimeo.

Mark Matienzo at Code4Lib 2010 Recommends “Ask Anything” from Matt Zumwalt on Vimeo.

Matt Cordial at Code4Lib 2010 - Lightweight Tools and Robust Testing from Matt Zumwalt on Vimeo.

A Little Blacklight Lovefest at Code4Lib 2010 from Matt Zumwalt on Vimeo.

1. Linked Data API/Data
2. EDINA - Unlock Places API & Geo/Data
3. Building the best IMS Basic LTI Tool
4. Blackboard/Learning Tools Interoperability API/Data
5. Memento: Time Travel for the Web
6. Internet Archive API/Data
7. Mobile API/Data
8. Microsoft Zentity
9. Eprints 3.2 API/DATA
10. MLA Data

The bounties include cash, an iPod Touch, an Acer 751 Netbook, and Amazon vouchers. Microsoft Research is offering the biggest prizes: both an Xbox 360 and a multi-touch Tablet PC.

Deadline for submissions is 10am tomorrow.

Although I hope that next year, Code4Lib and dev8D won’t overlap, if they should, I’d like to see if we can’t build some cross-conference teams to compete. There’s certainly enough intersection of interests that if we can bring some of the work of Code4Lib participants to bear on the challenges at dev8D, we’d see some pretty amazing things.

Aside from the pure geek-appeal of a self-replicating, 3D printer that accepts designs from Art of Illusion or Blender, is the amazing potential of RepRap to change the world as we know it. That sounds a little breathless and it’s certainly not entirely true: RepRap can’t fully replicate itself, yet (currently, just better than 50%). But ask yourself what might happen if everyone had a fabrication plant at home. At a cost of only £300 in materials to produce, RepRap could make manufacturing accessible to even some of the poorest communities. What happens to the sales of entire classes of goods and manufacturing if you, as Google’s Chris DiBona observed, have “China on your desktop”?

The final half-day of Code4Lib 2010 began with a roving, sublimely geeky keynote address by Paul Jones, Director of ibiblio.org (you know - host of project Gutenberg, etc.).  His opening challenge: “What is the levenshtein distance between Paul Jones and Cathy Marshall?” Little did he know that zoia the irc bot can compute levenshtein distances on the fly.  From this opening volley, he galloped through brain science, geek slang, asbergers, quality control, zombies, physical anthropology and catfish.

If you take a look at the code4lib irc log you can tell exactly when Paul posed the levenshtein distance challenge (14:17:31 UTC/GMT) and exactly when he asked the audience to provide their answers (14:48:49 UTC/GMT).

[14:48:49]	<gsf>	@levenshtein “cathy marshall” “paul jones”
[14:48:49]	<zoia>	gsf: The levenshtein distance of cathy marshall and paul jones is 12.

Lighting Talks

• Batch OCR using Open Source Tools - Jonathan Brinley
• VuFind at Western Michigan University - Birong Ho
• Please clean my data! - Vinita Tuteja, National Library of Australia
• Library a la Carte update - Kim Griggs and Michael Klein
• Serving Fedora content using Drupal and Fedora content models and disseminators - Alexander O’Neill, University of Prince Edward Island
• Library Values for the Internet - Jodi Schneider
• GroupFinder: Who/What/Where/When for patrons at your library - Joe Ryan, NCSU
• Digital Video Made Easier - Datastore and conversion using video APIs (YouTube, blip.tv): Jason Clark, Montana State University
• EAD, APIs, and Cooliris: providing access to digitized archival materials. - Tim Shearer

Final Presentations: Developing Mobile Apps and Developing bots for Google Wave

This year’s conference ended with one presentation about Google Wave and two about Mobile App Development.

Sean Hannan’s You Either Surf or You Fight: Integrating Library Services with Google Wave.  Wave is Google’s next-gen communication/collaboration/publishing platform built from this premise:

Email is based on the only metaphor we had for text communication — the postal mail system.  Given our contemporary idioms of emails, instant messaging, blogs, microblogs, VOIP, video teleconferencing, and collaborative document editing (ie. Google Docs), what would you get if you reinvented email today?

After a heraldic launch last year, Google Wave has been slow to gain traction even with the most adventurous of technology early adopters.  Wave is definitely visionary, but it also might be a technology before its time — Either that, or next year you will be reading these posts via Wave and you will wonder “How did people ever communicate without this?”  Time will tell.

When Sean polled the room to see who already had Wave accounts, almost all of the 250 people raised their hands but nobody was logged into their accounts. This bears testimony to both the tech fashion-forwardness of Code4Lib and Wave’s lack of traction as a tool that people might actually use.

Sean talked through the process of building, running and debugging an interactive bot for Wave that allows you to query some of Johns Hopkins web services on the fly from a “wavelet” in Google Wave.  His bot is called uncle-milty and is located at http://uncle-milty.appspot.com

This is what happens when smart people people come together, communicate often (year round), and respect each other.

Building a Better Advanced Search

Naomi Dushay & Jessie Keck’s presentation on building a better advanced search for SearchWorks is a key example of this unabashedly technical yet clear and concise communication.  They opened with a clear presentation of the real-world situation, including usability goals, etc.  They followed this with a breakdown of the desirable features and the possible ways to implement them.  Naomi then plunged into seriously complex solr config details that she managed to make intelligible step-by-step.  Hint: she used screenshots of the code with the salient items circled.

Drupal 7: A more powerful platform for building library applications

Cary Gordon from the Cherry Hill Company showed us the new features in Drupal 7.  I expected to be bored, since I don’t use Drupal and can’t stand the sight of PHP code. Instead, I was pleasantly impressed and intrigued by the features Cary walked through.  I keep hearing that Drupal has matured substantially in recent years, but I didn’t believe it until now.

Enhancing Discoverability With Virtual Shelf Browse

Andreas Orphanides, Cory Lown, and Emily Lynema from NCSU showed us their snazzy virtual bookshelf with an “infinite shelf” allowing you to scroll through book covers five at a time ad infinitum.  One wonderful idea is the use of “faux covers”, where they generated something that looks like a real book cover whenever a real cover image couldn’t be retrieved for a search result.

As with almost any Code4Lib presentation, this would have been intriguing if they only gave a demo of the software but they didn’t stop there.  80% of the time was spent explaining how they did it, what technologies they used, how they thought about the problem, what worked well and what went wrong.  Gosh I love hackers who share.

How to Implement A Virtual Bookshelf With Solr

In their third presentation of the day (remember, presentations were chosen by open popular vote), Naomi Dushay and Jessie Keck showed us their work around implementing the much-desired support for browsing through digital collections using a virtual shelf organized by call number.  This feature sounds simple, but the implementation proved more challenging.  To start with, the librarians couldn’t come to a consensus about what constitutes “correct” ordering.  Solution: ignore the arguing librarians and go straight to the users!

The real challenge lay in sorting out 8 million call numbers from numerous libraries using a variety of call number systems (ie. LC, Dewey, SUDOC, Thesis, etc.)  Who knew that call numbers were so complex? Not I, until now.

I knew these guys were good - I’ve collaborated with them, I’ve seen their code - but wow.  I didn’t realize how remarkably good they were until this third presentation.

Lightning Talks & Breakout Sessions

The day closed with 13 Lightning Talks followed by 6 Breakout Sessions.

Lightning Talks:

• LibX Update - Godmar Back
• How to build a Virtual Bookshelf Without Solr (or MySQL) - Maccabee Levine
• VIVO, an interdisciplinary national network - Paul Albert
• WolfWalk, two ways - Jason Casden
• Custom metasearch widgets - Alex Smith
• Node.js development - Gabriel Farrell
• Catalog Auto-suggest using SOLR - Jill Sexton
• EmeraldView, a PHP frontend for Greenstone - Yitzchak Schaffer
• Faceted browse on the cheap - Tom Keays
• EAD, APIs, and Cooliris: providing access to digitized archival materials. - Tim Shearer
• Kill the Search Button - Michael Nielsen, Jørn Thøgersen [facilitated by Roy Tennant]
• You Heard It Here First… - Roy Tennant
• File Information Tool Set (FITS) - Spencer McEwen

Breakout Sessions:

• Mobile Dev
• Blacklight - Bess Sadler
• Tools4Lib - the best tools used in your job - Devon Smith/decasm
• Let’s Link Our Data - If you have data, metadata, vocabularies, or just about anything else you want to link, show up and we’ll figure out what to do with it! - Ryan Scherle/ryscher
• Homebrew - building your own instead of using Fedora/DSpace/Blacklight/whatever - Esmé Cowles/escowles
• API queries to vocabularies - Ya’aqov Ziso

Linked Data was the dominant theme today, starting with the pre-conference Linked Data Meetup which included presentations and workshops from Tom Scott, Patrick Sinclair, and Silver Oliver of the BBC and John Sheridan & Jeni Tennison of data.gov.uk among many, many others.

Lin Clark of DERI ran a Drupal RDFa workshop in the afternoon. She demonstrated RDF Mapping, the RDF SPARQL Proxy, and the RDF SPARQL Endpoint modules and gave a preview of the RDF features to expect in Drupal 7 core.

One session I really wanted, but wasn’t able, to attend was Jim Downing’s Multicore Processing session. Not a terribly interesting title, but he worked through mining linked data from data.gov.uk, using the Clojure library Incanter to find correlations in the data, and finally, parallelizing the data processing.

The dev8D spirit of developer happiness was nowhere more evident than in this year’s new attention to electronics. I sat at a table watching Emma Tonkin build an electronic whiteboard with a nano-projector and Wiimote. Walking over to the projects room, I saw Ben O’Steen working with facial recognition software, Dave Tarrant turning off and dimming lights over the local network, Julian Cheal wiring each room’s entry with an RFID reader, and Ian Ibbotson programming an Arduino. Tomorrow, I’m told, we’ll have someone showing off a RepRap 3D printer.

I find the impromptu meetings as valuable as any of the structured components of dev8D. I sat down with Andy Turner, a researcher at the University of Leeds, who introduced me to GENeSIS, a mad-scientist level modeling and simulation project that I thought was absolutely fascinating. Just as serendipitously, as we wrapped up Day 1 of dev8D at the pub (where else?), I found myself drinking a pint with Scott Wilson and we got to talking about Apache Wookie and HTML 5.

See you tomorrow!

Chris Beer from WGBH (Boston’s Public Television; producers of Nova, Julia Child, etc.) walked us through the inner workings of the new WGBH website that runs on Blacklight, Fedora, and LightHTTPD.  Without getting bogged down in the details, he touched on many of the technical and organizational challenges they faced and how they addressed them.  Some of the technologies in the mix

• Blacklight
• Fedora
• PBCore: Metadata Standard for Media based on Dublin Core
• ODRL: Open Digital Rights Language
• JQuery

Chris did some amazing things with JQuery and FlowPlayer in his customized Blacklight views.  One example: You can browse to a point in a video by clicking in the transcript. If you then play the video, the transcript automatically scrolls with it — accurately.  This was made possible by the fact that WGBH already has detailed timecodes embedded in their transcripts.

The data model for a single episode in the WGBH archive fills an entire page — raw video, edited video, music, images, post-processing, transcripts, etc. with rights and permissions varying for each part.  This highlights the grace of Blacklight, which allowed him to simply leave that system in place and merely expose the public parts with a slick, faceted search & discovery interface.

One detail that Chris called out was the fact that Fedora has trouble with large (5GB+) datastreams.  He pulled up the corresponding (unresolved) tickets in Fedora’s Jira to establish the fact that this is a long-known unresolved problem.  He then added that WGBH’s own internal (proprietary) DAM system has the same problems.  At least with open source software, addressing a bug like this is a matter of public process and community initiative rather than being subject to the whims of a single organization.

WGBH will soon be launching a number of public websites based on this Fedora+Blacklight combination.

Becoming Truly Innovative: Migrating from Millennium to Koha

Ian Walls from NYU gave a humorous account of finally ditching proprietary Integrated Library Systems, which he represented as battling kittens, in favor of the open source ILS called Koha.  He went through the details of how they handled the challenge of migrating their data from iii (the proprietary ILS) to Koha and closed by telling the room that the process he used should work for anyone with a similar proprietary install.  The NYU migration took a total of 3-4 months without any all-nighters.

All of the code he used is available at contribs.koha.org. I don’t personally work with ILS systems, so I can’t say I have a solid read on the impact of this information, but I got the distinct impression that this was very encouraging news for a many of the people in the room.  When someone from the audience asked for a show of hands “How many people would like to migrate away from iii?” more than a third of the room raised their hands.

Ask Anything! (aka. Human Search Engine)

I know the Dev8D people will dig this one.

Dan Chudnov from Library of Congress facilitated an open-ended session where anyone with a question/request/missed connection could grab a mic and announce their desire.  In short, the “If there’s anything that you want to ask everybody, now is your chance to speak up.” The core idea was to give people an opportunity to tap the collective knowledge of the room.

The process was to hear a question, identify people who want to respond, and then move on to the next question.  The full-room discussion was kept to a minimum.  Here are most of the questions asked (I missed a few).

• Switching from Perl: What language should I switch to?
• How do I make an API accessible only from specific domains
• Who is using my library (pymarc) and how are you using it?  Please tell me so I can make it better.
• How do I extract files from Internet Archive .arc format?
• Is anyone interested in helping my organization port a custom staff app from Millenium ILS to Aleph ILS?
• Is there any interest in a 1-2 day Blacklight meeting?
• Has anyone used the California Digital Library digital curation microservices?
• Please tell us: What do Librarians (non-techies) need to know about software development?
• What’s the best way to do complex log analysis? (answer: splunk)
• I want to allow people to add data to my site without maintaining user info for them?  I want to use something like OAuth but it has to work with something like curl as the client? (recommendation: RPX)
• How do I model dates & date ranges in RDF?
• Proposal for a breakout session on building homebrew digital repositories (instead of Fedora, DSpace, etc)
• Request for 10 minute tutorial on 3D graphics programming
• Anyone interested in modeling archival description in RDF? If so, join http://groups.google.com/group/semantic-archives
• Is anyone using OpenCalais module for Drupal?
• Is anyone here going to be working on the OLE project?
• I’ve created software for hiding borrower information in library systems, but how can I share the code without making it vulnerable to “the man”?
• Is anyone doing work with HTML5? Especially geodata services, etc?
• Are there any libraries out there using Plone for their website?
• Who else is developing Facebook apps for their libraries?  Any ideas how to make a useful Facebook app for a library?
• Is anyone using MARCLogic or eXist in production systems?
• Since this seems to be really working, Should we start something like a Stackoverflow for library hackers? (answer: Yes.  Awesome idea.)
• We’re working on search & discovery with non-roman scripts in a solr/lucene context.  Please provide suggestions.
• Is anyone working on natural language processing implementations for machine learning?
• We’re moving all of our production systems to cloud environments.  Has anyone else done that?
• Is there a JSON library for MARC? (answer: it’s built into evergreen, OCLC is also looking into it.)

The verdict on Ask Anything: resounding applause, many smiles.

Surprise Trivia

Alex O’Neil from University of Prince Edward Island emcee’d a very fun round of trivia before we adjourned for lunch.  It was wicked fun.

For those who were not in attendance, I strongly recommend seeking out the slides (and the video) of these presentations.  They were all engaging, concise and informative reviews of topics that are important for any developer who wants to do their job with professionalism, confidence and agility.

Iterative Development Done Simply

Emily Lynema from North Carolina State University gave a rapid-fire account of agile development in the real world.  Two years ago, at a conference like this the presentation(s) about agile development tended to be spoken in the abstract - “Did you realize we could work like this?”.  Now that agile development has properly caught on in many organizations, it’s rewarding to hear someone give a breakdown of “Here’s how it actually works for us on our projects.”

Vampires vs. Werewolves: Ending the War Between Developers and Sysadmins with Puppet

Bess Sadler from Stanford (formerly of UVA) and the Blacklight Project gave her Vampires and Werewolves rallying call for ending the war between developers and system administrators.  Developers are responsible for making new apps, implementing new features, and offering new technologies that meet users needs; it’s their job to innovate.  System Administrators are responsible for ensuring the stability of systems; it’s their job to be risk-averse.  This is usually a recipe for protracted pain.  Bess proposes that we can solve this by changing the way we think about the relationship between these two groups.  She also proposes using a number of industry best practices.

• “Let go of the anger. Really listen.”
• Take the system administrators out to coffee
• Test your code & use continuous integration (ie. Rspec with rcov & Hudson)
• Use system monitoring (ie. nagios) to monitor specific features as well as apps
• Document your software (”You can’t RTFM if TFM doesn’t exist.”)
• Use an automated system configuration tool (ie. Puppet)

I am Not Your Mother: Write Your Test Code

Video of the Presentations

All of the Code4Lib presentations are being recorded.  They will be posted on the conference website sometime in the days/weeks after the event ends.

Possibly next year we will be able to stream the presentations live.  That would certainly make the irc channel easier for outsiders to follow.