Earlier this week I attended the Project Bamboo workshop at Princeton and thought I’d share a discovery I made while trying to complete the pre-workshop reading assignments attendees received:

1. Please read the proposal in its entirety. The proposal can be found at:

http://projectbamboo.org/files/docs/bamboo_proposal.pdf

2. Please read the Identifying Scholarly Practices handout. This handout can be found at:

http://projectbamboo.org/files/3/Identifying_Scholarly_Practices.pdf

All through the week leading up to the workshop I figured I’d surely get around to reading those documents but I never did. The night before I was to leave, I realized it just wasn’t going to happen.  Then I had a thought: why not run the text of these documents through my Mac’s “Text to Speech” service, capture the output and later listen to it as a podcast during the dead time of my  3+ hour drive up to New Jersey?

I launched WireTap Studio (to capture the sound) then opened the proposal PDF in Preview, highlighted the text of the document, selected “Services -> Speech -> Start Speaking Text” under Preview’s application menu and hit the record button.  I stopped after 30 seconds and imported the mp3 into iTunes. Sounded terrible—lots of ambient noise and sort of muddy sound quality. Oh yeah, I was just picking up the tinny sound of the Macbook’s speakers with the low-quality built-in mic, no wonder it sounded so bad.

Next tried to use WireTap Studio to intercept the audio stream (could also do this with Audio Hijack Pro) and found that I couldn’t seem to interrupt (and grab) the Speech services audio. It wasn’t associated with the application and no matter what I selected as input, it didn’t get the speech audio. I assume it can be done but I wasn’t having any success. Time to Google…

Doh! Turns out there’s a unix command baked right in to OSX (since 10.3) that not only does exactly what I was trying to do, it does is much faster than the real-time capture I was experimenting with. Meet “say”

say [-v voice] [-o out.aiff] [-f file]

So, I opened the PDF in Preview, used Command-A to select all the text, pasted it into a text file using BBEdit, chopped out the parts I didn’t care about then saved it to the desktop. Then in a terminal window, issued this command:

say -v Alex -o ~/desktop/bamboo.aiff -f ~/desktop/bamboo.txt

Alex is the “new and improved” voice in OS X 10.5 (Leopard). He has much better inflection and sounds much more human and much less Cylon. If you really get into this (or need a voice that deals with a language other than US English), you can purchase additional voices from Cepstral (http://www.cepstral.com). The voices are roughly $30 each.

In a little less than 3 minutes wall time, ’say’ produced the bamboo.aiff file that was easily imported into iTunes (2 hours, 5 minutes of audio). Here’s a representative sample of how Alex sounded with the material:

…information technologists to collectively tackle the question: How can we enhance arts and humanities research through the development of shared technology services? This proposal represents an 18-month planning and community design program, the Bamboo Planning Project, where through a series of conversations and workshops, we will map out the scholarly practices and common technology challenges across and among disciplines, and discover where a coordinated, cross-disciplinary development effort can best foster academic innovation. Input into the Bamboo process…

Digital Library Developer

We’ve posted a job advertisement for a Digital Library Developer and I encourage you to apply if you have an interest in building the sort of tools today’s library could really use but tomorrow’s digital library will absolutely require.

You’ll find the full posting (and online application form) at http://jobs.gmu.edu (position number FA730z).

Here’s the heart of the announcement:

George Mason University, University Libraries seeks a Digital Library Developer to join our innovative Digital Programs and Systems division as we build new ways to deliver library content and services.

Duties include: Anticipating and investigating trends in digital library technology so we can respond quickly to new opportunities. Provide primary support for new initiatives in resource discovery, digital preservation, knowledge management, and scholarly communication. This position reports to the Associate University Librarian for Digital Programs and Systems.

QUALIFICATIONS: ALA-accredited MLS or ALA-recognized foreign equivalent or Masters in information science or information systems or a related field required. (Other advanced degree may be considered.) Ability to work independently on technology implementation projects; excellent analytical skills to support problem solving and systems analysis; ability to handle multiple, simultaneous priorities. Progressively responsible information technology work experience and demonstrated competence with web development tools and technologies. Capacity to meet requirements for reappointment and promotion under Librarian’s Handbook.

PREFERRED: Experience with at least one of the following technologies: SOLR/Lucene; Java Enterprise Edition (JEE) to include servlets and JSP with Tomcat or similar technologies; LAMP (Linux, Apache, MySQL, PHP,/Perl/Python); experience with XML and related technologies (particularly XPATH and XSLT).

Salary range: $52,500 - $62,500

Strong Passwords

I’ve written previously about the value of a strong password (or to be more precise, about the hell that can break loose when you have a weak one).   I recently stumbled across a quick way  to generate a really strong password that you can then let your Mac’s keychain remember for you.    If you want an 8 character password, in a terminal window, type:

openssl rand -base64 6

You’ll get something like this:

36qOaUvH

For reasons I haven’t taken the time to discover, if I don’t run the command as root, I also get an “unable to write ‘random state’” message after the password displays.   That’s annoying but not really a problem.

Zotero’s syncing

A new preview build of Zotero (1.5) is available from my friends over at the Center for History and New Media.  Yes, this version lets you sync your Zotero database across multiple computers.   Hallelujah!

I installed it on my office MacPro earlier today and on my laptop at home this evening.   Once I had the 1.5 release on the second machine, I added my forum username/password in the Zotero add-on preference pane and hit the sync button…in seconds I had the 11 records that my desktop machine sent to the sync server earlier today.  Cool.

My personal research toolbox consists of DevonAgent (for web-based research), a ScanSnap (to convert paper to PDFs) and DevonThink Pro Office (to OCR the PDFs and then store and index the output of both the ScanSnap and DevonAgent).   This combination works really well for me and thus I never built a large Zotero database.  Now, with syncing, I’m thinking I’ll begin working Zotero into my information workflow (ingest into Zotero from whatever browser I’m using at the moment, then export relevant records to DevonThink as appropriate).

Project Bamboo

Off to Princeton next week for a Project Bamboo workshop.   I’m sure it will be an interesting and informative experience but right now I’m just in awe of the scope of planning.   For me, planning is the stuff you do after you’ve been working for a while and something begins to take shape—you start planning how it might actually be made useful.  Project Bamboo plans for eighteen months with workshops held around the country (and Paris, too)?   This is very different and I’m looking forward to seeing how it works up close.

It seems there’s some sort of new web development framework released every week or two but the other day I found one that shows a lot of promise: Sproutcore.

Odd name but an interesting concept. At the most recent WWDC (Apple’s World Wide Developer’s Conference), Sproutcore was revealed as the “engine” behind many of the new services on Apple’s .Mac replacement (MobileMe). Many are suggesting the real purpose is an open-source, plugin-free alternative to Adobe’s Flash and Microsoft’s SilverLight. If you’re interested in how something like Sproutcore fits in with cloud computing, Google, Flash, and the future, you should read the “Cocoa for Windows + Flash Killer = SproutCore” post on Roughly Drafted from June 14th.

From the Sproutcore site:

What is SproutCore?

SproutCore is a framework for building applications in JavaScript with remarkably little amounts of code. It can help you build full “thick” client applications in the web browser that can create and modify data, often completely independent of your web server, communicating with your server via Ajax only when they need to save or load data.

I spent an hour or so working through the “hello world” demo and it’s cool. You do development coding in Ruby with an interactive server process that simplifies the code-test-debug-code cycle. When done, there’s a standalone SproutCore utility that converts everything into static Javascript and CSS files—ready for deployment under Apache or whatever.  Here’s my ‘production’ version of the demo:

hello_world

I tested the look on both Windows (Firefox and IE7) and Mac (Firefox 3) and for this simple demo, at least, rendering was identical across platforms.  I think this is going to be a framework to watch.  It’s open source, doesn’t rely on plugins, is reasonably platform neutral (I’ve seen implementations on Ubuntu and Windows boxes) and relies on basic internet standards (Javascript and CSS).

http://www.sproutcore.com

I often hear my fellow librarians lamenting the fact that so many students fail to appreciate the wealth of resources available to them.   I agree, up to a point, but  they have to qualify the scope of their complaint the next time I hear it—every day I see students going to great lengths to use our e-resources.  Trouble is they’re not our students.

Earlier today I decided to do a quick scan of our log files, sorting for multiple logins on the same username from different IP addresses. Wasn’t long before my investigation led me to this page (excerpted below):

View yourself at: http://www.smso.net/forum/forumdisplay.php?f=317

If you go to this site, you’ll find 12 pages of username/password combinations for gaining illicit access to many, many libraries around the world. Seems the domain (smso.net) is registered to Mutib Al Tamimi on King Fahad Street in Riyadh, Saudi Arabia, and operates under the name Saudi Medical Site Online.

I especially enjoyed this: explaining how to help keep the SMSO Free Science Team working for all “Researchers” (after all, sharing and helping is their hallmark). It explains why the spelling in the first excerpt looks kinda odd (e.g., IOwa State Un1versity):

If you’re in library IT and responsible for authentication issues, I’ll recommend a quick visit to the Saudi Medical Site Online—just to make sure you haven’t left a door open. As I write this, the SMSO home page advertises a new login link for a university library every day so your number will probably come up sooner or later.

It has taken about a week off and on to get all the pieces in place: installation of the scanner, software installation, camera calibration, three days lost while we waited for a replacement lighting housing, RTFM’ing and what not but I finished generating my first e-book with the ATIZ Bookdrive DIY scanner late this afternoon (the title is one I picked up in a used bookshop in Boston a few years back).

It took roughly 20 minutes to scan the 240+ page book (the scanning software indicated I was working thru the book at a 632 pages-per-hour clip) and then another 30 minutes or so for post-processing (deskewing the jpg images, applying automatic crops, despeckling the page backgrounds, improving contrast and ultimately producing a PDF). While scanning is a hands-on endeavor, for the most part post-processing runs unattended (after you interactively test and then set processing parameters).

The “optimized” version of the PDF weighed in at 18 megabytes—a major improvement over the “raw” version (500Mb). I suspect I can get that down but will have to experiment with what lower resolutions scans will do (I was working at 300dpi). I still need to work through various post-processing options (e.g., although the pages were faded the sample could use some brightening and I think I needed to use a higher aperture for better focus), but I can tell this is going to work well once I figure out the optimal settings and workflow.

Here’s a link to a sample PDF with a few pages from the book. The schematics of older ballparks are kind of interesting.

Sample (800K)

If you have sysadmin duties in a library like Mason’s (where our core technologies actually reside in the library and not the computer center), backing up is one big part of what you do. Though heretical at the time, we abandoned tape for disk-to-disk backup in the early ’90’s so while it doesn’t take a lot of any one person’s time it’s still something that has to be scripted, tested, monitored and regularly thought about. When it comes to the actual work of getting the backups done, well, it’s pretty much just one machine talking to another at cron-induced intervals.

For some of our systems, backup is a simple script containing a find /cpio combination. Here’s the key:

cd /DirectoryToBackup
find . -depth -print | cpio -pdmv /PlaceWhereYouWantToPutStuff

‘find’ steps recursively through all the files and sub-directories, sending each file’s name to ‘cpio’ which then copies it to the destination if the file is newer than a copy in the destination directory. You probably see the gotcha here: If you have deleted a file that was backed up last week it will stay in the destination directory since there’s nothing newer to overwrite it.

That’s where rsync shines: it makes sure the destination is identical to the original (deleting files in the destination directory that don’t exist in the original). But I digress..

For some of our systems we’ve created elaborate backup/redundancy environments. Our Voyager ILS, for example, lives on a striped, mirrored, hot-spared RAID5 array inside a Sun V880 which gives an immediate measure of protection against any single disk failure. Each evening, we take the system down for about 13 minutes and run a backup from this mirrored array to yet another stripped partition on a different set of drives in the same machine. When that finishes, Voyager returns and a follow-on process copies that new “backup” partition to another machine in yet another part of the library. This copy of the backup goes to a different machine each night so that over the course of seven days, we have seven copies spread out across seven different systems. Were we to have catastrophic failure, we could cycle back through these backups and surely find one that could be restored.

For most of our other systems, like MARS (our DSpace institutional/digital repository) less manic measures are sufficient. The hardware environment for that system enables us to function well on a weekly backup.

MARS lives on a XServe RAID (RAID5) array with two hot spare drives so we’re really only talking about needing to survive a failure of more than three drives (a component of the array and both spares)–which isn’t likely in the course of a single week. At weekly intervals we mirror the bitstreams partition to another RAID5 partition on a nearby server. We follow a slightly different process for the postgres-based metadata but always have several restorable copies lying about.

For those services that live on Apple XServes (e.g., E-Reserves, our research portals, MARS and parts of the library’s website), we use SuperDuper! to clone the system bootdrive (with Applications) to a spare internal drive on weekly schedule and then mirror important data via rsync to another machine.

At the center of our backup strategy for every system is an Apple XServe with a 1.8 TB RAID5 array. Throughout the week this machine sets up and tears down NFS exports to our Solaris, Apple or Linux servers for rsync mirroring of important data.

Note: To improve security, we export the relevant directory from the backup server to the specific machine we want to backup. The backup process is launched on the production machine (via cron) which then mounts the exported drive from our backup server. We never export directories from our production machines and restrict access to the NFS daemons via hosts.allow. Not foolproof, of course, but combined with a few other measures it is much more secure than failing to make backups would prove to be.

Then, weekly this large data store of backups is mirrored to a 500GB external drive attached to the XServe via the firewire port. We have two of these firewire drives and rotate them weekly: When finished, this week’s copy goes across campus to another building and last week’s drive returns and is connected to capture next week’s data. In this way, we have a copy of all data stored off site and assume it will never be more than a couple of weeks old. We’ll next have to look into getting a drive completely off campus since we’re reasonably well protected in case of something like a building fire but still quite vulnerable to a disaster with a slightly larger footprint (e.g., meterorites, flood, etc.).

Earlier this week I incorporated a Drobo into the backup regimen: as one more spot where I can store the most important 500 or so gigabytes of library data. I decided to go with the Drobo since I also had a half-terabtye of video that I needed to back up somewhere and putting it on an expensive RAID unit seemed a waste of higher-throughput space. The Drobo seemed a reasonable alternative. The Drobo isn’t fast by any means (throttled by a USB 2 interface and a controller that doesn’t appear to offer much in the way of throughput) but it is dead-simple easy to use. To quote Howlin’ Wolf, “it’s built for comfort, it ain’t built for speed.”

I put four 500GB Seagate SATA-2 drives in the drobo (yielding roughly 1.36 terabytes of protected storage) and in no time had a functioning array ready for data (by the way, SATA-I would be fine since the Drobo can’t keep up with that level of throughput either). It’s not RAID but the Drobo uses some sort of proprietary storage virtualization system. According to the documentation, if a drive in the Drobo fails, doing a hot swap with a replacement drive will restore the data and reestablish redundancy. I’ll just think of it as a black box appliance.

Attached to my Mac Pro, I just completed a backup of close to 280 gigabytes of data from an external drive attached to the same computer via Firewire 800 and on larger files, throughput averaged roughly 15MB/s according to the rsync progress reports:

/weblogs/ezproxy/FY0506/01_31_06-03_03_06.log
  1015703564 100%   15.83MB/s    0:01:01  (65313, 100.0% of 268220)
/weblogs/ezproxy/FY0506/10_25_05-01_30_2006.log
  2147483647 100%   15.97MB/s    0:02:08  (65314, 100.0% of 268220)
/weblogs/ezproxy/FY0506/ezproxy.log.030306_050306
  1847436796 100%   16.10MB/s    0:01:49  (65315, 100.0% of 268220)
/weblogs/ezproxy/FY0506/ezproxy.log.082505_102405
  1561279782 100%   15.86MB/s    0:01:33  (65316, 100.0% of 268220)

The plan going forward is take the external drive that’s pulled from our XServe each week and rsync it to the drobo before sending it across campus to its “undisclosed” location. This will give me a nearby copy of the data and thanks to the drobo’s built-in redundancy, a safer feeling than I have exposing backup data to the failure of a single-spindle drive.