One Big Library.

What questions do you have about learning programming?

dchud — Sun, 01 Nov 2009 23:13:36 +0000

In the spirit of nanowrimo I'm going to jumpstart writing a book today. The goal of the book is to help librarians learn how to program. I've talked with many people about this and don't know if I'm up for the task but it's time to give it a go or stop dreaming about it.

I have some thoughts about where to start and what ground to cover but I want to be sure it answers many typical questions new coders have along the way. I also hope to develop the text online as I go, using comments, questions, and suggestions for improvements as they come in to help ensure that the book addresses its readers' needs usefully.

So we might as well start now. What questions do you have about learning to code? What goals would you want to achieve by learning? What specific problems are you unable to solve on your own now that you might be able to solve on your own eventually if you had the right kind of support along the way?

There are many options available for doing this kind of thing online but I want to start by keeping it simple. Post your questions here in the comments, hit me @dchud on Twitter, over email, phone, in person, or whatever works for you.

Please let me know what you think and we'll see where it goes. Thanks!

Time lapse.

dchud — Wed, 02 Sep 2009 03:04:48 +0000

Chicken Soup for the Old Webapp's Soul

dchud — Tue, 25 Aug 2009 04:34:15 +0000

I cut my teeth as a legit full-time programmer at my previous job. I'd written webapps before but nothing particularly complicated. I didn't know any one language much better than others. I hadn't built something substantial from scratch and seen it through from requirements to long-term (i.e. more than a year or two) maintenance.

All of this is to say - I still maintain two apps from that period. The thing about those apps is that they reflect how I didn't know what I was doing when I started, and I only knew enough about what I was doing when I finished regular coding on them to know what was wrong but not necessarily how to fix it. So I'm left with two apps people still care about. One's been broken for a year, and I made the mistake of rewriting it from scratch, so, natch, I haven't finished yet. Don't do that.

The other one broke last night. Badly. I just fixed it, finally, but I can see nothing but bad decisions and do-repeat-yourselfness and things that need Cleaning Up in there. Even so, and despite there being one horrific showstopper bug in there that needed a fix to get it running again, this other app kept chugging along, however pokily, until a perfect storm of long-term-maintenance-mode and indifferent-and-remote-and-slapdash-sysadminnery threatened to bring down the whole thing.

So, here are some pearls of idiot wisdom for you. By which I mean to say: don't make my mistakes. They weren't original mistakes when I made 'em, but if you go and do them now, they'll just be embarrassing *and* tired.

If you're leaving an app running indefinitely, e.g. for N years until somebody might say "okay, fine, you can shut it down," set up a regular log rotation scheme. If you don't, your logs will eat up your hard drives eventually. Maybe not overnight, maybe not in a month, but in 2.8 years since you've paid regular attention to it, yeah, maybe. Learn logrotate or whatever the kids use these days, and use it.
Before you run an upgrade, check your available disk space.
Whenever you check on the server, check your available disk space. (Get the point yet?)
If your old app fails suddenly after an upgrade, and you can't tell why, it might not be all the dependencies in your old app. It might, in fact, be that your logfiles filled up your disk, ya dope.
If you clean up that mess and find that your app still doesn't work, remember: if you filled your drive, things broke unpredictably. Your data and other subsystems might not be in a stable state. If you have a database, verify that your tables aren't corrupt using something like myisamchk and fix them if they are corrupt.
If you're still stuck, go onto your friendly nearest IRC channel or list and say "hey, I'm an idiot, this old app of mine failed and I see X, Y, and Z but not what's going wrong still after I cleaned all that up." Your friendly chatmates might say lots of useful things to help you rule out the myriad things that might have gone wrong. They will listen to you vent a bit and help you lighten up a bit. They will help you focus on the problem you have to fix instead of your emotional baggage of the half-baked app you wrote and never cleaned up real nice and the ridiculous non-rotated logs that filled up the disk and broke everything. They will remind you to pour yourself a drink and chill out. And if you listen well and stay patient with yourself, they'll share your joy when you fix your problem.

And finally: always keep a good bottle of bourbon at the ready for such times.

[Much thanks to robcaSSon, edsu, rangi, erikhatcher, and wickr for the patient support!]

Apple TV Duo

dchud — Fri, 07 Aug 2009 16:28:02 +0000

I used to think that Apple's presumably forthcoming tablet was going to have to either shrink a MacBook or grow an iPhone. Now I think that was wrong.

Once I saw this rendering of a docking tablet, it seemed clear that I was looking from the wrong direction. They aren't just going to make a smaller laptop or a bigger iPod.

They're going to make a portable Apple TV.

Take the Apple TV, put it into an iMac frame with a 24" or 30" screen, and make the guts of it a docking tablet. Everything converges. Even better, make the frame work with or without the tablet docked in... because you'll want to use the tablet while on your couch watching the TV. Why not?

Need to do work? Plug it in and use it like an iMac. Want to watch TV? Sit back and use it like a TV. Need to hit the road? Dock/sync up and drop it in your bag. It all fits together perfectly.

We've already used our 24" iMac like a TV for the two years we've had it. Netflix, Boxee, Hulu streaming, DVDs, espn360.com and mlb.tv, and video podcasts. The 2008 Olympics with that msnbc+silverlight thingy. For us, then, this would be a natural next step. We need a new portable for one of us, anyway, and an Apple TV Duo would match our needs well, especially if the price came in at around the same price as a low-end iMac or MacBook. The pad itself doesn't even need to be that powerful.

What to call it, though? "Apple TV Duo" is clunky but recycles the "Duo" of their early docking laptop (of which I sold many during my brief days as a CompUSA salesperson). I still like "MacPad" but that presumes a shrink-the-MacBook approach. Any of iPod Mega/Maxi/Multi could work in theory, but those presume a grow-the-iPod approach.

Bands I've Seen

dchud — Fri, 07 Aug 2009 14:34:27 +0000

Succumbing, but not on the 'book. And unable, natch, to stop at 50.

Rush
Marillion
John Prine
Arlo Guthrie
John Mellencamp
Billy Joel
Ray Charles
Sonny Rollins x2
Los Lobos x3
The National x3
Los Super Seven
Jane's Addiction
Jesus and Mary Chain
Philip Glass
Al Jarreau
Pat Metheny
Henry Rollins
Soundgarden
Ice Cube
Red Hot Chili Peppers
Pearl Jam
Ministry
Frank Morgan x2
The Books x2
Los Cubanos Postizos x2
Marc Ribot
Broken Social Scene x2
The Shins
Arcade Fire
Son Volt
Bob Mould x2
Wilco x2
Chris Lightcap Quintet x2
Tony Malaby Trio 
Craig Taborn Quartet
Steve Turre Shell Choir
James Carter Quartet x3
McCoy Tyner
Geri Allen
Buckwheat Zydeco
Robert Cray
George Winston
Johnny Mathis
Carl Perkins
Tim Berne's Big Satan
Ken Vandermark
Dave Brubek x2
Eliane Elias
Beans
Tortoise
Apostle of Hustle
Mark Kozelek
Bang on a Can All-Stars x2
Moxy Früvous
Norah Jones
Christopher O'Reilly doing Radiohead
Sufjan Stevens
Butch Morris Conduction
Honeyboy Edwards
Clogs
Bell Orchestre
Varnaline
Spoon
Juana Molina
Brian Wilson
Buddy Guy
Dave Matthews Band
Jeff Buckley
Morphine x2
Sugar
Jimmy Page
Santana
Los Van Van
The Iguanas
The Radiators
The Black Keys
The Pogues
Joe Jackson x2
sfSoundGroup
kd lang
The Orchid
Alejandro Escovedo
Guy Clark
The Cult
Metallica
Bruce Cockburn
The Jayhawks
R.E.M.
Throwing Muses
Richard Thompson x2
Bruce Springsteen
Bo Diddley
The Decemberists x2
Max Roach
Nick Brignola
Cubanismo
Vinx
Stars
Animal Collective
The Hold Steady x2
Cesaria Evora
Jim Hall
Califone x2
Ramblin' Jack Elliott
David Broza
Etta James
Joshua Redman
The Evens
Jimmie Dale Gilmore
Jon McDonald
Bela Fleck
Lyle Lovett x2
Mike Seeger
Warsaw Village Band
Leon Russell
George Shearing
Joe Pass
Joe Williams
Diane Schuur
Emily Haines
John Trudell
--- updated 2009-10
Akron/Family
Mose Allison
Jandek
The Stills
Metric

Dreaming of a read/write Linked Data web

dchud — Mon, 03 Aug 2009 05:24:54 +0000

I just turned in my fourth (count 'em, 4!) "Libraries in Computers" column for "Computers in Libraries" magazine about Linked Data in 2009. The third is due to appear in print next month, and this one's due in October. It took me four go-rounds, though, before I could get to the key payoff, answering a nagging Collect Underpants kinda question. Surely, less understanding editors would have already sacked such an idiot who needs to write the same story four times before making any sense of it. Shhh! Don't tell them.

All this talk about Linking Data is great, because we can all follow links. But I think what will put Linked Data over the proverbial top is opening an explicit feedback loop into the Linked Data web. Like the web itself and its transition through the Age Of Web 2.0 Hype Budgets, a read-only existence is only a temporary state of affairs. When people start figuring out how to write into the web as much as they read from it — and gaining value from such interactions — is when things start to get really interesting. So let's just skip the Trough of Disillusionment, here, and jump right into the read/write Linked Data web.

I've written here before about my most recent keynote-length talk about the value I see in Linked Data. Flip through those slides far enough and you'll see a long series of seemingly disconnected screenshots excerpted from an imagined user session in a well-known digital library application. Following that is a similarly imagined user session in a well-known OPAC at that app's same host institution. Following that is a series of screenshots from a diverse set of additional web resources about the same intellectual content — the same subject and the same authority records, but different institutions (several academic libraries, a few major museums, and online encyclopedias). And, finally, there's a slide that shows a search result set from a well-known search engine that shows barely a few of those diverse sites.

The intended effect of those slides is summed up simply — nobody will find your stuff, and even if they do, they'll never find all the other cool similar stuff out there about the same thing, because we're not feeding it all into the web correctly. We're not linking these things well enough. We haven't pulled our own thickly entangled yet beautifully informative traditional web of authorized headings and cross-referenced name forms up into the modern Web and used those as a target for linking our stuff together yet.

But wait… did you see it?

There it is again!

A *target* for linking!

A read-only target!

Hence my failure.

In suggesting that what we all need to do is to start pointing our web resources that have LCSH subject headings into sites like id.loc.gov I failed to mention one key thing: when we start pointing our stuff at sites like id.loc.gov, those sites need to start pointing back at sites like ours.

If today a reasonable search on a prominent search engine cannot find the eight great sites with archival materials about Billy Strayhorn, how is adding links from each of them to the same record about Billy Strayhorn at id.loc.gov going to help tie the sites together? That was my goal, my current problem statement. Actually, I'd better spell that out, since I've left it implicit.

I work in a big library. We have lots of collections. Collections of collections of collections. Materials so spread out that we honestly can't answer the question "what all do you have about Billy Strayhorn?" without knowing we're probably missing something. On one hand, this is reasonable, because it's a really big library. On the other hand, it's not reasonable, because lots of people come to libraries asking for "all the stuff you have about so-and-so" and even if some librarians in some libraries could say "right here, yes, right here, this is everything we have about so-and-so", there'd still be what all the other libraries and other sites share about so-and-so on the web that that person's maybe going to want to connect to also, so even that librarian's answer is still insufficient.

There's the question, then: how do we give people a better answer than "here's a few things, and you can search for it to find other stuff, but I know you'll be missing some really important stuff, and lots of things I don't even know about, so, yeah, search Google, but you'll still miss a ton of cool stuff"?

Popping that off the stack, we're back at the unstated earlier question: can't we just future-crawl the future-web and find the future-graph of sites future-pointing at that same id.loc.gov record about Billy Strayhorn and work back out from future-there? Wouldn't that give us a big picture of all the stuff out there about Billy Strayhorn, and help each of us at each of our libraries give a better answer to that patron's question, whether or not we even have anything about Billy Strayhorn?

Well, yes, maybe, but only if we each have a really big copy of the future-web's future-graph. But most of us don't. Even if I run a crawl seeded by the eight sites with materials about Billy Strayhorn, I'm not too confident that I wouldn't have to repeat the same process again for materials by or about George Russell. Or Charlie Christian, or Charlie Rouse. And so on. What are the odds?

Not good.

So lately my mind's stuck on models for doing something somewhere between manually trawling server logs for patterns in HTTP_REFERER values and automatically populating a list of links from whatever referring pages link in. The former is a lot of work, the latter is a public request for embarrassing spam. Which is what the Trackback spec can often lead to.

If I'm a leaf note in a Linked Data library collection world, I want somebody searching for Foo at Footown Library to find my interesting set of resources about Foo at Bartown University Library. If Footown and Bartown both point their Foo pages to id.loc.gov/foo-as-a-URI, then there will two sets of folks able to see that we've both got stuff about Foo: the biggest crawlers out there, the ones likely to see both links; and the host of id.loc.gov/foo-as-a-URI. That's not a lot of options.

The traditional record for Foo has long been used to make these kinds of connections in OPACs, but those connections tend to be tightly scoped at the boundaries of the local collection. Similar connections are made from the same record at other sites, but the connections an authority record enables among holdings at any one institution are not connected to the connections made at any other institution, or at least I don't think they are, at least not in the open web, in a crawlable-by-little-old-me sort of way.

I don't know how to do this, but I'm pretty sure we need to find a way.

Actual conversations with my Mom, early 2009

dchud — Mon, 13 Jul 2009 00:44:54 +0000

The following phone conversation excerpts took place between January 1, 2009, and mid-March, 2009. The names have been preserved to embarrass the participants.

"Hi Danny, happy new year, how are you?" "Hi Mom, happy new year, I broke my hand."

"Hi Mom, I'm feeling better, how are you?" "Hi Danny, I'm glad you're feeling better, but I broke my foot."

"Hi Danny, I'm really struggling with this foot, how's your hand?" "Hi Mom, my hand's getting better still, but now my back hurts like hell."

"Hi Mom, my back's feeling better and my hand's all healed, how's your foot?" "Oh it's finally getting better." "Oh good. Maybe things will get back to normal for both of us."

"Hi Danny, guess what, I broke my toe."

Fast forward to July:

"Hi Mom, guess what, I have bunions."

THATCamp 2009

dchud — Thu, 02 Jul 2009 03:16:23 +0000

Another THATCamp has come and gone and it was, again, a lot of fun. I've grown used to the dynamics of an unconference in the past five years or so because that's the kind of event I attend most of the time, now. JCDL 2009 was the first academic conference I'd attended in years, and though I enjoyed it as well and met a lot of interesting people and learned some useful stuff, it was missing the energy the mix of people at a good unconference can generate. And, though I feel like a self-important prig as I write this, I hated that though I'd made the effort to attend, there was no chance for me to get up and show off some stuff I'd worked on in front of the group. I use software that lets a user to become a committer; I value friendships that let a student become a teacher; I attend conferences that let an attendee become a presenter. Take out that dynamic and it's nowhere near as compelling.

Because it features this principle, as any good unconference does, the best part of THATCamp is the people. Both years I've met so many fascinating people and learned about so much amazing work that it's taken the whole week following for my brain to settle back down and follow up on all the threads left dangling on sunday afternoon like so many thesis topics. There's talk of franchised THATCamps to be staged in Austin and London among other places, and that's exciting. There's a #thatcamp channel on freenode that threatens to become a regular hangout. I've got about 50 more people I'm following on twitter all of whom already fill my screen with fascinating stuff to read and look at all day and some of them are even following me, too. What more could you ask for?

Well, there are a few things. I think there are a few tweaks to the formula that could improve the event a bit. I offer these only in the hopes of making THATCamp even better, not to complain or kill anybody's leftover buzz.

Shorter sessions. This year the sessions were 1:15 long; for intense topics that engage everybody in the room that's what you need to give everybody a chance to go deep. But for open-ended discussions where there's as much airing of concerns about how "this needs to happen" and "we have to do that", 1:15 is about 25 minutes too long. It might have just been the sessions I chose this year, but it seemed like I was in more of the latter type sessions than the former, and that was a bit of a let down. Also, there were as many as five or six sessions running concurrently in several slots on the first day, any three or four of which I would've liked to sit in on. Tightening the schedule could allow for more time blocks and cut down on the number of simultaneous tracks.
More hacking. When you go from having Bill Turkel teaching people how to fire code into an Arduino and the Omeka developers teaching how to write plugins and even me doing a simple tutorial on how to make little colorful balls dance around on screen with Processing one year to basically none of that the next year, it's a bit of a drop off to somebody like me who likes to learn by doing, especially in realtime at a moment when I'm jazzed up by all the amazing people and ideas in the air.
We talked about this a bit in #thatcamp on IRC last night - maybe if the sessions were a bit shorter and there were fewer concurrent tracks, one of the extra rooms could be a "hackin' room" or some such. Sorta like the chillout room at a rave with plenty of water and comfy couches where people can take a breather but, er, well, the exact opposite of that.

It might just be that I'm a little bit disappointed in myself for not prepping a hackier topic myself. I put a lot of time into hacks just for THATCamp last year and it was great fun pulling them off. I'd like to think that it was fun for the people in the room with me, too, and either way I learned a lot from the experience and I hope that was mutual. This year I was burned out on conference travel and work and didn't have the extra cycles to put something fun and new together, and I'm sorry I didn't. If I get to go again, I promise to do whatever I can to bring the hackin' back in!
Let us do our own scheduling. This is probably the biggest one. At the Foo Camp I went to the intro evening session ended with everybody mingling around big schedule boards where times, topics, and rooms get worked out among the attendees in realtime. It's messy and takes a while but it ends with drinks and everybody's just happy to be bumping into all the other fascinating people around them anyway so it serves as a nice icebreaker, too. At THATCamp, CHNM staff instead comb through ideas posted in advance to the blog and group and sort and lump and split topics into sessions with titles that don't necessarily match what the idea-posters had in mind. I wanted to talk about improving web sites with linked data but where do I go to talk about that in this schedule? "Standards"? "Publishing"? "Software Development"? "Libraries and Web 2.0"? (that's where I went, and did a bit of the talk, but I'm not sure my topic was what everybody else there had in mind, and I know I wasn't alone in this mode of confusion).
By cutting out this dynamic let-the-people-do-it-themselves step you minimize opportunities for catchy titles to draw people in, for people to negotiate whether or not they should merge their own topics, and for people to simply get to know each other and decide which other people they want to be sure to hear from and hang out with right off the bat. And imho you maximize confusion about which sessions to go to and where you can find the people you want to hear from.

I'd advocate for filling out a big whiteboard with a schedule with people putting the names of their talks and their names with it and leaving a good 60-90 minutes to work it all out. On a real board or on paper (vs. online), so we'd have to occupy the same physical space. With drinks nearby.

I know Jeremy put a ton of work into scheduling because I caught him in the act when I arrived late so I know it was no trivial feat. I just think opening it up would be easier on @clioweb and @digitalhumanist and better for the rest of us too.
Three word intros. Another nice thing they did at Foo was *very* brief intros of everybody in the room: your name, your affiliation, and *just* three words about who you are or what you're into. Mine would be: "Dan Chudnov, Library of Congress, One Big Library". It's a chance to put names to faces, it's another friendly icebreaker, and it's a chance for all of us 140-charsmiths to be clever.
The schedule. Maybe it might help to have an evening meeting the night before for the welcoming session, the scheduling, and maybe one or two lead talks to kick things off. Then everybody can go get dinner or drinks and talk and think about what's coming the next morning and maybe work on their slides or demos or whatever overnight. You'd know when your slot is the next day, and which sessions you want to be sure to get to.

I don't want to be all "they do it better at Foo Camp" but these last few points really do reflect things that Foo Camp does a little better that I think THATCamp could adopt to make it just that much better.

And not to repeat myself, but I offer all this up with the hope of leading folks to think about various ways to make a great event even greater. I ain't complaining - the organizers do a great job making a lot of people with diverse backgrounds comfortable in a terrific space with plenty of coffee and wifi and surprisingly good food and nicely designed t-shirts and as long as they'll have me, I'll keep applying to attend again. It's just that I'm a bit of a hacker at heart and I'm always thinking about little optimizations, so take this as nothing more than that.

I hope to see y'all again next year, or even sooner - and next time you're in DC please stop by LC to say hi if you like.

TCDL 2009 talk: Better living through linking

dchud — Fri, 29 May 2009 07:41:27 +0000

Wednesday I spoke at the TCDL 2009 conference about why I think Linked Data is important for libraries. I've given talks about this twice before, once at the code4lib 2009 pre-conference on linked data, and a variation on that talk at the TCDL 2009 developers forum pre-conference Tuesday.

This was the first time I spoke about this in a room not entirely filled with hackers, though, so I couldn't just start talking about conneg and RDF models. It needed more context. As far as I can tell, the context that matters most is that we've been building a web for fifteen years, now, and we've continually changed how we build the web as we've changed how we use the web. So I spent most of the talk stressing how adhering to the four rules of Linked Data can help us make our libraries' stuff more relevant, more connected, and more likely to be found and used by improving how we link things together.

First, though, a comment about the contents of the slides - I work for the Library of Congress, but I wasn't representing the library at this talk, which I traveled to and gave off work hours. So that second slide is for real - the opinions are my own. You'll see a lot of LC examples, there, though, for two reasons. One is that I see these sites and think about them a lot, much like the rest of you, just more so because I'm there. When I can show an example from an LC site, it's likely something most people in a room have seen before and understand. The other reason is that LC has a long history of doing digital library stuff, so long that a lot of what's up there looks prehistoric in some ways, but at the same time, there are a lot of cool new things happening there, not all of which get a lot of attention, like LCCN Permalink. I don't work directly on any of the systems which have screenshots in these slides, so when you see images of those systems, you're not seeing my work. I know a few scattered details about the systems and am lucky to get to interact with many of the people who work on building them, but when I spoke about them at TCDL I had no intention of representing their work, and said so. My comments probably seemed more critical than promotional, but I meant them to illustrate situations we all find ourselves in at all our institutions, that we all know well about already, so it's not news to anybody that we all need to improve how we do things.

So, right, disclaimer doubly disclaimed. On with the slides:

TCDL 2009 keynote: Better living through linking

I really enjoy events like TCDL - a single track, a healthy mix of public services, technical services, IT, managers, and administrators, and a tech focus but with a broad perspective necessary to talk tech in a roomful of diverse skills and interests. It really focuses my attention on the one or two issues that are at the core of the changes in technology coming at us. It seemed like people received the talk well, as I heard several comments from non-coders and coders alike about how it made sense that we should move in this direction.

Unfortunately I had to leave early but I'd encourage you to look at the abstracts and learn about all the great work being done in the Lone Star state.

Rot, rot, rot for the Natinals

dchud — Sun, 10 May 2009 16:27:09 +0000

Fanatic sports fan that I am it's been a pleasure to have become a pro hockey and pro baseball season ticket holder after moving to DC. I didn't particularly care for either the Nationals or the Capitals when I got here, but I went to see the Caps because hey, that's Ovechkin down there, and I went to see the Nats because hey, pro baseball in my own neighborhood.

Over the past two years I've come to enjoy rooting for both of these teams, and though my heart remains in Detroit (go Lions! etc.) I have no qualms about rooting for these two new teams, both of which were pretty bad when I first got here. The Caps are quite good, now, and though they have a ways to go (like, adding a few more fully-D-minded skaters), they're always exciting to watch. But you knew that already.

For my money, if the Nats were a stock, I'd be buying in big round lots right now. Their offense is loaded with all kinds of good hitters -- a healthy Nick Johnson should scare anybody, Dunn's a far more disciplined hitter than I ever realized, there's nothing to say about Zimmerman right now that the numbers don't say for themselves, Dukes still has more upside, and if Guzman could learn to take a few pitches, look out. Most of their starting pitchers keep looking better and better each time out, and they might actually get this guy in the draft, and at least the bullpen isn't losing *every* game for them anymore.

So, yeah, mark my words - look out for the Nats. Maybe if the Caps bow out early (not a done deal yet, I wouldn't count them out even going into Yannisburgh for a game six down 3-2), they could lend a few Os to their southeast neighbors.