NYPL Blogs: NYPL Labs

We Are Asking For Your Help With Technology Challenges at NYPL

David Riordan, NYPL Labs — Fri, 10 May 2013 04:45:40 -0400

Over a century ago, The New York Public Library was founded with a basic purpose: to provide free access to information, literature, and cultural resources for the enjoyment and enrichment of all New Yorkers.

In the late 19th century, this meant accumulating vast collections spanning all subjects and languages, erecting beautiful buildings to store these books, and hiring brilliant, dedicated librarians to serve them to the public. But what would it look like if we founded The New York Public Library today?

Look around you and you’ll notice that knowledge organizations of the early 21st century look radically different from their 19th and 20th century forebears. Twenty-first century institutions aren’t only marble sanctuaries, they’re also distributed networks. In addition to managing printed materials and catalogs, they compile big data and user relationships. Rather than only centralizing authority, they make it porous, and welcome collaboration.

At NYPL, we’re engaging the 21st century by reimagining the public in our name. Inspired by the accomplishments of the open source software movement and collective efforts like Wikipedia and OpenStreetMaps, NYPL wants to tackle some of its thorniest (and most interesting) challenges with your help. We began by building participatory websites and crowdsourcing apps to involve users more closely in the improvement of our collections. Now we're calling directly on engineers, hackers, tinkerers, designers, data scientists, new media artists and creators of all types to help find inspirational, yet functional solutions to some of our most future-oriented problems.

To test the waters, this summer we will be issuing a series of deadline-driven tech challenges. These will be open to all, with prizes for winning submissions. The winners may directly create or inspire new tools and services at NYPL, with the potential to impact libraries everywhere. There are a number of possible areas we could focus on, but in the spirit of the endeavor, we wanted to ask you first.

Here are some challenge areas we’re thinking about:

Building apps with historical data: Create innovative educational apps with one-of-a-kind materials and datasets digitized from NYPL collections

Build a historical ‘check-in’ app using old New York City atlases and other scanned materials

Hardware hacking: Build smarter tools for on-site use

Engineer an outdoor 24-hour book drop that checks library books on the way in, and keeps everything else out
Design lightweight scanning stations to digitize our legacy card catalogs

Data crunching/Machine learning: Process and analyze library data in creative ways

Consolidate records in our online catalog that describe versions of the same book (e.g. editions, formats)
Develop new methods to extract structured data from old card catalogs

Data visualization: Use library data to uncover new insights and tell new stories

Visualize the flow of physical books and ebooks as they're borrowed and read across the city
Map the flow of letters from Founding Fathers from archival collections

We want to hear from you about which of these challenges and challenge categories gets your wheels turning. We also would love to hear your specific ideas within these areas, or for entire new categories not mentioned here.

Leave feedback in the comments below or send along a note to challenges@nypl.org.

We’re very, very excited about these upcoming experiments and hope you are as well. Let’s think big and see what happens.

Digital Archaeology: Recovering your Digital History

Doug Reside, Digital Curator of Performing Arts, Library for the Performing Arts — Mon, 23 Jul 2012 09:22:52 -0400

If you've been using computers for a while, you've probably purchased quite a few devices for storing your work. My family's first computer (a Timex Sinclair 1000 purchased for about $40 in 1984 from our neighborhood grocery store) saved files to an ordinary audio cassette by transferring data over the same sort of cord you might use to connect your iPod to your car stereo. Since then I've used floppy disks, zip disks, CD-ROMS, DVD-ROMs, and memory sticks, and with each change I migrated most of my important files to the new format. Occasionally though, I, like most computer users, need to access files left behind on obsolete technology. I've written before in various places about the problem this poses for scholars and archivists working with the so-called "born digital" collections in our Library, but many of the tools and techniques I use as a digital curator, I also use to access my own digital history.

I suspect over the next few years there may be a couple unhappy souls desperately Googling for ways of accessing data on their old floppy drives, and so I thought I would submit some of my best practices to the corpus of online wisdom to help those future search engine supplicants in whatever way I can.

Connecting to old drives

In many ways, the hardest part about accessing old data is physically connecting old disks to new technology. The modern USB port has only been really common since 1998, and there are a wide range of older devices that never used the format. Although it is possible to purchase adapters to convert popular old connection types (such as serial, parallel, and SCSI) to USB, finding the appropriate software "drivers" to allow modern computers to run these devices once connected can be very difficult. Below are a few solutions I've found for some old formats.

1.44 Mb (High Density) Floppy Disks

Recommended solution: Purchase readily available drive

One of the most common media formats of the 1990s was the High Density 3.5 inch floppy disk. Fortunately, USB connected drives capable of reading these disks are readily commercially available and relatively inexpensive. (About $20 at the moment)

740K / 800K (Double Density) Floppy Disks

Recommended solution: Purchase older system with proper drive on Ebay

Most 3.5 floppy disks manufactured in the 1980s had only about half the capacity of the high density disks made in later years. These older, double density, disks were very similar in appearance to their high density counterparts (usually only distinguishable by the lack of a hole on the upper right hand corner of the disk); however, they depended on a different kind of drive technology. Drives made in the mid to late 1990s were often capable of reading both kinds of disks, but today's USB floppy drives can generally only read high density disks. To read older formats you will probably need to find a used older machine with the appropriate drive (the Mac Powerbook G3 laptop is my personal favorite). Sherlock Consulting, a small British company, has also released a tool called OmniFlop that will allow you to use high density drives to read low density disks, but it will not generally work with USB drives and most internal floppy drives require a special connection type not generally available on desktops manufactured in recent years.

5.25 inch floppy disks

Recommended solution: FC5025 connector and eBay for drive (most uses)

The 5.25 inch disks of the 1980s are iconic in the memories of those who used Commodore 64s and Apple IIs. Although solutions for connecting these drives to newer computers emerged in the late 1990s, two recently released tools make migrating data from old floppies much easier. The FC5025 from Device Side Data allows you to connect a 5.25 floppy drive to a USB port and copy all of the data on the disk. At about $55 + shipping, it's probably the most affordable option. However, for those with a little extra cash who are doing real archival work, the UK-made KryoFlux (90 euros, ~$115) may be a slightly better option. The device is at least as easy to use as the FC5025 and is capable of capturing a much more precise copy of the data on the disk. The precision can be important for archiving copy-protected disks or programs that exploit very low-level features of 1980s drive technology, but for most users it may not be worth paying almost twice the price of a FC5025. Moreover, for those who need a device for professional rather than personal reasons (e.g. librarians and scholars), the company has a different, unadvertised, pricing structure that may not be worth the hassle of negotiating for those who do not require the incremental improvements in accuracy. For both of the above options, you will also need to purchase a floppy drive on eBay (at present, the market rate seems to be about $40).

A note on Commodore, Apple, Atari, etc. "Flippy Disks"
In the early 1980s, most drives could only read one side of a disk at a time. You would actually need to physically "flip" the disk over and reinsert it into the drive to read the data on the opposite side. However, later drives, the kind you will usually find on eBay, were capable of reading both sides of the disk at once. These later drives also use the small round window on the right side of the middle of the disk to determine the position of the data on the disk. When you flip the disk over, the hole is on the opposite side and the drive can no longer position itself. There are several mechanical hacks (involving wire cutting and soldering things to your drive) that allow modern drives to read these "flippy" disks, but no really elegant solution has yet emerged.

Disk Imaging

Once you've found a way to connect the drive to your computer, you still need a way to copy the files. For older drives, copying a single file is sometimes much more difficult than making a complete duplicate of all of the data on your disk. Most of the solutions proposed above simply set up a stream in which the data (the 1s and 0s) flow from the disk into a new file on your modern computer. This new file is called a "disk image."

Even if you can access the files directly (for example, using a USB floppy drive to access a high density disk), you may still want to create a disk image. A disk image preserves all data on a disk, including hidden files and the remnants of deleted ones. There are commercial programs that will create various forms of disk images with varying levels of fidelity to the original source, but if you're using a Mac or a Linux machine, you can use the built-in disk imaging program "data description" or "dd." If you are using a Windows machine, you can use the same program in a Unix simulator called CygWin.

To create a disk image on Mac or Linux:

Open a Terminal/Command Line window.
Type "mount" at the prompt.
Find your drive in the list.

For instance, if your disk shows up as "Untitled 6" on the desktop, you want to look for a line that looks like:

dev/disk1 on /Volumes/Untitled (msdos, local, nodev, nosuid, read-only, noowners)

The first part of this line (/dev/disk1) specifies the path to the drive. Remember this.

The second part of (/Volumes/Untitled) specifies what Unix people call the "mount point" of the drive.

Eject the disk

In order to completely copy the drive, you'll need to "eject" it. You can usually do this with the "Eject" option to "Safely remove the drive" on the Mac. However, for some kinds of removable media (like CD-ROMs) this will also trigger the physical eject function on the drive and spit the disk out of the machine.

To be safe, you might want to issue the following commands in your terminal window:

sudo umount -f /Volumes/Untitled

(Remember, we saw the disk was "mounted" at /Volumes/Untitled").

The command "sudo mount -f /Volumes/Untitled" issues the command to eject the disks. You'll be prompted for your administrator password. If you don't have one, you might try leaving off the "sudo" at the beginning (typing "umount -f /Volumes/Untitled"). Putting sudo in front of a command in the Terminal window runs the command as an administrator or "Super User" ("Super User Do!---SUDO--get it?), but depending on how the disk was loaded you might not need this level of access to eject it.

Create the image

You are now ready to create the disk image. It's best if you can write protect the original disk at this point. For a 3.5 inch disk this means flipping the switch in the upper left to open the hole. On a 5.25 inch disk it means putting a small piece of opaque tape over the notch on the side of the disk.

Remember that the path to the drive above was /dev/disk1. I will now use it again in the command below.

BE VERY CAREFUL not to reverse the "if" (input file) and "of" (output) file parameter below. Doing so will cause you to overwrite the data on the original if you have not write-protected it.

dd if=/dev/disk1 of=~/Desktop/myimage.dmg

This command will stream every bit of data from the input file (if=) /dev/disk1 into an output file (of=) on the desktop which I have arbitrarily called myimage.dmg. It may look like nothing's happening for awhile (Unix commands aren't known for being especially good at letting you know what's happening), but eventually you should get a notice about the success or failure of the copy. If all went well, you can now double click the myimage.dmg file on the Desktop and it will launch as if it were a new disk (if you're using a Linux machine, you may want to change the file extension to .img instead of .dmg).

Using disk images

For operating systems, you may need to use an emulator to open the disk and view its files. An emulator is a program capable of simulating an older machine; they are often created by those who want to play old computer games, but can be very useful for recovering old files as well. In most cases, emulators use disk images rather than attempting to connect to physical drives, so the image you created above will be useful. Googling for the name of your computer or operating system along with the word "emulator" will likely turn up a few options for most systems. John A. Reder has a fairly good list of computer emulators for various operating systems.

Finding old software

Often, simply accessing old files isn't enough. You will also need to find the software that open them. Sometimes it's possible to recover much of the textual data in a file by opening it in a text editor like TextWrangler (for Mac) or Notepad++ (for Windows), but for other kinds of work (e.g. music, drawing, 3d designs), you'll need to find a copy of the original software to run in your emulator. Finding a legal copy of 10-20 year software can be tricky. In some cases, thoughtful rights holders release very old versions of their software for free (as Apple has done with many of the early versions of Mac OS), but in most cases the only way to find these programs is by buying a used copy and producing a disk image yourself or by assuming the legal risk of hunting for "ROMs" (disk images named after the Read-Only Memory cartridges many early game systems used) uploaded by others without clear authorization. Unfortunately, there's not much I can recommend here. However, if you are the rights holder of an out-of-print software title, especially those that were used to create content, I encourage you to consider releasing your old software to the public domain. Contact me at dougreside at nypl.org if you're interested in finding out how.

The New York City Historical GIS Project

Matt Knutzen, Stephen A. Schwarzman Building, Map Division — Wed, 13 Jun 2012 12:39:55 -0400

In 2010, the National Endowment for the Humanities (NEH) awarded The New York Public Library's Lionel Pincus and Princess Firyal Map Division a three-year grant for its New York City Historical Geographic Information Systems project, which builds digital cartographic resources from NYPL's historical paper map and atlas collections.

The project walks a portion of NYPL's New York City map collections through a series of workflow steps outlined in a previous blog post, Unbinding the Atlas. In a nutshell, maps are scanned (shooting a high resolution digital image), georectified (a.k.a. warped, rubber-sheeted, i.e. aligning pixels on an old map to latitude/longitude on a virtual map), cropped (removing extraneous non-map information from the collar area around a map), and digitized (think of this as tracing).

In the proposal, we committed to scanning 9,000 maps, but we were ultimately funded to image approximately 7,200 maps. Work has proceeded much faster than anticipated, however, enabling us to scan and mount 7,799 new maps so far. An additional 9,327 metadata records have been created for related collections, such as all of New York City’s zoning maps (a bibliography can be found at the bottom of this great post, or in this .doc file) dating to 1916; most of our public domain fire insurance atlases of areas outside of the city in New York and New Jersey; and our entire run of historical and contemporary New York state topographic maps. If the pace of imaging continues as expected, the project will have funded the digitization of 17,126 historical maps, most of which are concentrated on the five boroughs, but with significant coverage of upstate New York and New Jersey. The scanned maps can be found in a number of places, starting with the Library's Digital Gallery. We've also put together a collection guide. One of the richest veins of recently-imaged content — over 5,000 of our most detailed maps produced by the Sanborn Map Company — can also be found in the Digital Gallery. A favorite example is the atlas, whose gorgeous and graphically-complex index page, above left, shows a Brooklyn that has yet to fill in the scaffold of projected urban grid. A pleasant discovery in this was plate 175, below right, documenting what must be one of the only elephant-shaped buildings in the world — in this case an elephant bazaar, which once occupied the area on the north side of Surf Avenue, across from the Cyclone. (Explore the elephant and the elephant bazaar >>)

Once the maps are scanned, they are available for georectification, which happens using our web-based toolkit at maps.nypl.org. If you're interested in learning how to georectify maps, you're welcome to participate in this project. The home page includes a how-to instruction video, along with a more involved instruction sheet. There are also instructions on map cropping, a necessary step in the creation of map mosaics. We're also thrilled that our friend Liz Barry at the Parsons School of Design's Public Laboratory for Open Technology and Science has published this excellent instruction as part of her syllabus. If you'd like to take your work a step further and know a thing or two about Geographic Information Systems (GIS), you can follow her instructions for using maps from maps.nypl.org in ArcGIS, the industry standard GIS software. We've made great progress in georectifying NYC maps, warping about 2,400 sheets from some 162 atlases in total.

The image below shows some of the resulting georectified map layers from the project, including William Perris's Maps of the City of New York, 1857-1862; G.W. Bromley’s Atlas of the city of New York, Borough of Queens,… 1909; the Bronx Topographical Survey’s Topographic Survey and New Street System of the Borough of the Bronx…; G.M. Hopkins’s Detailed Estate and Old Farm Line Atlas of the City of Brooklyn, 1880; and G.W. Bromley’s Atlas of the City of New York, Borough of Richmond, Staten Island, 1917.

If you love map georectification, it's likely you'll also love map tracing. This is, as I pointed out in a previous post, where the proverbial rubber meets the road in terms of working with historical maps. If map warping is preparing pixels of digital images of old maps to relate to one another geospatially, map tracing is preparing machine readable data to be harvested, mined, analyzed, mashed, made a part of the semantic web, and related to itself, across time. We've created a handy guide to map tracing, if you're inclined to help us build this valuable resource. This type of data will eventually allow you to ask your phone a question, such as, "I'm standing in front of the Coney Island Cyclone. What other attractions would I see if I was here 100 years ago?" Ideally, you'd be presented with a reasonable answer, such as, "On the other side of the street, you'd see the colossal elephant bazaar that stood from 1885 to 1896," etc. And you might just see images like the items highlighted below.

The last part of the NYC Historical GIS project is tracing, and we've done quite a bit of it through our own efforts, through partnerships with local colleges and through our Citizen Cartography workshop series. At the bottom of this post are some of the resulting datasets that require a bit of explanation. The large majority of maps we're working with in this project are called fire insurance maps. They were designed to give insurance adjusters a reasonable metric by which to assign fire insurance values to buildings. They did so by documenting, among many other things, each building structure's street address, construction materials (e.g. wood, brick, etc.), the property's height (in stories), and the location of fire hydrants, as well as the general locations and width of water mains (to gauge available water pressure levels at each hydrant location). The goal of the project is to trace from a series of atlases all of the buildings' spatial footprints and transcribe all of the accompanying feature data. The "why" is explained at length in my earlier post.

The short explanation is that, while these types of maps are no longer used for fire insurance, they find continued relevance among our readers and, in truth, among any number of users along a spectrum of popular and scholarly domains. Realistically, anybody doing a bit of historical and geographic detective work can find these maps and their contents useful. And having this data as searchable (i.e. machine readable) information both breathes new life into these wonderful historical documents and enables their use in a wider, linked-data universe.

The three images below show a set of building footprint data derived from William Perris's first edition Maps of the City of New York, published in a 98 sheet, seven volume series between 1852-1854. The data was traced at maps.nypl.org, cleaned up using what are called regular expressions and a tool called Google Refine. The resulting data was then loaded into Google Fusion tables, where it was stylized. The image below (at top) shows the map data from the Perris atlas colored based on the attributes in the "material type" field. The image below (center) is of exact same data, but filtered to only show buildings that are Mixed Use (residential and commercial), those places where small business people — sole proprietors — lived, either in a back apartment or upstairs, and ran some sort of business from home. The image below (at bottom) is again the same data, but filtered again to show only Mixed Use buildings that are also framed (wooden). This type of filtering and potential for analysis of building data is a powerful means to explore and understand the historical city and is really at the core of the NYC Historical GIS project. Taking a step back, this type of methodology, i.e. moving paper, analog collections from the material to the digital world and making them machine readable, queriable, linkable, and generally more intelligent and more than the sum of their parts, is at the core of much of the exciting work being done by NYPL Labs, on the web (the Google Books Ngram Viewer, for example), and in the wider world of the digital scholarship.

So, if you're interested in learning more, please come to NYPL's Map Division or join us for a Citizen Cartography workshop.

Happy Birthday to... Us! A Year of Menus

Rebecca Federman, Collections Strategy, Stephen A. Schwarzman Building — Fri, 20 Apr 2012 09:10:18 -0400

It's hard to believe, but a year ago this week the New York Public Library launched What's on the Menu?!

Two days in from our very first Tweet, we had 1,000 dishes transcribed. As of this writing, we have 866,636 dishes dishes transcribed and we're not done yet.

We still have many more menus to digitize and we're working hard on new ways to make the site even easier to navigate and use.

But before the next year begins, Michael Inman, Ben Vershbow, and I wanted to take the time to say a sincere, bow-down, "you rock" thank you to those who have put in time to make this project a success, starting with all of you.

When we launched WOTM? last April, we had no idea what the response of the project would be. Apparently, you like it. And we're beyond thankful to all of you who have spent time transcribing menus, writing in with suggestions, questions, and helpful comments. We wouldn't be here without your help.

But like a real restaurant, there are many cooks in the WOTM? kitchen who have done so much to help move this project forward. We'd like to thank:

Michael Lascarides and Kris Kelly for building the site as a labor of love.

The National Endowment for the Humanities for their financial and intellectual support.

To the American Historical Association for giving us the honor of the Roy Rosenzweig Prize.

To the Institute of Museum and Library Services for their Sparks! support.

Our interns: Edith Bellinghausen, Jayme Hall, Leslie Harker, and especially Amanda Glassman who kept the engine running and Twitter tweeting — even while low on gas.

Rebecca Austin, Kate Summers, and Lucy Daniel Anderson for re-cataloging the physical menus to include the proper metadata.

Pat Rader and Abigail Meisterman in our metadata team for helping us in those choppy waters.

Eric Shows, Terrance D'Ambrosio, and the entire Digital Imaging Unit for scanning the menus and pushing them live.

Rebecca Holte and Caryn Gedell in the Registrars Office for moving the menus safely.

NYPL Labs: Dave Riordan, Mauricio Giraldo, and Zeeshan Lakhani for their skills and good humor.

And finally, to Ann Thornton and the New York Public Library staff for their continued support and encouragement.

The next course on this tasting menu will be arriving shortly.

All Hands on Deck: NYPL Turns to the Crowd to Develop Digital Collections

Vicky Gan, Intern, Strategic Planning Office — Thu, 15 Sep 2011 12:51:48 -0400

Users are generating that reality every day at The New York Public Library through two landmark crowdsourcing endeavors, What’s on the Menu? and Map Rectifier. The former enlists the public in the transcription of historical menus, and the latter allows users to “rectify” historical maps by overlaying them on modern ones. Both projects stand out amid a glut of competitors as refreshingly guilt-free and subliminally educational uses of online time. With every menu transcribed and map rectified, users are supporting research in the humanities.

In the time it takes to deploy an angry bird, a user can identify and transcribe a dish dating back to the mid-nineteenth century. The recipe for the menu project is simple — click, type, submit, repeat — and has proven a runaway success, yielding almost half a million plates of Blue Points, porterhouses, croquettes, et al. The average visit to What’s on the Menu? is slightly under eight minutes and thirty page clicks long — an eternity on the web — and patrons are hungry for more. The initial release of 8,700 digitized menus was provisionally transcribed in just four months. NYPL recently ramped up digitization efforts to meet demand.

“There’s some kind of thrill about it,” observes Rebecca Federman, project curator and culinary collections librarian at NYPL. “Menus have an everyday nature but are also extraordinary.” They are extraordinarily rare; most restaurants scrap them after each service. Menus are a form of ephemera — printed materials, such as flyers, posters, and programs, that were not meant to be kept. They reveal the past through its quotidian details and prove, dish by dish, that we are what we eat.

For the transcriber, the collection is a treasure trove of little discoveries — the antiquated use of “farinaceous” instead of today’s “pasta”; the remarkable preponderance of oyster dishes; the revelation that steaks cost twenty-five cents, not twenty-five dollars. For the scholar, it is an invaluable source of historical data. Author William Grimes used the collection to produce a culinary history of New York. Texas A&M marine biologist Glenn Jones scoured seafood menus to study fish populations. What’s on the Menu? has also inspired users outside the academic sphere. Chefs Mario Batali and Rich Torrisi are fans of the project, and a fourth-grade class in Texas has been transcribing menus as a typing exercise. By codifying and enhancing digital collections, crowdsourcing spawns new applications for historical information.

NYPL’s Geospatial librarian, Matt Knutzen, is excited about crowdsourcing’s potential for the map collection. “You can study anything through geography, through a spatial lens,” he says. Using NYPL’s Map Rectifier, amateur cartographers align old maps with current ones to “create a historical framework for geographical information.” The maps convey much more than the locations of roads and landmarks; each layer incorporates multidisciplinary data to illuminate a specific period in a region’s development. Demographic data could chart patterns of migration. Nautical data could track changes in a harbor’s bathometry. Maps of defunct factories could have real-world implications for environmental remediation. “We’re not positioning ourselves to answer those questions,” says Knutzen. “We’re enabling people to find those answers.”

The Map Rectifier has already made a concrete impact in Haiti. In 2010, the Library contributed historical maps and georectification software to the Haiti OpenStreetMap used by earthquake aid workers. Citizen cartographers the world over collaborated to create an open-source map with up-to-date information on Haiti’s resources and infrastructure. NYPL added layers of data that helped rescue teams locate victims and coordinate relief efforts. In its way, crowdsourcing helped save lives.

Ben Vershbow sees many more such surprises down the road. Vershbow is the director of NYPL Labs, the Library’s experimental digital humanities unit. According to Vershbow, cultural heritage institutions are at last starting to step more boldly into the collaborative web. “After testing the waters on third-party services like Flickr Commons,” he says, “we are beginning to see libraries, museums, and other organizations investing in their own tools and communities, and going deeper with particular collections.” While enthusiastic about drawing users more directly into library initiatives, he is quick to note that crowdsourcing is not a goal in itself but a “solution to particular sorts of problems,” and can often open up a host of new challenges. “In asking for the public’s help in extracting the menus data,” he says, “we are making an implicit promise to do something interesting and useful with it. That means more investment in technology and in library staff dedicated to overseeing, growing, and explaining the menu database. What it means fundamentally,” Vershbow continues, “is re-imagining the very roles of librarians and curators, positioning them not only as custodians of physical collections, but as leaders of online communities.”

NYPL Labs staff are already on the hunt for their next “blockbuster” project. They’re considering collections to be tagged, transcribed, and curated by the public. They’re floating ideas for “maker sites,” which would allow users to create new media using items from NYPL’s Digital Gallery. They envision, in the distant future, a fully searchable, cross-referenced map of historical information — an online time machine. Click on a restaurant to pull up its 1902 menu. Pinpoint a teacher who lived on Mott Street. See the shows from the Lyceum Theatre’s opening night.

“It’s uncharted territory,” says Vershbow. And that’s what’s so thrilling about it.

Announcing: Musical of the Month

Doug Reside, Digital Curator of Performing Arts, Library for the Performing Arts — Wed, 18 May 2011 06:36:13 -0400

Growing up in St. Louis, Missouri, my favorite part of the week was visiting the Florissant Valley Public Library and checking out cast recordings. I remember flipping through the bins of LPs, staring down at the big black album with glowing cat eyes, and wondering what in the world that show might be about. It was always a little disappointing when the liner notes were missing or the plot summaries were particularly sparse. In such cases, I would make up a story to fit between the songs (which led to some surprises when I finally saw these shows in their entirety). Sometimes I would go to the shelves to try to find a libretto, but, with the exception of the titles in Stanley Richard's excellent Great Musicals of the American Theater anthologies, I was usually unsuccessful.

Now I work in a library that has the text (often in multiple versions) of nearly every Broadway show produced in the last 100 years. We have audio recordings of shows I've never heard of and the Theater on Film and Tape Archive preserves a large portion of the last 30 years of professional New York (and some regional) theater. I can't help but think, though, about the new versions of my younger self, who live in places with libraries perhaps even less well-stocked than the one of my childhood. I'm also struck by the number of times I have received a mass email to theater scholars with a plea from a University professor asking "Does anybody know where I can find a copy of the libretto of [a semi-obscure, older musical]."

So, I'm instituting a new blog series I'm calling the Musical of the Month. My plan is to publish on this blog an electronic edition of a libretto about once a month in formats you can read online, on your smart phone, or on most ebook readers. In some cases I'll also be able to provide digitized images of piano/vocal scores for at least a few songs. Additionally, for each play, I'll provide a short history of the original production, explanatory notes for references that may be obscure to modern readers, as well as links to additional related material digitized from library collections (either at NYPL or elsewhere) and links commercial material (such as cast recordings) available for purchase.

By the end of the year I hope there will be about semester's worth of material that could serve as a kind of textbook for early musical theater history. To begin with, the plays will necessarily be selected from out-of-copyright titles (that is, texts published before 1923). My hope, though, is that arrangements may eventually be made with intellectual property holders to make accessible some later texts as well. I will begin next month with the 1866 musical that many early historians called (for very questionable reasons) the first American musical. As a preview, you can check out Harvard's copy of an early promptbook for the play. Enjoy!

The Queen B: Miss Buttolph and Her Menus

Rebecca Federman, Collections Strategy, Stephen A. Schwarzman Building — Thu, 28 Apr 2011 08:05:44 -0400

If you've transcribed even one menu, you've likely seen her stamp. A blue oval bearing her name, "Buttolph Collection", as graceful as a branding iron over asparagus, Russian caviar, or Boston baked beans.

Miss Frank E. Buttolph stamped nearly every menu she collected for the New York Public Library, twenty-three years worth, amounting to roughly 25,000 menus under her tenure alone.

But who was Miss Buttolph and why did she collect menus?

Neither question is easy to answer. We know from records that she was about fifty when she began her menu project, she was educated (she translated Tasso), and she was an avid collector of postcards with lighthouses.

Her most notable collection, her menu collection, began on January 1, 1900, with lunch. In a letter dated February 14, 1900 she writes:

"On New Year's Day I stopped in the Columbia Restaurant for lunch and thought it might be interesting to file a bill of fare at the library. A week later the thought occured, why not preserve others? As a result 930 have passed through my fingers to the Astor Library."

By August, Miss Buttolph was taking out ads in hotel and restaurant trade journals soliciting menus from their readership. This ad from Hotel Monthly (August, 1900) stresses the physical condition of the menus, or cards:

"It is of the highest importance the cards should be well wrapped and then placed between stiff card-board of a larger size, else they are sure to be soiled and broken in the mail, which condition renders them worthless. One beauty of this collection is, nearly all of the 3,600 cards [in the collection] are perfect, but I have had had to fight harder then Gen. Otis did in the Philippines to keep my standard in position. When it has to be lowered I shall discontinue the work."

The full ad is reproduced below:

Miss Buttolph's colorful personality, which is suggested in the ad, was both the reason for her success and the cause of her downfall. Her diligence in hunting down menus (writing to restaurants, putting up advertisements, and speaking to the press), and her commitment to high quality (she did not hesitate to send menus back if they did not meet her standards) insured that the Library's collection was both comprehensive and pristine.

However, even though she was never an employee of the Library, Miss Buttolph's idiosyncricies and negative behavior ("Is this museum maintained by the city to afford whistling space for the cleaners, instead of for students?) upset many on staff and in the Library administration who felt that her behavior was too disruptive ("[Miss Buttolph] is contantly complaining about something and when she gets started, it is almost impossible to get rid of her.")

Miss Buttolph was dismissed from NYPL in 1923. She died of pneumonia at Bellevue Hospital the following year, on February 27, 1924.

* * *

Despite her tumultuous relationship with the Library, her committment to her collection never wavered. In one of her last letters to the administration, she writes:

"For many years my library work has been the only thing I had to live for. It was my heart, my soul, my life. Always before me was the vision of students of history, who would say 'thank you' to my name and memory...."

Thank you, Miss Buttolph. Your incredible stamp continues to be felt, indeed.

Continue transcribing her collection on What's on the Menu?

New Feature! Unlock Menus to Continue Editing

Ben Vershbow, Manager, NYPL Labs — Wed, 27 Apr 2011 08:43:34 -0400

We've gotten a number of questions over the past week of What's on the Menu? about menus marked as "done." Do we really mean done? As in finished, vetted, archived for posterity? Fear not, we've cleared up this confusion with some new language. What we really meant to say was "under review."

On several occasions, a volunteer e-mailed us saying they'd spotted errors, or missing dishes, on menus marked as complete. I happily re-opened the menus in question (a facility only open to site administrators) and invited them to continue their work. After doing this a few times ad hoc, we decided to just add this as a feature.

Now, for any menu you find that's under review, you can click the little lock icon to its left to re-open it for further editing, correcting or transcribing. So if you are feel like donning your proofreader's glasses, we invite you to dive into the menus formerly known as finished to hunt out typos and problematic transcriptions (as ameliorated by these helpful tips), or to insert decimal points in the price fields to bring the cost of living and dining down to appropriate circa 1900 levels (as required, say, in this menu), or other tidying and correcting tasks not yet anticipated. Don't forget to re-submit the menu for review (via the button below the dishes list, left sidebar) when you're "done"!

And then what?

As is often the case in libraries, and on the always-evolving web, the work is never completely and totally done. For the menus, finishing transcription is in fact just the beginning of a long and only partially mapped out journey of data cleanup and rectification — not to mention subsequent tasks we may open up, such as the identification of sections (appetizers, desserts), categorization (breakfast menu, dinner menu), and other things still TBD.

Our philosophy all along has been to launch the project as openly and simply as possible, build a big data pile, and then to start finding solutions for navigating and improving the data. Any brave souls who want to jump in now to start polishing and tweaking are more than welcome! Please report back any common issues that you find (in the comments field below or via e-mail at menus@nypl.org).

We're also racing to post a detailed Help page, much of it informed by valuable user feedback, and soon, we expect, by the forthcoming insights of menu unlockers as well. The key is in your hands!

Go to What's on the Menu?

Doin' the Dishes!

Rebecca Federman, Collections Strategy, Stephen A. Schwarzman Building — Tue, 19 Apr 2011 11:33:26 -0400

Saratoga Chips. Corned Beef Hash. Large Pot of Oolong Tea.

Okay, so they’re not included in the works of Shakespeare (as far as I know), but that doesn’t mean these dishes aren't of value to researchers and scholars and the generally curious who read menus in order to learn more about the food served and consumed in restaurants throughout history.

But until now this kind of information (the food!) was difficult - if not impossible - to search in our digitized menu collection.

Yes, you can search “oyster”, but you’ll get “Oyster Bay” instead of saddlerocks, or “Shanley Bros. Oyster House” instead of blue points. In other words you get the location and restaurant name, but not the very content (the food!) of the menu.

So we’ve built a website where you can tell us “What’s on the Menu?” and help create what we like to call a “database of dishes.” From your transcriptions, I hope we can learn about the foods of the last century to see what these historic menus can teach us about the culinary landscape today. Here are some items I’m excited to track over the next few months:

Tutti-frutti
Rise and fall of oysters
Vichy water
Battle Creek Sanitarium dishes
Moselle wine

And this is just the beginning! We'll be following many more foods and wines from these unique primary sources of our dining history. So help us “do the dishes” and we can all reap the rewards.

Go to What's on the Menu? and jump in!

Candide 2.0: A Reading Experiment Begins

Alice Boone, Curator, 'Candide at 250: Scandal and Success' — Wed, 17 Feb 2010 09:52:15 -0500

For the next ten weeks, the New York Public Library will host a public, interactive reading of Candide, in connection to its ongoing exhibition at 42nd St.. This edition will look familiar to readers who remember the story, or even just its famous lines about “the best of all possible worlds” and “we must cultivate our garden.” But the innovative format, which facilitates reader annotations and discussions in the digital margins, will also yield surprises, as we have taken that closing line and used it as inspiration for a “cultivated” edition, with “seeds” of discussion sown by readers, opening up the text for public participation. We are thus extending the invitation to NYPL readers to add to these annotations: to make our garden grow, to paraphrase Leonard Bernstein.

To begin the annotation-cultivation of Candide, Nicholas Cronk, president of the Voltaire Foundation at Oxford, takes a new look at the opening chapter of the book, noting how what seems like a familiar narrative structure of the story immediately becomes unsettling, as the narrator disappears in the first paragraph (click "text" link at end of quote to jump to this place in the book):

“The narrator leaves us on our own after this brief and apparently pointless appearance. We are not, after all, to be guided through the story; the narrator has let us down by stealing any old plot from Tom Jones; nothing is quite as it seems; and we are on our own, left to make sense of things as best we can…” [text »]

Candide has been expelled from his family castle in Westphalia and has lost his love, Cunegonde. He has some questionable guidance from his tutor, Dr. Pangloss, whose metaphysico-theologico-cosmolo-nigology insists on a circular understanding of cause and effect: “the nose is formed for spectacles—thus we have spectacles!” [text »]

Cronk shows how the examples Pangloss uses to support his philosophy are designed to appeal to the Baron’s small universe in his Westphalia castle:

“The argument from design is meant to prove the existence of God: here it only proves the existence of German barons.”

The narrative perspective—through Candide’s eyes— shows both the limitations of the naïve hero’s experience and the universality/banality of the Panglossian system, as Candide describes “this best of all possible worlds, [as] the Baron's castle was the most magnificent of castles, and his lady the best of all possible Baronesses” [text »] and Pangloss as “the greatest philosopher of the whole province, and consequently of the whole world.” [text »]

Cronk asks what we are to make of these repeated phrases—“sufficient reason,” “cause and effect,” “best of all possible…” The repetition is humorous, but more than humor is involved. Cronk puts it, “It seems that the more we play with these terms, the more they lead a life of their own.” [text »]

It was a similar sense of how Candide’s philosophical debates and peripatetic travels were adapted and led lives of their own that inspired my own interests as curator of the NYPL Candide at 250: Scandal and Success exhibit in Wachenheim Gallery. I saw a counter-history of the novel in the way that its readers had transformed it, as if its canoncity were reflected in a funhouse mirror: the odder iterations of the story (Esperanto experiments, 1960s countercultural rambles, contemporary human rights campaigns) could be encapsulated in the opening line of the second chapter: “Candide, driven from terrestrial paradise, walked a long while without knowing where…” [text »]

With Candide’s errant wandering as my inspiration, I became interested in the errors that the text has picked up in its 250 years of translation. For example, in some translations, the Bulgars become Bulgarians; the puerile wordplay on Bulgars/buggery (in reference to Frederick the Great) [text »] would have been familiar to some readers, but it did not translate for others who did not consider the context. Most translation differences, of course, should not be seen as errors, as translations have wandered around a (supposed) central or original sense with different emphases, references, overtones, and felicities of language—vagaries of translation that are new sources of life for the text.

For this annotation project, we have used a public domain translation available in digital form. If it is not the best of all possible translations, it nevertheless facilitates what could be the best of all possible discussions, as the annotating facilities at each paragraph allow interested readers to communicate with others about how translation choices affect reading.

The history of Candide is a history of adaptation for different cultural periods and different philosophical and political arguments. Playwright Stanton Wood has updated the story to the past 10 years of world events in his Candide Americana with Rabbit Hole Ensemble, which showed at the NYC Fringe Festival in 2009. In Wood’s retelling, the Seven Years War becomes the Balkan Wars of the 1990s, and Candide witnesses the fall of the World Trade Center on September 11, Hurricane Katrina, and other events. In his annotations to chapter 3, Wood reflects on the relevance of eighteenth-century philosophy to considering contemporary events. Wood uses another type of counter-history as his inspiration: Susan Nieman’s Evil in Modern Thought: An Alternative History of Philosophy (2002). [text »]

Wood is also attuned to the strangeness of the story when he notes Candide’s encounter with a miserly preacher who advocates charity: “Incorporating this kind of ironic moment into the natural flow of the action was one of the challenges of adaptation - every moment in the book seems to include an example of someone saying one thing while doing the opposite in a particularly delicious way.” [text »]

Like Cronk, Wood is interested in how the features we expect to see in a novel—here, a genuinely nice character in Jacques the Anabaptist—disappear almost immediately. [text »]

What is the effect of these destabilizing devices in these early chapters: baits and switches, irony, and wandering as novelistic structure? What do they reveal about Voltaire’s work in the conte philosophique, the philosophical tale? How do these devices lend themselves to further adaptation?

Drawing on the Past: Enlivening the Study of Historical Geography at maps.nypl.org

Matt Knutzen, Stephen A. Schwarzman Building, Map Division — Wed, 03 Feb 2010 11:50:12 -0500

On behalf of The Lionel Pincus and Princess Firyal Map Division, the NYPL’s Director of Digital Strategy and Scholarship and our partners EntropyFree LLC, I am proud to announce the launch of maps.nypl.org

This new website is a parallel snapshot of all maps currently available on the Digital Gallery as well as a powerful set of tools designed to significantly enhance the way we access and use maps and the cartographic information they contain.

The first such enhancement is in how historic maps are viewed. The user interface of maps.nypl.org allows zooming and panning in a way that has come to be expected by users of web maps (Google Maps, Bing Maps etc...)

The next is georectification, which we are calling here “warping”, a familiar term to GIS professionals and few others. Map “warping” is the process where digital images of maps are stretched, placing the maps themselves into their geographic context, rendered either on the website or with tools such as Google Earth.

Illustrated here is the 1915 Redraft of the 1660 Castello Plan documenting early lower Manhattan, “warped” using the Map Warper and rendered in Google Earth.

And below here, a set of 98 detailed sheet maps of the New York City published by William Perris in the early 1850s, overlaid in Google Earth after having been “warped” and mosaicked using maps.nypl.org

Once historical map has been digitally “warped,” users of the Library’s digital maps can virtually “trace” features, such as cities, farm boundaries, rivers, ponds and even buildings, converting them into digital geospatial data. And that data, in turn can be easily linked to other digital information, such as building photographs, text citations or any other information that relates to the same geographic location.

Illustrated below is a photo of Phenix Bank at 45 Wall Street, digitally “pinned” to one of the maps in the series shown above.

One of the most exciting aspects of this project is its participatory nature, meaning that anybody with a computer can create an account, log in, and begin warping and tracing maps, whether for a school or personal project or otherwise. And when the project is complete, the contribution remains in place (à la Wikipedia and openstreetmap.org ), adding one more piece to this new historical geographic data model.

I’ll blog again about some of the ways maps.nypl.org is already being employed by a variety of user groups. Also, follow us on twitter for project updates and events.

For now, however, feel free to create an account, watch the how to video, and enjoy.

World Series warm-up: historic New York-Philadelphia baseball images on Flickr

Ben Vershbow, Manager, NYPL Labs — Wed, 28 Oct 2009 11:28:26 -0400

The 2009 World Series brings together two cities uncommonly rich in baseball history. Though you might guess which team NYPL is rooting for this year, we've posted a selection of images on The Commons on Flickr representing a variety of New York and Philadelphia ball clubs of yore.

Some of the game's earliest years are chronicled in over 500 photographs, prints, drawings, caricatures, and printed illustrations donated in 1921 to the New York Public Library by early baseball player and sporting-goods tycoon A. G. Spalding (whose name to this day is printed across every ball used in the National League). As the contemporary Yankees and Phillies clash on the field, here you'll find Philadelphia Quakers, Athletics and Keystones in a gentlemanly mix with New York Giants, Knickerbockers and Metropolitans, and of course Brooklyn Excelsiors and Atlantics. Each one of these images of course has an enormous back story, which we hope the baseball history buffs among you will help fill in through comments, links, tags and annotations. Also check out a smaller set, Proto-baseball, which gathers images of baseball's ball-and-stick forebears like cricket and Old Cat. Here's "Six boys with a ball and three bats, playing Three Old Cat":

We also invite you to explore the full Spalding Collection on the NYPL Digital Gallery and through this finding aid (PDF) from the Manuscripts and Archives Division. Now let's just pray for the rain to stop so Game 1 can get underway... ***UPDATE*** Picked up by Gothamist!

Mapping New York's Shoreline: The Storied River

Matt Knutzen, Stephen A. Schwarzman Building, Map Division — Fri, 16 Oct 2009 08:42:37 -0400

Staff of the New York Public Library recently hand picked a set of nearly 500 images, collected from across our Digital Gallery, composing them as a curated set of images at the Commons on Flickr. They represent the Hudson River Valley through several hundred years of history and complement Mapping New York's Shoreline, 1609-2009, now up in the Gottesman Exhibition Hall at the Stephen A. Schwarzman Building.

The images depict landscape scenes in stereoscopic vision, a popular 19th century format; everyday and commemorative menus from restaurants and catering halls; postcards of scenic places and buildings; and engravings of important estates, prominent citizens and dramatic turning points in historical events. These images have been geocoded and are part of map-based bibliography, The Storied River, coming soon to the NYPL. Stay tuned, the launch will be posted on the NYPL's map blog...

In the meantime, enjoy the same photos at the Commons on Flickr, perused as a gallery of images... Mapping New York's Shoreline: The Storied River ...or, my favorite, pinned to a map on the Flickr website.

Learn more about the NYPL Map Division.

General Motors and Chrysler images on Flickr Commons

Ben Vershbow, Manager, NYPL Labs — Thu, 04 Jun 2009 08:16:32 -0400

As we watch with astonishment the "restructuring" of two American automotive titans, take a look back at the first four decades of their history, a time which saw multiple breaking waves of innovation in both engineering and design, and a steady absorption of manufacturing brands into the conglomerates we now see in crisis today.

Over the years, General Motors Corporation donated photographs and related materials as a public service to contemporary and future researchers, and to create an ongoing record of the company's output. The Library, in turn, mounted these photographs and texts in albums. General Motors was one of several transportation corporations that donated public relations materials to the Library department that is now the Science, Industry and Business Library. The General Motors holding is particularly comprehensive. See here the first Oldsmobile (1897)!

NYPL's latest contribution to the Flickr Commons is a group of 425 images from these albums, ported over from our Digital Gallery and organized into six Flickr sets and one overall collection: G.M. and Chrysler Cars and Trucks, 1897-1938. Peppered through some of them you'll find the ghostly portraits of engineers long gone, and delightful staged shots of motorists in action.

Spread the word and enjoy!

Milstein joins the Flickr Commons!

Tue, 19 May 2009 08:24:20 -0400

Just last week, the New York Public Library updated their Flickr Commons photostream. The newest images are from the Milstein Division and include construction photographs of the Woolworth Building as well as block by block street views of both Fifth Avenue (1911) and Broadway (1899).

Not only are these images aesthetically beautiful, they are also valuable historical objects which are useful for historians and genealogists alike. The latter two collections allow detailed study of the storefronts that peppered the sidewalks of Broadway and Fifth Avenue. Chances are you may also spot an ancestor’s shop if they conducted business there during the turn of the century.

Since joining the Commons late last year the New York Public Library has been met with an enthusiastic response from the Flickr community. Additionally some of the comments we have received have enriched our understanding of our collections. Having our items within Flickr Commons also makes them available for other creative purposes. We look forward to viewing future comments and innovative reuse of our collections!

NYPL joins Flickr Commons

Ben Vershbow, Manager, NYPL Labs — Tue, 16 Dec 2008 04:22:50 -0500

Chances are, if you spend any time online you've come across Flickr. Flickr is a wonderful site for storing, sharing and building community around photographs. It's similar to online photo services like Kodak Gallery or Shutterfly except with a greater social focus and tools and features reminiscent of Facebook. About a year ago Flickr launched the Flickr Commons, a project dedicated to sharing and describing the public photo collections of the world's leading cultural heritage institutions.

Starting this past January with The Library of Congress, and continuing with places such as The Smithsonian Institution, The Brooklyn Museum, The National Maritime Museum, The National Library of New Zealand, the Nationaal Archief of the Netherlands and numerous others, the Commons has grown steadily over the past year into a truly remarkable public photography resource. We are delighted to be the latest institution to join in this endeavor, with an initial contribution of 1,300 images culled from various areas of our diverse photographic collections.

We think of this as a sort of appetizer course, a sampler of collections accessible in greater breadth and depth on the NYPL Digital Gallery, and on-site in our network of libraries. Lush images of modern dance pioneers; haunting early cyanotypes of algae (the first photographic works to be produced by a woman); majestic geographical surveys taken along the Union Pacific Railroad, iconic Depression-era images taken under the Farm Security Administration's famed photography program; Berenice Abbott's epic documentation of 1930s New York for the Federal Art Project; stunning 19th century vistas of the Egypt and Syria; scenes and portraits of Ellis Island Immigrants, the Statue of Liberty under construction... These and more are now available to view, tag and discuss in the Flickr Commons, and are offered as an invitation to explore further on our own site or in our actual libraries. After this initial road test, we expect to post many more images into the Commons pool.

When Flickr launched the Commons this January with the LOC as pilot partner, photos were intentionally posted without descriptive tags (except for "Library of Congress"). LOC was curious to see what would happen when users were presented with a relatively blank slate. The response was overwhelming. In addition to thousands of comments, "favorites" (users bookmarking images in their personal accounts) and image annotations (notes appended directly to the surface of the digital images), tens of thousands of tags were posted, describing the images in myriad ways. Later, in the culmination of this first experimental phase, more than 500 LOC catalog records ended up being updated and enhanced with information provided by the Flickr community. This was incredibly inspiring to behold. Having studied the results of the LOC experiment, and the participation of subsequent Flickr partners, we decided to tweak the conditions slightly. NYPL librarians have already spent a ton of time describing these photos, particularly with subject headings that convey the contents of the images. Rather than discard this information, we've added a selection of these headers, repurposing them as tags, and posted them as a nucleus for Flickr viewers to build from. The hope is that this will stimulate rather than stifle activity on the Commons, with librarians and non-librarians collaborating on the description of this material. Judging by initial reactions (we went live last night and have been tracking since then), Flickr users seem unfazed by the presence of this seed data and are tagging and commenting vigorously. A couple of encouraging quotes:

"These are wonderful. It's great to see how libraries with valuable collections like this are making them available more widely through digitization. Thanks for the links to the NYPL Digital Gallery." »»

And:

"Yes, they are so beautiful. Libraries really have a wealth of stuff waiting to be shown. Thank you New York Public Library! We love this!" »»

The next step is for our curators to jump into the fray to answer questions (and perhaps ask some too), and to sift through all the tags, comments and annotations. We expect to learn a lot from Flickr users and are thrilled at the exposure that this project will give to our photographic collections. We also see the Flickr Commons as a sort of training ground for our staff — a place to get some serious hands-on experience collaborating with users in a vibrant social Web community. Down the road, we expect to implement similar tools and features on our own site, say, for example, in the Digital Gallery. Stay tuned for further reports from our Flickr experiment. We're excited to see where this goes.

Library of Congress + Flickr = tagging for everyone

Sat, 19 Jan 2008 09:52:32 -0500

The Astor Library was opened to the public almost 150 years ago. One reason it was not viewed as a success is expressed in the illustration left.

Most of us, I think, would agree that democratization of information is a good thing. Making books, art, music freely available to more people can only bring about societal enrichment. The New York Public Library has a history of doing just that.

Working with the idea of social collective knowledge, libraries seem to be pushing the boundaries further. Even the Library of Congress is considering the benefits of allowing patrons to manipulate and create content. Recently the LOC collaborated with Flickr, uploading two collections of photographs with no applicable copyright restrictions, and allowing other Flickr users to tag the photos with their own descriptions. It is an interesting idea and many of the photographs are great, especially the color transparencies.