NYPL Blogs: NYPL Labs

Open Audio Weekend Hackathon: Exploring Audio Accessibility for the Public Good

Shana Kimball — Thu, 07 Jul 2016 11:12:03 -0400

On June 25 and 26, the Library welcomed developers, designers, data scientists, audio producers, and others to a two-day hackathon designed to advance the cause of audio accessibility and explore future uses of media archives online. This event built upon and extended our work on Together We Listen, a project generously supported by a Knight Foundation prototype grant which was awarded to NYPL and our incredible partners The Moth to crowdsource corrections to computer-generated transcripts.

Kicking Off the Day

Open Audio Weekend attracted a diverse group of participants, with backgrounds in libraries, oral history, software development and more.

Image by Jemma Brown

All Open Audio Weekend participants received t-shirts designed by NYPL's Brian Foo. The design is five different representations of the spoken phrase "Open Audio Weekend" which Brian stitched together from audio clips in the NYPL Community Oral History project. The first representation is the words, the second is the phonetic transcription, the third is musical notation based on the pitches and rhythms of the speakers, the fourth is the pitch analysis based on raw frequency data of the speakers, and the last is the waveform or amplitude of the speech.

We shared an audio mashup (created by Tim Lou Ly) of Moth stories and excerpts from NYPL’s Community Oral History Project.

Open Audio Weekend Themes

Participants formed into small groups around the following prompts, designed to focus work around broad themes:

Discover: What new experiences can we make around discovery of audio?
Find: How can we make it easier to search audio?
Listen: What are some meaningful ways we can augment the experience of listening to audio?
Share: How can we make it easier to share audio?
Learn: What can we learn from audio? How can audio be used in an educational context?
Engage: How can we engage a community through audio?
Access: How can we make audio collections more usable for people with disabilities?

Participant-Designed Projects

The projects presented during our shareout on Sunday afternoon represented a rich array of prototypes and creative imaginings of the future of audio accessibility. The full list of projects is available on the GitHub repo for the event, and here are the highlights!

CrowdScribe

Crowdscribe is a proof of concept for a Chrome extension that supports crowdsourced transcriptions. Users can request transcriptions of media using the extension, and users who are on the same webpage at the same time will get a notification to help transcribe media on the page. This prototype raises awareness around accessibility, allows for the crowdsourcing of transcription, and is designed with live events in mind. By targeting live events, the extension builds upon existing communities and audiences.

BPL Sampler

Have you ever wondered what an oral history would sound like overlaid on top of hip hop beats? Check out the BPL Sampler, which remixes voices excerpted from the Brooklyn Public Library Our Streets, Our Stories project.

P.I.T.C.H.Y. D.A.W.G. (Perfecting Interactive Technology for Content Heard by You Despite Awkward Word Groupings)

In addition to being an imaginatively named project, P.I.T.C.H.Y. D.A.W.G. combines the experience of listening to audio with exploring related media, with the option of 3 modalities: Audio Only, Highlights, Full Experience.

Storynode

Mapping place names from oral history collections. To build this prototype, the group ran transcripts through the Stanford Named Entity Recognizer. These place names were then plotted on a map, creating a cartographic way of browsing audio and transcriptions.

InstaBurns

InstaBurns is an experiment in auto-generating common terms and their frequency from transcripts in order to explore the relationship of terms within and across audio files. The InstaBurns platform also uses significant terms to automatically generate a slideshow of related images using the Google Image API.

A-to-V

A-to-V is a one-stop central database where collectors of oral histories provide searchable information about their audio files and make those files directly available to users.

Homemade History

A project modeling potential engagement and reuse activities around oral history collections. Building on NYPL's Open Transcript Editor, this model would allow for users to clip two minutes of an oral history and record their own complimentary response to the clip which would be ingested back into the larger collection.

Last but not least, the entire event was transcribed in real time by Stanley Sakai from the Open Steno Project.

Thanks to all the participants for making our weekend-long hackathon a great success. Thanks also goes to our great Together We Listen partners PopUp Archive and The Moth, as well as contributing partners Gimlet Media, Buzzfeed Audio, Fresh Air, WNYC Archives, Library of Congress, Brooklyn Public Library, PRI's The World in Words, Columbia University Master of Arts in Oral History Program and to the support of the Knight Foundation for making all of this possible.

More scenes from the event below!

National Photo Month at the Digital Imaging Unit

Eric Shows, Assistant Director, Digital Collections Services — Tue, 31 May 2016 15:22:46 -0400

The Digital Imaging Unit at The New York Public Library is an extraordinary place filled with talented artists and photographers who are dedicated to providing the public with images from the library’s special collections. I’m particularly fond of the notion that the work we do helps to release information from the page and put it at the fingertips of a new kind of internet-connected public library patron. This flow of information and its impact also has a reverse component, for we often and unexpectedly find ourselves transformed in the process.

Interacting with the special collections materials in the way we do, carefully and expertly handling the rarest and most fragile artifacts of our shared cultural heritage, and putting these objects in front of the highest-resolution cameras available reveals details and moments that inevitably stop us in our tracks. We see a person in a window looking back at the camera, an erasure, inky fingerprints on the back of a manuscript, the otherworldly skill and precision required to accomplish a particular drawing or print, or we pause in front of the overwhelming beauty of an object. We find ourselves seeing the objects, photography, the world, and ultimately ourselves differently after these encounters. As professional photographers, nothing brings us more pleasure than to be faced with the prints of photographic luminaries and to be able to attend to their translation into the networked landscape.

Here are a few highlights from our most beloved encounters with the library’s photo collections that we’ve seen along the way. —Eric Shows, Digitization Services Manager

Sharecroppers' children on Sunday, near Little Rock, Arkansas. Image ID: 5251626

Arkansas sharecropper. Image ID: 5338462

Waiting for relief agent, Scott's Run, Monongalia County, West Virginia. Image ID: 5326678

I was very fortunate to handle most of the Library’s collection of Ben Shahn’s Depression-era FSA photographs. Shahn was primarily a painter and illustrator, which I think made him uninhibited behind the camera, but also very observant. He photographed his subjects in such a thoughtful way that they do not come across as victims from a bygone era, but as real and relatable people. —Martin Parsekian, Collections Photographer

George Avakian and Anahid Ajemian during the Ajemian sisters' first European tour. Image ID: 5665559

George Avakian recording Sidney Bechet. Image ID: 5649287

Earliest photo of George Avakian, with his parents in Tiflis. Image ID: 5649237

I selected these images because I find it fascinating that we can appreciate and witness through them the life, work, and legacy of American music producer/writer, George Avakian. I think is great that not only was he recognized for playing a major role in the development of jazz, but also for impacting the lives of many great artists through his work as a music producer. —Jenny Jordan, Collections Photographer

Joan Mitchell. Image ID: 5154611

Helen Frankenthaler. Image ID: 5154371

Willem DeKooning. Image ID: 5154297

I selected photographs of Willem DeKooning, Helen Frankenthaler, and Joan Mitchell by Walter Silver. These images capture the people behind iconic New York School paintings. The casual studio shots help me to imagine living and working with the abstract expressionists in the 1950s New York. —Rebecca Baldwin, Collections Photographer

From forest to mill. Image ID: 110149

View of Log-raft. Columbia River. Image ID: G91F306_021F

Sawing timber. Image ID: 110132

I am always amazed by how we do what we do, for better, for worse, all the while. —Steven Crossot, Assistant Manager, Digitization Services

Housetop life, Hopi. Image ID: 433127

“Mobile anti-aircraft searchlights…” Image ID: 5111981

Storm. Image ID: 5147219

These three images represent the peculiarity of the library’s photographic collection. The romance and exoticism of Edward Curtis’ images of Native Americans from the early 20th century, official U.S. government press photos of military from WWI, and an annotated work print from the Walter Silver collection, with unintentionally ironic subtext. —Adam Golfer, Collections Photographer

Blossom Restaurant, 103 Bowery, Manhattan. Image ID: 482799, New York Public Library

Testing meats at the Department of Agriculture. Beltsville, Maryland. Image ID: 3999921

Scott's Run mining camps near Morgantown, West Virginia. Domestic interior. Shack at Osage. Image ID: 5233691

These are a few of the photographs that have stuck with me over the years, by Berenice Abbott, Carl Mydans, and Walker Evans. Whether capturing the graphic signage and intensity of expression on people’s faces, the oddity of a testing scene or the subtle beauty and pride portrayed through a domestic scene, they all resonate with me in different ways. —Pete Riesett, Head Photographer

Chicken Market, 55 Hester Street, Manhattan. Image ID: 482844

Bread Store, 259 Bleecker Street, Manhattan. Image ID: 482591

Pingpank Barber Shop, 413 Bleecker Street, Manhattan. Image ID: 482595

I love exploring the NYPL's photography collections because of the historical and pictorial relevance of the works they hold, including Berenice Abbott's Changing New York—a series of stunning, iconic black and white photographs of the "old" city. Abbott was an extraordinarily skilled architectural photographer, but I especially enjoy her methodical documentation of storefronts as an integral part of the city, featuring visually glorious layers of texture and content. —Allie Smith, Collections Photographer

Tiger Man: Animal Graffiti, 14 St. Image ID: 5038710

Pretty Long Haired Woman Looking Up: Pretty Man in White Polo Looking Down. Image ID: 5038738

Woman Through Graffiti Window is Caught Unaware By Camera: Crowded Group in Car, Woman in Fur Coat and Wool Hat Looks at Camera. Image ID: 5038756

These are just a few images in a wonderful series by photographer Alen MacWeeney that were taken in 1977 in the NYC subway. At first glance I love these photographs because of how cool and stylized they look depicting the 1970s graffiti covered New York. Then you peer in closer and you see how MacWeeney added his own twist to the images by pairing two separate images to create diptychs which at first sight might appear to be one image. That creates an interesting narrative between the cast of characters. Added plus with these images is that no one is typing away on their phones and it doesn’t appear as crowded and jam packed with people as it is today. But there are things in the photos that never change on the subway and are timeless. —Marietta Davis, Collections Photographer

The OldNYC App Is Here! We Spoke with Its Creators

Shana Kimball — Wed, 04 May 2016 10:32:53 -0400

In May 2015, Brooklyn-based developer Dan Vanderkam launched OldNYC, providing users a new way to experience NYPL's Photographic Views of New York City collection and discover the history behind the places New Yorkers see everyday. Now Orian Breaux and Christina Leuci—fans of NYC history, maps, and technology—have brought the OldNYC experience to mobile phones. I spoke with them to find out more about the development of this app, which is now available in the iTunes App Store.

Who are you, and what do you do?

Orian: We both work in NYC’s tech world. I’m a product manager, currently focused on user growth at LiveAuctioneers. I love exploring topics like data visualization, urban science, and user behavior. On the side, I mentor aspiring technologists at a tech school called General Assembly and build projects like NYC Time Machine. Before entering tech, I studied aeronautical engineering at Rensselaer Polytechnic Institute (RPI).

Christina: As for me, I'm a software developer at Cyrus Innovation, where we help companies build and scale their software products. Previously, I studied web development at Flatiron School, after attending Rutgers University for two years. When we’re not working, you can find us dancing Swing or Argentine Tango!

How did you decide to make a mobile app for OldNYC?

Orian: It was totally spontaneous. Since OldNYC launched last May, I had spent many hours admiringly exploring its photographs. In a November mailing list update, creator Dan Vanderkam asked if anyone wanted to build a mobile app for OldNYC. As someone who loves this intersection of NYC history, maps, and technology, it was an automatic “Yes!” for me. Coincidentally, Christina had been talking about creating an iPhone app, so I immediately volunteered ourselves.

Christina: It was an exciting project to start, and we’re thankful for Dan’s openness to our building off his work. After exchanging emails, we all met for coffee one winter morning, then Orian and I got to work. Admittedly, neither of us had built an iOS app before, but with our combined coding, product, and design experience, we thought we could learn quickly enough to build a solid app.

What was your process?

Orian: We started with a question: “How can we use the mobile platform to enable an experience not possible on a regular desktop website?” From the beginning, we had this vision where you could walk down a street and easily access historical photos nearest to your location. Like a self-guided historical tour, where you’re encouraged to discover “what was there” anywhere you go.

We worked backwards from that idea, breaking down the pieces of design, development, and research work we needed to do. Dan had already built some infrastructure for storing photos and related data, so we spent some time figuring out what we could use and what we had to build ourselves.

Christina: For our version 1 release, we focused on foundational features, like map navigation, photo viewing, and allowing the user to center on their location. We did several iterations of wireframes on paper, thinking about how the user would interact with each of these features. From there, mocking up the experience in Sketch, a digital design tool, helped us refine the user interface and interactions further.

Of course, we spent most of our time coding the app. We were both new to iOS development, so for everything we built, we spent time teaching ourselves.

Orian: All throughout, we tested the app often. I made it a point to test not only while walking in the streets, but on boats, in the subway, and deep inside buildings. Since our app relies on location and downloading images from the NYPL’s website, testing helped us find and fix bugs to ensure the app runs smoothly.

What is most interesting about bringing 1900s photographs to 2016 phones?

Orian: The idea that technological progress is constantly redefining our relationship with the past. To retrieve a historical record 100 years ago meant pinpointing its specific location within a specific library or museum. Only in the last few decades have computers and the Internet allowed us to centralize, organize, and distribute that information throughout the world.

Beyond making information accessible and searchable, I think the next problem is discovery. With so much of the world’s information available online, it’s easy to find something when you generally know what you’re looking for. But how do you find information that you would love when you don’t know it exists?

This fundamental problem of discovery is why Amazon and Spotify build recommendation engines and why Pinterest tells you about trending pins. They are methods for surfacing content you’re unlikely to find by yourself. I see OldNYC’s map interface + location tracking as an extension of that idea, where a user can discover new photos highly relevant to where they are in the present moment.

It’s fun to envision what new ways future technology will help us discover the past. I’m personally waiting for the day I can take a virtual reality tour of 1600s New Amsterdam!

Christina: For me, the most interesting thing is the discussion that will rise from these resurfaced pieces of history. By exploring photos from New York City’s past, you really get a sense of the society that was present from the late 1800s into the early 1900s and how it has changed throughout the years.

People are now able to step through the city’s life and watch as it has grown and changed into the metropolis it is now. Can you imagine, hearing stories from your Grandmother who lived in New York City in the 1930s and being able to find the intersection she lived at and explore that photo in detail?

The stories we discover are preserved through their documented photos and text and provide a sense of identity for the city. As we expand our dataset, users will be able to trace any NYC intersection’s life from a drawing of farmland to a modern skyscraper.

What’s next for your app?

Christina: We’re just getting started! We have a full backlog of features and improvements to make the app experience more special. Our app users can give feedback from within the app, which also helps us prioritize what to build next.

For incremental improvements, we’d like to increase photo resolution, improve the photo viewing interface, and enable search by location. We also need to include Dan’s feature of allowing users to submit corrections of photograph descriptions.

Not all improvements are user-facing though; there’s a lot we can do to improve our data architecture and make the app run more efficiently.

Orian: That said, we’re most excited for the bigger initiatives, including:

Commenting on photographs, so app users can tell their stories. The OldNYC desktop site captured thousands of comments, so we’ll make these readable from the app, as well.
Enabling users to easily generate and share before-and-after photos. It’s a thrill to discover “what was there” in the areas we live and visit often. We want to help people share that moment of discovery with the world.

What other NYPL data sets or resources would you like to explore or build on?

Christina: If we could wave a magic wand, every historical record would be discoverable by location. All datasets are fair game, but the next logical step is to incorporate more photographs. Currently, OldNYC only uses photographs from the Milstein Collection’s “Photographic Views of New York City, 1870s-1990s”

Beyond that, we’d love to include lithographs that capture scenes from the 1600s to 1870s, as well as any oral records. Imagine walking around the city with the ability to explore every facet of your current location’s history!

Orian: We’re also exploring using rectified maps from NYPL Map Warper, like the 1920s Aerial survey map. The idea is that app users can use a “timeline slider” to sift through the layers of history, to see photographs in their historical context.

How can libraries—or any holder of large open datasets—make it easier for developers to make new things with our collections?

Christina: I think the key ingredients are: 1) helping developers feel supported and 2) maintaining clean and open datasets.

As a developer, I feel supported when an organization is actively giving me the tools and instruction I need to start working with the data. A library can do this by offering tutorials/code samples and walkthroughs and by describing interesting use cases for the datasets. All this cuts down on “research time” and empowers developers to take action. The NYPL Digital Collections API is a great approach for libraries to model. It’s an open database that gives developers the chance to build applications that access NYPL data.

Over time, as developers build applications off a dataset, you can foster community by showcasing work. This encourages more developers to join and extend the work of others, like how Orian and I have done with Dan’s OldNYC.

Thanks for speaking with us!

Together We Listen: Make Hundreds of NYC Stories Accessible—One Word at a Time

Alexandra Kelly, Outreach Services and Adult Programming — Tue, 05 Apr 2016 12:05:35 -0400

The following blog post is co-authored by Willa Armstrong (NYPL Labs) and Alex Kelly (Adult Programming and Outreach Services).

Are you familiar with the NYPL Community Oral History Project? Take a few moments to listen to some highlights or just dive right into our full collection of stories.

The NYPL Community Oral History Project is truly the people’s project. It’s powered by the public, as hundreds of engaged community members come together to gather oral histories from each other in order to preserve the rich and constantly changing history of New York City. Beginning in 2013 at Jefferson Market Library in Greenwich Village and building momentum, oral histories have been collected in six additional neighborhoods. Visible Lives, an oral history project on the disability experience is another large scale collection effort, based out of Andrew Heiskell Braille and Talking Book Library. Read more about our growing collection at oralhistory.nypl.org/about.

To date, the Community Oral History collection contains over 1,000 stories, with more on the way! This is very exciting, but here’s the issue: We’re faced with the challenge of making this large corpus of audio accessible and searchable to the public. It’s a challenge faced by many organizations and institutions with audio-based collections and archives.

When working to make audio accessible, transcripts are important because they make audio content searchable online, and accessible to people with hearing disabilities. Recent advances in speech-to-text technologies have made great progress in opening audio to the web, but the transcripts they produce are still error-prone and can only be considered first drafts. Though they’re a good start, careful human editing is required to provide polish these computer-generated drafts and ensure accurate, high quality transcripts.

So, how are we going to transcribe these hundreds of audio hours quickly and cost effectively? People and computers need to collaborate.

And this brings us to our big announcement: NYPL Labs has built a brand new Open Transcript Editor to engage the public in helping to make our oral history collection accessible—one word at a time. The Open Transcript Editor is an interactive transcript editor allowing multiple people to perform the final layer of polish and proofreading on computer-generated transcripts. It’s a big undertaking and we’re inviting the public to pitch in and help correct computer-generated transcripts from our NYPL Community Oral History Project.

Visit transcribe.oralhistory.nypl.org to get started and help make this public treasure trove of NYC stories accessible.

Since we’re not the only ones tackling this challenge, the NYPL has teamed up with The Moth, a live storytelling organization with its own growing audio archive, for Together We Listen, a community program that invites our respective audiences to correct transcripts online using this new tool. To get our initial, computer-generated transcripts, both partners have sent their stories through Pop Up Archive, a speech-to-text service that works extensively with the public media and cultural heritage sectors. This project was made possible with generous support provided by the Knight Foundation Protoype Fund, an initiative of the John S. and James L. Knight Foundation. The Open Transcript Editor codebase itself is open source and is available to be used and further developed around other audio archives.

Anyone can contribute to this effort online and, for the first time ever, NYPL is also organizing in-person events at various library locations to encourage people to get together and help transcribe the oral histories for their own communities. We hope you can join us at an event in your neighborhood!

By editing transcripts, you're helping to create truly accurate transcripts in order to share hundreds of stories from the ongoing Community Oral History Project. Once they have been edited with agreement from enough contributors, completed transcripts will be available to read and download at oralhistory.nypl.org, along with the audio recordings.

Pitch in now: Tune in and transcribe!

Join the Together We Listen project!
Join our initiative to make New York City history accessible one story at a time! We've partnered with The Moth to create a transcription tool that will allow you to help us improve upon computer-generated transcripts for over 1,000 stories from our Community Oral History Project. Be a part of history today: http://on.nypl.org/1RZCn8A
Posted by NYPL The New York Public Library on Tuesday, April 5, 2016

Want to receive updates about NYPL digital initiatives? Sign up for our e-mail newsletter.

Introducing the Photographers’ Identities Catalog

David Lowe — Fri, 25 Mar 2016 14:27:36 -0400

Today the New York Public Library is pleased to announce the launch of Photographers’ Identities Catalog (PIC), a collection of biographical data for over 115,000 photographers, studios, manufacturers, dealers, and others involved in the production of photographs. PIC is world-wide in scope and spans the the entire history of photography. So if you’re a historian, student, archivist, cataloger or genealogist, we hope you’ll make it a first stop for your research. And if you’re into data and maps, you’re in luck, too: all of the data and code are free to take and use as you wish.

Each entry has a name, nationality, dates, relevant locations and the sources from which we’ve gotten the information—so you can double check our work, or perhaps find more information that we don’t include. Also, you might find genders, photo processes and formats they used, even collections known to have their work. It’s a lot of information for you to query or filter, delimit by dates, or zoom in and explore on the map. And you can share or export your results.

Blanche Bates. Image ID: 78659

How might PIC be useful for you? Well, here’s one simple way we make use of it in the Photography Collection: dating photographs. NYPL has a handful of cabinet card portraits of the actress Blanche Bates, but they are either undated or have a very wide range of dates given.

The photographer’s name and address are given: the Klein & Guttenstein studio at 164 Wisconsin Street, Milwaukee. Search by the studio name, and select them from the list. In the locations tab you’ll find them at that address for only one year before they moved down the street; so, our photos were taken in 1899. You could even get clever and see if you can find out the identities of the two partners in the studio (hint: try using the In Map Area option).

But there’s much more to explore with PIC: you can find female photographers with studios in particular countries, learn about the world’s earliest photographers, and find photographers in the most unlikely places…

Often PIC has a lot of information or can point you to sources that do, but there may be errors or missing information. If you have suggestions or corrections, let us know through the Feedback form. If you’re a museum, library, historical society or other public collection and would like to let us know what photographers you’ve got, talk to us. If you’re a scholar or historian with names and locations of photographers and studios—particularly in under-represented areas—we’d love to hear from you, too!

Emigrant City: A Collaborative Resource

Willa Armstrong — Mon, 07 Mar 2016 14:59:35 -0500

Emigrant City, a project for transcribing 19th and early 20th century real estate records from the Emigrant Savings Bank, launched last November. As the Library’s newest participatory project, we were excited to share it and are even more excited about the support and contributions of the last four months. There have over 475,000 contributions by over 5,000 participants! Those are impressive numbers! A hearty thanks to all the remarkable participants collaborating on this project!

The work of accurately transcribing these historic mortgage records is broken into three tasks. With every box drawn in Mark, entry made in Transcribe, and vote cast in Verify, that contribution count grows and we rapidly approach the end goal of a complete, searchable index of names and mortgage details.

As of this week, contributors have identified some 9,370 records. It’s time to start sharing these records! We’ve built an interface for you to browse the records after searching names, addresses, and other fields. If you’re eager to find your great-great-uncle or the former owner of your apartment building, our Tips & Tricks page will get you started.

You can also download a bulk data set of all the project data in JSON format. Remember, with your help this resource is still growing! You can still contribute to the project by transcribing and verifying data. The browsable and downloadable records will be refreshed weekly with new contributions as we continue to complete the dataset.

This resource is only possible with your help. In addition to building the records available to browse, your contributions informed the design and format of these records and the project. Frequently asked questions, such as how to deal with multiple mortgage numbers on one record, have helped hone help text and clarify the process for the next volunteer. (If you do come across multiple mortgage numbers, mark them each individually!) If you’re interested to hear more about how the project was built, read Paul Beaudoin’s post on Scribe, the new framework powering Emigrant City. Thanks again for lending and hand and working with us. We hope you enjoy perusing the records you’ve helped create.

Additional Resources

Emigrant City provides a resource for discovering individual records. The entire physical collection is accessible at the Library's Archives & Manuscripts portal, and digitized materials are available for exploration and download on our Digital Collections website (including the records indexed in Emigrant City.) The Library also provides on-site access to Ancestry Library Edition which includes digitized resources from the Deposit-Account Ledger, Index Book, Test Books, and Transfer, Signature, and Test Books of the New York Emigrant Savings Bank records from 1850-1883.

Nomadic Classification: Classmark History and New Browsing Tool

Matt Miller, NYPL Labs — Wed, 27 Jan 2016 14:58:19 -0500

In the past few months, NYPL Labs has embarked upon a series of investigations into how legacy classification systems at the library can be used to generate new data and power additional forms of discovery. What follows is some background on the project and some of the institutional context that has prompted these examinations. One of the tools we’re introducing here is “BILLI: Bibliographic Identifiers for Library Location Information;” read on for more background, and be sure to try the tool out for yourself.

Then there is a completely other type of distribution or hierarchy which must be called nomadic, a nomad nomos, without property, enclosure or measure. Here, there is no longer a division of that which is distributed but rather a division among those who distribute themselves in an open space — a space which is unlimited, or at least without precise limits… Even when it concerns the serious business of life, it is more like a space of play, or a rule of play… To fill a space, to be distributed within it, is very different from distributing the space. —Gilles Deleuze, Difference & Repetition

Classification, the basic process of categorization, is simple in theory but becomes complex in practice. Examples of classification can be seen all around us, from the practical use of organizing the food found in your local grocery store into aisles, to the very specialized taxonomy system that separates the hundreds of different species of the micro-animal Tardigrada. At their core these various systems of categorization are simply based on good faith judgments. Whoever organized your local grocery thought: “Cookies seem pretty similar to crackers, I will put them together in the same aisle.”

A similar, but more evidence based process developed the system that categorizes hundreds of thousands of biological species. Classification systems are usually logical but are inherently arbitrary. Uniform application of classification is what makes a system useful. Yet, uniformity is difficult to maintain over long periods of time. Institutional focus shifts, the meaning of words drift, and even our culture itself changes when measuring time in decades. Faced with these challenges, in the age of barcodes, databases, and networks, the role of traditional classification systems are not diminished but could benefit by thinking how they could practically leverage this new environment.

**Nerd Alert! If 19th century classification history is not your thing you might want to skip to A Linked Space.**

Problem Space

Libraries are founded on the principle of classification, the most common and well known form being the call number. This is the code that appears on the spine of a book keeping track of where it should be stored and usually the subject of its content. The most well known form of call number is, of course, the iconic Dewey Decimal System. But there are many other systems employed by libraries due to the nature of the resources being organized and the strengths and weaknesses of a specific classification system. Just as there is no single tool for every job there is no universal system for classification.

The New York Public Library is a good example of that realization as seen in the adoption of multiple call number systems over its 120-year history. The very first system used at the library was developed in 1899 by NYPL’s first president, John Shaw Billings. He wanted to develop a system that could efficiently organize the materials being stored in the library’s soon-to-open main branch, the Stephen A. Schwarzman building. In fact, Billings also contributed to how the new main building should be physically designed:

John Shaw Billings, sketch of the main building layout. Image ID: 465480

The classification scheme he came up with, known as the Billings system, could be thought of as a reflection the physical layout of the main branch circa 1911. Billings was very practical in the description of his creation, writing:

Upon this classification it may be remarked that it is not a copy of any classification used elsewhere; that it is not especially original; that is is not logical so far as the succession of different departments in relation to the operations of the human mind is concerned; that it is not recommended for any other library, and that no librarian of another library would approve of it.

The system groups materials together by assigning each area of research a letter, A-Z (minus the letter J, more on that later). This letter, the first part of the call number, is known as a classmark.

For example, books cataloged under this system that are Biographies would have the first letter of their classmark, “A”. History is “B”, Literature is “N”, and so on through “Z”, which is Religion. More letters can be added to denote more specific areas within that subject. For example, “ZH” is about Ritual and Liturgy and “ZHW” is more specifically about Ancient and Medieval Ritual and Liturgy.

While this system was used to classify the materials held in the main branch’s stacks, there were also materials held in the reading rooms or special collections around the building. To organize these materials, he reused the same system but added an asterisk in front of the letter to make what he called the star groups — i.e., a classmark starting with a “K” is about Geography, but “*K” is a resource kept in the Rare Books division though not necessarily about Geography. With these star groups, the Billings system became a conflation of a subject, location, and material-based systems. This overview document gives a good idea of the large range classification that the Billings system covered:

Top level Billings classmarks

In the 1950s, the uptick in the acquisition of materials made the Billings system too inefficient to quickly catalog materials. While parts of the Billing system continued being used, even through to today, a general shift to a new fixed order system was made in 1956 and then refined in 1970.

The idea behind the fixed order systems is to group materials together by size to most efficiently store them. The library decided that discovery of materials could be achieved not by the classmark but by the resource’s subject headings. Subject headings are added to the record while it is being cataloged and provide a vector of discovery if the same subjects are uniformly applied.

In the United States the most common vocabulary of subject headings is the Library of Congress Subject Headings. Enabling resource discovery through subject headings obviates the need for call numbers to organize materials. The call number can just be an identifier to physically locate the resource. The first fixed order system grouped items only by size:

Old fixed order system classmarks

The call numbers would look something like “B-10 200”, meaning it was the 200th 17cm monograph cataloged. This system was refined in 1970 to included a bit more contextual information about what the materials were about:

Revised fixed order system classmarks

Since the J letter was previously unused in the Billing system it was used here as a prefix to the size system to add more context to the fixed order classification. For example, a “JFB 00-200” means it is still a 17cm monograph but is generally about the Humanities & Social Sciences.

While this fixed order system is used for the majority of the monographs acquired by the library there are special collections in the research divisions that use their own call number systems. Resources like archival collections or prints and photographs have records in the catalog that help locate them in their own division. For example, archival collections often have a call number that starts with “MssCol” while rare books at the Schomburg Center start with “Sc Rare”. This final diverse category of classification at NYPL drives home the obvious problem: the sheer number of classification systems at work reduces the call number to an esoteric identifier—especially for obsolete and legacy systems—useful to only the most veteran staff member. These identifiers have great potential to develop new avenues of discovery if they can be leveraged.

A Linked Space

An ambitious 19th-century librarian faced with this problem might come up with a simple solution: Let’s invent a new classification system that incorporates all these various types of call numbers into one centralized system. With the emergence of linked data in library metadata practice, however, when relationships between resources are gaining increased importance, a 21st-century librarian has an even better idea: Let’s link everything together!

In most existing library metadata systems, the call number is a property of the record; it is a field in the metadata that helps to describe the resource. The linked data alternative is to make the classmark its own entity and give it an identifier (URI), which allows us to describe the classmark and start making statements about it. This is nothing new in the library linked data world; the Library of Congress for example, started doing this for some of their vocabularies (including some LCC classmarks) years ago. But to accomplish this task at NYPL it took a combination of technical work, institutional knowledge sleuthing, and a lot of data cleanup.

The first step is to simply get a handle on what classmarks are in use at the library. By aggregating over 24 million catalog records and identifying the unique classmarks, we are able to create a dataset that contains all possible classmarks at the library. This new bottom-up approach to our call number data enables the next step of organization and description.

Institutional knowledge is hard to retain over 120 years of history. It takes the form of old documents and outdated technical memos. The Billings classification was first documented in a schedule (a monograph book) that lists each classmark and its meaning. Some of these original bound resources are still around the library and have gone through their own data migration journey.

Page from the bound Billings schedule

While the bound books are still in use today, most with copious marginalia, over the years this resource was converted to a typed document and then converted into the digital realm in the form of a MS Word document. This data was stewarded by our colleagues in BookOps who are the authority for cataloging at NYPL. We took this data and converted it into an RDF SKOS vocabulary and reconciled it with the raw classmark data aggregated from the catalog. This new dataset is comprehensive because it aligns what classmarks are supposed to be in use, from the documentation, with what is actually in use, from the data aggregation. Each classmark now has its own URI persistent identifier which we can begin making statements about:

Example triple statements for *H classmark - Libraries

This confusing looking jumble of text is a bunch of RDF triples in the Turtle Syntax talking about the *H (Libraries) classmark. It is describing the name of the classmark, the type of classmark it is, what narrower classmarks are related to it, how many resources use it in the catalog and some mappings to other classifications systems among other data. Now that we have all the classmarks in a highly structured semantic data model we can publish all these statements and start doing even more interesting things with them.

BILLI Home page

All of these statements about our classmarks are hosted on a system we are calling BILLI: Bibliographic Identifiers for Library Location Information, an homage to NYPL’s mustachioed first president John Shaw Billings.

The BILLI system allows you to explore the classmarks in use at the library, traverse the hierarchies, and link out to resources in the catalog. But the real power of having our classmark information in this linked data form is the ability to start building relationships by creating new data statements.

A logged in staff member can add notes or change the description of a classmark but, more importantly, is able to start linking to other linked data resources. Right now, staff members are able to connect our classmarks to Wikidata and DBpedia, two linked data sources connected to Wikipedia:

Staff interface for mapping classmarks

The system auto-suggests some possible connections, here for the *H Library classmark, and the staff member can select or search for a more appropriate entity. Once connected, we pull in some basic information to enrich the classmark page:

Classmarks page displaying external data

As well as the the mapping relations:

New mapping relationships created

Now that a link has been established we can use the metadata found in those two sources in our own discovery system and even apply some of it to the resources in our catalog that use this classmark. While the system is currently only able to build these connections with Wikidata/DBpedia, we can explore other resources that it would make sense to map to and expand the network to library and non-library based linked data sources.

While these classmark pages are here for us humans to interact with, computers can also “read” these pages automatically as they are machine readable through content negotiation. To a computer requesting *H data, the page would appear like this: http://billi.nypl.org/classmark/*H/nt

A Space of Play

If classification is arbitrary it is true there are a number of other systems that the library’s materials could be organized under. The only limitation is that it would be cost prohibitive to apply a new system to millions of resources. But in this virtual space, existing and even newly-invented systems can be easily overlaid and applied to our materials.

The first classmarks listed on BILLI are grouped under “LCC Range.” This classmark system is based on the existing Library of Congress Classification, a system which has historically not been used at NYPL. LCC is traditionally used at most research libraries, but because NYPL used the Billings and then Fixed Order systems it was never adopted here. Due to new linked data services, however, we were able to retroactively reclassify our entire catalog with LCC classmarks.

Using OCLC’s Classify—a service that returns aggregate data about a resource from all institutions in the OCLC consortium—we are able to find out a resource’s LCC classmark from other libraries that holds the same title. While we were not able to match 100% of our resources to a Classify result we were able to apply a significant number of LCC classmarks to our materials.

Of the materials for which we were unable to obtain a LCC, we can use some simple statistics to draw concordances between Billings and LCC. For example, if we have enough resources that have the same LCC and Billings classmark we can assume that the two are equivalent and apply that LCC to all the materials with that Billings classmark. One caveat is that LCC numbers can be very specific in their classification, much more so than Billings. What we need to do in order to map Billings to LCC is create more generalized, or coarser, LCC classmarks.

To accomplish this we parsed the freely available PDF LCC outlines found online into a new dataset with each classmark representing a large range of LCC numbers. Using this invented, yet still related, LCC range classmark we can map our Billings classmarks to their coarse LCC equivalents. Now, a researcher who is perhaps more familiar with LCC can easily navigate to corresponding Billings resources.

The LCC range classmark example represents an exciting opportunity to organize our materials in new ways within a virtual space. In this networked environment classification shifts from rigid hierarchy to a fluid interconnected mappings—the difference between dividing space and filling it.

Free for All: NYPL Enhances Public Domain Collections For Sharing and Reuse

Shana Kimball — Tue, 05 Jan 2016 18:07:46 -0500

Today we are proud to announce that out-of-copyright materials in NYPL Digital Collections are now available as high-resolution downloads. No permission required, no hoops to jump through: just go forth and reuse!

The release of more than 180,000 digitized items represents both a simplification and an enhancement of digital access to a trove of unique and rare materials: a removal of administration fees and processes from public domain content, and also improvements to interfaces — popular and technical — to the digital assets themselves. Online users of the NYPL Digital Collections website will find more prominent download links and filters highlighting restriction-free content; while more technically inclined users will also benefit from updates to the Digital Collections API enabling bulk use and analysis, as well as data exports and utilities posted to NYPL's GitHub account. These changes are intended to facilitate sharing, research and reuse by scholars, artists, educators, technologists, publishers, and Internet users of all kinds. All subsequently digitized public domain collections will be made available in the same way, joining a growing repository of open materials.

To encourage novel uses of our digital resources, we are also now accepting applications for a new Remix Residency program. Administered by the Library's digitization and innovation team, NYPL Labs, the residency is intended for artists, information designers, software developers, data scientists, journalists, digital researchers, and others to make transformative and creative uses of digital collections and data,and the public domain assets in particular. Two projects will be selected, receiving financial and consultative support from Library curators and technologists.

To provide further inspiration for reuse, the NYPL Labs team has also released several demonstration projects delving into specific collections, as well as a visual browsing tool allowing users to explore the public domain collections at scale. These projects, which suggest just a few of the myriad investigations made possible by fully opening these collections, include:

a "mansion builder" game, exploring floor plans of grand turn-of-the-century New York apartments;
a then-and-now comparison of New York's Fifth Avenue, juxtaposing 1911 wide angle photographs with Google Street View; and
a "trip planner" using locations extracted from mid-20th century motor guides that listed hotels, restaurants, bars, and other destinations where Black travelers would be welcome.

The public domain release spans the breadth and depth of NYPL's holdings, from the Library's rich New York City collection, historic maps, botanical illustrations, unique manuscripts, photographs, ancient religious texts, and more. Materials include:

Berenice Abbott's iconic documentation of 1930s New York for the Federal Art Project
Farm Security Administration photographs by Walker Evans, Dorothea Lange, Gordon Parks, and others
Manuscripts of American literary masters like Walt Whitman, Henry David Thoreau, and Nathaniel Hawthorne
Papers and correspondence of founding American political figures like Alexander Hamilton, Thomas Jefferson, and James Madison
Sheet music for popular American songs at the turn of the 20th century
WPA-era lithographs, etchings, and pastels by African American artists
Lewis Hine's photographs of Ellis Island immigrants and social conditions in early 20th century America
Anna Atkins' cyanotypes of British algae, the first recorded photographic work by a woman (1843)
Handscrolls of the Tale of Genji, created in 1554
Medieval and Renaissance illuminated manuscripts from Western Europe
Over 20,000 maps and atlases documenting New York City, North America, and the world
More than 40,000 stereoscopic views documenting all regions of the United States

We've made it easier than ever to search, browse, and download public domain items in Digital Collections. This video shows you how:

Explore the Public Domain

Visit nypl.org/publicdomain for information about the materials related to the public domain update and links to all of the projects demonstrating creative reuse of public domain materials. Go forth, and reuse!

Have you been inspired by or created new works by re-using items in NYPL’s Digital Collections? We would love to hear about your work! Please share it with us using #nyplremix or by email at digitalcollections@nypl.org.

Scribe: Toward a General Framework for Community Transcription

Paul Beaudoin — Mon, 23 Nov 2015 17:08:30 -0500

A couple of weeks ago, NYPL Labs was very excited to release Emigrant City, the Library's latest effort to unlock the data found within our collections.

But Emigrant City is a bit different from the other projects we’ve released in one very important way: this one is built on top of a totally new framework called Scribe, built in collaboration with Zooniverse and funded by a grant from the NEH Office of Digital Humanities along with funds from the University of Minnesota. Scribe is the codebase working behind the scenes to support this project.

What is Scribe?

Scribe is a highly configurable, open-source framework for setting up community transcription projects around handwritten or OCR-resistant texts. Scribe provides the foundation of code for a developer to configure and launch a project far more easily than if starting from scratch.

NYPL Labs R&D has built a few community transcription apps over the years. In general, these applications are custom built to suit the material. But Scribe prototypes a way to describe the essential work happening at the center of those projects. With Scribe, we propose a rough grammar for describing materials, workflows, tasks, and consensus. It’s not our last word on the topic, but we think it’s a fine first pass proposal for supporting the fundamental work shared by many community transcription projects.

So, what’s happening in all of these projects?

Our previous community transcription projects run the gamut from requesting very simple, nearly binary input like “Is this a valid polygon?” (as in the case of Building Inspector) to more complex prompts like “Identify every production staff member in this multi-page playbill” (as in the case of Ensemble). Common tasks include:

Identify a point/region in a digitized document or image
Answer a question about all or part of an image
Flag an image as invalid (meaning it’s blank or does not include any pertinent information)
Flag other’s contributions as valid/invalid
Flag a page or group of pages as “done”

There are many more project-specific concerns, but we think the features above form the core work. How does Scribe approach the problem?

Scribe reduces the problem space to “subjects” and “classifications.” In Scribe, everything is either a subject or a classification: Subjects are the things to be acted upon, classifications are created when you act. Creating a classification has the potential to generate a new subject, which in turn can be classified, which in turn may generate a subject, and so on.

This simplification allows us to reduce complex document transcription to a series of smaller decisions that can be tackled individually. We think reducing the atomicity of tasks makes projects less daunting for volunteers to begin and easier to continue. This simplification doesn’t come at the expense of quality, however, as projects can be configured to require multiple rounds of review.

The final subjects produced by this chain of workflows represent the work of several people carrying an initial identification all the way through to final vetted data. The journey comprises a chain of subjects linked by classifications connected by project-specific rules governing exposure and consensus. Every region annotated is eventually either deleted by consensus or further annotated with data entered by several hands and, potentially, approved by several reviewers. The final subjects that emerge represent singular assertions about the data contained in a document validated by between three and 25 people.

In the case of Emigrant City specifically, individual bond records are represented as subjects. When participants mark those records up, they produce “mark” subjects, which appear in Transcribe. In the Transcribe workflow, other contributors transcribe the text they see, which are combined with others’ transcriptions as “transcribe” subjects. If there’s any disagreement among the transcriptions, those transcribe subjects appear in Verify where additional classifications are added by other contributors as votes for one or another transcription. But this is just the configuration that made sense for Emigrant City. Scribe lays the groundwork to support other configurations.

Is it working?

I sure hope so! In any case, the classifications are mounting for Emigrant City. At writing we’ve gathered 227,638 classifications comprising marks, transcriptions, and verifications from nearly 3,000 contributors. That’s about 76 classifications each, on average, which is certainly encouraging as we assess the stickiness of the interface.

We’ve had to adjust a few things here and there. Bugs have surfaced that weren’t apparent before testing at scale. Most issues have been patched and data seems to be flowing in the right directions from one workflow to the next. We’ve already collected complete, verified data for several documents.

Reviewing each of these documents, I’ve been heartened by the willingness of a dozen strangers spread between the US, Europe, and Australia to meditate on some scribbles in a 120 year old mortgage record. I see them plugging away when I’m up at 2 a.m. looking for a safe time to deploy fixes.

What’s next?

As touched on above, Scribe is primarily a prototype of a grammar for describing community transcription projects in general. The concepts underlying Scribe formed over a several-month collaboration between remote teams. We built the things we needed as we needed them. The codebase is thus a little confusing in areas, reflecting several mental right turns when we found the way forward required an additional configuration item or chain of communication. So one thing I’d like to tackle is reigning in some of the areas that have drifted from the initial elegance of the model. The notion that subjects and workflows could be rearranged and chained in any configuration has been a driving idea, but in practice the system obliges only a few arrangements.

An increasingly more pressing desire, however, is developing an interface to explore and vet the data assembled by the system. We spent a lot of time developing the parts that gather data, but perhaps not enough on interfaces to analyze it. Because we’ve reduced document transcription into several disconnected tasks, the process to reassemble the resultant data into a single cohesive whole is complicated. That complexity requires a sophisticated interface to understand how we arrived at a document’s final set of assertions from the the chain of contributions that produced it. Luckily we now have a lot of contributions around which to build that interface.

Most importantly, the code is now out in the wild, along with live projects that rely on it. We’re already grateful for the tens of thousands of contributions people have made on the transcription and verification front, and we’d likewise be immensely grateful for any thoughts or comments on the framework itself—let us know in the comments, or directly via Github, and thanks for helping us get this far.

Also, check out the other community transcription efforts built on Scribe. Measuring the Anzacs collects first-hand accounts from New Zealanders in WWI. Coming soon, “Old Weather: Whaling” gathers Arctic ships’ logs from the late 19th and early 20th centuries.

Emigrant City: An Introduction

Willa Armstrong — Wed, 04 Nov 2015 12:09:30 -0500

NYPL Labs and the Irma and Paul Milstein Division of United States History, Local History and Genealogy are excited to announce the launch of Emigrant City, the Library's newest, online participatory project. Emigrant City invites you to help transcribe recently digitized mortgage and bond record books from the Library’s collection of Emigrant Savings Bank records. Your transcriptions will help make the materials digitally accessible to all, including genealogists, educators, historians. In the process, you'll get a detailed glimpse of real estate transactions and immigrant life during a foundational period of New York City's history. Help the Library build this exciting new resource!

Still operating today, Emigrant Bank is the oldest savings bank in New York City and the ninth-largest privately owned bank in the country. It was founded in 1850 by 18 members of the Irish Emigrant Society with the goal of serving the needs of the immigrant community in New York. NYPL’s Manuscripts and Archives Division houses the Library’s collection of the early records from the bank. The collection’s first mortgage record is dated February 20, 1851. From the mid-19th century through the 1920s, there are an estimated 6,400 mortgages, each telling a story of upward mobility in a rapidly expanding city. (Two of these stories are found in mortgages 1 and 87, belonging to Francis A. Kipp and Mary O'Connor, respectively. Stay tuned for a forthcoming blog post detailing their stories.)

These real estate records have remained largely invisible and difficult to search. However, the full Emigrant Savings Bank collection is frequently consulted by genealogists and historians, among others. This collection contains a wide variety of materials about the bank's depositors and borrowers, including minutes of the board of trustees and finance committee. Portions of this larger collection, the test books, have even been digitized and made available through Ancestry.com. (This resource is available onsite at all NYPL locations.) Through digitization of the real estate records, and transcribing the hand-written information they contain, we hope to expose this underused portion of the collection to enable new discoveries and research.

Emigrant City is also an experiment. Digitizing materials is much more than simply creating a digital image of a manuscript or artifact. Though computers have made fantastic advances in automatically converting digitized pages into searchable text, vast troves of information exist in libraries and archives that require careful human labor to unlock their deeper contents to search engines and digital researchers. So here at NYPL Labs, we’ve been working with the citizen science mavens at Zooniverse, with generous support from the National Endowment for the Humanities, to prototype of a highly configurable crowdsourcing framework called Scribe that could be used on a wide range of historical and archival material.

Emigrant City joins a collection of crowdsourcing projects launched by NYPL in recent years, including Building Inspector and What's on the Menu? Go to emigrantcity.nypl.org to get started! There are lots of records to go through, and when finished, we’ll have a robust data-set of verified, structured data. Meanwhile, the team is working to create browsing and bulk download options for this wealth of information. With the growing data set, we’ll be able to find myriad stories, like those of Francis Kipp and Mary O'Connor, and to ask innumerable questions. It’s a lot of work, but we’re confident we can do it. Join us!

NYPL Labs Residency with Triple Canopy

Shana Kimball — Thu, 15 Oct 2015 10:41:07 -0400

NYPL Labs is pleased to announce Anjuli Raza Kolb, Jaffer Kolb, and Kameelah Janan Rasheed as the recipients of first residency co-hosted Triple Canopy magazine. Engaging materials in The New York Public Library’s holdings both critically and imaginatively, as documents of past events and practices as well as bases for new creative work, these projects enact new modes of reading and writing history.

Over the next year, recipients will work closely with Triple Canopy editors toward the creative and technical realization of their projects, which may be published in a variety of formats, ranging from public performance to print pamphlet to hand-coded Web work. To this end, recipients receive six months of access to one of the research study rooms at the Stephen A. Schwarzman Building; access to reference librarians and the NYPL Labs team for one-on-one consultations; and an honorarium of $2,000 from Triple Canopy. Their projects will be published in Triple Canopy's online magazine and also represented through public programs at the New York Public Library, as part of NYPL Labs’ ongoing effort to engage digital practitioners in new forms of knowledge creation and dissemination.

Triple Canopy received numerous well-conceived proposals in response to our call. We would especially like to acknowledge finalists Jenna Bliss, Adeola Enigbokan, Alan Felensthal, and Alan Reid, who submitted particularly strong and compelling projects.

We thank everyone who applied to the call and congratulate this year’s recipients!

Victor Prevost, Men Standing on Willowdell Arch, 1862. Image ID: psnypl_rbk_881

Anjuli Raza Kolb and Jaffer Kolb are siblings working at the intersection of visual and scholarly practice. Anjuli is a professor of English and comparative literature at Williams College and Jaffer is a designer and lecturer at Princeton University's School of Architecture. For their Triple Canopy commission, the Kolbs will explore the social and architectural history of the Central Park Ramble. Frederick Law Olmsted and Calvert Vaux, along with master gardener Ignaz Anton Pilát, designed the 38-acre Ramble to bring the “mysterious illusion of lush, tropical vegetation” into an otherwise highly rational park. The perfect combination of wildness and density, invisibility and navigability, the Ramble has served as one of the city’s most significant wilds, a space for orgiastic reverie, for community and solidarity, and for refuge and love during the darkest days of the AIDS crisis. The Kolbs’s research will draw upon the library’s Pilát papers and the Gay Activists Alliance records.

Prophet Noble Drew Ali (standing center) and temple members, at religious service of the Moorish Science Temple of America, circa late 1920s. Image ID: 5217112

Kameelah Janan Rasheed is an artist-archivist whose work has been exhibited at the Studio Museum in Harlem, the Queens Museum, the Bronx Museum, the Schomburg Center for Research in Black Culture, and the Weeksville Heritage Center, among other venues. Rasheed’s project focuses on printed matter, sermons, and religious iconography produced in the late nineteenth and early twentieth centuries within black religious movements in the United States. She will explore how self-anointed black prophets created and sustained post-slavery communities, inspiring a sense of belonging on the part of followers. Her research will involve sermons, song lyrics, proselytizing materials, photographs, FBI investigation files, and ephemera related to the Church of God and Saints of Christ, the Moorish Science Temple of America, and the International Peace Mission, all housed at the New York Public Library’s Schomberg Center. Rasheed will write a sermon for the eve of the twenty-second century that draws on these documents, as well as an essay that includes annotated archival materials and video interviews with contemporary adherents of these movements.

The Internet Loves Digital Collections: April 2015

Josh Hadro — Mon, 04 May 2015 10:15:15 -0400

January 18, 1969 menu from the Oak Room at the Plaza Hotel

What was the most viewed image on NYPL's Digital Collections platform in April 2015?

It was a menu.

Specifically, a menu from the Oak Room at the Plaza Hotel, dated January 18, 1969.

How did that happen?

Seems that a little show called Mad Men featured a mention of the Oak Room at the Plaza Hotel in the third episode of this final season. Following the airing of that episode a few weeks ago, Gothamist posted a recap and review titled "Unpacking Last Night's Mad Men: From The Oak Room To Port Authority," featuring some color commentary about the various settings and details in the show. That recap featured a link to the menu above from our Digital Collections, and made that image the most viewed in April 2015.

What sorts of things were the Mad Men characters dining on in January 1969 (or similarly in 1970, where the show's chronology has now arrived)?

Shrimp Cocktail ($2.10)
Terrine of Imported Foie Gras ($3.65)
Sirloin steak (for 2, at $19.70)
Pot of coffee ($0.70)

Conveniently, you can see all the dishes on the Oak Room menu transcribed via our "What's on the Menu?" transcription tool.

Guide to symbols in a 1910 Tour Book from the Automobile Club of America

Meanwhile, a close second was a 1910 "Tour Book" from the Automobile Club of America (better browsed via our book viewer). That traffic came via a post on Slate's "Vault" blog (featuring "historical treasures, oddities, and delights") titled "The Complex Series of Symbols Early Motorists Used for Wayfinding," which showcased the fascinating set of symbols used in early motorist route descriptions.

Another popular collection that got a lot of notice last month is our nearly complete set of "The Green Books" from 1936-1967, which was just recently put online.

Digital Curatorial Assistant K Menick described the collection in a blog post, noting how these historical documents highlight the contours of a segregated nation listing "hotels, restaurants, beauty salons, nightclubs, bars, gas stations, etc. where black travelers would be welcome. In an age of sundown towns, segregation, and lynching, the Green Book became an indispensable tool for safe navigation."

That's the story for this month! Check back in a few weeks for more stories from our Digital Collections.

Du papier au Web : créez vos propres cartes interactives

Mauricio Giraldo Arteaga — Mon, 13 Apr 2015 14:26:55 -0400

This is a translation to french of “From paper maps to the Web : A DIY Digital Maps Primer” by Claire Chemel, library curator and digital manager at the Département des Cartes et plans of the Bibliothèque nationale de France.

Leer versión en español

En novembre 2014, j’ai été invité à la 2e Semaine du livre numérique organisée par la Bibliothèque nationale de Colombie. J’y ai présenté les projets développés par le NYPL Labs, et animé un atelier consacré aux outils actuels de cartographie numérique. Voici le contenu de cet atelier, qui vous expliquera comment créer vos propres cartes géo-référencées à l’aide d’outils Web gratuits.

En bref

Nous allons créer ceci. Ce tutoriel nécessite que vous disposiez d’une carte numérisée et de données à lui superposer. Les principales étapes décrites sont:

géo-référencer la carte numérisée pour générer des tuiles,
créer des données GeoJSON à superposer à la carte numérisée,
créer un fond de carte personnalisé comme référence actuelle,
assembler tous les éléments sur une page Web interactive.

Note: Ce tutoriel suppose que vous utilisiez Mozilla Firefox, Apple Safari ou Google Chrome. Nous allons utiliser la console développeur, et je ne dispose pas d’instructions pour tous les navigateurs.

Pour commencer

Voici ce que nous voulons obtenir. Il s’agit d’une carte de Bogota en 1891, conservée à la Bibliothèque nationale de Colombie (lien Flash Player), enrichie de données trouvées dans un annuaire de Bogota de 1888.

1) Geo-référencement

Après numérisation de la carte, la première étape est de lui ajouter des données géographiques, d’établir des correspondances entre les pixels de l’image et les lieux qu’ils représentent : il s’agit du géo-référencement. Ce procédé va déformer l’image numérisée :

Le scan original (réduit, évidemment)

…pour l’aligner sur la projection de Mercator qui est utilisée dans la plupart des projets de cartographie en ligne comme OpenStreetMap ou Google Maps:

Scan géo-référencé aligné sur la projection de Mercator

Le degré de déformation dépendra de la qualité du relevé, de l’état de conservation et de la projection originelle de la carte. Vous vous demandez sans doute comment cela fonctionne. Des logiciels commerciaux ou open-source permettent de géo-référencer des images, mais le principe de ce tutoriel est d’obtenir le même résultat en utilisant uniquement votre navigateur Internet. Voici donc Map Warper! Map Warper est un outil en ligne qui permet de télécharger vos cartes numérisées et fournit une interface simple pour les géo-référencer (ou les “rectifier”). Géo-référencer consiste à faire correspondre une partie de la carte numérisée (à gauche) à une partie du fond de carte Mercator (à droite) :

L’interface de rectification en vis-à-vis de Map Warper

Vous remarquerez les repères sur chaque image. Chaque repère porte un numéro, et figure dans les deux vues. Ils permettent de savoir que le nord se trouve à gauche sur l’image numérisée, et l’est vers le haut. Plus vous ajouterez de repères, plus le géo-référencement sera précis, mais plus l’image finale prendra du temps à générer. Toutefois, la génération d’image est une opération unique, donc ne vous inquiétez pas trop à ce sujet. Il s’agit plus de savoir combien de repères vous voulez ajouter. La carte utilisée dans ce tutoriel en a 101.

Dernier point, il faut être sûr d’obtenir une image géo-référencée de haute qualité après qu’elle ait été modifiée. L’action de déformer l’image de départ s’appelle le réechantillonage 1 (“resampling” en anglais). Dans les options avancées de Map Warper, vous pouvez choisir entre deux méthodes, “Nearest Neighbour” (qualité basse mais rapide) ou “Cubic Spline” (qualité haute mais lente) :

Choisir “Cubic Spline” dans l’option “Resampling Method”

Vous pouvez voir le résultat final ici. Vous pouvez aussi télécharger cette carte en haute résolution dans l’onglet “Export”.
Mais à mon avis l’élément le plus important à obtenir de Map Warper, ce sont les tuiles. Voici le modèle de leur URL :

Vous trouverez l’URL de la tuile dans l’onglet “Export”.

Le modèle de leur adresse est :

http://mapwarper.net/maps/tile/4949/{z}/{x}/{y}.png

Gardez précieusement cet URL, vous en aurez besoin. L’outil de création de tuiles de Map Warper utilise l’image géo-référencée pour produire des tuiles carrées, à différents niveaux de zoom et avec des coordonnées propres, de manière à ne montrer que les parties utiles de votre carte interactive quand vous la consultez2.
En voici un exemple :

Les cartes en lignes sont constituées de millions de tuiles comme celle-ci.

2) Extraction des données

Nous avons la carte, maintenant il nous faut choisir quelles données y afficher. Notre exemple utilise l’annuaire de la capitale de la Colombie, Bogota, en 1888. Cet annuaire regorge d’informations, regroupant les noms de dizaines de milliers de personnes (chacune avec son adresse et sa profession), des dizaines de métiers (décrits en page 4) et des publicités, listant aussi de nombreux commerces (avec leurs adresses et propriétaires).
Cet annuaire nous donne un aperçu intéressant de la vie en Colombie à la fin du XIXe siècle : avocats, photographes et comptables en partagent les pages avec selliers et forgerons.
Je n’ai pas été très original, et j’ai cherché des hommes politiques importants de l’époque, comme le président en exercice (page 222, premier nom de la seconde colonne). Il y a sept personnes sur notre liste : quatre présidents, un vice-président, un ministre et un président par interim3. Cette liste précise les :

nom
fonction (poste le plus élevé occupé au sein du pouvoir exécutif colombien)
mandat
page (d’apparition dans l’annuaire)
profession (telle qu’indiquée dans l’annuaire)
adresse
URL de la photo dans Wikimedia Commons
latitude, longitude (une valeur défaut située au centre-ville de Bogota, que nous allons modifier).

Télécharger la liste en format CSV

Vous pouvez bien sûr créer votre propre liste avec des données plus intéressantes ou plus utiles pour vous.
N’oubliez pas de créer des colonnes latitude et longitude.
Sauvegardez sous forme de liste CSV (“comma-separated values”).

GeoJSON

Pour le moment, nos données sont contenues dans une liste CSV, mais les outils cartographiques en ligne utilisent généralement le format GeoJSON. GeoJSON est fondé sur la norme JSON, un des formats de données les plus courants sur le Web. GeoJSON utilise le concept d’entités (“features”) pour décrire une information géographique. Ces entités peuvent être des points (comme dans notre exemple) ou des éléments géométriques plus complexes comme des lignes simples ou des multilignes, des polygones… Chaque entité est définie par une géométrie –geometry– (point, ligne, polygone…) et des propriétés –properties– liées, qui sont les données que vous voulez lui associer (pour nous, les nom, adresse, photo… d’un individu). Par exemple⁴:

Nous devons convertir notre tableau en un objet GeoJSON, puis remplacer les valeurs par défaut de latitude et longitude par les valeurs exactes, qui nous seront indiquées par la carte elle-même. Il nous faut un outil permettant de générer et de manipuler facilement du GeoJSON : voici donc GeoJSON.io! Il s’agit d’un “outil rapide et simple pour créer, visualiser et partager des cartes”. Il possède une interface bien pratique pour créer le GeoJSON dont nous avons besoin.

Ouvrez donc GeoJSON.io dans une nouvelle fenêtre de votre navigateur. Vous verrez la carte par défaut, sans aucun zoom. Maintenant, bricolons un peu. Faites un clic droit n’importe où sur la carte, et choisissez Examiner l’élément (Inspect Element en anglais):

Clic droit→ Examiner l’élément

Ceci ouvrira un affichage avancé permettant de lire et de modifier le code de la page vue (ici, l’interface cartographique). GeoJSON.io comprend une interface de programmation (API) pour contrôler la carte affichée. Le noyau du site est MapBoxJS, lui-même construit sur Leaflet, une “bibliothèque JavaScript open-source pour des cartes interactives adaptées aux mobiles”. Je mentionne les deux, puisque, la plupart du temps, ce qui fonctionne dans l’un fonctionne dans l’autre (documentez-vous bien avant de choisir!), et j’utiliserai le terme Leaflet au lieu de MapBoxJS.

Dans l’onglet Console vous verrez du texte et, en bas, un curseur là où vous pouvez exécuter du code JavaScript. Vous remarquerez également des commentaires du créateur de GeoJSON.io. Une ligne est prévue pour entrer du JavaScript supplémentaire, tapez-y le contenu du GIF suivant et appuyez sur Entrée :

La carte sera centrée et zoomée sur Bogota, Colombie, la zone couverte par la carte de 1891. Maintenant tapez ceci :

…et faites Entrée. Cela ajoutera la couche de tuiles proprement dite. Cette ligne de code comprend l’URL que vous aviez copiée à l’étape 1. Le résultat donnera quelque chose comme ça :

Un petit “bricolage” dans GeoJSON.io

Vous pouvez maintenant fermer la console de développement (mais pas le navigateur!).

Note: Il faudra entrer ce code à chaque fois que vous accéderez à GeoJSON.io, puisqu’il ne sauvegarde pas les modifications faites dans la console. Vous pouvez par contre conserver les données que vous ajoutez à la carte en vous connectant.

Ajouter des données dans GeoJSON.io

Nous allons maintenant utiliser cette version modifiée de la carte comme base pour géo-localiser correctement notre liste CSV des présidents.
Faites glisser sur la carte le fichier CSV que vous avez téléchargé :

La magie du glisser-déposer dans GeoJSON.io

Vous remarquerez que les données sont automatiquement converties en GeoJSON (à droite) et que la carte zoome sur les repères correspondant à chaque président (à gauche). Un message (sur fond vert en haut à gauche) nous informe que sept entités ont été importées.

Mais où est passée la carte de 1891? Pas de panique, la carte est juste à un niveau de zoom trop avancé, et notre jeu de tuiles ne dispose pas d’images à cette échelle. Dézoomez un peu, et vous verrez réapparaitre la carte de 1891.

Localiser les points de repère

Les points indiqués dans notre CSV sont tous localisés les uns sur les autres, sur la Plaza de Bolívar. Il faut les déplacer sur leur emplacement correct. Si vous cliquez sur le marqueur gris, vous verrez les données attribuées au repère du dessus (General Rafael Reyes). Il habitait alors au 50, Calle 16. Trouvons cette adresse sur la carte.

Il est relativement facile de trouver les adresses puisque les coins de chaque pâté de maisons indiquent les numéros des immeubles correspondants. La numérotation des “carreras” (rues verticales) va du sud vers le nord, et se répartit entre pairs à l’ouest et impairs à l’est ; alors que celle des “calles” (rues horizontales) va de l’est à l’ouest, avec les numéros pairs au nord et impairs au sud :

Plaçons-nous sur une adresse approximative, en nous repérant sur les coins de rues. Pour ce faire, ouvrez le mode édition en cliquant sur l’icône . Les marqueurs sont encadrés en rose, et vous pouvez les déplacer. Placez-les à l’endroit voulu, cliquez sur “Save” pour enregistrer les modifications :

Certaines adresses sont délicates à placer, mais il est plaisant de pouvoir se perdre ainsi dans le Bogota de 1891.
Vous remarquerez que sur cette carte les bâtiments gouvernementaux portent les couleurs du drapeau colombien. Quand on place Rafael Núñez Moledo, le président en exercice à cette date, son adresse correspond à un de ces bâtiments, la Casa de Nariño.

Sauvegarder le fichier GeoJSON

Maintenant il faut générer le GeoJSON final, que nous utiliserons pour créer notre carte interactive. Choisissez simplement Save > GeoJSON dans le menu. Un fichier nommé map.geojson sera généré et enregistré sur votre ordinateur. Pour tricher, vous pouvez aussi télécharger celui que j’ai créé…

3) Créer une carte personnalisée de 2014 (facultatif)

Nous voulons comparer cette carte de 1891 avec le Bogota d’aujourd’hui, afin d’étudier les changements dans le temps. Il nous faut une carte de base, qui est ce que GeoJSON offre quand vous chargez la page : une carte du monde toute simple (et exacte, espérons-le), affichant les rues actuelles. Nous pouvons utiliser les tuiles standard d’OpenStreetMap ou un service comme MapBox pour produire une carte complétement personnalisée (MapBox s’appuie sur les données d’OpenStreetMap). MapBox peut faire beaucoup de choses : il permet de changer les couleurs, de choisir ce qui est affiché (rues, bâtiments, parcs, etc.) et même d’avoir recours à l’imagerie satellite! Je ne vais pas expliquer comment utiliser MapBox, vous pouvez vous référer à leur excellent tutoriel.
Quand vous aurez fini, notez bien l’identifiant de la carte (“Map ID”), qui se présentera sur le modèle username.k53dp4io. La page “Projects” de MapBox permet de voir toutes vos cartes et de copier directement les identifiants dans votre presse-papiers :

NOTE: Si vous ne voulez pas créer votre propre carte personnalisée, je donnerai plus loin un exemple de MapBox ID.

4) Assemblage final

Nous disposons maintenant de tous les éléments nécessaires à l’assemblage de notre carte interactive :

des données géographiques en format GeoJSON,
des tuiles pour la carte de 1891,
des tuiles ou une MapBox ID pour la carte de 2014.

Nous afficherons un prototype dans JSFiddle, un outil permettant de rapidement créer et tester du code HTML, JavaScript et CSS. Familiarisez-vous avec l’interface grâce à ce tutoriel.

JSFiddle affiche quatre volets principaux :

code HTML (en haut à gauche),
code CSS (en haut à droite)
JavaScript (en bas à gauche),
et le résultat final (en bas à droite).

JSFiddle se charge d’assembler les trois codes et d’en afficher le résultat chaque fois que vous cliquez sur “Run” (en haute, dans la barre bleue).

HTML et CSS

Dans cet exemple, les éléments HTM et CSS sont très simples. Nous avons uniquement besoin d’une zone rectangulaire de la page affichant la carte et ses commandes.
Il nous faut un élément HTML là où sera placée la carte. Tapez ou collez ceci dans le volet HTML :

Avec ce code nous créons un élément div identifié comme une carte et qui, vous l’aurez compris, contiendra la carte. Il faut ensuite donner un style à cet élément : une hauteur, une largeur et, si vous voulez, une bordure ou d’autres attributs. Le style est contrôlé par le code CSS. Tapez ou collez ceci dans le volet CSS :

Une hauteur et une largeur de 400 pixels seront attribuées à l’élément dont l’identifiant est map (le préfixe # signifie identifiant en CSS). Bien sûr vous pouvez créer un rectangle plus grand (si votre écran peut l’afficher) et indiquer d’autres attributs entre ces accolades { } (par exemple, background-color: #f00; pour un arrière-plan rouge si vous souhaitez voir l’élément sans carte), mais j’ai voulu garder les choses simples.

Si vous cliquez sur “Run” maintenant, vous ne verrez pas grand-chose (à moins que vous n’ayez ajouté une couleur d’arrière-plan ou une bordure). Nous n’avons plus besoin d’HTML et de CSS pour le moment.

Ajouter MapBoxJS

Pour visualiser la carte et la rendre interactive, nous avons besoin d’éléments supplémentaires et de JavaScript. J’ai déjà parlé de Leaflet et de MapBoxJS: Leaflet est inclus dans MapBoxJS, donc nous nous préoccuperons uniquement de ce dernier. MapBoxJS est composé de deux fichiers: un fichier JS et un fichier CSS. Vous savez déjà ce que fait le CSS. Le fichier JavaScript renferme tout l’interactivité magique de notre carte. Voici les URL des fichiers en question (NB : il ne s’agit pas de la dernière version de MapBoxJS, mais cela fonctionnera quand même) :

Fichier CSS :
https://api.tiles.mapbox.com/mapbox.js/v1.5.0/mapbox.css

Fichier JavaScript :
https://api.tiles.mapbox.com/mapbox.js/v1.5.0/mapbox.js

Dans la colonne de gauche de JSFiddle, ouvrez la section “External Resources”. Copiez ces URL et collez chacune d’elle dans la zone JavaScript/CSS URI, puis cliquez sur le bouton “+”. Vous verrez alors quelque chose comme cela :

Votre page JSFiddle après l’ajout des deux fichiers MapBoxJS

Désormais, JSFiddle chargera ces fichiers quand vous cliquerez sur “Run”.

Bonjour la carte!

Et maintenant, le moment que vous attendez tous! Créons un peu de JavaScript pour voir la carte de 1891. Écrivez ceci dans le volet JavaScript :

…et cliquez sur “Run”. Voici ce que vous devriez voir :

Votre première carte Web!

Grâce à Leaflet, créer des cartes en ligne est aussi facile que ça.

Note: Je n’entre pas ici dans les détails des API Leaflet ou MapBoxJS. Il existe des tutoriels et des exemples pour les deux. A la place, je vais vous donner des fragments de code, en expliquant brièvement ce qu’ils font. Vous allez copier, coller, cliquer sur “Run” et le résultat sera magique5. Plus tard vous découvrirez par vous-même comment faire encore mieux.

Combiner plusieurs jeux de tuiles

Vous avez vu que la carte est vide, sauf pour la partie correspondant à 1891 : c’est normal. L’URL de ce jeu de tuiles ne contient que la carte rectifiée, et rien d’autre. Il nous faut un jeu de tuiles supplémentaire pour comparer avec 2014 (je vais utiliser un exemple de MapBox Map ID, au cas où vous n’auriez pas créé le vôtre à l’étape 3). Nous allons remplacer le code JS par un nouveau, contenant :

les informations de provenance de la carte,
les tuiles pour 2014,
une commande pour passer d’un jeu de tuiles à l’autre.

Ce code doit remplacer votre JS précédent :

Vous remarquerez que ce code est assez similaire à l’autre. Les différences principales sont l’attribution des données et les jeux de tuiles MapBox (par le biais de la Map ID). La commande elle-même consiste en deux lignes : une pour créer une variable baseMaps qui supportera les tuiles (vous pouvez ajouter autant de jeux que vous voulez) et une autre pour créer une commande de bascule et l’ajouter à la carte. La voici en action:

La mention de provenance de l’image change avec le jeu de tuiles.

Nous y sommes presque! Il faut maintenant afficher nos données. Leaflet nous facilite le travail, puisque il est par défaut compatible avec GeoJSON. Cette étape tient en quelques lignes, mais d’abord, supprimez la fonction de zoom map.setView([4.598056, -74.075833],14). Maintenant collez ce code en bas du volet JS :

Copiez le GeoJSON depuis le fichier texte que vous avez téléchargé dans GeoJSON.io et collez-le là où vous lisez 'paste_geojson_here_keep_quotes'. Gardez bien les guillemets! Cette ligne devrait ressembler à ceci :

Nous avons remplacé la fonction zoom par map.fitBounds(geolayer.getBounds()). Cela rend la carte plus “intelligente” : nous ne rentrons pas manuellement longitude, latitude et niveau de zoom, nous laissons Leaflet calculer la zone occupée par l’ensemble des marqueurs avec getBounds() et ajouter cette valeur à la fonction fitBounds(). Voilà, la carte est maintenant zoomée pour montrer tous nos marqueurs. Si vous en ajoutez de nouveaux, les limites de la zone s’ajusteront automatiquement!

Vous pouvez également ajouter des points de repères ou d’autres informations à la bascule de couches. Il faut juste créer une variable comparable à celle que vous avez créée pour les jeux de tuiles et mettre à jour le code de création de commande :

Vous verrez quelque chose comme ceci en cliquant sur “Run” :

Votre carte et ses données personnalisées

Note: Pensez bien à déplacer le code de création de commande L.control.layers en dessous de l’analyse de GeoJSON. La variablegeolayer a besoin d’exister pour être ajoutée à d’autres couches (overlays). Consultez mes résultats JSFiddle pour plus de détails.

Une autre ligne importante est celle contenant la fonction L.geoJson(). Celle-ci analyse toutes les entités décrites par le fichier map.geojson. Leaflet/MapBoxJS affiche par défaut des marqueurs bleus pour les entités points, que vous pouvez personnaliser. L.geoJson() permet aussi d’ajouter de l’interactivité aux marqueurs, car pour le moment il ne se passe rien si vous cliquez dessus.

Animer les marqueurs

Nous voulons pouvoir cliquer sur nos points de repère et afficher une fenêtre pop-up avec les données que nous leur avons associées (dans les propriétés de l’entité). Il y a deux choses à faire :

créer une fonction qui créera et affichera un pop-up pour chaque entité point,
et modifier l’appel L.geoJson() pour utiliser cette fonction.

C’est ce que fait la fonction bindPopup() de Leaflet : afficher une boite de texte pour une couche donnée. Le texte peut être balisé en HTML. Collez ce code sous tout ce que vous avez jusqu’à présent :

Cette fonction showPopup()reçoit une entité (feature), l’élément de GeoJSON qui renferme toutes nos informations (une géométrie et des propriétés), et une couche (layer), dans notre exemple, le marqueur bleu. Ces deux paramètres sont traités automatiquement par la fonction L.geoJson(). showPopup() extrait ensuite les propriétés de chaque entité (nom, adresse, etc.) et construit une chaîne HTML, utilisée pour créer le pop-up.

Modifiez votre ligne L.geoJsoncomme ceci pour connecter showPopup:

…vous ajoutez , {onEachFeature: showPopup} après geodata. Ceci indique à Leaflet d’appliquer la fonction showPopup à chaque entité GeoJSON.

Note: Si votre GeoJSON comporte plusieurs catégories d’entités (points, lignes, polygones…), il faut savoir que la même fonction sera appliquée à toutes. Par exemple, les polygones ont des limites, mais pas les points. Vérifiez si l’entité sur laquelle vous avez cliquée a bien des limites avant d’utiliser la fonctionfitBounds.

Si vous chargez la carte et cliquez sur un repère, vous verrez quelque chose comme cela :

C’est déjà très bien, mais ce serait encore mieux en affichant la photo, et en liant le numéro de page à l’annuaire numérisé. C’est ce que nous allons faire. Remplacez la fonction showPopup par celle-ci :

Nous avons juste ajouté une action sous condition : si key est égal à “Page” un lien est créé vers l’annuaire, et si key est égal à “Photo” une vignette est affichée, avec une hauteur limitée à 150 pixels (au cas où l’image serait trop grande).

Voici à quoi ressemble M. Núñez maintenant :

…tout à fait présidentiel!

Et nous en avons fini!

Pour conclure

Il vous faudra compiler ces trois fragments de code dans une page HTML pour publier votre nouvelle carte quelque part. Pas d’inquiétude, voici du code avec les zones nécessaires pour y coller vos CSS, HTML et JS. Sauvegardez le tout dans un fichier .html et publiez-le :

Voici la carte finie. J’ai fait quelques petites modifications au CSS pour remplir la fenêtre du navigateur.

J’espère que ce tutoriel vous aura été utile. N’hésitez pas à me contacter pour tout commentaire ou question!

A comparer avec ce qui se passe lorsque l’on convertit du son depuis un CD vers un MP3. ↩
Voici une introduction au fonctionnement des tuiles des cartes Web. ↩
Je n’ai pas fait de recherche poussée sur ces noms, donc il est possible (mais peu probable) qu’il s’agisse d’homonymes. ↩
Depuis GeoJSON.org ↩
Si tout pouvait fonctionner comme ça! ↩

The Internet Loves Digital Collections (March 2015)

Josh Hadro — Wed, 01 Apr 2015 10:49:03 -0400

West 52nd Street between 5th Avenue and 6th Avenue. Odd Numbers.

What was the most viewed image on NYPL's Digital Collections platform in March 2015?

It was a door.

Specifically, a door on the north side of 52nd Street between 5th Avenue and 6th Avenue. (Pictured at right; you can see what it looks like today at the bottom of this post.)

Why was that image the most viewed? Here's the story: The image comes from "The Roy Colmer New York City doors photograph collection," which includes 3,122 images related to a set of "photographic prints used in Colmer's conceptual art piece, Doors, NYC (1976)" (from the collection description).

A blog post from early 2014 commemorating Colmer and his work describes the project a bit more fully:

From November 1975 to September 1976, Colmer photographed more than 3,000 doors, inclusive and in sequence, on 120 intersections and streets of Manhattan from Wall Street to Fort Washington. The project, although documentary in nature, was essentially conceptual to Colmer, for whom Doors, NYC was as much an exploration of the serial possibilities of photography as of its ability to capture a place.

Meanwhile, for quite some time David Lowe, a specialist in our photography division, has been working with the division's metadata to create what he calls the Photo Geographies. Colmer's door project was among the first mapping projects of Lowe's geodata work (see map embedded below).

This project in turn attracted the attention of NPR's History Dept. (among others). And it was this NPR post that drove the most traffic to our Digital Collections site, and the photo above in particular.

That's the story for this month! Check back in a few weeks for more stories from our Digital Collections.

The view of 52nd st. today:

Del papel a la web: haz tus propios mapas interactivos.

Mauricio Giraldo Arteaga — Tue, 24 Feb 2015 11:42:42 -0500

This is a translation to spanish of “From Paper Maps to the Web: A DIY Digital Maps Primer” written by Daniela Schütte, manager of the Memoria Chilena project of the Biblioteca Nacional de Chile.

NOTA: hay terminología que debe mantenerse en inglés puesto que así debe escribirse en el código.

Lire en Français

En noviembre del 2014, fui invitado a la 2^a semana del libro digital organizado por la Biblioteca Nacional de Colombia. La idea era presentar los proyectos que estamos desarrollando en el NYP Labs y también, dictar un taller sobre herramientas para geolocalización de mapas. Pienso que puede ser útil compartir los contenidos de ese taller, ya que integra diversas herramientas y procesos que permiten que la cartografía digital sea, hoy en día, accesible para todos.
Se trata de un paso a paso que, utilizando herramientas de georeferenciación gratuitas, te enseñará a hacer tu propio e increíble mapa georeferenciado.

RESUMIENDO…

Esto es lo que haremos. Este tutorial asume que ya tienes un mapa digitalizado y que tienes claro qué información quieres incorporar. Usualmente, los pasos son:

Georeferenciar el mapa escaneado para poder generar la cuadrícula (“tileset” en inglés) del mapa;
Procesar los datos en GeoJSON para sobreponerlos al mapa escaneado;
Crear un mapa personalizado que servirá de referencia como mapa de la actualidad;
Integrar todos los elementos en una página web interactiva.

NOTA: Este tutorial asume que estás usando Mozilla Firefox, Apple Safari o Google Chrome. Tendremos que jugar con la consola de desarrollo y no tengo instrucciones multinavegador.

¡Empecemos!

Esto es lo que queremos hacer. Un mapa de Bogotá de 1891 conservado en la Biblioteca Nacional de Colombia (el link requiere Flash Player) enriquecido con datos extraídos de un directorio de Bogotá de 1888.

1) Geo-localizando.

El primer paso, luego de digitalizar el mapa, es agregar la información geográfica. Para esto, es necesario establecer la equivalencia entre pixeles y ubicación exacta que ellos representan. Esto se conoce con el nombre de geolocalización. Este proceso distorsionará la imagen digitalizada:

Imagen original digitalizada (encogida, por supuesto)

…Al hacerla calzar con la proyección de Mercator que es la que se utiliza en la mayoría de los proyectos que utilizan mapas digitales como OpenStreetMap or Google Maps:

Proyección Mercator de la imagen original geolocalizada

El porcentaje de distorsión dependerá de la calidad de la muestra –el mapa digitalizado–, su estado de conservación y la proyección del mapa original. Es posible que te estés preguntando ¿y cómo ocurrirá esto? Existen softwares propietarios y open-source que permiten geolocalizar imágenes de un modo sencillo. Sin embargo, el objetivo de este tutorial es lograrlo sin que tengas que instalar ningún programa que no sea tu navegador.

Para empezar, anda a: Map Warper. Esta es una herramienta web que te permite subir tu mapa digitalizado para luego, a través de una sencilla interfaz, geolocalizarlo (o “rectificarlo” para hablar como cartógrafos). En el lado izquierdo verás el mapa digitalizado y en el lado derecho, la proyección Mercator.

Así se ve la interfaz dividida para la rectificación en Map Warper

Fíjate en los marcadores de la imagen. Cada pin tiene un número y el mismo marcador se ve en ambos lados de la imagen. Asimismo, cada uno de ellos te servirá para darte cuenta que en el mapa digitalizado, el Norte está hacia el lado izquierdo, mientras que el Este está hacia arriba. Si bien, cuantos más marcadores agregues, más precisa será la referenciación; debes considerar que también será más lento el procesamiento de la imagen final. No obstante, la imagen se genera sólo una vez, así que yo no me preocuparía demasiado por eso. De lo que sí debes preocuparte es de cuántos marcadores estás dispuesto a agregar. El mapa de este tutorial, por ejemplo, tiene 101.

Una consideración final en este proceso es estar seguros que, luego de la distorsión producida por la geolocalización de la imagen, tienes una imagen de alta calidad. El proceso de distorsionar la imagen original se llama resamplear ¹. En las opciones avanzadas de Map Warper puedes definir si quieres que el proceso sea rápido pero de inferior calidad (seleccionando la opción “Nearest Neighbour”) o un poco más lento, pero de mejor calidad (seleccionando “Cubic Spline”):

Selecciona “Cubic Spline” en el menú “Resampling Method option”

Puedes ver el mapa final aquí. También puedes descargarlo en alta resolución en la pestaña “Export”. Sin embargo, pienso que el insumo principal que se obtiene de Map Warper es la cuadrícula o baldosas (“tileset” en inglés), como prefieran llamarlas. Y que en el fondo, es la URL que se ve aquí:

Puedes encontrar la URL del tileset en la pestaña “Export”

Y el template es: http://mapwarper.net/maps/tile/4949/{z}/{x}/{y}.png

Para seguir trabajando necesitarás esta URL así que guárdala en un lugar seguro, un bloc de notas por ejemplo, para ir dejando todos estos datos, que luego ocuparemos. Map Warper tiene un motor que utiliza la imagen geolocalizada para generar un tileset en el mapa que opera a distintos niveles del zoom. Así que sólo se nuestra la parte en la que estás trabajando ². Esta es una baldosa o una parte del tileset de ejemplo:

Un mapa Web está hecho de millones de estas

2) Extracción de datos.

Ya tenemos el mapa. Ahora tenemos que pensar qué información queremos mostrar en él. Nuestro ejemplo utiliza este directorio de la ciudad de Bogotá de 1888, la capital de Colombia. Este documento tiene información valiosa sobre: decenas de miles de personas, cada una de ellas con su nombre completo, ocupación y dirección; docenas de ocupaciones diferentes (descritas en la página 4) y avisos publicitarios (junto con la dirección de locales comerciales y los nombres de sus dueños). Este directorio constituye un interesante retrato de la vida colombiana en la segunda mitad del siglo XIX: abogados, fotógrafos y contadores comparten páginas con talabarteros y herreros. Yo me fui por el camino aburrido y decidí buscar políticos influyentes de la época, como por ejemplo el presidente en ejercicio (página 222, primero en la segunda columna). La lista tiene los nombres de siete personas: cuatro presidentes, un vicepresidente, un ministro, y un presidente en ejercicio ³. En ella se incluye:

nombre;
cargo (El cargo más alto en el poder ejecutivo colombiano);
período de ejercicio;
página (en la que aparece en el directorio);
ocupación (de acuerdo con la tipificación del directorio);
dirección;
La URL de una fotografía de Wikimedia (en caso de existir una);
latitud, longitud (un marcador de posición del centro de Bogotá que modificaremos en esta etapa).

Descarga la lista en CSV.

Puedes crear tu propia lista con otros datos que te parezcan más interesantes o más útiles. Sólo debes asegurarte de incluir una columna para indicar la “latitud” y la “longitud” y guardarla en una lista de valores separados por comas.

GeoJSON.

Hasta acá, tus datos están almacenados en una lista de valores separados por comas, pero las herramientas para geolocalización en la web, generalmente utilizan el estándar de datos GeoJSON. GeoJSON está basado en JSON que es una de las formas más populares de estructurar datos en la web. GeoJSON utiliza el concepto de “atributos” para describir los datos geográficos. Estos atributos pueden ser puntos (como en este caso) o formas geométricas complejas como líneas sencillas, compuestas o polígonos. Cada atributo es descrito por su geometría, en inglés geometry, (el punto, la línea o el polígono en sí mismo) y por sus propiedades, en inglés properties, que en el fondo son todos los datos extra que quieras asociar a ellos (en este caso, el nombre de la persona, su dirección, fotografía, etc.). Por ejemplo⁴:

Ahora tenemos que convertir la hoja de datos en un objeto GeoJSON y actualizar los valores de latitud y longitud, en inglés latitude y longitude, de los marcadores de posición. Para esto, utilizaremos el mismo mapa. Pero necesitaremos una herramienta que nos permita generar el archivo GeoJSON y manipularlo fácilmente.

Lo que nos lleva a GeoJSON.io. Es “una rápida y sencilla herramienta para crear, visualizar y compartir mapas”. GeoJSON.io tiene una elegante interfaz que nos permitirá crear el archivo GeoJSON que necesitamos.

Carga la página de GeoJSON.io en una nueva pestaña del navegador. Lo primero que verás será un mapa por defecto en un zoom out completo. Entonces, tendremos que intervenir un poco. Haz click con el botón derecho en algún punto del mapa y selecciona Inspeccionar elemento (o Inspect element si tienes el navegador en inglés):

Click botón derecho → Inspeccionar elemento

Al hacer esto, tendrás una visualización avanzada para desarrolladores, esto significa que podrás ver y modificar el código de la página que estás viendo (en este caso, la interfaz del mapa). GeoJSON.io incluye una interfaz para programación (API) que te permitirá también, controlar el despliegue del mapa. El núcleo de este sitio es MapBoxJS, que está construido en Leaflet, una “librería JavaScript de código abierto para el desarrollo de mapas interactivos compatibles con dispositivos móviles”. Menciono ambos asuntos porque, en la mayoría de los casos, lo que funciona en uno de ellos, funcionará también en el otro. (Es importante que leas toda la documentación antes de tomar cualquier decisión). De ahora en adelante, me referiré a Leaflet en lugar de MapBoxJS.

En la pestaña “Consola” (o “Console” en inglés) verás algo de texto y, al final, un cursor en el que podrás ejecutar el código JavaScript. También te darás cuenta que hay algunos comentarios del creador de GeoJSON.io. Escribe esto en el área que mencioné y presiona ENTER (fíjate en el GIF animado de más abajo):

Con esto, lograrás que el mapa se centre y haga un “zoom” en Bogotá, Colombia, el área comprendida por el mapa de 1891. Ahora escribe esto:

y presiona ENTER. Al hacerlo, agregarás el tileset que ya habíamos creado. Fíjate que la línea de código que acabamos de escribir incluye la URL que copiamos en el paso 1. El resultado, debiera verse más o menos así:

Una rápida “intervención” a GeoJSON.io

Ahora, puedes cerrar la ventana de consola (No la ventana del navegador).

NOTA: Será necesario que apliques este código cada vez que cargues GeoJSON.io ya que no guarda las modificaciones vía consola. Lo que sí puedes hacer es registrarte y así guardar los datos que agregaste a tu mapa.

Agregando datos a GeoJSON.io.

Ahora usaremos la versión modificada del mapa como base para geolocalizar correctamente la lista CSV de presidentes. Arrastra la lista CSV que descargaste sobre el mapa:

La magia de arrastrar y soltar en GeoJSON.io

Te darás cuenta cómo los datos inmediatamente son convertidos a GeoJSON (panel derecho) y el mapa muestra los marcadores de cada presidente (panel izquierdo). Aparecerá un pequeño mensaje verde (arriba a la izquierda) que indica que siete atributos fueron importados.

Pero… ¡El mapa de 1891 desapareció! No te asustes. Esto sólo significa que el mapa ha hecho un zoom “demasiado cerca de la tierra” y la URL del tileset no tiene imágenes a esa escala. Haz un zoom out y verás que el mapa de 1891 aparece otra vez.

Moviendo lo marcadores.

Como podrás darte cuenta, los marcadores de la lista CSV quedaron geolocalizados, uno encima de otro, en la misma ubicación, la Plaza de Bolívar en Bogotá. Así que tendremos que moverlos a su ubicación correcta. Si haces click sobre el marcador gris, verás los datos adicionales del primero en la lista (el General Rafael Reyes). Su dirección, en ese entonces, era 50, Calle 16 (el número 50 de la Calle 16). Encontrar la dirección en el mapa será relativamente sencillo ya que cada cuadra tiene la numeración de inicio y término escrita en cada esquina. Si te fijas, la numeración de las “Carreras” (vías verticales) aumenta hacia el Norte con los números pares e impares de Este a Oeste, mientras que la numeración de las “Calles” (vías horizontales) aumenta hacia el Oeste con los números pares e impares de Norte a Sur:

Como puede ser que no sepamos exactamente dónde estaba una dirección, pondremos los marcadores en su ubicación estimada. Para hacerlo, es necesario que actives el modo de edición, haciendo click en el ícono editar:

Los marcadores tendrán ahora un borde rosado y los podrás mover donde quieras. Una vez ubicados en su posición correcta, recuerda guardar, haciendo click sobre el botón “Save”:

Hay algunas direcciones más complejas que otras, pero como podrás imaginarte, esta es una tarea bastante agradable, porque literalmente, te pierdes en las calles de Bogotá de 1891. Un aspecto interesante de este mapa es que los edificios de gobierno aparecen coloreados con los tonos de la bandera colombiana. Cuando encuentres la casa de Rafael Núñez Moledo, el entonces presidente en ejercicio, notarás que su dirección coincide con uno de esos edificios del color de la bandera colombiana; esa es la Casa de Nariño.

Guardando el GeoJSON.

Ahora, tenemos que generar la versión final del archivo GeoJSON para crear el mapa interactivo. Simplemente selecciona Save > GeoJSON en el menú de GeoJSON.io. Un archivo llamado map.geojson se descargará en tu computador. Si prefieres, puedes descargar el que hice yo.

3) Haciendo un mapa del 2014 personalizado (opcional).

En verdad, lo que queremos es poder comparar cómo ha cambiado Bogotá desde 1891 a nuestros días. Entonces, lo que necesitamos es un “mapa base”, que básicamente, es lo que aparece en tu pantalla cuando cargas GeoJSON.io: un mapa (esperemos, exacto) “color vainilla” de calles del mundo actual. Puedes usar la cuadrídula estándar de OpenStreetMap o también, puedes usar un servicio como MapBox para generar un mapa completamente a tu gusto (MapBox usa datos OSM). MapBox ofrece bastantes alternativas: te deja cambiar los colores, elegir qué se muestra (calles, edificios, parques, etc.) e incluso te permite usar imágenes satelitales.

No voy a describir cómo hacer tu propio mapa en MapBox. Mejor, puedes leerlo en su excelente tutorial. Una vez que estén listos, tienen que guardar el identificador del mapa que crearon, que es algo así como usuario.k53dp4io. Puedes usar la página de proyectos de MapBox para ver todos sus mapas y copiar fácilmente el ID al bloc de notas:

NOTA: Si no quieren explorar el proceso de personalizar su propio mapa, pueden usar el ID de alguno de los ejemplos de MapBox.

4) El montaje final.

Ya tenemos todas las piezas para ensamblar nuestro mapa interactivo. Para eso tenemos:

los datos del mapa en formato GeoJSON;
el tileset del mapa de 1891;
el tileset del mapa actual o el ID de MapBox del mapa de 2014.

El prototipo del mapa interactivo lo armaremos en JSFiddle, una herramienta que permite crear y testear rápidamente código HTML/JavaScript/CSS. Revisa este breve tutorial para que te familiarices con la interfaz.

JSFiddle tiene cuatro paneles principales:

Código HTML (arriba a la izquierda);
Código CSS (arriba a la derecha);
Código JavaScript (abajo a la izquierda);
Resultado final (abajo a la derecha).

JSFiddle se encarga de ensamblar los componentes de los tres códigos en el resultado, cada vez que presionas “Run” (arriba, barra azul).

HTML & CSS.

En este ejemplo, la parte de HTML y CSS son muy sencillas. Sólo necesitamos un área rectangular en la página, en la que ubicaremos el mapa y todos sus controles.

Primero, el elemento HTML en el que ubicaremos el mapa. Escribe o copia y pega esto en el panel HTML:

Con este código, creas un elemento div cuyo identificador es map y, como te podrás imaginar, contendrá al mapa. Ahora, necesitamos darle “estilo” a este elemento (debe tener alto, ancho, altura y, si queremos, bordes y otros atributos). El estilo se controla con el código CSS. Escribe o copia y pega lo siguiente en el panel CSS:

Al hacer esto, el elemento, cuyo identificador es map, tendrá una altura y un ancho de 400 pixeles (el prefijo # significa “ID” en CSS). Por supuesto que pueden hacer el rectángulo más grande (si tu monitor es lo suficientemente grande) o aplicarle algunos otros atributos entre las llaves { } (por ejemplo: background-color: #f00; para un fondo rojo, si quieres ver el elemento sin el mapa) pero en realidad, preferiría que lo mantuvieras lo más sencillo posible.

Si presionas “Run” en este momento, no verás mucho (a menos que hayas añadido algún color de fondo o borde al elemento). En cualquier caso, ese es todo el HTML y CSS que necesitas por el momento.

Agregar el MapBoxJS.

Para visualizar el mapa y hacerlo interactivo necesitaremos de algunas piezas adicionales y un poco de JavaScript. Mencioné hace un rato Leaflet y MapBoxJS. Leaflet está incluido en MapBoxJS, así que más tarde nos ocuparemos de él.

MapBoxJS se compone de dos archivos separados: uno JS y otro CSS. Ya tenemos una idea de qué es lo que hace el CSS. El archivo JavaScript contiene toda la magia del mapeo interactivo. Estas son las URLs de los archivos en cuestión (Esta no es la más reciente versión de MapBoxJS pero no hay que preocuparse, funcionará):

Archivo CSS:

https://api.tiles.mapbox.com/mapbox.js/v1.5.0/mapbox.css

Archivo JavaScript:

https://api.tiles.mapbox.com/mapbox.js/v1.5.0/mapbox.js

En la columna de la izquierda de JSFiddle busca la sección “External Resources”. Ahora, copia estas URLs y pégalas en la caja JavaScript/CSS URI y luego presiona el botón +. Verás que algo como esto aparece después de hacerlo:

Este es tu “fiddle” una vez has agregado los dos archivos de MapBoxJS

Esto hará que JSFiddle cargue los archivos la próxima vez que hagas clic en el botón “Run”.

¡Hola mapa!

¡Ahora sí viene la parte que hemos estado esperando! Escribamos unas líneas de JavaScript para poder ver el mapa de 1891. Copia y pega esto en el panel de JavaScript:

…y presiona “Run”. Esto es lo que deberías ver:

¡Tu primer mapa web!

Gracias a Leaflet, es así de fácil trabajar con mapas web.

NOTA: No voy a entrar en detalles respecto de los diferentes aspectos de las APIs de Leaflet o MapBoxJS. Cada uno tiene sus propios tutoriales y ejemplos.

Lo que sí voy a hacer es dar algunos snippets y explicar, en términos sencillos, para qué sirven. La idea es que copies, pegues, hagas clic en “Run” y que la magia ocurra⁵. Luego podrás investigar, probar y resolver cómo hacer otras cosas por tu cuenta.

Trabajando con múltiples tilesets.

Habrás notado que el mapa es completamente blanco, a excepción del mapa de 1891. Esto es bueno y ocurre porque el tileset sólo tiene el mapa rectificado y nada más. Necesitamos un tileset adicional, del 2014, para poder comparar (podemos usar el ID de un mapa de MapBox, en caso de que no hayas creado el tuyo en el paso 3 de más arriba). Reemplazaremos el código JS con uno nuevo que incluirá:

Información de la fuente y/o atribución del mapa;
el tileset del 2014;
una herramineta de control que nos permitirá cambiar entre un tileset y otro.

Este código debe reemplazar el anterior código JS:

Si miras con detención este código, te darás cuenta que es bastante similar al que habíamos usado antes. La diferencia principal está en los datos de atribución y en los tilesets de MapBox (que incorporamos con su respectivo ID del mapa). La herramienta de control, son sólo dos líneas: la primera, que crea la variable “mapa base” y que soporta los tilesets (puedes agregar cuantos sets quieras); y la segunda, que crea “el control” y lo incrusta en el mapa. Veamos el control en acción:

Fíjate cómo cambia la atribución (al pie del mapa) al cambiar de un tileset a otro

¡Ya casi terminamos!

Ahora lo único que falta es que podamos desplegar nuestros datos. Leaflet hace este proceso bastante sencillo porque es soporte nativo de GeoJSON. Bastan unas pocas líneas de código… pero primero debes borrar la linea que hace zoom map.setView([4.598056, -74.075833],14). Ahora pega este código en la parte de abajo del panel JS:

Debes copiar el código GeoJSON del archivo de texto que descargaste de GeoJSON.io y pegarlo en donde dice 'pegar_geojson_aca_mantener_comillas'. Asegúrate de conservar las comillas sencillas: ''. Hecho el cambio, esa línea debiera verse como esta (por brevedad reemplacé el contenido de las “features” con un …):

Ahora, reemplacemos la función del zoom con map.fitBounds(geolayer.getBounds()). Esto hace al mapa “más inteligente”: en lugar de especificar de forma manual la longitud, latitud y el nivel del zoom, dejamos que Leaflet calcule el área que ocupa el set de marcadores con getBounds() y entregue ese valor al mapa mediante la función fitBounds(). Voilá, el zoom del mapa muestra ahora todos los marcadores. Si quieres agregar algunos más… ¡el área cambiará automáticamente!

También puedes agregar puntos u otra información adicional sobrepuesta en el control de capas. Sólo tienes que crear una variable similar a la que creamos anteriormente y actualizar el código de creación, por ejemplo:

Cuando presiones “Run” deberás ver algo así:

El mapa con datos

NOTA: Asegúrate de mover el código de creación del control L.control.layers por debajo de donde está el GeoJSON. La variable geolayer necesita existir para poder ser añadida a as otras capas. Puedes guiarte por mi resultado en JSFiddle si necesitas más detalles.

Otra línea importante es la de la función L.geoJson() que es la encargada de regular el comportamiento de todos los atributos descritos por map.geojson. Leaflet/MapBoxJS tiene por defecto íconos azules para los marcadores, pero pueden ser modificados si quisieran. L.geoJson() también permite agregar interacción a los marcadores. Si ahora hacemos click con el botón derecho, no pasa nada.

Marcadores que cobran vida.

Me gustaría que, al hacer clic sobre un marcador, apareciera una ventana o “popup” con la información que asociamos a cada uno de ellos (en las properties de cada punto). Para lograrlo, tenemos que hacer dos cosas:

Crear una función que construya y despliegue el popup para cada punto (marcador).
Modificar el llamado L.geoJson() para que utilice esta función.

Leaflet tiene la función bindPopup(), que hace precisamente eso: dibuja una caja con un texto para un determinado layer (el término en inglés para los elementos que se dibujan en el mapa). Este texto puede incluir HTML. Copia y pega este código, debajo de todo lo que llevas hasta el momento:

Esta función showPopup() recibe un feature, la pieza de GeoJSON que contiene toda la información (geometría y propiedades), y un layer (en nuestro caso, el marcador azul). Estos dos parámetros son procesados automáticamente por la función L.geoJson(). showPopup() extrae las propiedades de cada atributo (nombre, dirección, etc.) y las articula en una cadena HTML que luego es usada para crear el popup.

Pero para que funcione, tenemos que conectar la función showPopup a algo. Para hacerlo, debemos modificar la actual línea de L.geoJson de la siguiente forma:

…solamente estamos agregando , {onEachFeature: showPopup} después de geodata. Lo que hicimos fue decirle a Leaflet que aplique la función showPopup a cada atributo en GeoJSON.

NOTA: Si nuestro GeoJSON tiene multiples tipos de atributos (por ejemplo: puntos, líneas o polígonos) es importante tener en mente que la misma función será aplicada a cada uno de ellos. Por ejemplo, los polígonos tienen un área, pero los puntos no; entonces será necesario que revisemos si el atributo sobre el que clickeamos tiene área antes de intentar usar la función fitBounds en el mapa.

Al cargar el mapa y hacer clic sobre un marcador, debiera verse así:

Se ve bastante bien y todo, pero… ¿no sería mejor ver la foto y quizás linkear a la página que corresponde del directorio?

¡Hagamos eso!

Remplacemos la función showPopup con esta:

Lo que acabamos de hacer fue agregar una cosa más para el popup: Si key es igual a “Page” linkeamos al directorio y si key es igual a “Photo” construimos un contenedor para la imagen que la despliega a 150 pixels de alto (sólo en caso de que sea demasiado grande).

Así es como se veía el Sr. Núñez:

…digno de un presidente!

Y, ahora sí, casi terminamos.

Empaquetando.

Lo último sería compilar estos tres snippets de código en una página HTML para que puedas publicar tu nuevo mapa en alguna parte. No te preocupes, para esto hay otra plantilla que tiene marcados los sitios en los que debes pegar el CSS, HTML y JS. Guarda todo el código como un archivo .html y publícalo donde quieras:

Puedes ver el mapa terminado aquí. Sólo hice algunas modificaciones mínimas al CSS para que se viera mejor en la ventana del navegador.

Ojalá este tutorial sea útil. Cuéntame si tienes algún comentario o sugerencia.

Similar a lo que ocurre con la música cuando se convierte de calidad CD a MP3. ↩
Esta explicación puede ser útil para una introducción más profunda sobre cómo funcionan los tilesets en los mapas web. ↩
No hice una investigación acuciosa de los nombres, así que puede ser (aunque es improbable) que existan homónimos. ↩
De GeoJSON.org ↩
¡Todo debiera funcionar así! ↩

NYPL Digital Collections Platform: An Introduction

Josh Hadro — Wed, 21 Jan 2015 16:01:13 -0500

Nearly ten years ago, in March 2005, The New York Public Library (NYPL) debuted an amazing site of digitized images called Digital Gallery.

Just a few weeks ago we released a new iteration—the spiritual successor to Digital Gallery, which we call Digital Collections.

Digital Collections contains more than 800,000 digitized items, and that number grows every day. While that’s a small fraction of the New York Public Library’s overall holdings, the aim of Digital Collections is to provide context for the materials we have digitized and to inspire people to use and reuse the media and data on offer there to advance knowledge and create new works.

While the aim of Digital Gallery was similar, we’ve now moved beyond just static images, and the platform includes video, audio, and texts, as well as more interplay with our collections experiments (more on that below).

I couldn’t be more proud of this joint effort of a variety of teams across the library, including librarians, curators, the NYPL Technology Group, NYPL Labs, and many others. I’m pleased to present this post as a kind of primer to the features large and small that we’ve included in the platform.

As ever, this is a work in progress, and we’re eager for any feedback you may have—leave a comment below, or let us know at DigitalCollections@nypl.org.

Browsing Digital Collections

Let’s start with one of our main goals with the revamp of the site, which was to make collections browsing and discovery a core element of the Digital Collections experience.

Materials on our Digital Collections site are currently organized in a simple hierarchy. Every item belongs to at least two defining hierarchical levels: it must belong to a parent Division, and within that Division it must belong to a top-level Collection. Many collections have levels of hierarchy beyond that, but those are the only two that are required.

As a result, you can browse by Division, which essentially corresponds to the organization unit within the Library that is responsible for the physical materials represented on this site. On each individual Division page (e.g., the Irma and Paul Milstein Division of United States History, Local History and Genealogy), you’ll find a brief description of the division with a link to Contact Info for that location, as well as a lane of recently digitized items and the Collections of digitized material within the Division.

You can also browse by Collection, which allows you to start exploring at the highest level of how we’ve arranged the materials, across all Divisions. If you’re interested in something particular, you can filter by collection title using the filter box at the top of that page.

We’ve also built some navigational tools that allow you to traverse the sub-containers within many collections. The Vinkhuijzen collection of military uniforms is a great example: this collection is divided into containers by country. Within those countries are examples of military costumes from different periods in that country’s history. Take a look at Denmark: within the Denmark container, there are nine sub-containers that will filter the results set to periods between 1760 and 1885.

Anytime you see a “+” next to an element of hierarchy in this browse view, that means there’s at least another level of hierarchy to explore—just click the container with the “+” in the title, and more options will expand beneath it. The image set on the right will automatically filter to show you just the images within the section of the hierarchy you’ve selected. (You’ll also notice this same hierarchy displays on every item page as well; we’ll get to those features in a minute.)

A couple of tabs appear above the Collections options panel on the left. If a collection has any hierarchical elements, you’ll be able to browse them via the “Navigation” tab.

Meanwhile, all collections (and search results, for that matter) have a “Filters” tab, which gives you options to further narrow the results set by keyword, topic, name, date, material type, and more.

The “About” tab features as much information as we can derive from the metadata of that collection. In many cases, there is descriptive text giving some history and background for the collection. In other cases there will only be some basic collections details, such as the owning division, local identifiers, and a few more tidbits.

You can see all three of these possible tabs on the Renaissance and medieval manuscripts collection, for example.

One other way we’ve attempted to inject some serendipitous discovery and highlights into the browsing experience is via the “lanes” on the home page. There you’ll find groupings of collections on certain topics and themes, though these are not necessarily the full extent of what Digital Collections has to offer—for that, try searching the full breadth of the content.

Searching Digital Collections

You can search Digital Collections from the home page, or via the persistent search box in the upper right hand corner of any other page.

For all searches, you’ll see a body of search results on the right and a set of faceting tools on the left. The tools on the left help you filter and narrow your search. Each time you choose a facet, you’ll see the search results on the right side refresh to include just the ones that match the keyword and other criteria you’ve selected. To undo a facet selection, just click the little “x” next to the label in the line above the search results, or in the lefthand pane.

When there are a lot of options under a given facet, you can click “more” which will expose an expanded list of possibilities. Here, you can filter the options available before making a selection that will apply to all the search results on the right.

If you search for a general topic—posters, for example—you’ll see the search results on the right, the set of faceting tools on the left, but you’ll also see some suggestions at the top of the results set under the “Looking for one of these collections?” label. If there’s one or more collections that include a lot of items matching the searched keyword, Digital Collections will suggest those collections as browse paths above the search results. Give it a try—it works for lots of general topics, and some specific ones, too.

One other search that we’re particularly proud of including with this revamp of Digital Collections: a search for Public Domain materials (technically, for a variety of reasons, these are labeled on our site as “no known US copyright restrictions,” and that’s how I’ll refer to them from here on out).

The New York Public Library is actively reviewing and labeling materials in our Digital Collections with statements that indicate how you may reuse the image, and what sort of permission, if any, you need to do so. You can find all items similarly marked as having no US copyright restrictions via the search above (you can also see the label indicating as much on the individual item pages, which we’ll get to momentarily).

Meanwhile, below the filters on all search results pages, you’ll also see a button you can select labeled “only Public Domain.” For any search, selecting this option will filter to only the subset of results you searched for that have so far been marked as having no known US copyright restrictions. We’re reviewing more material every day to allow you to use and reuse as many of the materials from our collections as possible, so we hope you’ll use that filter to promote the progress of science and useful arts and lots of other things, too!

Item information, tools, and citations

Every item page on the site has a couple of sections, all designed to bring you more contextual information about the material and to connect you to other items relevant to your interests. (If you don’t already have an item page handy, check out one of my favorite example items, in case you want to follow along at home.)

First, of course, is an image of the item itself (or a streaming media player for audio and video files). If an item has more than one image associated with it, you’ll see thumbnails for those just below and to the right of the image. (You can also navigate these using the arrow keys.) You’ll also find a few tools below and to the left which allow you to perform actions like zoom, rotate, print, and more.

Below this are some basic metadata elements—think of these as the basic facts about the item, including creator, date, library location, and more. When an image has no known US copyright, you’ll see a label to that effect here as well. There is a tab with “sharing” options which lets you send a link to the item to Twitter, Pinterest, and Facebook. Finally, there’s a tab that gives you options to download versions of the file, as well as links to order decorative prints or high-resolution files.

At the very bottom of the page, there’s a more extensive metadata section labeled “item data.” Here we list everything we know about an item including topical connections, material type listings, various identifiers, collections hierarchy, notes, and more. Wherever possible, we link these values to filtered searches so that with one click you can see other similar items with those same attributes.

Next to the section heading “Item Data,” there’s also a “Cite this Item” tab. Here we provide our best attempt at an automatically generated cut-and-paste citation in MLA, Chicago/Turabian, and APA formats. However, as with all citations, you should should double check the generated elements of the citation to make sure all the information is correct. We’ve also included a pre-formatted Wikipedia citation code block which should make it easy for you to make reference to an item from Digital Collections on any Wikipedia page.

Finally, at the very bottom of each item page, we’ve included an automatically generated timeline which examines the item metadata and pulls out all of the relevant dates we can parse to try and give you a snapshot view of the physical and digital life cycle of the item you’ve found.

Collections context and collections experiments

If you’ve been following along up to this point, you’ll notice that, in describing the various elements and their functionality, I skipped over a big chunk of the item page. Let’s return to it now, and wrap this up with some pointers to future connections we hope to make.

The middle section of every item page has two columns: one on the left labeled “Library Division & Collection with this item,” and one on the right labeled “View this item elsewhere.”

The left-hand section here displays what we know about an item in terms of the division that owns it and the collection that contains it, including all the layers of containers between the item and the library division. This reflects the same hierarchy elements mentioned earlier in this post. If you hover over any level of this hierarchy, you’ll see some example thumbnails of other items belonging to those containers and, of course, you can click through to see all the matching items via the collections navigation view.

Some items have deep hierarchies and some shallow, but in every case we hope this gives you some insight into how the item relates to other materials held by the same division and highlights other parts of our collections that you might not otherwise have encountered.

The “View this item elsewhere” section is also meant as an exercise in making contextual links to other NYPL resources and beyond. The most common references you’ll see in this section:

The NYPL Catalog (essentially, our inventory tracking and bibliographic description system across all Library divisions)
The NYPL Archives Portal (describing 10,000 archival and manuscript collections comprising over 50,000 linear feet of material in nearly every format imaginable)
The Digital Public Library of America (a national effort that makes NYPL collections searchable along with those of hundreds of other institutions)

Meanwhile, we’re also hard at work extracting data from historical sources and materials, and we’ve incorporated contextual links to those data extraction experiments wherever possible. Some of those include:

As we undertake more research and development on the possibilities of working with our collections data and helping make as many connections among materials as possible, we hope to expand the list of “See also” references included on the site.

What are the connections you’d like to see? How can we make the digitized portions of NYPL’s collections more useful and relevant to you?

Let us know in the comments, or email us at DigitalCollections@nypl.org.

From Paper Maps to the Web: A DIY Digital Maps Primer

Mauricio Giraldo Arteaga — Mon, 05 Jan 2015 16:38:06 -0500

Leer versión en español

Lire en Français

I was invited to the National Library of Colombia’s 2^nd Digital Book Week as a speaker and to give a workshop on digital mapping tools. I thought it would be useful to share that workshop since it encompasses a lot of different processes and tools that make digital cartography today very accessible. It is a primer on working with various free web mapping tools so you can make your own awesome maps.

TL;DR

You will make this. This tutorial assumes you have a digitized map and some data you want to overlay on it. The general steps covered are:

geo-referencing the scanned map so that web tiles can be generated
generating GeoJSON data to be overlaid
creating a custom base map (to serve as reference/present day)
integrating all assets in an interactive web page

Note: This tutorial assumes you are using Mozilla Firefox, Apple Safari or Google Chrome. You will be playing with the developer console and I don’t have multi-browser instructions.

Let’s get started!

This is what we want to make. It is an 1891 map of Bogotá available in the National Library of Colombia (link requires Flash Player) annotated with some data found in an 1888 Bogotá City Directory.

1) Geo-referencing

The first step after scanning a map is to add geographical data to it; to establish an equivalence between its pixels and the geographic location they represent. This is called geo-referencing. This process will distort the scanned image:

Original scan (shrunk, of course)

…to match the Mercator projection which is used in most web mapping projects such as OpenStreetMap or Google Maps:

Geo-referenced scan in Mercator projection

The amount of distortion will depend on the quality of the survey, preservation state and original projection of the map. You may be asking: how did this magically happen? There’s commercial and open-source software that allows you to geo-reference images but the point of this tutorial is doing all of this without installing any software other than your web browser. Enter: The Map Warper! Map Warper is a web tool that lets you upload your scanned maps and provides a simple interface for you to geo-reference them (or “rectify” in geo parlance). Referencing boils down to you telling what part of the scanned map (left) corresponds to what part of the Mercator projection (right):

The split-view rectification interface in Map Warper

Notice the pins in the image. Each pin has a number and the same pin is present in both views. From them you can tell that North in the scan is pointing leftwards while East is pointing upwards. The more pins you add, the more precise the referencing will be but the slower the final image generation. However, image generation happens only once so I wouldn’t worry too much about that. It’s more an issue of how many pins you are willing to add. The map in this tutorial has 101 pins.

A final consideration in this process is to make sure you get a high-quality geo-referenced image after distortion. The process of distorting the original image is called resampling ¹. In the Map Warper’s Advanced options you can set the method from the low-quality but fast Nearest Neighbour to high-quality but slow Cubic Spline:

Select “Cubic Spline” in the Resampling Method option

You can view the final map here. You can also download high-resolution assets in the Export tab. However, I think the main perk you get from the Map Warper are the tiles. It’s that URL template you see here:

You can find the tile URL in the “Export” tab

The template is:

http://mapwarper.net/maps/tile/4949/{z}/{x}/{y}.png

You will need this URL! Keep it somewhere safe. Map Warper has a tile-generating engine that uses the geo-referenced image to produce square map tiles at different zoom levels and coordinates so that only the necessary parts of the interactive map get displayed as you use it². This is an example tile:

Web maps are made up of millions of these

2) Data extraction

We have the map. Now we want to figure out what data to show on it. Our example uses this 1888 City Directory of Bogotá, Colombia’s capital city. This directory is information-rich, containing tens of thousands of person names (each with address and occupation), dozens of different occupations (described in page 4) and advertisements (along with many store addresses and owner names).

The directory provides an interesting view of life in late XIX century Colombia: lawyers, photographers and accountants share pages with saddlers and blacksmiths. I went the boring route and looked for some prominent politicians of the time, such as then-sitting president (page 222, first in the second column). The current list contains seven people: four presidents, a vice-president, a minister and an acting president³. The list includes:

name
office (highest office held in the Colombian executive branch)
term
page (where it appears in the directory)
occupation (as displayed in the directory)
address
Wikimedia Commons photo URL
latitude, longitude (a placeholder set to downtown Bogotá that we will change in this step)

Download the CSV list

You can create your own list from other data you find more interesting or useful. Make sure to include latitude and longitude columns and save it as a comma-separated list.

GeoJSON

So far our data is contained in a comma-separated list, but web mapping tools generally use the GeoJSON standard. GeoJSON is based on JSON which is one of the most popular ways of structuring data in the web. GeoJSON uses the concept of “features” to describe geographic data. Those features can be points (as is our current case) or more complex geometries such as lines, multilines and polygons. Each feature is described by its geometry (the point, line, polygon itself) accompanied by its properties which is whatever extra data you want to associate with it (in our case, a person’s name, address, photo, etc.). For example⁴:

We need to convert our spreadsheet into a GeoJSON object and then update the placeholder latitude and longitude values to the proper values. We will use the map itself to help us figure out those. We need a tool that lets us generate GeoJSON that we can easily manipulate.

Enter GeoJSON.io! This is “a quick, simple tool for creating, viewing, and sharing maps”. GeoJSON.io has this nifty interface we can use to create the GeoJSON we need.

Go ahead and open GeoJSON.io in a new browser window. you will see the default map at full zoom out. Now we need to do a little hacking. Right-click somewhere on the map and select Inspect Element:

Right-Click → Inspect Element

This opens an advanced developer view that let’s you view and modify the code of the page you are viewing (in this case, the map interface). GeoJSON.io includes a programming interface (API) that lets you control the map being displayed. The core of this site is MapBoxJS, which is itself built on top of Leaflet, an “Open-Source JavaScript Library for Mobile-Friendly Interactive Maps”. I mention both because, for the most part, whatever works on one of them works on the other (do read the documentation before making any decisions!) and I will be referring to it as Leaflet instead of MapBoxJS.

In the Console tab you’ll see some text and, at the bottom, a cursor where you can execute JavaScript code. You’ll see some comments from the creator of GeoJSON.io and a row where you can type new JavaScript commands. Type this in that area and press ENTER (refer to the animated GIF below):

This will center and zoom the map in Bogotá, Colombia, the area covered by the 1891 map. Now type this:

…and press ENTER. This will add the tile layer itself. Notice that line of code includes the URL you copied in step 1. The end result will look something like this:

A quick “hacking” of GeoJSON.io

You can now close the development window (not the browser window!).

Note: You will need to re-apply this code every time you load GeoJSON.io since it doesn’t save modifications made via console. You can save the data you add to the map by logging in.

Adding data to GeoJSON.io

Now we will use this modified version of the map as a base to properly geo-locate the CSV list of presidents.

Drag the CSV file you downloaded on the map:

Drag and drop magic in GeoJSON.io

You will notice how the data is immediately converted to GeoJSON (right pane) and the map zooms in to show the points that represent each president (left pane). You can see a small green message (top left) showing seven features were imported.

But the 1891 map disappears! No worries. This just means that the map is zoomed in “too close to the ground” and the tile URL template does not have images up to that level. Zoom out a bit and you will see the 1891 map appear again.

Moving the points around

The points in the CSV are all geo-located on top of each other on the same point in Bogotá’s Plaza de Bolívar. We need to move them to their proper location. If you click the gray pin you will see the additional data for the topmost one (General Rafael Reyes). His address at the time was 50, Calle 16 (50 16th Street). We need to find that address in the map.

Finding the address will be relatively easy since each block has its starting and ending address numbers written on the corners. You will notice that “Carrera” (vertical-ish streets) numbers increase northward with odd numbers east and even numbers west while “Calle” (horizontal-ish streets) numbers increase westward with odd numbers south and even numbers north:

We will place the point in the approximate location between corners in a given block. To do so, activate editing mode by clicking the icon. Pins will have a pink outline and you can move them around. Place the pins in the desired location and click “Save” to commit the changes:

There are some tricky addresses but this task can be quite enjoyable since you literally get lost in 1891 Bogotá. An interesting aspect of this map is that government buildings are colored with the Colombian flag. When you place Rafael Núñez Moledo, the sitting president at the time, you will notice that his address matches one of those flag-colored buildings (the Casa de Nariño).

Saving the GeoJSON

Now we must generate the final GeoJSON that we will use to create our interactive map. Simply select Save > GeoJSON in the editor menu. A file called map.geojson will be generated and downloaded to your computer. You can also just download the one I did, cheater!

3) Creating a 2014 custom map (optional)

We want to be able to compare this 1891 map with present day Bogotá so we can see how things have changed over time. We need a “base map” which is basically what GeoJSON.io has when you load it: a (hopefully accurate) “plain vanilla” street map of the present day world. You could use the standard OpenStreetMap tiles or use a service such as MapBox to produce a completely custom map (MapBox uses OSM data). MapBox is quite powerful: it lets you change colors, customize what gets shown (streets, buildings, parks, etc.) and even use satellite imagery!

I’m not going to describe how to create your own map in MapBox. I will leave that to their excellent tutorial. When you’re done, you will need to write down the Map ID which looks something like username.k53dp4io. You can use the MapBox projects page to see all your maps and easily copy the ID to clipboard:

NOTE: If you don’t want to go through the process of customizing your map, you can use an example MapBox ID later.

4) Final assembly

We now have all the assets required to assemble our interactive map:

map data in GeoJSON format
a tile template for the 1891 map
a tile template or MapBox ID for the 2014 map

We will prototype the interactive map in JSFiddle, a tool that lets you quickly create and test HTML/JavaScript/CSS code. Check out this quick tutorial to familiarize yourself with the interface.

JSFiddle has four main panes:

HTML code (top left)
CSS code (top right)
JavaScript code (bottom left)
The end result (bottom right)

JSFiddle takes care of assembling the three code components into the result every time you click “Run” (in top, blue bar).

HTML & CSS

In this example the HTML and CSS parts are very simple. We only need a rectangular area in the page that will display the map and all its controls.

We need an HTML element where the map will go. Type or copy/paste this in the HTML pane:

With this code we create a div element whose identifier is map and, as you can imagine, it will contain the map. We now need to “style” the element (give it a width and a height and, if you want to, borders and other attributes). Styling is controlled with CSS. Type or copy/paste this in the CSS pane:

This applies a width and a height of 400 pixels to the element whose identifier is map (the # prefix means “id” in CSS). Of course you can make the rectangle bigger (if your monitor is big enough) and apply other attributes between those { } brackets (e.g.: background-color: #f00; for a red background if you want to see the element with no map) but I just wanted to keep it very simple.

If you click “Run” now you won’t see much (unless you added a background color or a border to the element). That’s all the HTML and CSS you will need for now.

Adding MapBoxJS

To present the map and make it interactive we will need some external assets and JavaScript. I mentioned Leaflet and MapBoxJS before. We are going to need them in order to present and control the map. Leaflet is included in MapBoxJS so we just need to worry about the latter. MapBoxJS is composed of two separate files: a JS file and a CSS file. You already have an idea of what the CSS file does. The JavaScript file contains all the interactive mapping magic. These are the URLs to the files in question (note that it is not the latest MapBoxJS version but no worries, it will work):

CSS file:
https://api.tiles.mapbox.com/mapbox.js/v1.5.0/mapbox.css

JavaScript file:
https://api.tiles.mapbox.com/mapbox.js/v1.5.0/mapbox.js

In the left column in JSFiddle find the “External Resources” section. You need to copy those URLs and paste each in the JavaScript/CSS URI box and click the + button. You will see something like this after you do it:

Your “fiddle” once you add the two MapBoxJS files

This will make JSFiddle load those files the next time you click “Run” and from then on.

Hello map!

Now comes the part we’ve been waiting for! Let’s write some JavaScript so we can see the 1891 map. Write this in the JavaScript pane:

…and click “Run”. This is what you should see:

You first web map!

Thanks to Leaflet, it’s that easy to work with web maps.

Note: I’m not going into details here about the different aspects of the Leaflet or MapBoxJS APIs. They each have their own tutorials and examples. I will instead give some code snippets and superficially explain what they do. You will copy, paste and click “Run” and magic will happen⁵. You will later figure out how to do more awesome things on your own.

Managing multiple tile sets

You may notice that the map is all white except for the 1891 map and that is good. The tile set URL only has the rectified map on it and nothing else. We need to have an additional 2014 tile set to compare (I will use an example MapBox Map ID, in case you did not create your own in step 3 above). We will replace the JS code with new one that will contain:

some attribution information for the map (useful for when you want to, you know, attribute data in the map)
the 2014 tile set
a control that will let us swap one tile set for another

This code should replace your previous JS:

If you look throught this code you will notice it is quite similar to what we had before. The main differences are the addition of attributions and MapBox tile sets (via the map ID). The control itself is two lines: one to create a baseMaps variable that will hold the tile sets (you can add as many tile sets as you want) and another to create the control and add it to the map. Behold the control in action:

Notice how the attribution changes when you toggle the tile sets

We’re almost there! We now need to display our data. Leaflet makes this process quite easy since it natively supports GeoJSON. The process is just a few lines, but first remove the map zoom function map.setView([4.598056, -74.075833],14). Now paste this code at the bottom of the JS pane:

You need to copy the GeoJSON output from the text file you downloaded from GeoJSON.io and paste it where you see 'paste_geojson_here_keep_quotes'. Make sure you keep those quotes! That line should end up looking something like:

We replaced the zoom function with map.fitBounds(geolayer.getBounds()). This makes the map “smarter”: instead of us typing longitude, latitude and zoom level by hand we let Leaflet calculate the bounding area for the set of points provided with getBounds() and pass that as a value to the map’s fitBounds() function. Voilá, the map now zooms to show all the points in the set. If you add more points the bounds will change automatically!

You can also add the points and any other data overlay to the layer toggler. You just need to create a variable similar to the one you created for the tile sets and update the control creation code:

You will see something like this when you click “Run”:

Your map with custom data on it

Note: Make sure to move the control creation code L.control.layers to a point below where the GeoJSON is being parsed. The geolayer variable needs to exist for it to be added to the overlays. Refer to my JSFiddle result for details.

Another important line is the one with the L.geoJson() function. This function parses all the features described by the map.geojson. Leaflet/MapBoxJS have default blue pin icons for point features which you can customize if you want. L.geoJson() will also let us add some interaction to the pins. Right now clicking them does nothing.

Making the pins come alive

We want to click on the pins and show a popup box with the data we have associated to it (in the feature’s properties). We need to do two things:

a function that will build and present the popup for a given feature (point)
modify the L.geoJson() call to use this function

Leaflet’s bindPopup() layer function does just that: draws a box with text next to a given layer. This text can be marked up with HTML. Copy/paste this code below all you have so far:

This showPopup() function receives a feature, the piece of GeoJSON that contains all the information (geometry and properties), and a layer, the same GeoJSON as displayed by Leaflet (in our case, the blue pin). These two parameters are passed automatically by the L.geoJson() function. showPopup() then loops through each property in the feature (name, address, etc.) and builds an HTML string. This string is used as the markup for the popup.

We have not connected showPopup to anything. Modify your current L.geoJson line as follows:

…you are just adding , {onEachFeature: showPopup} after geodata. This tells Leaflet to apply the showPopup function for each feature in the GeoJSON.

Note: If your GeoJSON contains multiple types of features (e.g.: points, lines and polygons) you need to keep in mind the same function will be applied to all of them. For example, polygons have bounds but points do not. You will need to check to see if the feature being clicked has bounds before trying to fitBounds the map.

Running the map and clicking on a pin will result in something like this:

This is nice and all but wouldn’t it be better to actually see the photo and maybe link that page number to the directory itself? Let’s do just that! Replace the showPopup function with this one:

We just added a check in the loop: if key equals “Page” we build a link to the directory and if key equals “Photo” we build an image tag and constrain the height to 150 pixels (just in case the image is too big).

This is how Mr. Núñez looks like now:

…worthy of a president!

And we’re done!

Wrapping it all up

You will want to compile these three code snippets in an HTML page to publish your new map somewhere. Worry not, below is a code snippet that has the requisite spots for you to paste CSS, HTML and JS. Save all the code as a .html file and publish it somewhere:

You can see the finished map here. I made minor modifications to the CSS to fill the browser window.

Hope you found this tutorial useful. Drop me a line if you have any comments or questions!

Similar to what happens with music when converted from CD quality to MP3. ↩
Read this explanation for a better introduction on how web map tiles work. ↩
I did not thoroughly research the names in question so it may be (however unlikely) that they are homonyms. ↩
From GeoJSON.org ↩
Everything should work this way! ↩

Generative eBook Covers

Mauricio Giraldo Arteaga — Wed, 03 Sep 2014 12:50:45 -0400

Finding better covers for public domain ebooks

Here at NYPL Labs we’re working on an ebook-borrowing and reading app. On the technical side, Leonard Richardson is doing all the back end magic, consolidating multiple data sources for each book into a single concise format: title, author, book cover and description. John Nowak is writing the code of the app itself (that you will be able to download to your phone). I am doing the design (and writing blog posts). Many of the ebooks we will be offering come from public domain sites such as Project Gutenberg. If you spend a few minutes browsing that site you will notice that many of its ebooks either have a really crappy cover image or none at all:

Book covers weren’t a big deal until the 20th century, but now they’re how people first interact with a book, so not having one really puts a book at a disadvantage. They are problematic, and not only in ebooks. It’s difficult to find high-quality, reusable covers of out-of-print or public domain books. There are some projects such as Recovering the Classics that approach this problem in interesting ways. However, we at NYPL are still left with very limited (and expensive) solutions to this problem.

Given that the app’s visual quality is highly dependant on ebook cover quality (a wall of bad book covers makes the whole app look bad) we had to have a solution for displaying ebooks with no cover or a bad cover. The easy answer in this situation is doing what retail websites do for products with no associated image: display a generic image.

This is not a very elegant solution. When dealing with books, it seems lazy to have a “nothing to see here” image. We will have at least a title and an author to work with. The next obvious choice is to make a generic cover that incorporates the book’s title and author. This is also a common choice in software such as iBooks:

Skeuomorphism aside, it is a decent book cover. However, it feels a bit cheesy and I wanted something more in line with the rest of the design of the app (a design which I am leaving for a future post). We need a design that can display very long titles (up to 80 characters) but that would also look good with short ones (two or three characters); it should allow for one credited author, multiple authors or none at all. I decided on a more plain and generic cover image:

Needless to say this didn’t impress anyone; which is OK because the point was not to impress; we needed a cover that displayed author and title information and was legible to most people and this checked every box… but… at the same time… wouldn’t it be cool if…

10 PRINT “BOOK COVER”

While discussing options for doing a better generative cover I remembered 10 PRINT, a generative-art project and book led by Casey Reas that explores one line of Commodore 64 (C64) code:

10 PRINT CHR$(205.5+RND(1)); : GOTO 10

This code draws one of two possible characters (diagonal up or diagonal down) on the screen at random, over and over again. The C64 screen can show up to 40 characters in a row. The end result is a maze-like graphic like the one seen in this video:

At the 2012 Eyeo festival, Casey Reas talked about this project, which involves nine other authors who are collected in this book. I highly recommend watching Reas’s presentation (link jumps to 30:11 when 10 PRINT is mentioned). The two characters–diagonal up and diagonal down–come from the C64 PETSCII character list which is laid out here on the Commodore keyboard:

Each key on the PETSCII keyboard has a geometric shape associated with it. These shapes can be used to generate primitive graphics in the C64 operating system. For example, here is a rounded rectangle (I added some space to make it easier to see each character):

In terms of the letters on the same keyboard, that rectangle looks like this:


UCCCI
B   B
B   B
JCCCK

10 PRINT was the starting point for my next ebook cover generator. In 10 PRINT a non-alphanumeric character is chosen by a random “coin toss” and displayed as a graphic. In my cover generator, a book’s title is transformed into a graphic. Each letter A-Z and digit 0-9 is replaced with its PETSCII graphic equivalent (e.g. the W gets replaced with an empty circle). I used Processing to quickly create sketches that allowed for some parameter control such as line thickness and grid size. For characters not on the PETSCII “keyboard” (such as accented Latin letters or Chinese characters) I chose a replacement graphic based on the output of passing the character into Processing’s int() function.

Colors and fonts

In order to have a variety of colors across the books, I decided to use the combined length of the book title and the author’s name as a seed number, and use that seed to generate a color. This color and its complementary are used for drawing the shapes. Processing has a few functions that let you easily create colors. I used the HSL color space which facilitates generating complementary colors (each color, or hue in HSL parlance, is located in a point on a circle, its complementary is the diametrically opposite point). The gist code:

int counts = title.length() + author.length();
// map the count to a number between 30 and 260
// (seemed to give the best results)
int colorSeed = int(map(counts, 2, 80, 30, 260));
// use HSL color space
colorMode(HSB, 360, 100, 100);
// main color is darker
shapeColor = color(colorSeed, baseSaturation, baseBrightness-(counts%20));
// complementary color
baseColor = color((colorSeed+180)%360, baseSaturation, baseBrightness);

This results in something like:

To ensure legibility and avoid clashes with the generated colors, I always use black on white for text. I chose Avenir Next as the font. The app as a whole uses that font for its interface, it’s already installed on the OS and it contains glyphs for multiple languages.

There are more (and better) ways to create colors using code. I didn’t really go down the rabbit hole here but if you feel so inclined, take a look at Herman Tulleken’s work with procedural color palettes, Rob Simmon’s extensive work on color, or this cool post on emulating iTunes 11’s album cover color extractor.

Shapes

I created a function that draws graphic alternate characters for the letters A-Z and the digits 0-9. I decided to simplify a few graphics to more basic shapes: the PETSCII club (X) became three dots, and the spade (A) became a triangle.

I wrote a function that draws a shape given a character k, a position x,y and a size s. Here you can see the code for drawing the graphics for the letter Q (a filled circle) and the letter W (an open circle).

void drawShape(char k, int x, int y, int s) {
  ellipseMode(CORNER);
  fill(shapeColor);
  switch (k) {
    case 'q':
    case 'Q':
      ellipse(x, y, s, s);
      break;
    case 'w':
    case 'W':
      ellipse(x, y, s, s);
      s = s-(shapeThickness*2);
      fill(baseColor);
      ellipse(x+shapeThickness, y+shapeThickness, s, s);
      break;
    // plus all the other letters below
  }
}

My cover generator calls drawShape repeatedly for each character in a book’s title. The size of the shape is controlled by the length of the title: the longer the title, the smaller the shape.

Each letter in the title is replaced by a graphic and repeated as many times as it can fit in the space allotted. The resulting grid is a sort of visualization of the title; an alternate alphabet. In the example below, the M in “Macbeth” is replaced by a diagonal downwards stroke (the same character used to great effect in 10 PRINT). The A is replaced by a triangle (rather than the club found on the PETSCII keyboard). The C becomes a horizontal line offset from the top, the B a vertical line offset from the left, and so on. Since the title is short, the grid is large, and the full title is not visible, but you get the idea:

There is a Git repository for this cover generator you can play with.

Some more examples (notice how “Moby Dick”, nine characters including the space, does fit in the 3x3 grid below and how the M in “Max” is repeated):

MOB
Y D
ICK

MA
XM

And so on:

The original design featured the cover on a white (or very light) background. This proved problematic, as the text could be dissociated from the artwork, so we went for a more “enclosed” version (I especially like how the Ruzhen Li cover turned out!):

We initially thought about generating all these images and putting them on a server along with the ebooks themselves, but 1) it is an inefficient use of network resources since we needed several different sizes and resolutions and 2) when converted to PNG the covers lose a lot of their quality. I ended up producing an Objective-C version of this code (Git repo) that will run on the device and generate a cover on-the-fly when no cover is available. The Obj-C version subclasses UIView and can be used as a fancy-ish “no cover found” replacement.

Cover, illustrated

Of course, these covers do not reflect the content of the book. You can’t get an idea of what the book is about by looking at the cover. However, Leonard brought up the fact that many Project Gutenberg books, such as this one, include illustrations embedded as JPG or PNG files. We decided to use those images, when they are available, as a starting point for a generated cover. Our idea is to generate one cover for each illustration in a book and let people decide which cover is best using a simple web interface.

I tried a very basic first pass using Python (which I later abandoned for Processing):

This lacks personality and becomes problematic as titles get longer. I then ran into Chris Marker and Jason Simon’s work, and was inspired:

I liked the desaturated color and emphasis on faces. Faces can be automatically detected in images using computer-vision algorithms, and some of those are included in OpenCV, an open-source library that can be used in Processing. Here’s my first attempt in the style of Marker and Simon, with and without face detection added:

I also tried variations on the design, adding or removing elements, and inverting the colors:

Since Leonard and I couldn’t agree on which variation was best, we decided to create a survey and let the people decide (I am not a fan of this approach, which can easily become a 41 shades of blue situation but I also didn’t have a compelling case for either version). The clear winner was, to my surprise, using inverted colors and no face detection:

The final Processing sketch (Git repo) has many more parameters than the 10 PRINT generator:

Conclusion

As with many subjects, you can go really deep down the rabbit hole when it comes to creating the perfect automated book cover. What if we detect illustrations vs. photographs and produce a different style for each? What about detecting where the main image is so we can crop it better? What if we do some OCR on the images to automatically exclude text-heavy images which will probably not work as covers?

This can become a never-ending project and we have an app to ship. This is good enough for now. Of course, you are welcome to play with and improve on it:

NYPL Labs and Map Division host first library Net Artist Residency

Ben Vershbow — Thu, 07 Aug 2014 07:39:02 -0400

<UPDATE> Submissions are now closed. We will be announcing the selected resident in mid-September. Thanks to all who applied!</UPDATE>

NYPL Labs is pleased to announce the Library's first-ever Net Artist Residency, in partnership with local hardware startup Electric Objects.

Based in the East Village, Electric Objects (EO) has an intriguing idea: what if there was a computer dedicated to the sole purpose of exhibiting digital artwork? To find out, they're making one. The E01 will be digital environment -- a screen -- that is free from the distractions and constant pings and alerts of our daily devices. A slowed-down part of the internet that you can display in your home, office, or what have you, as you would a painting or photograph.

EO is in the early stages of designing and prototyping, and they're just wrapping up a Kickstarter campaign to raise funds to manufacture their first generation of screens. To begin exploring the possibilities of the platform, they've started an artist residency program. NYPL Labs is excited to be among their first partners.

As the first of what we hope to be many such opportunities, we're asking applicants to see what beautiful things can be made from our map and NYC history collections. This includes:

More than 20,000 historical maps available in high resolution
19th century city data (old NYC building footprints, addresses, residential & business info) from our Building Inspector project
an 1854/5 Doggett's New York City directory containing tens of thousands of ghosts of mid-19th century Gotham

We're accepting applications until August 20 (2014). The selected artist will receive:

EO Prototype Kit (approx. $400)
$500 stipend
Full rights to re-sell / re-share / re-display any work produced
Time with Jacob Bijani, who leads the Electric Objects development team
Time with the NYPL Labs team and Geospatial Librarian Matt Knutzen
The opportunity to present your work at NYPL (physically and online)

And we've assembled an awesome judges panel working at the intersection of digital maps and digital arts+crafts:

Alyssa Wright, Mapzen
Aaron Cope, Cooper-Hewitt Labs
Matt Knutzen, NYPL Map Division
Zoë Salditch, Electric Objects
Jacob Bijani, Electric Objects
Mauricio Giraldo, NYPL Labs

Get the full details at the Electric Objects artists page.

The Networked Catalog

Matt Miller, NYPL Labs — Thu, 31 Jul 2014 13:46:18 -0400

The catalog could be considered the backbone of any working library. Performing an essential service, the catalog informs staff and patrons what materials are held and where in the library they can be found. As libraries have changed throughout history, the catalog, and how it has been used, has evolved as well. Some of the earliest catalogs still in existence, originating from medieval libraries, were simply lists written on the inside covers and margins of books in the collection. For these early libraries, containing a couple hundred volumes at most, this system was sufficient to keep track of the collection. However, during the Renaissance, as libraries grew in size, external catalogs started being produced which listed all the works held in the library. This guide was primarily for the library staff to consult when a visitor was interested in a specific volume. They would then go off and retrieve the item. It wasn't until Victorian Britain that the printed catalog was produced with the thought of patron use in mind, ushering in a new concept: The catalog as tool for discovery. Of course catalogs continued to evolve, inhabiting the iconic card catalog in the late 19th century and then into computerized systems in the late 20th century, but with the same mandate, organization and discovery.

At NYPL Labs, we are fascinated with our catalog and the possibilities its data represents. Just as the catalog has changed in the past we wonder what other possible forms it could take today, and in the future. With this driving thought we conducted a preliminary experiment: what if the catalog had a "See All" button? What if you could see everything at once, to get the big picture about what subjects the library has information on and what are the related topics? To even imagine this conceptually is no easy task, unlike our medieval predecessors, The New York Public Library holds many millions of items. But the catalog comes to the rescue, every item has been manually assigned subject headings or terms that describe what that resource is about. For example, here are the subject headings for Doris Kearns Goodwin's A Team of Rivals:

Subjects Headings for A Team of Rivals

If we were to aggregate the subject headings for all of our resources we would be able to see the big picture, what subjects the library has the most information about. This idea of knowledge mapping has been around for hundreds of years, usually emerging from the desire to map the history of ideas or time. They have traditionally been visually conveyed as a temporal map or timeline. For example, Joseph Priestley's A New Chart of History from 1769.

Joseph Priestley's A New Chart of History from 1769. Courtesy of MediaWiki Commons

Returning to our subject headings we can notice another quality, not only do they categorize the item, they also indicate a relationship between the subjects. Because of this relationship we can rightly assume that the topic of Abraham Lincoln is related in some way to United States.

Possible subjects heading relationships for A Team of Rivals

Combining these two concepts we get a complete statement: The more a subject heading is used indicates a strength in NYPL's collection, and a large co-occurrence of subjects points to a strong relationship between those two subjects. When thinking about the catalog in these terms it takes on the form of a large network, a galaxy of interconnected subject headings. This network metaphor fits well when it comes to thinking about the library and all the interconnected knowledge it holds.

To transform this thought experiment into reality we used tools from the field of Network Analysis to process our catalog. Simply put, the form of network analysis we used could be thought of as a physics simulation. You put in a bunch of connected subject headings of various sizes (number of uses), set the gravity and repulsion strengths, and press play. The result is a self-organization of the subject headings based on how large they are and their relationships to other subjects. For example, if Abraham Lincoln was in one part of the network and the United States is in another, and they have a very strong relationship (meaning used together many times) then they are going to want to be together in the network. Using networks in this way is a fantastic way to start seeing patterns and groupings of related subjects.

We ran this simulation for over 430,000 subject headings that were interconnected with 11 million relationships. Below is a time-lapse video of the simulation playing out over the course of five days:

NYPL Subject Heading Network from NYPL Labs on Vimeo.

This is a extremely zoomed out view of the process, each tiny speck represents a subject headings trying to find a place in the larger network based on its relationship. After the dust settled we took that information and drew an image based on where every subject heading ended up. We also ran some algorithms to group the subject headings together into communities based on their shared connection and assigned that group a color. The result was an impressive view of our catalog:

Click to view interactive version.

We also created an interactive view of the network, allowing you zoom in and explore. Click on subject to see some possible resources and a search tool that gets you in the vicinity of a requested subject. This visualization could be considered a map of NYPL's knowledge, showing what the major subjects we have holdings for:

Mapped sections of the network.

As this is the first pass at thinking of the catalog as a network you notice a few kinks, with some subjects overly bunched together or overlapping. But it is an exciting first step. We will continue to refine this network approach to the catalog but we are always thinking ahead. What other new forms could the catalog take to help explore the vast materials living at NYPL? Stay tuned.

Notes

If you're a tech person and want some more details on how we made our network check out the slides of this presentation that outlines more of the technical side.
For an evocative introduction to library history and the source of the catalog history facts check out Matthew Battles, Libraries: An unquiet history.
For more old charts and timelines check out Daniel Rosenberg's Cartographies of Time (ebook version)
If you’re interested in the concept of network analysis take a look at Wikipedia for: Network theory and Social network analysis to get a basic intro.