DigitalNZ - Case studies

View unique NZ aerial photos from V.C. Browne & Son

2009-11-13T10:21:14Z

To make hard-to-find New Zealand content easier to ‘find, share, and use’, you have to know that the content has been digitised. Herein lies the paradox of Digital New Zealand.

That’s why the following note from Roger Barclay made our day:

"I have read with interest your comments about making NZ content easier to find. I have recently launched a website that provides access to V.C. Browne & Son's aerial photograph collection and it sounds like this site fits well with what you are trying to achieve. How do we go about getting this content reflected in your search facilities?"

The V.C. Browne & Son Collection

The V. C. Browne and Son NZ Aerial Photograph Collection contains approximately 26,000 images – and you can now discover the digitised ones through Digital New Zealand.

Victor Carlisle (V.C.) Browne was one of New Zealand’s pioneering aerial photographers. His son William (Bill) took over his Aerial Photograph business in the 1970s.

The collection spans a period of over 50 years, from the 1930s to the late 1970s. Most of the images are black and white aerial photographs, but there are some portraits and architectural works as well.

The photographs mainly focus on the South Island, particularly Canterbury where the Brownes were based. But most New Zealand cities and major towns are also imaged.

One of the great things about the collection is that it’s in sequence (Roll No/Photo No) – so browsing the collection is like looking out the window of your own plane onto the intriguing landscape of an older New Zealand.

Our favourite image

Roger notes that he is still doing work to improve the metadata about the images and identify the contents – it’s a big job.

Nevertheless one of our favourites is this photograph, which by our counts is right on top of where DigitalNZ is located, inside the National Library of New Zealand.

[Probably Wellington somewhere (PB0615/5): Copyright V.C. Browne & Son, used with permission]

Here's what the same area looks like today (click & hold mouse on map to move):

View Larger Map

The big square building in the centre on Aitken Street (not there in the earlier view) is the National Library, where we are.

Adding value to the collection

Roger Barclay has been steadily scanning and creating metadata to increase access to the Browne family photograph collection since the early 2000s. You can read the background to the project on his website.

To bring this content into the Digital New Zealand search experience, we asked him to provide us with a single (static) URL for each content item – previously you couldn’t link directly to a large image and metadata; which also made it hard to share links to the images with other people.

Now that the site has this new structure, Digital New Zealand can more easily collect information about these unique pictorial representations of the country.

In addition, Google will also be able to find its way to the images; bringing the content to the eyes of even more interested people.

More please

If Roger hadn’t contacted us, it might have been a while before we found out about the V.C. Browne & Son collection. We’re glad he did (and thanks to the DigitalNZ content provider who referred him on).

Do you have a digital collection of New Zealand-related content we don’t know about?

We’re here to help people find it through the DigitalNZ discovery system, including the developer APIs that enable links to your content to appear in other relevant contexts such as Te Papa’s collections online.

Drop us a line if you’d like to bring your content into view.

You can find out more about being part of DigitalNZ here

Or get advice about digitising your family collections on Make it Digital

DigitalNZ APIs in action

2009-07-27T10:06:20Z

One of the best ways to explain the benefits of having an open API (Application Programming Interface) to data in the DigitalNZ discovery system is to show it in use.

What’s an API? In short, it’s a way for software applications to ‘talk to’ each other, and a way for developers to ‘talk to’ applications. We use an API to share data with other applications.

The Indicommons website summarises it like this:

"Open APIs allow services and collections to become interconnected, the experience of outside developers to be engaged, and new tools and spaces to be fashioned to benefit the community at large."

Over the past couple of weeks we’ve seen two new examples that really demonstrate the joy of a ‘joined up’ web of data.

Both Te Papa and Auckland City Libraries have recently launched systems that draw links to content from other organisations into their local search experiences. Take a look below to see the DigitalNZ API in action.

Te Papa Collections Online

http://collections.tepapa.govt.nz/

Try a search for something like kiwi and check out the right hand panel. If you can’t find what you’re looking for in the Te Papa collections, maybe it’s available from another DigitalNZ content provider.

If you’ve got questions about how they did this, drop them a line on the Te Papa blog.

Chinese Digital Community

http://chinesecommunity.org.nz/

This is a really neat application of the new External Search Sources Rails Engine developed by Katipo Communications for Kete and other Ruby on Rails applications. It uses an RSS feed to draw results in from other sites, including the DigitalNZ search system via our API.

Seeing tools like these coming to light is fantastic, and we look forward to seeing more examples of developers working with the DigitalNZ API to make NZ content easier to find share and use.

Let us know if you’ve got something in the pipeline too.

Interview with Paul Hagon

2009-06-24T21:48:05Z

One of the questions we get asked most when working with New Zealand content creators is ‘what’s an API’?. Quickly followed by… ‘so, why would you do this?’

When a developer we’ve never met picks up on the DigitalNZ API to create a map-based interface for discovering New Zealand content, an answer starts to form.

In short, an API (Application Programming Interface) is a way for applications to talk to each other; and a way for developers to talk to applications.

As the Indicommons website so aptly summarises: ”Open APIs allow services and collections to become interconnected, the experience of outside developers to be engaged, and new tools and spaces to be fashioned to benefit the community at large.”

Paul Hagon’s ‘experiment’ with the DigitalNZ API brings these words to life. And DigitalNZ didn’t have to do anything except give developers and applications permission to ‘talk to’ information about and links to New Zealand digital content sources!

So, with a nod to Brooklyn Museum who looked to DigitalNZ when releasing their public API, what follows is a Q & A across the trans-tasman divide. We try to delve deeper into Paul’s creation than 140 characters of twitter permits…

You can also find out about the DigitalNZ location search on Paul's own blog, here.

How did you find out about the DigitalNZ API?

I found out about the DigitalNZ API from Seb Chan at the Powerhouse Museum when he was in New Zealand last year for the National Digital Forum. I can’t remember if it was via twitter or via the write up on their Fresher + New(er) blog. I can remember getting quite excited about it, not just from the API point of view, but from what was actually achieved in getting all the contributors involved. It’s very impressive.

Tell us about what you’ve created, thought process behind it, etc?

I've been sitting on this idea since the start of the year, but have been away on holidays for a fair bit of time and never got around to implementing it.

Typically interfaces into large amounts of data, like DigitalNZ, are based around entering something into a search box. It’s a relatively effective but rather boring interface. You need to know exactly what you want to search for and there is very little sense of discovery by accident or exploration (when was the last time you started randomly entering search terms into a search box?). What I’ve created is a map based interface into the collection that focuses more on this discovery aspect rather than getting perfect results.

The interface works by using a combination of API’s from Google and Flickr to extract the name of a location for a given latitude, longitude and accuracy on a map (based upon how far you have zoomed in to the map). This is why if you position the map over Taupo, on the zoomed out map it returns results based on Waikato, then as you zoom in further on the same spot it searches for Taupo District, then Taupo and finally Taupo Central. As each search takes place, a rough guide to the area being searched is displayed on the map using a Flickr shapefile. These shapefiles are generated from the locations people have tagged photos in Flickr.

By using an interface like this, you can see other areas on the map, that you are likely to explore. It’s easy to get results without having to do a lot of typing or thinking.

Is this it, or are there other things you want to do with it?

With the web there is never any ‘this is it’, there are always things that can be changed. It’s still a bit rough around the edges so needs a bit of cleaning up of the code it’s built on. In the short time since developing this, Yahoo have released all their geographic data (WOE data) under a Creative Commons license and Flickr have also released their shapefile data under a Creative Commons license. I’ll incorporate both of these into the service as I’m kind of using and abusing the Flickr API in its current format by not really incorporating Flickr content, which is a bad thing. By Yahoo & Flickr opening up and sharing their content under these licenses, I can use the data without feeling guilty about doing the wrong thing.

The big thing is going to be improving the accuracy of the search results. The DigitalNZ API allows me to search over data that has been given specific location information, but at the moment, not a lot of records have this data. I’ve had to set up 2 search options - one limits it to give an accurate result, and another that searches over everything and picks up place names in the title or description of records. It’s not as accurate, but provides better results. It’s a bit clunky filling up an interface with these options. I like to keep things as simple as possible.

Through the Google Group that has been set up for DigitalNZ, I’ve found that people are attempting to extract location data from the DigitalNZ records by running them through services like Open Calais and Yahoo placemaker. The community that has formed is working on filling these holes in the data and hopefully this work will be incorporated back into DigitalNZ to produce better results for both the participants and the developers that are extending DigitalNZ.

Tell us about yourself!

(I kind of know who you are because you created this extra cool Then and Now mashup… but other than that all I know is constrained to 73 x 73 pixels and a twitter profile…)

Thanks for that – I’m glad you like the mashup. I live in Canberra and I work as a web developer at the National Library of Australia. I’ve been here for 3 years and love it. I previously worked at the Australian War Memorial, so I’ve spent 10 years working with large collections on the web. I studied industrial design in the days before the web existed and have a fascination about how we interact with things. When I’m away from the computer I’m a keen photographer and a semi-retired ultra endurance triathlete.

Any final thoughts?

It’s a fantastic step forward that’s been taken with DigitalNZ. The real power of doing this type of thing comes once an ingest API is provided and we can easily build applications where corrections or further information can be added to the records. I think of it as information karma, if I use your resources to build interesting things, I also have a responsibility to add back to the service and help fix the deficiencies.

Thanks for the opportunity to participate in the interview.

Memory Maker... a remix showcase

2008-12-16T16:12:47Z

We ran the 'Coming Home' Memory Maker campaign to demonstrate what is possible when content providers ‘free up’ selected public cultural content for people to remix with permission; and used the remix editor to deliver the content to users.

The remix editor (developed by Ideum) lets people mix together and recombine video, audio, text, music, and graphic content to create their own 60 second online video that can be saved and shared (by sending a link, or embedding a link to it on another website, just like a YouTube video). We used it to help us crack open the issues around rights and licensing that prevent New Zealanders from being active participants in the creation of digital heritage experiences.

Like our first search showcase, we filled the Memory Maker with content relating to celebrations for the 90th anniversary of Armistice Day on 11 November 2008. We've been delighted to watch as schools and other web users make their own commemorative videos out of New Zealand digital content - not by stealing it, but because they know they are allowed to and we made it easy for them.

Getting content from contributors

Following a difficult search for New Zealand content that we were clearly allowed to remix (few NZ search tools letting us filter by licence, and many confusing or downright scary rights statements), we put a call out to National Digital Forum members and other organisations.

We asked these content providers to help us find content relating to the return of troops from World War I, including the devastating flu epidemic that came after. More importantly, we asked for content that could be released for people to use, adapt, and mix up with other content legally – either because it was out of copyright or in copyright but licensed for remix under Creative Commons.

“No known copyright restrictions”

A lot of content in cultural institutions is out of copyright, which means you cannot license it using Creative Commons. Also, not everyone realises that if you make a simple digital copy (like a photocopy) of an item, such as a picture or sound recording, you are unlikely to create new copyright because the copy lacks originality.

We encouraged content providers to identify content with “no known copyright restrictions” to participate in the Memory Maker campaign, a similar approach to that taken by Flickr: The Commons. You can read about what “no known copyright” means on the Kete Digital New Zealand rights and permissions page, a repository we use to put content licensed (or free) for remix into. You are welcome to borrow from this page to modify your own rights and use pages.

Creative Commons licensing

Where content providers held, or administered, copyright in digital content we encouraged them to license it under one of the New Zealand Creative Commons licences that allows people to share, remix and reuse legally:

Because users of the Memory Maker are able to crop, overprint, blend and merge the content, we stayed away from content licensed under Attribution-Non-commercial-No Derivative Works (BY-NC-ND). Sometimes we also had to modify the content to fit it into the landscape orientation of the Memory Maker and make it useful at a smaller resolution, so decided to play it safe - advice we provided to our content providers as well.

What has been learnt?

Our greatest learning was probably that we, and our content providers, have lots more to learn about digital rights, permissions, and licences. We're proud to have begun to chip away at this barrier to use of New Zealand content, but we've stil got a long way to go. Nonetheless, on the basis of our small foray, we can say that:

Creative Commons is not always the right answer, especially in the cultural heritage sector. You can’t licence what you don’t hold copyright in.
It can be difficult for content providers to be certain if an item is out of copyright - information is not always forthcoming. We advise ‘if in doubt, leave it out’ for now – but we also have a fast take down policy.
Convincing content providers to pre-approve heritage content for change is difficult when you can’t predict how the content might be used. We were glad some were willing to take a leap of faith with us.
Content providers often need advice on how to use New Zealand Creative Commons licenses appropriately before being willing to adopt them.

What's next

We were amazed at the generosity of the content providers who tested the waters with us and freed up even just a small amount of content for remix. But having to ask them to tell us what we could and couldn't do with their digital content before we used it in the Memory Maker was not ideal.

Just think how great the web would be if everyone took on the task of enabling users to know when they can (and can't) copy and re-use digital content legally. It's not hard to put this information in your metadata or content once you know how. The hard bit is deciding what licence or rights statement to use.

What do you think? Are you willing to step up to the challenge and start addressing the rights issues that make it difficult for New Zealanders to interact with digital New Zealand content? Perhaps you are doing this already and are willing to share your knowledge and expertise - either here or on a Creative Commons forum? Or maybe you need some advice?

We'd like to run more remix campaigns. Maybe one day we won't have to ask - we could just use the DigitalNZ search and browse by rights!

Coming Home... a search showcase

2008-11-26T13:43:13Z

The Coming Home search experience went live on 11 November 2008, to celebrate the 90^th Anniversary of the World War One Armistice. This was a sneak preview of the DigitalNZ tools that allow users to build their own NZ search tool, finding hidden and buried NZ content about their topic of interest. We undertook to connect together content from a whole range of sources, and in the process learn about how to open up more of NZ's rich digital treasures.

How did we do it and what did we learn?

Getting metadata content

The Coming Home search was mainly fuelled with content contributed by members of the National Digital Forum (NDF). A key feature of the DigitalNZ search approach is that we focus on aggregating metadata content, and on collecting descriptions of content objects, so we can drive visitors to websites around NZ. We had a three-pronged approach to gathering that content:

Direct harvesting via OAI-PMH, XML sitemaps, or RSS feeds
Collaboration with aggregating sources that contributors were already part of, including Matapihi, KRIS, NZ Museums, and NZ On Screen
Providing a repository website for content that couldn't immediately get online otherwise. We call this service Kete DigitalNZ, and if you don't have a website we can host your organisation's digital treasures for you

What did we ask from content contributors?

We approached organisations and individuals to participate and asked them to let DigitalNZ do one, some, or all of the following:

Regularly harvest and cache metadata and thumbnails so the content is discoverable in search experiences powered by DigitalNZ (a bit like Google).
Establish and maintain RSS feed services so the content can be discovered through other websites and online services
Allow public API access to the harvested metadata so it can be queried, discovered, and connected with other data sources (including in third party applications).

So what does all this mean? We asked that people share their metadata with us, so that we could share it with everyone. This is not a straight-foward thing to agree to, and even now we are still working through the details. We thank everyone who gave us the benefit of the doubt when all we had were concepts and ideas.

Building a search tool

Very early on in the project we decided building another website or online exhibition around 'Coming Home' would be a wasted effort. Instead we set out to allow anyone to build a search tool that explored the NZ content that they were interested in. We created the search builder and then used it to build our very own 'Coming Home' search. The search builder uses a straight forward boolean search query to gather up all the content we we're interested in, for example here's the query for 'Coming Home':

((War OR WW? OR "great war" OR peace OR flu OR influenza OR day? OR victory OR treat* OR parade OR celebration OR armistic* OR soldier* OR army OR anzac OR wounded OR gallipoli OR commemorat* OR memorial OR monument? OR carillon OR medal* OR military OR (maori AND battalion)) AND (191? OR 192?)) OR ComingHome2008 OR (returned AND servic*) OR "R.S.A" OR RSA OR veterans

At the time of writing this post, the query pulls together 14,156 related items for the 'Coming Home' search. The result is unique New Zealand content on a chosen topic being brought together and accessed through discovery points all over the web, from educational blogs such as LunchBox through to institutional websites such as Archives New Zealand.

What has been learnt?

There is so much more to say, but we are only at the beginning of the journey. Here are a few things we've noted:

Build solutions to fit the environment rather than expect the environment to fit the solution. More methods of harvesting give us more flexibility.
Developing a good search experience is not just about being able to harvest metadata. Good quality metadata aids discoverability of content.
Good quality metadata is not just about structure. Shared approaches to data values in dates, subject terms, and so on also aid discoverability of content.
Letting users enhance metadata might allow us to benefit both content providers and users – a test for another day.
A lot of metadata exists telling people what they can’t do with content and far less telling people what they can do. We need to make both obvious.
Explaining new generation web concepts such as ‘API’ in a vacuum is difficult without examples.

What's next?

You can of course use the search builder to create your own search experience, it's a demonstration of what's possible. What can you add? Can you become a contributor? Can you develop using the APIs? Do you know of a burning issue in this digital space that needs attention? Have something to share or comment on? We'd love to hear from you.