DigitalNZ

In the spotlight: Tauranga City Libraries

Wed, 08 Feb 2012 13:00:00 +1300

This week's spotlight on awesome content contributors shines on Tauranga City Libraries.

Tauranga City Libraries contribute digital content to DigitalNZ from their online digital 'knowledge basket' called Tauranga Memories. The aim of the site is to capture the spirit of Tauranga through the community's contribution of personal stories, photographs, video and audio footage.

A real highlight of Tauranga Memories is the content that documents and reflects on the grounding of the Rena oil tanker on the off the coast of Tauranga in October 2011. The collection includes a photo of oil on Mount Maunganui Beach, the diaries of a rescue worker trying to save injured wildlife, a poem from a 9 year-old student reflecting on the disaster and much, much more.

Little Blue Penguin, Oiled Wildlife Response team trip to Motuotau Island, Rena Oil spill, Tauranga. Photo by Paul Cuming. CC BY-NC 3.0.

Tauranga Memories is a treasure trove, and we're proud to make it available through DigitalNZ. Hooray!

Safer Journeys for Teens: NZTA Remix competition for Secondary School Students

Wed, 01 Feb 2012 11:15:00 +1300

The New Zealand Transport Agency (NZTA) has just launched a competition for secondary schools students (Years 9-13) focusing on a theme of 'Safer Journeys for Teens'. The competition encourages the creative re-use of content and data from the NZTA websites and publications.

The incredibly popular 'Legend' advert is just one example of the content available for remixing in the competition.

In addition to creating a remix or mashup, students will have to provide evidence that they have used their work to engage with students at their school.

There are three categories in this competition. The creative remix category includes remixed images, videos, music and animation. The data mashup category includes an infographic that incorporates NZTA data. The literary remix category includes the remixed works of William Shakespeare.

The top prize is $10,000 in vouchers for the winning school. Five students will also be selected to receive vouchers worth $500 each. There is also a prize of $2000 to go towards teachers and some students presenting their work at the International Conference on Thinking 2013 in Wellington. There are separate prize pools for Auckland schools, Rural schools and all other schools with Year 9-13 students.

Students and teachers can begin working on their entries when the school term begins and can be submitted from September 1st 2012 to October 31st 2012.

Students are not limited to remixing NZTA content and data only. They are also welcome to use other content and data so long as it is appropriately licensed or permission is granted by the copyright owner. There is lots of amazing, remixable content on DigitalNZ. A quick search for "car" images (that can be modified) yields over 3,000 results!!

First motor-car in Auckland. Sir George Grey Special Collections, Auckland Libraries, AWNS-19001005-7-5. No known copyright.

More information about other remixable resources will be available on the competition site in the future.

In the spotlight: The University of Auckland Library

Thu, 26 Jan 2012 09:00:00 +1300

Here at DigitalNZ HQ, we're constantly adding new content from amazing digital collections around New Zealand (and the world!). With over 25 million items already available through the DigitalNZ search site, we thought it'd be useful to highlight various content partners and their awesome contributions!

This week's content contributor close-up: The University of Auckland Library.

Over the past month or so, we have greatly increased the amount of content from the University of Auckland Library. DigitalNZ now includes access to digital items from 11 of their digital collections. The wide range of formats and content make them great additions to DigitalNZ.

The Anthropology Photographic Archive collection contains more than 30,000 images recording staff research in New Zealand and the Pacific Islands in the field of archaeology and social anthropology from 1950 to date.

Potter at work 18. 1965-1970? Anthropology Photographic Archive, Department of Anthropology, The University of Auckland. CC BY-NC-ND 3.0.

The Cuthbert Collection contains photographs of the Fanning Island cable station, buildings and people, and Norfolk island scenes.

The Early NZ Books Collection includes significant books published about New Zealand in the 19th century, including John Savages 'Some account of New Zealand' from 1807 and Charles Heaphy's 'Narrative of a residence in various parts of New Zealand' from 1842.

The Early NZ Statutes Collection covers 100 consecutive years of New Zealand law-making from 1841-1940. It comprises Ordinances 1841-1854, Acts of Parliament 1854-1940, Auckland Provincial Acts 1853-1876 and Reprinted Statutes 1908.

The History of the University of Auckland Collection includes PDF texts of centennial histories of faculties and departments, and photographs of staff and buildings.

People working on computer machines 1. 1974. The University of Auckland Library. CC BY-NC-ND 3.0.

The Journal of the Polynesian Society contains a rich repository of indigenous texts and traditions contributed by Pacific peoples, as well as by missionaries and other sojourners, often published in local languages with English translations. The New Zealand Journal of History is New Zealand's premier journal for academic writing on New Zealand history and includes contributions from New Zealand's most famous historians.

ResearchSpace includes full text theses and other research outputs by students and researchers at The University of Auckland.

The Bookshelf, also known as the Digitised Texts Collection, contains online versions of a variety of books, journals and newsletters with New Zealand relevance as selected by University of Auckland Library staff.

If you're up for the challenge, check out all the items we have available from The University of Auckland Library (currently over 50,000!).

Other notable contributions include a series of oral histories, public lectures, seminars and interviews from the University of Auckland.

A picture's worth a thousand words

Wed, 21 Dec 2011 10:51:00 +1300

It's our last week of work here at DigitalNZ headquarters, and we're busy tying up loose ends and eating way too much chocolate. We thought we'd make our last blogpost of 2011 a fun romp through some of the team's favourite images in DigitalNZ.

Woman modelling Daphne lingerie

Woman modelling Daphne lingerie. K E Niven and Co :Commercial negatives. Ref: 1/2-211738-F. Alexander Turnbull Library, Wellington, New Zealand. http://beta.natlib.govt.nz/records/22872947

Right now, this has to be my favourite image. A lingerie model and a pole-dancing monkey puppet? I don't even know where to begin.

Pokeno the Dog

Regimental Pets; Pokeno, Dog of the 50th Regiment, A New Zealand Hero © Victoria and Albert Museum, London.

Fi has a soft spot for Pokeno: "He kicked off my librarian hunting instinct for the first time in a while. Why would the Victoria and Albert Museum have an an engraving of a wee dog named after a small North Waikato NZ settlement? I had to know. So by searching DigitalNZ I found that Pokeno was the site of the Queen's Redoubt, an important military base during the New Zealand Wars. The Queen's Redoubt was built by the 50th Regiment that Pokeno belonged to. I also found this wee snippet in PapersPast (Otago Daily Times , Issue 4150, 7 June 1875 ) that I think refers to that engraving: "...Another Waikato hero had his portrait engraved in the Graphic of the 3rd April. This was Pokeno; the- pet dog of the 50th Regiment, who attached himself to that corps in 1864, followed his- masters throughout the campaign of that and the succeeding year, and has remained faithful to them throughout their numerous-.wanderings since then." I also discovered that military mascots have quite a long tradition. So I wonder if Pokeno was New Zealand's first one? Maybe someone out there knows?"

Benson and Hedges Fashion Design Awards

Polytech fashion design students at the Benson and Hedges Fashion Design Awards - Photograph taken by Ross Giblin, Ref. EP/1976/3581/22A-F.
Alexander Turnbull Library, Wellington, New Zealand. http://beta.natlib.govt.nz/records/23141361

This is Chris's favourite image. When asked for his reason for liking it so much, he said, "What's not to love? Lauren of Arabia hangs out with Starman and local Wiccan." Fair enough!

Steve Goodfellow and Daniel Charles lean into the gale

Steve Goodfellow and Daniel Charles lean into the gale - Photograph taken by Ross Giblin. Dominion post (Newspaper) :Photographic negatives and prints of the Evening Post and Dominion newspapers. Ref: EP/1992/2132/1-F. Alexander Turnbull Library, Wellington, New Zealand. http://beta.natlib.govt.nz/records/23193639

This probably isn't Dan's "favourite image", but we think it's pretty cool because it's an image of Dan when he was 10 years old (he's on the left). Unbelievably, this photo was taken by Ross Giblin, the same Dominion Post photographer who also took the above photograph of the fashion design students 26 years earlier. Cool!

A search for the 'unidentified'

When searching DigitalNZ, a few of us like to search for images of people that include the term 'unidentified'. As a former cataloguer, I was interested to know how many images contained people who could not be identified. There are thousands. Then my curiosity turned to the content of the images. I found myself wanting to talk directly to the people in the photos. Who are you? Where are you from? What is your story?

Unidentified woman sitting on cow

Unidentified woman sitting on a cow. Horowhenua Historical Society Inc.

Unidentified spaniel

Unidentified spaniel. Whites Aviation Ltd :Photographs. Ref: WA-25136-G. Alexander Turnbull Library, Wellington, New Zealand. http://beta.natlib.govt.nz/records/23251125

We hang out on DigitalNZ everyday and have heaps of favourite images. These are just a small selection. We'd love to know what your favourites are. So leave a comment and share them with us.

Happy Holidays, and we'll see you in 2012!

DigitalNZ's Mighty Big 2011

Mon, 19 Dec 2011 13:50:00 +1300

2011 has been a mighty big year for Team DigitalNZ.

One of the biggest achievements, (though hidden to most) was a complete overhaul of our technical infrastructure to improve performance and set DigitalNZ up for major growth. This was a very big and important job and that fact that you probably haven't noticed it is a very good thing.

Photographs for a brochure, Working model of computer controlled factory, 1980, University of Auckland Library, CC BY ND SA

Being a non-techie I always imagine the new infrastructure looks a lot like this image. But I'm assured it isn't quite this exciting and that there is no blackboard.

Another significant project that kept us busy was was helping National Library of New Zealand develop and launch the new National Library Beta - where you can search the full collections of the National Library and the Alexander Turnbull Library. Beta is built on the DigitalNZ search infrastructure and we worked with the Beta team every step of the way.

DigitalNZ now points to well over 25 million digital items. When we launched 3 years ago we had just over 30,000 items. How’s that for an increase?! To celebrate we gave DigitalNZ search some extra special attention. So the DigitalNZ website and our search service got a tummy tuck, a nose job and some brand new magic knickers. We are very keen on iterative development, and this is just the first step. We’ve got some ideas about improvements, and we’ve had some extremely useful feedback too (keep it coming!). Over the next year we’ll be coming up with new useful ways to make amazing NZ content easier to find, share and use.

In 2011 two fantastic new digitisation projects launched as a result of winning Make It Digital digitisation awards in 2010. We’re really proud to have supported the digitisation of two extremely rich New Zealand resources.

One award-winner was a project to digitise the The Nelson Photo News, which celebrates the Nelson region’s rich social history over the 1960s and early 70s.

The second winning entry was a project to digitise the The British Parliamentary Papers relating to New Zealand, which contains a huge amount of valuable information on a large range of subjects covering 1835 to 1882 - a crucial period in New Zealand’s history.

Copy of Schedule 1280 [title page], Auckland. 14 August 1850. British Parliamentary papers. Colonies: New Zealand. University of Waikato Library. http://digital.liby.waikato.ac.nz/bppnz. [Accessed Dec 19th, 2011].

2011 also saw the team work hard to bring New Zealand relevant content held overseas into DigitalNZ. We’ve worked with the National Library of Australia, the Powerhouse Museum and Museum Victoria in the last couple of years, and this year we were delighted to use the Victoria and Albert Museum’s very cool API to bring in a small, but truly beautiful, collection with a connection to New Zealand. We have plans for doing more of this in 2012 – so let us know if you are aware of New Zealand content held overseas that you would like to be able to search via DigitalNZ.

It was also a year of hellos and goodbyes. We said farewell to Elliott Young, our Technical Lead. Elliot did a power of work on Beta and the new technical infrastructure. He also built the New Zealand Picture Show which brilliantly demonstrates the power of the DigitalNZ API. Chris McDowall has now joined us and is bringing great insight into the treasure that is your metadata.

We also welcomed Chelsea Hughes as our Community Manager. Chelsea is the brains behind those tricky weekly quiz questions, and she was the classy curator of the excellent Mix and Mash exhibition at the National Digital Forum.

Mix and Mash Exhibition at the 2011 National Digital Forum

Of course, Mix and Mash was a major activity for us this year. The competition encouraged people to mix and mash a whole lot of amazing data and content into fantastic entries. We’re supremely proud of the way the competition turned out.

We’re all taking a short break over the Christmas period, and then we’ll be back to get started on the exciting plans we have for 2012. We'll post more about this in January. Happy and safe holidays everyone!

Friday Fun with DigitalNZ: Quiz #6

Fri, 16 Dec 2011 11:51:00 +1300

This is our last quiz for 2011. Happy Hols everyone!

Slow food, small data

Tue, 13 Dec 2011 13:33:00 +1300

I recently spoke at the New Zealand National Digital Forum (view on Slideshare). This blog post covers some of the things I discussed.

In August, Max Ogden, an American developer and open web advocate, tweeted this: "I would love for 'small data' to be the 'slow food' of the internet".

At the time I thought, "Right on. That’s a really interesting idea." Then I did what most of us do on Twitter when we see something poignant. I re-tweeted it, I even favourited it… and then I forgot all about it. A month later I watched Candy Elsmore's video entry to Mix & Mash, "A Grand Mother". The tweet and the remix seemed related somehow.

As I watched Elsmore's video, Ogden's tweet began making a strange sort of sense. In the context of DigitalNZ, I began thinking that small data might refer to intimate, hand-picked collections of photographs, audio, newspaper articles, and video. There are parallels between the values of the slow food movement and the acts of creating, curating and sharing personal datasets. I am not sure that this is what Max meant, but this is how I have come to understand the relationship between small data and slow food.

Touching History

Slow food is about preserving and promoting traditional food products, often through the formation of seed banks to preserve heirloom varieties. There are ways of preparing food that have been handed down for generations and there is something special about old varieties of fruits and vegetables that makes people want to continue growing them. To take part in these practices is to touch history. Similarly, small data is frequently motivated by the desire to hold onto and share special things from the past. We need better tools for curating, contextualising and sharing these treasures.

Hangi Revealed. Joann Ransom, 2007. Kete Horowhenua. CC-BY-NC-SA.

Place Matters

Slow food also celebrates local cuisine within regions. It embodies a notion that place matters. Although many large datasets have a geographic dimension, their very comprehensiveness dilutes their ability to represent place. A layer of Picasa images draped over a Google map gives clues into the structure of geographic space but it is largely silent on the nature of our subjective experiences. One way to represent our understandings of place is to focus and edit. Instead of showing everything all at once, you convey a sense of place by concentrating on the small set of things that matter and weaving them together into a coherent visual narrative. If big data is a mind-boggling layer of zoomable thumbnails spread across a global interactive map, small data is a cork board pinned with photographs from our trip to Takou Bay.

DIY Processing

Slow food often involves organising small-scale processing. Doing it yourself is empowering. There is just you, your tools and a vision. In an age when we angst over how to make sense of an ever-increasing deluge of data, working with a small dataset can be liberating. Candy Elsmore's remix worked with a data set so small that she could print the individual items on pieces of paper and arrange them on a table by hand. The scale of the data enabled her to devise her own methods and tools that. Working with large data in unconventional ways requires specialist technical knowledge; small data is democratic.

None of this is to imply that we should turn our backs on big data. DigitalNZ currently indexes 27 million items and that number is growing by the day. Instead consider this a gentle reminder that we should enable people to work with the small and intimate alongside the large and comprehensive. Big and small data require different approaches. We should make sure that we collectively providing tools that can support both.

Friday Fun with DigitalNZ: Quiz #5

Fri, 09 Dec 2011 12:25:00 +1300

2nd to last Friday Fun quiz for 2011. Enjoy this week's question!

Friday Fun with DigitalNZ: Quiz #4

Fri, 02 Dec 2011 13:19:00 +1300

Have a great Friday everyone!

Friday Fun with DigitalNZ: Quiz #3

Fri, 25 Nov 2011 13:00:00 +1300

Another Friday, another quiz! It's a bit trickier this week, but we're confident you'll get it right!

A day in the life of Dan, the Harvester Man

Thu, 24 Nov 2011 13:45:00 +1300

This is my first blogpost. Hi, I'm Dan. I'm a content analyst in the DigitalNZ team, but people around here call me "The Harvester Guy".

Most people (including me before I started here) associate the word "harvester" with those big agricultural combine harvester things. If you do a google image search for "harvester" you get a pretty awesome assortment of these machines, with all sorts of different attachments, shapes and sizes for harvesting a variety of different crops. I particularly like the look of the one with the scorpion digger claw that walks on spider legs.

Anyway, back to explaining what I do…

When a new set of NZ relevant content (a new crop) is identified and an agreement is reached with the content provider, it's my job to harvest that content into the DigitalNZ system. This content can come in a wide range of forms, from basic excel spreadsheets to giant metadata rich rdf/oai repositories to simple sitemaps of regular web pages.

Some content sources have so much metadata (information) about each record that the challenge is how to deal with it all, and where to put all the potentially useful bits of information. Other content sources may offer great images or resources but contain very little associated information. Sometimes it's a struggle to find even a title for each record. With over 25 million items (!!!) harvested into DigitalNZ so far we have built up quite a variety of tools and techniques for dealing with this vast variety of content.

So in a way I do get to drive around in those awesome harvesting machines (in my head anyway .. the spider scorpion one) harvesting up great NZ content into DigitalNZ, making it easier to find, share and use.

Friday Fun with DigitalNZ: Quiz #2

Thu, 17 Nov 2011 23:52:00 +1300

It's week two of our DigitalNZ Quiz! Take a second, give it go... and good luck!

These are a few of my favourite (Mix and Mash) things

Mon, 14 Nov 2011 13:00:00 +1300

I’ve recently emerged from 7 months immersed in the second Mix and Mash – the great New Zealand remix and mashup competition. Mix and Mash challenges people to build something completely new by reusing existing NZ digital content and data. This year over 100 entrants, including an entire primary school class, vied for $50,000 in cash and prizes.

DigitalNZ runs Mix and Mash because we want to:

see more New Zealand content and data used in innovative and surprising ways,
get tangible examples of the great things that can happen when people and organisations license their content for people to be creative with, and
show New Zealanders the amazing open data and content that is already out there for them to use.

That's quite a lot we want to achieve. I’m going to need a series of posts to cover everything I want to say about Mix and Mash.

This first post is about some of the Mix and Mash things that made me really happy. In future posts I’ll talk much more about the winners, give detail about the categories and cover some of the lessons we’ve learned over the last couple of years.

Data are stories that haven’t been told yet

When Archives New Zealand got in touch to say that they had Excel spreadsheets of all of the signatories of the Women’s Franchise petitions (1892 & 1893) I was pretty excited. On face value it is a big old list of names and addresses. But, of course, each one of those names has a story and, really, data are just stories that haven’t been told yet.

I was really hoping that someone would do something great with this dataset. I was thinking people could do something clever by linking it with metadata using the DigitalNZ API or mapping the addresses in some clever way. What I wasn’t expecting was that this dataset would spark a truly beautiful digital story that would win the Supreme Creative Remix (sponsored by Creative Commons NZ). I’m so very pleased that Candy Elsmore stumbled upon that big old list and discovered Mary Edith’s name. It meant she could turn this dataset into a story that made me, and others, shed tears. You can read more about Candy’s entry A Grand Mother in Chelsea’s Anatomy of a Remix post and if you haven’t watched this digital story then you really, really, should.

Re-imagining beautiful works

One of my favourite Mix and Mash categories was Literature Remix, sponsored by New Zealand Post. I plan to dedicate an entire future post to this category, so this really is just an introduction. With the help of the very clever Pip Adam we gathered gorgeous works of literature from 17 outstanding New Zealand writers. Each author took the plunge and licensed their work for reuse so that we could make them available for mixing and mashing.

The quality of entries was superb and wide ranging in style. It was Allan Xia’s Crossed Cultures that all three of our judges (Pip Adam, Fergus Barrowman & Paul Diamond) fell in love with.

Allan remixed a stunning work by comic artist Dylan Horrocks, a beautiful poem by Renee Liang, with his own illustrations. He created a captivating illustrated work in a very original vertical scrolling style. As judge Paul Diamond commented,”…[the scrolling style] aided the narrative, creating a sense of uncertainty and a compulsion to see how things turned out". Crossed Cultures sends shivers down my spine every time I experience it.

Unexpected delights

Two remix entries in our open category really lit my fire. The New Zealand’s Great Walks application used existing mobile tour software and carefully curated a huge amount of fantastic content from Department of Conservation (DOC), Te Ara, Flickr, and Wikimedia. They even got permission from the DOC Spokesbird Sirocco the Kakapo to use birdcalls from the Department of Conservation's audio collection.

What I love about this entry is that it shows the value of organisations allowing the reuse of textual content as well as images and data. Because DOC and Te Ara have allowed reuse of their website text it could be re-crafted into an excellent, free tool that promotes New Zealand as a destination. Mix and Mash Judge Lawrence Lessig was quite taken with the entry and commented that he is looking forward to using it when he comes back to New Zealand.

Probably my favourite entry of all was the Oakura font designed by Andrew McMillan. Andrew took a photo of the Oakura Hall while on holiday. He was taken with hall's typeface and decided to make a rather gorgeous new font based on it. I would never have guessed that someone would enter a font into Mix and Mash. It is such an original and practical idea. I’ve downloaded and used the font myself.

Oakura Hall, by Andrew McMillan (_Karora)

Helping people learn new skills

Mix and Mash isn’t just about the entries (as much as we love, love, love all the entries!). We want people to learn new things about using digital content and data. This year we developed two great new tools to do just that. I'm super proud of them.

The first of these tools was Freedom to Mix: An educators' guide to reusing digital content which aims to improve awareness about copyright, licensing and using content and data respectfully. We worked closely with Esther Casey and the team at National Library Services to Schools to develop this.

Freedom to Mix provides information, activities and ideas to confidently create a remix from material you know you have the rights to reuse. Even though it was written for teachers, it's suitable for anyone wanting to learn more about remixing digital content. The guide has a Creative Commons Attribution 3.0 New Zealand License, so if you like the cut of its gib then feel free to tailor it for your own needs.

The second tool we developed was the Beginners' guide to data mashups which was sponsored by the fabulous Charities Commission. The guide is designed for absolute beginners, like me, to work through and understand the nuts and bolts of taking open data and turning it into something useful. It even walks you through the development of an application that uses the Charities Commission's Charities Register. If you’re wondering what all the fuss is about mashing up data then here’s your chance to learn more about it. If I can do it anyone can!

So there you have it, those were just a few of my favourite Mix and Mash things. In my next post I’ll talk in more detail about the winners of both the 2010 and 2011 competition.

Friday Fun with DigitalNZ: Quiz #1

Fri, 11 Nov 2011 12:02:00 +1300

Here at DigitalNZ HQ we think you deserve to have a bit of fun on a Friday afternoon. Every Friday over the next few weeks we'll be posting a one-question quiz that can be answered by searching for clues using DigitalNZ's shiny new search service. We hope you have as much fun answering the questions as we did creating them!

Happy 3rd birthday, DigitalNZ!

Thu, 10 Nov 2011 13:35:00 +1300

Today is DigitalNZ's 3rd birthday! The years have flown by, and yet, we find it hard to remember a time when there was no DigitalNZ. On November 11th, 2008, we took our first steps as part of the 90 year commemoration of Armistice day. Anyone remember the Coming Home search service that started it all?

A lot of decisions have been made over the years that have shaped the DigitalNZ you have come to know (and hopefully love). One of the very first decisions we ever made was deciding what the DigitalNZ logo would look like. We thought it would be fun to share with you some of the designs we did not select. It makes you wonder how DigitalNZ might be different had we gone with these alternatives.

Once that decision had been made, the logo was of course tweaked a bit more... after all who can resist messing with designers! Here were some of our alternative colour options (WARNING: logos may cause dizziness or 1980s fluorescent nostalgia):

There is a member of our extended team who still maintains that hot pink goes with everything... hmmm.

Thank you

As we share with you a slice of virtual birthday cake, we just want to say thanks. To the content partners who make everything possible, thank you so much... the services would not exist without you. To all the folk and companies who have worked on DigitalNZ over the years, you are legendary. To our crowd of fans and visitors, kia ora! We hope we'll do you all proud in the years ahead.

Visualising DigitalNZ content over time

Fri, 04 Nov 2011 12:40:00 +1300

DigitalNZ indexes a lot of stuff. As I type this blog post, it's midday on Friday, November 4, 2011. If I visit the DigitalNZ website and search for every item we index, I get 25,661,157 results. That's a mind-boggling number.

In preparation for a talk I'm giving at the National Digital Forum in a few weeks, I am trying to understand what exactly are the 25 million pieces of New Zealand digital content we link to. I've been asking myself questions like "How are these things distributed over geographic space?" and "What aspects of life in New Zealand are represented by this material?" In order to answer these questions I have been using the DigitalNZ developer API to interrogate our metadata repository.

One of the first questions I had was "What is the temporal distribution of publication dates for the various images, articles, videos, manuscripts and so on that DigitalNZ indexes?" It turned out the DigitalNZ API makes this question easy to answer.

The histogram at the top of the post displays the volume of items we index organised by the year the content was originally published. I've restricted the figure to only show items that were created between 1840 and 2011.When reading these charts you should keep in mind that not every item of content that DigitalNZ indexes has a publication date in its metadata. If we don't know when the item was originally published then it is not represented in the charts below.

Here is a very high-level account of how I interpret the chart above. DigitalNZ has metadata describing a large volume of digital content that builds steadily from the 1850s, reaching a peak in the early 20th Century, before dropping away in the mid-1940s. We can see another rise in content beginning at the start of the 21st Century. I was a bit confused as to what the mountain of data represents but when I showed this chart to a few DigitalNZ colleagues and they each said, "Oh, that data mountain is obviously Papers Past".

Papers Past is is a collection of digitised articles and images from New Zealand newspapers and periodicals published between 1839 and 1945. It is an amazing treasure and by far the largest set of items indexed by DigitalNZ. The second chart visually distinguishes between Papers Past and all other items DigitalNZ collects metadata on that has a publication date.

I created these charts using the DigitalNZ developer API. To get the data for the first chart I constructed a query that looks like this:

http://api.digitalnz.org/records/v2.json?search_text=*:*&facets=year&facet_num_results=-1&num_results=0&api_key=[YOUR_API_KEY]

To access the DigitalNZ API requires a key which you append to all of your calls (i.e. the bit where it says [YOUR_API_KEY] ). You can grab a key here. This API call asks DigitalNZ for a JSON file that provides a count for every record summarised by year facets. It should be noted that that API allows you to swap "json" for "xml" if you are so inclined.

The second call that I needed to make looked like this:

http://api.digitalnz.org/records/v2.json?search_text=collection:"Papers Past"&facets=year&facet_num_results=-1&num_results=0&api_key=[YOUR_API_KEY]

This query is very similar to the earlier search except it is restricted to just data from the Papers Past collection. Once I had this data under my belt, I simply transformed the JSON into a spreadsheet and produced a stacked histogram.

You can learn more about the DigitalNZ API by reading the developer documentation. Feel free to get in touch with me if you have questions about any aspect of it. I'm always keen to discuss ideas that people might have or to figure out ways to overcome hurdles you might face.

Educators' eyes opened at ULearn 2011

Tue, 25 Oct 2011 16:44:00 +1300

Auckland Weekly News, 21 June 1939. Sir George Grey Special Collections, Auckland Libraries, AWNS-19390621-46-1. No known copyright.

Last week I was given the fantastic opportunity to show educators DigitalNZ and Mix and Mash at the annual ULearn conference in Rotorua. ULearn is the largest teaching and learning conference in New Zealand with over 1800 educators in attendance.

I have presented these topics before and I am always amazed that there are some teachers who haven’t used DigitalNZ before. Amazed and excited actually, because I know that I am about to show them something hugely useful and relevant that they will take away and be able to implement with their classes immediately.

At Services to Schools, we discuss the skills students need to follow an inquiry process, like finding information from a variety of sources and in a variety of formats. This lends authenticity and credibility to the information and when using it to make something new, different sources provide multiple perspectives and a deeper layering and understanding of a story. DigitalNZ, of course, makes this easy.

We have been very keen to encourage student entries into the Mix and Mash competition as it is such a great outlet for creative use of New Zealand digital content. It also ties in beautifully to many aspects of the curriculum. To increase confidence in teachers this year, we created Free to Mix; An educator’s guide to reusing digital content. The guide provides a whole heap of tips and ideas and links that will enable teachers and librarians to help students understand, find and use New Zealand digital content. We discuss copyright and Creative Commons, the best places to find material for reuse, what to do to enter the competition and a whole lot more that will keep a school’s creative remix community buzzing well beyond the six weeks of the competition.

One thing that surprised and delighted the teachers at the ULearn workshop was the achievability of some of the entries. When they saw the Mix and Mash Supreme Creative Remix winning entry, A Grand Mother, they realised that you don’t need advanced technical skills when you have a great narrative. Year 12 student Casey Carsel’s entries showed history, heritage and humour as well as a huge variety of well attributed resources.

Our favourite Mix and Mash primary school entry was Pt England School embodies the spirit of the competition and just looks like a lot of fun. Another entry reflected work that was completed for NCEA credits, and others demonstrated learning that began in class and extended beyond their normal coursework.

In lots of different ways, the teachers at the session felt positive and empowered and challenged in a variety of ways. One of the participants, a school librarian, became motivated to create a Digital NZ custom search related to the school wide topic for Term 4 and publish it on her library blog.

Another participant felt motivated to show her entire school staff the digital stories that were entered this year and use them to inspire digital storytelling work in all the classes in his school.

One teacher found an image of some students in her school from about 100 years ago and decided this image would be a centrepiece in their jubilee work.

Even as I was delivering the workshop, one teacher began uploading information from my talk to her Learning Management System. Her students were easily able to find links to Creative Commons, Digital NZ and inspirational digital stories in their own learning environment before she’d even left the workshop.

These things are easy to do for educators but hugely empowering for the students who will learn about the rich resources in New Zealand’s digital collections. Students will use these resources to make their own heritage materials and become an active part of the global creative community.

Anatomy of a Remix: How "A Grand Mother" is a grand reuse of New Zealand digital content

Thu, 13 Oct 2011 14:00:00 +1300

The video, “A Grand Mother” by Candy Elsmore, won this year’s Mix and Mash Supreme Creative Remix. We highly recommend that you check it out (you may need a box of tissues).

It’s a beautifully told story about the pride and delight Candy’s family felt after discovering their grand mother / great grandmother, Mary Edith Aberhart, had signed the 1893 Suffrage Petition.

New Zealand’s digital content put to good (re)use

The video remixes music from band Wet Wings, photographs from Candy’s own family photo collection and content from a number of DigitalNZ content partners.

Candy included an image of Grove Road in Blenheim, from the Alfred & Walter Burton Collection at the Auckland Art Gallery Toi o T?maki:

An image of Candy’s ancestor, J H Aberhart, was found in The Cyclopedia of New Zealand [Nelson, Marlborough & Westland Provincial Districts] made available through the NZETC:

Candy found her great grandmother’s listing in the 1893 Suffrage Petition, made available by Archives New Zealand specifically for the Mix and Mash competition:

The narrative of the video was influenced by information from ‘Brief history - women and the vote' published on the Ministry of Culture and Heritage website New Zealand History Online.

Candy also incorporated quotes from articles in the Marlborough Express newspaper (Women's Franchise - Mr. Buick's Views (21 March 1891), Female Franchise (27 September 1892), and Local and General News (13 July 1893), made available through The National Library's PapersPast website.

“A Grand Mother” is a brilliant demonstration of the use of openly-licensed digital content and presents a simple, compelling and highly personal story about a woman who had a small but vital role in New Zealand’s suffrage movement.

This entry may have won the Mix and Mash competition’s top creative remix prize, but perhaps more importantly, it is also now an heirloom that Candy can share with her family, now and in the future.

Launched! British Parliamentary Papers. Colonies: New Zealand

Thu, 29 Sep 2011 06:14:00 +1300

Make it Digital Award winners - University of Waikato Library have been blogging their experience of digitising the The British Parliamentary Papers relating to New Zealand. They've now reached the finish line! You can read about their project progress and process in previous posts.

We have reached the finish line - we are now live! Let the celebrations commence.

The website is looking brilliant, the metadata has all been loaded, the appendices have been compiled which provide:

Links to online biographies of Government Officials during the period of the British Parliamentary Papers, 1830 to 1882
Government Officials during the period of the British Parliamentary Papers, 1835 to 1882.

Here is the URL: http://digital.liby.waikato.ac.nz/bppnz

Our thanks to all involved.

We are signing off and await your feedback and trust you have productive
searches on the database.

Best regards,

The team at the University of Waikato Library

DigitalNZ website has been updated

Wed, 28 Sep 2011 01:30:00 +1300

So you will have noticed that our website just got a refresh. Hurrah! We're planning a full post on the ins-and-outs of the redesign, and what we are trying to accomplish, a little down the track. But for now we'll just let the site speak for itself, and let you explore.

We have quite a long list of things we want to come back to and improve, but are really interested in your initial reactions. What is working well, or not so well for you? Any bugs or issues we haven’t spotted? We are certainly trying some new things with this interface, and we’ll be monitoring the use closely to see what we can learn. Comment on this page, or email your thoughts to info@digitalnz.org. Thanks!

P.S For now you can still access the old search service at search.digitalnz.org

The final stretch: Digitising the British Parliamentary Papers relating to New Zealand

Wed, 10 Aug 2011 13:16:00 +1200

Make it Digital Award winners - University of Waikato Library have been blogging their experience of digitising the The British Parliamentary Papers relating to New Zealand. This post describes their final sprint towards the finish line! You can read about the project progress and process in previous posts one, two and three.

It has been some time since our last update but we can report that we have been making progress.

Various key tasks have been worked on or completed including:

The webpages have been built and developed
Correcting of metadata for completed pages
Checking that all files have been added
Rights management -thanks Fiona and Lewis (from DigitalNZ) for our discussion on videoconference
We did have a few technical glitches but these have all been solved now.

We have recently sent out a test version to friends and colleagues and this resulted in some useful responses and has given us tweeks to work on and investigate.

And we can report another successful 'product' -a baby has arrived. Samia was born on 28 May 2011 to Abdi and his wife.

We hope to go live with a pilot soon.

Nelson Photo News Launched

Wed, 10 Aug 2011 10:53:00 +1200

Make it Digital Award winners - The Friends of the Nelson Library Microfilm Sub-committee have been blogging their experience of working collaboratively to digitise the Nelson Photo News. This, their last post, tells us the happy news about their launch. You can read previous posts about the digitisation project process and progress here and here.

After a long gestation Nelson Photo News on-line was finally launched to an enthusiastic crowd on 18th July 2011.

The team responsible was surprised and delighted that Barry Simpson, the Nelson Photo News chief editor and photographer, was able to attend despite being in poor health in recent years. As Joanna Innes-Walker chairperson of the team remarked “my day was made when I saw Barry Simpson himself coming through the door”.

Barry Simpson (centre-left), the Nelson Photo News chief editor and photographer, with his daughter Annette Savage (centre-right) enjoying themselves with guests at the launch event.

To get the website up and running the project team had to resolve various issues including how to deal with copyright issues around images of the 1960’s era. The team are pleased they have shown a way for other cities around New Zealand to digitally publish their own Photo News. Usage figures from the website show that around 1000 people viewed the site in the week after the launch.

Feedback has been very positive and among other requests received has been one for more information on a long lost relative and one from the UK for permission to re-use a rare image of All Black coach Ivan Vodanovitch.

Some of the comments received:

Thanks Karen. I thought last night went off exceptionally well. Dad (Barry Simpson) enjoyed himself and we will look forward to showing him the site. Many thanks for all your hard work. I'm sure this is a site that will be well used. Annette Savage

What a terrific project! Congratulations to you for doing this. I have sometimes seen the odd, battered copy of the magazine in a second hand store and often thought what a great resource they would be. I think this would be great to use as a resource for high school students doing a specific study on past times in Social Studies or History. I hope that that does happen. I hope that other photo news do likewise. Josephine Maplesden

Nearly there - digitising the British Parliamentary Papers relating to New Zealand

Thu, 16 Dec 2010 10:09:00 +1300

Make it Digital Award winners - University of Waikato Library are blogging their experience of digitising the The British Parliamentary Papers relating to New Zealand:

Work has been proceeding apace on all aspects of the project. The scanning is all done. Thanks NZ Micrographics!

Here in Hamilton at the University of Waikato Library, two teams have been beavering away.

The busy Waikato University Library Team (with Licorice Allsorts) responsible for digitising the British Parliamentary Papers relating to New Zealand

One has been taking the scanned image files and breaking them up or combining them into their constituent reports and documents to create printable pdf files for each despatch. This is nearing completion though we have noted a small number of images that needed to be rescanned - a few pages with tears and holes and some others with blemishes have been found and new, replacement images are being prepared.

The other team has been reading through the text files after they had been ocr-ed. This has proved a tedious but necessary task and it has shown to staff how much useful and previously unrecognised material is lurking in these pages. We are about three quarters of the way through that proofreading which is correcting only key terms and names, not every word.

Mostly the ocr-ing has produced pages that are true to the original but there have been a few mistakes that we have spotted. We thought you might enjoy some of them:

Scanned text	What it should read:
F. Dillon Hell	F. Dillon Bell
Banged by the neck	Hanged by the neck
And this sum is required to make god	And this sum is required to make good
Such a policy requires a highly organised farce	Such a policy requires a highly organised force
Wellington Bangers	Wellington Rangers
I was assuming powers to which I had no light	I was assuming powers to which I had no right
A lime at least	A time at least
Aimed rebels	Armed rebels
I herewith enclose Major Eraser’s despatch	I herewith enclose Major Fraser’s despatch
Secretary of Slate	Secretary of State
for giants of confiscated lands	for grants of confiscated lands
Loo Papers relating to New Zealand	100 Papers relating to New Zealand
From these farts, you are the responsible tribunal	From these facts, you are the responsible tribunal

Finally, we have begun the process of building the collection and creating the web interface using Greenstone software developed by our colleagues in the University of Waikato Faculty of Computing and Mathematical Sciences.

Given that some documents have been completed we hope to have at least part of the collection live and accessible in the very near future. We will then add further material as it is finished.

Hamilton 15 December 2010

API Enhancement brings geotagging to search results

Fri, 15 Oct 2010 15:55:00 +1300

Geotags for search results are now being returned in the Search Records API. We're still updating the docs, but the field is called geo_co_ords and it returns a semicolon-separated list of lat,lng pairs.

This means you can finally create awesome google maps of locomotives with the DigitalNZ API.

You can even jump to a street view (drag the small orange humanoid from the top left onto the map pin) and see how things have changed.

But perhaps the most exciting thing is that You can geotag any image on DigitalNZ and have it appear on the map moments later. Try this:

start with a subject you love, let's say lighthouses. Everyone loves lighthouses.
click the Geotagged: No filter. This will show all the non-geotagged images of lighthouses in DigitalNZ, as thumbnails.
scroll through the list until you see find an image that you think you could place on a map.
click the small map pin in the bottom right of the image. You might like to try this lighthouse at Tairoa Head. Scroll down the page and click the Add a Location tab above the map.
sign in (or register: it's quick and painless and you know you can trust us not to spam you).
zoom in on the map to where you think the image belongs (the mouse scroll wheel can help), and click where you want to place the pin. Click Save to save the location.

When you return to the lighthouses map a few minutes later, you should see a clickable pin where you placed your lighthouse image. Take a moment to appreciate that feeling of having made the world a better place.

To build something yourself with the DigitalNZ API and google maps, view source on any of my map pages to see the javascript and see the Google Maps API documentation for details.

Elliott Young
DigitalNZ Tech Lead

Enabling Remix - how to make your content available for Mix and MashNZ

Thu, 14 Oct 2010 11:38:00 +1300

We've been talking to all our wonderful content providers about making content available for reuse in the upcoming Mix and Mash Competition - due to launch on 3 November 2010.

This post outlines the 4 small steps to get onboard and make content easily available through DigitalNZ for Mix and Mash.

Step 1) Read the Make It Digital Remix Guide :
We have just published a guide which explains how to enable remix*. We strongly encourage you to read the guide and then come back to this post for the next 3 steps.

Step 2) Understand the competition and what it lets people do with your content:
Mix and Mash will have categories for a range of content types and formats. Entrants need to locate content that is appropriately licensed for remixing.
DigitalNZ is currently improving http://search.digitalnz.org so everyone can much more easily find items by licence/rights status. This will be live in the next week.
If an entrant finds an item in http://search.digitalnz.org that is appropriately licensed then they will be able to click on it and go straight to the item's page on your website.
They will then copy/download the item directly from your site and use it for the competition.
We have built in help text that asks people to read and follow the licence/rights statements for individual items before they use them.

Step 3) Tell us about the content you want to make available for use:
All you need to do is send me (fiona.rigby@digitalnz.org) an email telling me:
1) details of the items or collection, including information about whether they are already discoverable in DigitalNZ or not. Include URLs from your website if possible.
2) the license/usage statement** that you would like to apply. Include the URL to that statement if possible.
(Do still get in contact if you have content that you'd like to make available but can't host online. We may be able to give it a home at http://kete.digitalnz.org/ or another of our hosting options.)

Step 4) Wait a wee bit. We'll get back to you with next steps so you can get involved in this exciting iniative.
We will try to keep this as simple as possible for you!

Mix and Mash will be the biggest push ever to get people using our NZ digital content and data. It will not only create a bucket-load of new content but also give huge exposure to the wealth of NZ content DigitalNZ partners are already making available.
We're really excited about working with you on this.

If you have any specific questions you can always email us, anytime, at info@digitalnz.org

Cheers Fiona
DigitalNZ Content Manager

*Remix: A new version of a song, book, picture, video (you name it) made by adding to, or otherwise changing, the original version (licence permitting of course!)
**Terms of Use Statement excerpt from our Remix Guide (we do encourage you to read this guide in full):
"Many terms of use on websites were written in an era when it was easier to put up a blanket statement as a legal disclaimer and leave it to users to worry about copying. Today this is a disservice to you and your users, as it discourages them from using your online services and respecting what your website says. Rather than relying on wordy legal disclaimers, you can use your terms of use to describe the kind of community and behaviour you would like to see built around your content. As part of the Creative Commons work on a public domain mark, they have drafted an excellent set of user guidelines that can be used as a basis of drafting a Terms of Use statement for your remix content. You are free to copy and adapt this text, provided you attribute Creative Commons.org for the original text."

Great progress digitising the British Parliamentary Papers relating to New Zealand

Thu, 07 Oct 2010 15:25:00 +1300

Make it Digital Award winners - University of Waikato Library are blogging their experience of digitising the The British Parliamentary Papers relating to New Zealand.

We have been making steady progress with the digitisation of the British Parliamentary papers relating to New Zealand. New Zealand Micrographics were selected to scan and OCR the papers for us and this new partnership was kicked off with an audioconference to discuss the project and refine the details. Many questions were asked and answered, and we all came away feeling more knowledgeable and positive about where we are heading. In order to get the papers to Wellington, New Zealand Micrographics said they would despatch a baby freight container. When it arrived we decided if this was the baby one we did not want to see the grown up version! Our project papers filled approximately one third of the box but with plenty of packaging to tuck them in they were on their way south.

A pilot set of papers have been scanned, passed Quality Control and had the OCR completed. We were impressed with the clarity of the images and agreed that the OCR was sufficiently accurate. We were chomping at the bit, ready to get on with the mass scanning by this time, but there were still some questions to consider, particularly around the file naming and the handling of the non-standard pages (e.g. foldout maps). Eventually, these were resolved after a good number of emails flew along the wires. We really appreciated the input of New Zealand Micrographic Services here, as their experience and suggestions enabled us to make some quick decisions, particularly around the tricky nomenclature issues.

While NZMS started scanning in earnest, back in the Library we were thinking about the search parameters and web interface for the digital collection. National Library’s webpage for their Appendices to the Journals of the House of Representatives project was launched recently and we were all impressed by its layout and ease of use. Given the similarity of the two projects and our intention to also use Greenstone software, it seems sensible that we should produce a webpage similar in style and function. (We are also great proponents of the library philosophy of making use of shared expertise and not reinventing the wheel!) The master access images have recently arrived from New Zealand Micrographics so we now have two big tasks ahead of us – to combine (and in some cases breakup) page images to create printable PDFs of each despatch, and to proofread and carry out essential edits of the OCRed text.

It definitely feels that we are making progress.

Nelson Photo News Project Now in Full Swing

Mon, 06 Sep 2010 15:04:00 +1200

Make it Digital Award winners - The Friends of the Nelson Library Microfilm Sub-committee are blogging their experience of working collaboratively to digitise the Nelson Photo News.

Following many years of fundraising, digitisation of the Nelson Photo News by the Nelson Library Microfilm Subcommittee is now in full swing.

Once we had secured sufficient funding, the first step was to put together a full set of magazines and Nelson Public Library were very helpful in handing over their set for the project to use.
To ensure that contaminants weren't passed on to the Nelson Provincial Museum collection, the magazines had to undergo a requisite period of freezing and treating, before condition reporting could commence.

Volunteers at Nelson Provincial Museum then began the task of checking through each copy of the Nelson Photo News checking for marks, defects or missing pages to ensure that the set which was digitised was as complete and pristine as possible.

Nelson Provincial Museum Digital Cataloguing Technician Megan Hansen-Knarhoi and Museum Volunteer Pam Mason work on condition reporting Nelson Photo News issues loaned by Joan Inwood from Richmond for the project

At the end of the process it was found that several additional issues were needed to complete the set. Following a call in local newspapers for owners of Photo News collections to come forward, we were able to secure the missing issues from a private collection.

"Taming the Mighty Maitai" Photo spread Nelson Photo News. 1969, Nelson N.Z. : Logan: 99 (Feb 6), 60-61

"Terrific Fun for a Tremendous Crowd" Image by Barry Simpson, Nelson Photo News Collection: 1394 fr 15

While condition reporting work continued, three issues chosen from the early, middle and end publishing periods were sent away to the digitisation supplier to be used for a pilot test study. These three issues will be digitised in line with the proposal submitted by the supplier and at the end, a working sample of the proposed website will be available for us to look at online. The sample will also include examples of various options so that we can evaluate these and establish our preferences before the full digitisation commences. It will also allow us to check how images appear, the resolution at which they appear and how images could be suppressed if we can not run images because of copyright issues. Once this example has been refined to a working standard, the remainder of the issues will be sent away for the wholesale digitisation work to begin.

"Nelson's Music Industry" photo spread in Nelson Photo News. 1965, Nelson N.Z. : Logan: 51 (Feb 6), 38-39

"Final tests are made by Mr H F Roberts" Barry Simpson, Nelson Photo News Collection: 3285 fr 4

Building a Website on the DigitalNZ API

Wed, 11 Aug 2010 17:19:00 +1200

Introduction

After an eighteen-month spell at Weta Digital I am back at the National Library, or more precisely the hallowed ground of Digital New Zealand.

When I left, Digital New Zealand was a newborn widget. I have returned to find a fully-fledged search service and API suite, with over 100 content partners.

That's one hundred organisations who have signed on and made their digital material searchable through the DigitalNZ APIs. Just wrangling all that disparate data into a searchable common metadata standard is a mind-boggling achievement, let alone the background work required to make such things happen. Having so many organisations on board makes the search API more valuable than ever.

So let's build something with that DigitalNZ Search Records API, and see how it all works.

We could do something client side with javascript, perhaps some kind of play on the DigitalNZ Search Widget.

Or we could do the obligatory google maps mash up, although the search API isn't yet geographically potentised. Hopefully soon.

Or we could just make a website for discovering images and videos, since the vanilla standard http://search.digitalnz.org isn't particularly exhilarating when used for this purpose. Yes, that sounds like fun.

Fast forward a few days and I had what I thought was a pretty reasonable website built on the DigitalNZ search API. There was only one problem.

When I told the DigitalNZ team about it:

So here's this website I built on the DigitalNZ API: http://elliottyoung.com/labs/nzpictureshow/

They said:

Looks good!

Good? Didn't you see how fast it was? It searches half a million images and videos in a few hundred milliseconds! And shows hundreds of thumbnails at a time!

oh, yep

Users can even narrow their search by decade, content partner, placename and rights, using cool retro tag clouds!

and geotag any result by clicking on the small map pin!

uh-huh

So did you notice that it's all done with only 150 lines of code? That's a pretty incredible website for 150 lines of code. This API changes everything!

yeah

and do you know I could actually host this awesome website for like $1/week because the API is doing all the heavy lifting and the actual images and videos are delivered to the user's browser directly from the content providers thanks to the miracle of hypertext?

sure

This went on.

At first I couldn't tell what was happening, but eventually I realised that what I thought was so great about my website was really just what was so great about the DigitalNZ APIs, and the team have been living and breathing them for quite some time. They've become accustomed to these miraculous APIs.

But I think that something this awesome has to be shouted about. I knew just what to do.

I'm going to blog about it

It will start with a story about how I came to DigitalNZ and built this picture show website with their API. Then there will be this weird conversation between the DigitalNZ team and me that possibly never happened. Last of all, we'll unpick the website, show all of the source code, and hopefully help everyone on their individual ways to doing really cool things with the DigitalNZ API.

And so it begins.

This may reveal that developing with the DigitalNZ APIs is easier than you thought. At the very least, you can surely pinch some code to use in your own projects.

I need to apologise to users of older versions of Internet Explorer, for how the code snippets appear on your screen. For the rest of us they should just overflow the righthand margin in an ugly but perfectly functional manner.

Building a Website on the DigitalNZ API

Use any Language

I have used PHP and MySQL because they're widely known, friendly, and come for free on this $1 per week hosting plan I found. Best of all, PHP won't snipe at you for a hundred minor things that fancier languages will, so you can write code that just gets it done. Oh, and when it hits a bug, it won't print a whole page of barely intelligible stack trace to the screen without giving a clue to where the error even occurred. No, it'll just say something like: "on line 15 in index.php you missed a semicolon" or something like that. It wins.

If you're making a widget — a small "panel" that sits within another website — you will have to use Javascript or Flash, since widgets run inside the user's browser and the options are limited. But if you're making a normal website with the DigitalNZ API, almost any language and database you can think of will be fine.

Of course, if developing on your own computer, PHP comes pre-installed on Mac OS X and Linux, and you can download a perfectly good setup for Windows called xampp. Perhaps best of all, you can google site:php.net <what I need> and you'll almost always find documentation about the function you need, with lots of examples.

Start with a "Hello World"

Before we look at the NZ Picture Show in detail, we'll start with a really simple script that uses the DigitalNZ Search API to display some results and thumbnails for the search_string given in the URL. Here's how it looks in action - and here's the script:

Notes

Code


you can get your API key here •

retrieve the parameter from the URL •
search for sunrises if nothing else •

set up the API call •
make the API call •
dump the results to the screen (disabled) •

loop through the results •
linking each one to its original •
displaying a thumbnail •
and a title •

<?php

$api_key = '<your api key>';

$search_text=stripslashes($_GET['search_text']);
if(!$search_text) { header('Location:'.$_SERVER['PHP_SELF'].'?search_text=sunrise'); exit; }

$api_call = 'http://api.digitalnz.org/records/v2.json?search_text='.urlencode($search_text).'&api_key='.$api_key;
$api_response = json_decode(file_get_contents($api_call),true);
//var_dump($api_response['results']);

foreach($api_response['results'] as $result) {
	echo '<a href="'.$result['source_url'].'">
	      <img src="'.$result['thumbnail_url'].'"/>
	      <br />'.htmlspecialchars($result['title'],ENT_COMPAT,'UTF-8').'</a><br /><br />';
}
?>

One of the great things about PHP is that you can probably copy the code above, paste it into a new text editor window, save it as helloworld.php to the www or htdocs folder on your own computer or on your own webhost, and point your browser at http://localhost/helloworld.php. The only thing you'll need is your own API key, which is just a long string of characters by which the API will know you. Luckily, getting one is free, quick and easy - just follow the instructions at http://digitalnz.org/developer/getting-started/.

If you're wondering how this API call actually works:

$api_response = json_decode(file_get_contents($api_call),true);

file_get_contents is a PHP function that retrieves a URL (just like a web browser, only it puts the output in a string instead of on the screen). Because we want the data in an array (which allows us to use the super handy foreach loop), we then pass it to json_decode. Job done!

Use what you like from the Source Code of the NZ Picture Show

The website comprises 4 PHP files, included in full below:

index.php - the main script
functions.php - a bunch of functions used by index.php
constants.php - just some constants, such as API key, number of results to return, etc
db.php - a small script that adds a suggestion to the database for the 'Popular' section on the front page. Very optional; I just wanted to show MySQL in use.

There's also a simple css file, some javascript (jquery and jquery.infinitescroll, to automatically load more images as you approach the bottom of the page) and a logo. But we're not concerned with them.

Let's look at the php files one at a time

The Source Code, Part I: index.php

The default landing page is called index.php for historic reasons; it doesn't have anything to do with indexing.

Note that any raw html in a php file is just output directly to the browser as html; to invoke php code you need to enclose the code in <?php and ?>.

Notes

Code




get the
parameters
from the
url


call the API •




output the html to set up the 
page, its search forms and so on.







































if there's some results...

display the 'tag clouds' at the top •

display the thumbnails •


make a link to the next page •

or there's no results...

this was a user search •
that found nothing
or the API returned an error •

or this is the landing page so
display the 'Popular' cloud •

and let them add a Popular term •










load the javascript for twitter,
and the infinite scroll feature

and configure it

<?php
include 'constants.php';
include 'functions.php';

$search_text=stripslashes($_GET['search_text']);
$applied_filters=stripslashes($_GET['applied_filters']);
$page=max(1,floor($_GET['page']));
$suggestionadded=floor($_GET['suggestionadded']);

if($search_text or $applied_filters) {
	$api_response=call_api($search_text,$applied_filters,$page);
	if($api_response) $result_count=$api_response['result_count']; else $api_failure=1;
}
?>

<!DOCTYPE html>
<html>
<head>
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
	<link rel="stylesheet" type="text/css" href="css/basic.css" media="screen" /> 
	<script type="text/javascript" src="js/showhide.js"></script>
	<title>NZ Picture Show | <?php if ($search_text) echo htmlentities($search_text); else echo 'Home';?></title>
</head>
<body OnLoad="document.searchform.search_text.focus();">
	<div class="block banner">
		<div class="headline"><a href="?">NZ Picture Show</a><br />
		<a href="http://twitter.com/share" class="twitter-share-button" data-count="none" data-via="digitalnz">Tweet</a></div>
		<div class="search">
			<form name="searchform" action="">
					<input name="search_text" type="text" class="textbox" value="<?php echo htmlspecialchars($search_text,ENT_COMPAT,'UTF-8'); ?>"><br />
					<input class="Search button" value="Search" type="submit"> &nbsp;
					<?php if ($result_count>0) echo $result_count.' Results found in '.$execution_time.' seconds. &nbsp; 
					<a target="_blank" href="http://search.digitalnz.org/search?search_text='.
					urlencode(DEFAULT_SEARCH_TEXT.' '.$search_text.' '.$applied_filters).'">View on DigitalNZ</a>'; 
					else echo 'Type a word or two and click Search'; ?> 
					 &nbsp; <span onclick="showhide('abouttext'); return true;">About</span> &nbsp; <a href="?">Popular</a>
			</form>
			<div id="abouttext" style="display:none;">
					<div class="h">About this website</div>
					This website demonstrates use of the <a href="http://digitalnz.org" target="_blank">
					Digital New Zealand</a> API to query and discover Digital New Zealand content.  A tutorial, including the 
					full source code, is available at <a href="http://digitalnz.org" target="_blank"
					>http://digitalnz.org</a>.
					<div class="h">Terms of Use</div>
					Use of this website is governed by the terms of use of DigitalNZ at 
					<a href="http://digitalnz.org/terms/" target="_blank">http://digitalnz.org/terms/</a>.
			</div>		
		</div>
		<div style="float:right;">
			<a target="_blank" href="http://www.digitalnz.org"><img style="border:0;" src="images/DNZ_Logo.jpg" alt="Powered by Digital New Zealand"></a>			
		</div>
		<div style="clear:both;"></div>		
	</div>

		<?php 
	
	if($result_count>0) {

		display_clouds($api_response,$search_text,$applied_filters); 
		echo '<div id="content" style="clear:both; ">';
			display_thumbs($api_response); 
		echo '</div>';
		echo '<div style="clear:both;"></div>';
		echo '<div class="navigation"><a href="?search_text='.urlencode($search_text).'&applied_filters='.urlencode($applied_filters).'&page='.($page+1).'">Next</a></div>';

	} else {
	
		if($api_response) echo '<div class="block"><span class="heading">NO RESULTS</span>
			<br />&nbsp;<br />Sorry, your search returned no results.  Please try again, or click a suggestion below to get started.</div>';
		elseif($api_failure) echo '<div class="block"><span class="heading" style="color:red;">ERROR</span>
			<br />&nbsp;<br />Sorry, your search returned an error from the search service.  Please try another search.</div>';
		echo '<div class="block"><span class="heading">POPULAR</span>';
			display_popular();
		echo '</div>';
		echo '<div class="block"><span class="heading">SUGGEST A SEARCH TERM</span><br />&nbsp;<br />
			'.($suggestionadded?'Thanks!  Your suggestion has been queued for moderation. ':'').'Add your favourite search to the Popular section:
			<form name="suggestionform" action="db.php">
					<input name="suggestion" type="text" value=""><br />
					<input class="Search button" value="Add" type="submit">
			</form>
			</div>';
		
	}
	?>

	<script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>
	<script type="text/javascript" src="js/jquery-1.3.2.min.js"></script>
	<script type="text/javascript" src="js/jquery.infinitescroll.min.js"></script>
	<script type="text/javascript">
		$('#content').infinitescroll({
		navSelector  : "div.navigation",            
		nextSelector : "div.navigation a:first",    
		itemSelector : "#content div.post",         
		bufferPx     : 100,
		loadingImg   : "images/blank.gif",          
		loadingText  : "",    
		donetext     : "" 
		});
	</script>
</body>
</html>

The Source Code, Part II: functions.php

This contains the functions used to render the search results and facet tag clouds.

Notes

Code


call_api
Calls the Search API and
returns the results


All these API parameters 
are documented here.




The actual API call


If there's an error, return false


tag_size
returns a font-size %


sanitise
takes care of macrons etc


display_clouds
displays a tag cloud for each
facet in the results

the facets in the array can best be
seen with a var_dump from the api, 
but they are returned as an array 
of facets, where each facet is made 
up of a facet_field (eg decade) and 
an array of values, where each value 
has num_results (the number of results 
that would be returned if that facet were
applied) and the string (eg "1960-1969")



array_multisort puts the results in
alphabetical order within each facet

loop through the tag_fields, tag_values 
and tag_counts arrays











Add a string to the search to filter
the results based on the tag clicked, eg
decade:1960-1969






display_thumbs
Displays a page of thumbnails with
captions beneath


display the thumbnail •




the pin •






the title •
and the content provider •







display_popular
Display a tag cloud of popular options


Retrieve all the 'published' suggestions •




Print them out at the appropriate size •

<?php

function call_api($search_text,$applied_filters,$page) {
global $execution_time;
	$api_call='http://api.digitalnz.org/records/v2.json?num_results='.NUM_RESULTS.
		'&search_text='.urlencode(implode(array(DEFAULT_SEARCH_TEXT,$search_text,$applied_filters),' ')).
		'&facets='.SHOW_FACETS.
		'&facet_num_results='.FACET_NUM_RESULTS.
		'&start='.max(0,($page-1)*NUM_RESULTS).
		'&sort='.SORT_KEY.
		'&direction='.SORT_DIRECTION.
		'&api_key='.API_KEY;
	$time_start = microtime(true);
	$result=json_decode(file_get_contents($api_call),true);
	$time_end = microtime(true);
	$execution_time = round($time_end - $time_start,2);		
	if(is_array($result)) return $result; else return false;
}

function tag_size($tag_count,$max_count) {
	return min(TAG_MAX_SCALE,floor(TAG_SIZE*(1.0+(TAG_RATIO*$tag_count-$max_count/2)/$max_count)));
}

function sanitise($text) {
	return htmlspecialchars($text,ENT_COMPAT,'UTF-8');
}

function display_clouds($api_response,$search_text,$applied_filters) {
		
	$max_count=0; 

	foreach($api_response['facets'] as $facets) {
		$facet_field= $facets['facet_field'];

		foreach($facets['values'] as $value) {
			$facet_value=$value['name'];
			$facet_count=$value['num_results'];
			$tag_values[]=$facet_value;
			$tag_fields[]=$facet_field;
			$tag_counts[]=$facet_count;
			$max_count=max($max_count,$value['num_results']);
		}
	}
	array_multisort($tag_fields,$tag_values,$tag_counts);
	$c=count($tag_values);

	echo '<div class="block">';
	for($i=0;$i<$c;$i++) {
		# Add Tag Category heading if it's the first tag of its category
		if ($tag_fields[$i]<>$last_tag_field) { if($last_tag_field) echo '</div><div class="block">'; 
			echo '<span class="heading">'.strtoupper($tag_fields[$i]).'</span>'; $last_tag_field=$tag_fields[$i]; }
		if(!(strpos(urlencode($applied_filters),$tag_fields[$i].'%3A%22'.urlencode($tag_values[$i]).'%22')===FALSE)) { 
		# Tag has already applied
			echo '<a class="tag'.($i%2).'" title="remove '.htmlspecialchars($tag_fields[$i].':"'.$tag_values[$i].'"',ENT_COMPAT,'UTF-8').' 
				from filters" style="font-weight:bold; color: #c8470f; font-size: '.tag_size($tag_counts[$i],$max_count).'%;" 
				href="?search_text='.urlencode($search_text).'&amp;applied_filters='.str_replace($tag_fields[$i].'%3A%22'.
				urlencode($tag_values[$i]).'%22','',urlencode($applied_filters));
		} else {
		# Tag has not been applied
			echo '<a class="tag'.($i%2).'" title="'.$tag_counts[$i].' items" style="font-size: '.tag_size($tag_counts[$i],$max_count).'%;" 
				href="?search_text='.urlencode($search_text).'&amp;applied_filters='.urlencode($applied_filters).
				($applied_filters?'+':'').$tag_fields[$i].'%3A%22'.urlencode($tag_values[$i]).'%22';
		}
		echo '">'.str_replace(' ','&nbsp;',$tag_values[$i]).'</a> ';
	}
	echo '</div>';
}		

function display_thumbs($api_response) {
	foreach($api_response['results'] as $value) {
		echo '
			<div class="post">
				<a target="_blank" href="'.sanitise($value['source_url']).'">
					<img src="'.sanitise($value['thumbnail_url']).'" 
						title="'.sanitise($value['title']).' ('.sanitise($value['content_provider']).'): '.sanitise($value['description']).'"/>
				</a>
				<div class="metadata">
					<a target="_blank" href="http://search.digitalnz.org/records/show/'.$value['id'].'.html">
						<img class="pin"  src="images/pin.gif"/>
					</a>
					<div>
						<a title="'.sanitise($value['title']).'" target="_blank" href="'.sanitise($value['source_url']).'">'.
							substr(sanitise($value['title']),0,45).'</a>
					</div>
					<div>
						<a class="provider" title="'.sanitise($value['content_provider']).'" target="_blank" href="'.
							sanitise($value['source_url']).'">'.substr(sanitise($value['content_provider']),0,45).'</a>
					</div>
					
				</div>
			</div>';		
	}
}

function display_popular() {
	$connection = mysql_connect('localhost', MYSQL_USER, MYSQL_PASSWORD);
	$max = mysql_fetch_row(mysql_query('select max(hitcount) from younge_dnz.suggestions where published=1'));
	$max_count=$max[0];
	$suggestions = mysql_query('select suggestion,hitcount from younge_dnz.suggestions where published=1 order by suggestion');
	$i=0;
	while($row=mysql_fetch_row($suggestions)) {
		$suggestion=$row[0];
		$hitcount=$row[1];
		echo '<a class="tag'.($i%2).'" title="'.$hitcount.' items" style="font-size: '.tag_size($hitcount,$max_count).'%;" href="?search_text='.
			urlencode($suggestion).'">'.str_replace(' ','&nbsp;',$suggestion).'</a> ';
		$i++;		
	}
}

?>

The Source Code, Part III: constants.php

This one's simple enough. A bunch of things you can configure.

Notes

Code


your API key •
how many thumbnails to grab at once •
the filter that is applied to all the searches •
how many tags are in each cloud •
how the results are sorted - a field•
how the results are sorted - asc/desc •
database username •
database password •
which tag clouds to show •
tag cloud largest font % scale •
tag cloud overall font size •
tag cloud size differential •

<?php

define('API_KEY','put your key here');
define('NUM_RESULTS',50);
define('DEFAULT_SEARCH_TEXT','(category:Images OR category:Videos)');
define('FACET_NUM_RESULTS',20);
define('SORT_KEY','date');
define('SORT_DIRECTION','asc');
define('MYSQL_USER','put your database username here');
define('MYSQL_PASSWORD','put your database password, if any, here');
define('SHOW_FACETS','decade,placename,rights,category,content_partner');#choose from category,century,decade,year,creator,rights,collection,content_partner
define('TAG_MAX_SCALE',250);
define('TAG_SIZE',200);
define('TAG_RATIO',1.6);

?>

The Source Code, Part IV: db.php

Takes a script parameter and adds it to the MySQL database of popular searches, if valid. Very optional.

Notes

Code




get the parameter from the URL •
if there's a suggestion and it's not too long •
connect to the database •
see how many results this search would return •

if it would return more than one result •
add it to the database •
and flag the fact that we added it •



redirect back to the home page •

<?php
include 'constants.php';
include 'functions.php';

$suggestion = stripslashes($_GET['suggestion']);
if($suggestion<>'' and substr_count($suggestion," ")<4) {
	$connection = mysql_connect('localhost', MYSQL_USER, MYSQL_PASSWORD);
	$api_response=call_api($suggestion,'','',0,0);
	$result_count=$api_response['result_count'];
	if($result_count>0) {
		mysql_query('insert into younge_dnz.suggestions (suggestion,hitcount,published) values ("'.mysql_real_escape_string(ucfirst($suggestion)).'",'.$result_count.',0)');
		$suggestionadded=1;
	}
}

header("Location: http://".$_SERVER['HTTP_HOST'].rtrim(dirname($_SERVER['PHP_SELF']), '/\\')."/?suggestionadded=".$suggestionadded);
exit;

?>

I'm using a MySQL database to store the popular searches that appear on the homepage. Like PHP, MySQL is free and ubiquitous. The table can be created by executing the following SQL, using PHPMyAdmin or similar:

Notes

SQL


the key words •
how many hits this search gets •
whether this search is visible •

CREATE TABLE `suggestions` (
  `suggestionid` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `suggestion` varchar(255) NOT NULL,
  `hitcount` int(10) unsigned NOT NULL DEFAULT '0',
  `published` tinyint(1) NOT NULL DEFAULT '0',
  PRIMARY KEY (`suggestionid`),
  UNIQUE KEY `suggestion` (`suggestion`),
  KEY `published` (`published`)
) ENGINE=MyISAM AUTO_INCREMENT=132 DEFAULT CHARSET=latin1

The Wrap Up

The DigitalNZ APIs open up an unbelievably rich and deep content source for you to use in your own projects.

As you've seen here, you can use the APIs to create something new, and then share that website or widget with the world.

Using the same techniques, you can also combine our APIs with a growing number of other organisations' APIs, to create a "mash up" (or something so cool it doesn't even have a name yet) that is greater than the sum of its parts. This is a very exciting area for development.

Whatever you're doing, or thinking about doing, with the DigitalNZ APIs, I hope you've found something useful in this post, and I look forward to seeing what you've made! Please send me a link, or post a comment below.

Elliott Young
DigitalNZ

Nelson Photo News - Make it Digital Award Project Update

Mon, 09 Aug 2010 14:15:00 +1200

This is the first in a series of posts from our second Make it Digital Award winners - The Friends of the Nelson Library Microfilm Sub-committee. The Make it Digital award winners are sharing their real life experience of getting their digitisation projects off the ground.

Nelson Photo News. 1963, Nelson N.Z. : Logan:30 (Apr 27)

The first run of the Nelson Photo News is a rare and unique
photographic record of life in Nelson in the early 1960’s and 1970’s –
a time of rapid social and cultural change. The serial captures
personal and community events through photographs and lively
commentary.

The Project Steering Group

Friends of the Nelson Library Microfilm Sub-committee - Image Courtesy of Nelson Mail

The Friends of the Nelson Library Microfilm Sub-committee responding to
community pressure to do something about preserving and making
accessible this valuable resource, have been fundraising for a number
of years to enable the digitisation of the issues from 1960 - 1972 to
proceed. Winning the grant from Digital NZ was an exciting time for the
committee as it meant the project can now come to fruition.

The project is very much a collaborative effort, with the The Friends
of the Nelson Library Microfilm Sub-committee composed of members from
the Nelson Public Libraries: nga whare matauranga o whakatu, the Nelson
Provincial Museum: Pupuri Taonga o Te Tai Ao, Nelson Historical
Society, Nelson Institute and the Nelson Branch of the New Zealand
Society of Genealogists.

To date the committee has been responsible for several microfilming and
digitisation projects including the microfilming of the Nelson Mail and
the digitisation of the Nelson Historical Journals.

Why We Are Digitising?

As the editor Barry Simpson stated in the first issue: “Our experience
…. indicates that many people allow the first issue or two to be
destroyed, which is regretted later…subscribers grow to value the
pictorial record of community life.”
Unfortunately most early issues of the Nelson Photo News have been
discarded, and over the run only a handful of complete sets are
believed to exist in private collections; currently there are no full
sets held in the public domain.

The publication of each issue of the Nelson Photo News was a popular
event and over the years the publication has reached almost iconic
status locally. With a 'roving cameraman' style to the publication,
many Nelsonians feature in past editions and feedback indicates that
they are keen to see themselves again in their 1960s glory!

Nelson Photo News. 1960, Nelson N.Z. : Logan:2 (Dec 10).

"The Photo News magazines are part of my history. Oh the excitement on
the day they went on sale each month, and the competition in our house
to be first to read it. I fully support this publication being
digitised. On a personal level it would be great to be able revisit
these magazines. On a wider aspect, there is huge content of historical
value to the Nelson region and its people. "

Nelson Photo News. 1963, Nelson N.Z. : Logan:27 (Feb 2).

With limited issues available, digitisation of this serial will ensure
that this important snapshot of 1960s life will be not only preserved,
but readily accessible to the wider community.

Despatches from Down Under- the British Parliamentary Papers from and to New Zealand

Tue, 27 Jul 2010 16:38:00 +1200

This is the first in a series of posts from Make it Digital Award winners - University of Waikato Library . We've invited both award winners to share their real life experience of getting their digitisation projects off the ground.

We were delighted to hear the news that we had won one of the “Make it Digital” awards. As our colleague Emma Pooley described it -we had made it as NZ Digitisation Idols!

Once we knew we had the funding though it was time to form the project implementation team and get our planning and work underway. John Robson, the Map Librarian, began preparing the documents ready for scanning while Vye Perrone (Project Co-ordinator) and Abdi Ali (Library Computer Systems Consultant) worked on the administration and technical aspects of the project. Kathryn Parsons, the New Zealand Collection Librarian is principal advisor to the project with her past experience in digitisation and her New Zealand history expertise. Fiona Rigby and Lewis Brown from National Library have offered much valuable information and support.

Our task is to digitise the British Parliamentary Papers Relating to New Zealand, covering the period from 1835 to 1882. This set of documents contains a huge amount of valuable information on a large range of subjects covering a crucial period in New Zealand’s history. It is material that only a few libraries in New Zealand hold so digitisation will make it available to a far wider audience and much easier to use.

The process:

The material to be scanned has all been removed from its old original bindings. Each document has been separated into single sheets ready for sending to the company chosen to scan it.

For each document a contents list of all the despatches, letters, etc therein has been compiled noting the sender, recipient, date, pagination of each item as well as any enclosures and maps associated with it. The project has been structured so that in the future, work can continue to cover the documents not forming part of the Library’s original set.

In the meantime too, the full specifications for the project have been developed and test pages scanned using a range of resolutions. This has been a bit of a learning curve as we dived deeply into unfamiliar technical terminology and detail. We wanted to be sure that we had a solid understanding of what would need to be done to ensure long term preservation while maximising access for current researchers. With this complete, and our understanding advanced, we are now selecting a company to do the scanning and OCRing of the Papers. As soon as that selection is made, the original sheets will be packaged and dispatched.

Kathryn Parsons
University of Waikato Library

A Century! Our 100th Content Provider

Thu, 24 Jun 2010 13:34:00 +1200

We are delighted to announce that Kete Selwyn, part of the Aotearoa People's Network Kaharoa, is our 100th content provider.

Kete Selwyn aims to be a vibrant and lively online community that works together to capture the memories and stories of Selwyn District’s past and present. With over 1700 images, audio and descriptions - all Creative Commons licensed - Kete Selwyn is certainly making their community’s content easier to find, share and use!

Glen Walker, Selwyn Libraries Manager, says Kete Selwyn has "...been a labour of love". Glen did some careful budgeting and was able to create a short term position for a knowledgeable and experienced Heritage Librarian.
Heritage Librarian, Lynda Seaton, catalogued cabinets crammed full of "misc" history stuff, and digitised as much as possible as she went.

Glen says they are particularly proud of the Soldiers of Selwyn collections which is the first time that accurate information has been brought together about every single WW1 soldier listed on a memorial in the Selwyn area.
Now working in a different library Lynda is still volunteering her time to add to Kete Selwyn and is about to start WW2 soldiers - which is very exciting!

Kete Selwyn was publicly launched in December 2009 and they are planning on new ways to work with the Selwyn community to add more content.
They have big plans for Heritage Week this year and next ANZAC Day they will promote the Soldiers of Selwyn collection to local schools. It is hoped the schools will use the collection to create new content experiences that will commemorate the sacrifice of their community's soldiers.

Check out Kete Selwyn's results in DigitalNZ Search

Congratulations Kete Selwyn and welcome to DigitalNZ!

Coronation Celebrations at Annat from Kete Selwyn

Launch of Housing Research for New Zealand portal

Thu, 22 Apr 2010 13:19:00 +1200

We’re pleased to have been part of building the new Housing Research for New Zealand online housing research register, now live at http://housingresearch.digitalnz.org.

Digital New Zealand is always excited when other organisations see a use for the solutions we’ve developed to help make New Zealand digital content easier to find, share and use.

Housing Research for New Zealand is a joint venture between a number of Government agencies and other organisations who aim to make all housing research produced in New Zealand available through a single search website.

It’s a big goal, but a worthy one - and a great fit with our aim to increase the amount of New Zealand digital content available online. A large number of public sector agencies in particular both generate and use housing research.

Our part is twofold: we work with Housing Research for New Zealand members to bring their content into the website; and provide the API plug-in that the portal runs on behind the scenes.

The result is a really nice example of inter-agency collaboration and re-use of existing government information (metadata about research publications), not to mention a ‘one stop shop’ for people to freely access topical housing research.

Here’s what you can expect when you go to the site:

Working with some key players in the housing sector has been a great experience for us - and we’re looking forward to collaborating with more organisations to grow the portal.

If you’re interested in making housing research available alongside over 1000 other research papers from New Zealand organisations, just drop as a line at info@digitalnz.org.

Announcing the Make It Digital Award recipients

Wed, 21 Apr 2010 16:04:00 +1200

DigitalNZ is delighted to announce that the Make It Digital Awards will be making some excellent New Zealand content easier to find, share and use.

We put together a top notch panel of judges to take on the unenviable task of selecting the winners from the 10 really interesting entries.

The Make it Digital Judges:

Jock Phillips: Jock is the General Editor of the Te Ara - The Online Encyclopedia of New Zealand. Jock is also the author of A man’s country? and other published works in New Zealand history. He took up the gauntlet to create the world’s first born-digital national encylopedia after many years as a historian teaching at Victoria University of Wellington (where he founded the Stout Research Centre) and serving as Chief Historian for the Ministry for Culture and Heritage.

Adrian Kingston: Adrian is Collections Information Manager Digital Assets and Development at Te Papa, he is responsible for the development of the Museum's Collection Information System, work with thesauri and controlled vocabulary, management of the Museum's digital assets, and getting Te Papa's collections out into the digital world. Adrian has been at Te Papa since 2001, working in the Art and Photography collections before moving into Collections Information in 2004.

Virginia Gow: Virginia is DigitalNZ’s Content Manager. She has worked on digital initiatives at the National Library of New Zealand since 2005, specialising in collaborations across the cultural heritage sector such as Matapihi, and web content development. Prior to working at the National Library, she was a production editor for Te Ara, the online encyclopedia of New Zealand.

The Make it Digital Judges went through rigorous selection process that included:

Checking for basic entry requirements
A pre-selection assessment against the award criteria
Prioritisation using the Make it Digital Scorecard

And the winners are:

Digitisation of Nelson Photo News – Microfilm sub-committee of the Friends of the Nelson Library
Nelson Photo News is a rare and unique photographic record of life in the Nelson region in the early 1960s and 1970s.

Our judges said:
While this is a regional project we believe it will provide access to pictorial content from a period that is not commonly available online. Therefore, it will have significance to New Zealanders and contribute to a wider understanding of our culture and heritage. The project process has been well thought out and you have demonstrated significant community support. We were also pleased to note the discussion and understanding of rights and licensing issues.

Despatches from Down Under: The papers of the New Zealand colony – from and to the British Colonial Office - University of Waikato Library
The British Parliamentary Papers (BPP) are a significant source of important and largely untapped information about New Zealand in the mid-nineteenth century. They comprise reports, correspondence and other documentation sent to the Government in Britain and cover all aspects of life and events in the colony at the time.

Our judges said:
This project has significance to New Zealanders and will contribute to a wider understanding of our culture and heritage. The project process has also been well thought out and demonstrated community support via the Make It Digital voting tool. Judges also noted that this content will complement other digitisation efforts that are in progress.

These two projects will be providing updates and reports through Make it Digital so that we can follow their progress and learn from real life digitisation projects.

We wish all of the Make it Digital entrants the very best with progressing their projects.

DigitalNZ is more than happy to give advice about digitisation projects via Make it Digital. We’d love to hear from you!

Help to map DigitalNZ content – new tool!

Tue, 16 Mar 2010 13:54:00 +1300

Please help us to make content from Digital New Zealand partners more discoverable... Find an item. Check for a location. Put it on the map!

Suggest or view more details

Search Digital New Zealand and you’ll now see a “Suggest or view more details” link as well as the existing links through to the item.

Use this link to see locations already provided and to start adding locations for items that haven’t been placed on the map yet.

Watch how it works

Why are we doing this?

Rich and comprehensive metadata + content = discoverability. But not all of the 1.5 million content items available through DigitalNZ have extensive metadata.

Developers and content users are also seeking ‘place’ or location-based experiences, such as this google-map based application from Paul Hagon.

By putting items on the map in DigitalNZ, you create placenames and geo-coordinates behind the scenes, making the metadata richer and the content just that extra bit more findable.

Some ways to use the tool

Remember to map items that don’t have locations when you find them (if you know the location)
Make a custom search for your favourite topic, click on “Search” with the search box empty to bring up all the items for that search, and work your way through

Let us know other helpful tips or ideas based on your own experiences.

Not working as it should?

Drop us a line at info@digitalnz.org or leave us a comment below – you can ask for help there too.

On behalf of our content providers who want to make their content easier to find, share and use - thanks!

2010 DigitalNZ Roadmap

Wed, 17 Feb 2010 16:41:59 +1300

Since launching DigitalNZ at the end of 2008 we've been operating in 12-week cycles... we'd take several weeks to plan a cycle, complete a 12-week burst of activity, and then plan the next one. It's enabled us to be really responsive, but of late we've felt the need to look further ahead as well. So we're pleased to share with you our roadmap for 2010 (PDF), a four-page outline of what we hope to deliver for the year. We should point out that this is just a guide, it's not fixed, and we will be reviewing it constantly to make sure it stays relevant.

What's guiding our activities this year?

Several things were identified as having a big impact on the direction of DigitalNZ over the year, and it was determined that the follow four points should guide our activity:

Prioritise content for inclusion in DigitalNZ
Emphasise distributed tools over a centralised portal
Balance maintenance with new service development
Maintain the status quo of the Digitisation Advisory Service

Planned activities

With these points in mind, we set out to determine what we'd actually be working on... and here's the summary:

Maintain Make it Digital Guides
10k digitisation grants
Develop Collaborative Digitisation Framework (based on Appendices to the Journals of the House of Representatives project) to assist future digitisation projects
Maintain existing content harvesting and hosted search services
Inclusion of all Papers Past digital newspapers
Inclusion of research outputs and government publications
Inclusion of significant archives or subscription services where digital content is difficult to find
Investigate including significant catalogue and database records for content that is not yet digital
Reconfigure search algorithm and interface to support content scaling
Release open source package and investigate community management options
Reduce number of items with unknown copyright status
Implement user generated metadata services
Investigate new methods of metadata generation
Raise awareness of the Shared Repository solution amongst Crown Research Institutes and other Government Agencies
Investigate developing shared national workflow to support digitisation on demand services
Scale infrastructure to support increased API usage and content growth
Review metadata licensing arrangements and formats to encourage usage (caching, storing, commercial use options)
API and usage reporting dashboard
48 hour Hackathon competition for end user applications
Marketing to focus on distributed tools, APIs, and data reuse
Confirm staffing contracts for FY 2010/11
Update evaluation process

It's a fine balancing act... holding or extending some services, while also exploring a few new areas. We're really excited about the year ahead! Comments, thoughts and suggestions are all welcome on the blog, or drop us a line at info@digitalnz.org.

Download the 2010 DigitalNZ Roadmap (PDF)

$10,000 to kick-start your digitisation project!*

Tue, 01 Dec 2009 16:26:59 +1300

Make it Digital has two awards of up to $10,000 on offer for organisations who have NZ content they want to digitise and make easier to find, share and use.

To enter, organisations need to register their Make it Digital project on makeit.digitalnz.org/voting and then encourage votes and comments from their community along with completing a supporting application to tell us why the project is important.

Find out more.

*Terms and conditions apply

Collaborative digitisation of the Appendices to the Journals of the House of Representatives

Fri, 20 Nov 2009 16:18:00 +1300

The National Library of New Zealand recently announced that it was beginning a project to digitise the Appendices to the Journals of the House of Representatives (AJHRs). The AJHRs are important official records of New Zealand’s social, economic and political history, but they are difficult to access and are not freely available online.

The National Library and DigitalNZ are providing seed funds, totalling $100,000 in 2009/2010, to initiate the digitisation programme. However, the complete digitisation of the AJHRs is a long-term project that will take several years of effort, and will need contributions from many stakeholders to be successful.

The DigitalNZ connection

DigitalNZ has a particular interest in exploring how many organisations can work together to tackle large-scale digitisation projects. There are small and very finite resources available for digitisation projects in New Zealand, and the pooling of resources to reach economies of scale may sometimes be appropriate.

We are working with the National Library to pilot this as a collaborative digitisation project. The project team is already working with Parliamentary Library, with significant contributions from the University of Otago and Hocken Library. A big thank you for your early offers of collaboration!

Get involved

We are still thinking about options for involvement, but possibilities include:

AJHR volumes - Do you have access to volumes of the AJHRs that could be used in the digitisation process?
Existing digital copies - Do you know of any existing digital versions of any AJHR volumes?
Distributed digitisation - Could your organisation digitise some of the volumes using local staff and resources?
Funding - Would your organisation or your customers benefit from online versions of the AJHRs? Does your organisation have funding it can contribute?
Specialist staff - The project team is looking for people with specialist knowledge of image quality assurance. Does someone at your organisation have these skills to contribute?

Want to know more?

If you're interested in getting involved, it would be great to hear from you. For those who are attending the 2009 National Digital Forum in Wellington, you can find out more from Fiona and Gordon at the DigitalNZ and National Library stands. Alternatively please drop us a line at info@digitalnz.org.

View unique NZ aerial photos from V.C. Browne & Son

Fri, 13 Nov 2009 10:04:00 +1300

To make hard-to-find New Zealand content easier to ‘find, share, and use’, you have to know that the content has been digitised. Herein lies the paradox of Digital New Zealand.

That’s why the following note from Roger Barclay made our day:

"I have read with interest your comments about making NZ content easier to find. I have recently launched a website that provides access to V.C. Browne & Son's aerial photograph collection and it sounds like this site fits well with what you are trying to achieve. How do we go about getting this content reflected in your search facilities?"

The V.C. Browne & Son Collection

The V. C. Browne and Son NZ Aerial Photograph Collection contains approximately 26,000 images – and you can now discover the digitised ones through Digital New Zealand.

Victor Carlisle (V.C.) Browne was one of New Zealand’s pioneering aerial photographers. His son William (Bill) took over his Aerial Photograph business in the 1970s.

The collection spans a period of over 50 years, from the 1930s to the late 1970s. Most of the images are black and white aerial photographs, but there are some portraits and architectural works as well.

The photographs mainly focus on the South Island, particularly Canterbury where the Brownes were based. But most New Zealand cities and major towns are also imaged.

One of the great things about the collection is that it’s in sequence (Roll No/Photo No) – so browsing the collection is like looking out the window of your own plane onto the intriguing landscape of an older New Zealand.

Our favourite image

Roger notes that he is still doing work to improve the metadata about the images and identify the contents – it’s a big job.

Nevertheless one of our favourites is this photograph, which by our counts is right on top of where DigitalNZ is located, inside the National Library of New Zealand.

[Probably Wellington somewhere (PB0615/5): Copyright V.C. Browne & Son, used with permission]

Here's what the same area looks like today (click & hold mouse on map to move):

View Larger Map

The big square building in the centre on Aitken Street (not there in the earlier view) is the National Library, where we are.

Adding value to the collection

Roger Barclay has been steadily scanning and creating metadata to increase access to the Browne family photograph collection since the early 2000s. You can read the background to the project on his website.

To bring this content into the Digital New Zealand search experience, we asked him to provide us with a single (static) URL for each content item – previously you couldn’t link directly to a large image and metadata; which also made it hard to share links to the images with other people.

Now that the site has this new structure, Digital New Zealand can more easily collect information about these unique pictorial representations of the country.

In addition, Google will also be able to find its way to the images; bringing the content to the eyes of even more interested people.

More please

If Roger hadn’t contacted us, it might have been a while before we found out about the V.C. Browne & Son collection. We’re glad he did (and thanks to the DigitalNZ content provider who referred him on).

Do you have a digital collection of New Zealand-related content we don’t know about?

We’re here to help people find it through the DigitalNZ discovery system, including the developer APIs that enable links to your content to appear in other relevant contexts such as Te Papa’s collections online.

Drop us a line if you’d like to bring your content into view.

You can find out more about being part of DigitalNZ here

Or get advice about digitising your family collections on Make it Digital

Seeking Make it Digital Scorecard users

Thu, 12 Nov 2009 10:52:00 +1300

Has your organisation used (or thought about using) the DigitalNZ Make it Digital Scorecard to help with digitisation decision making? The Scorecard is a decision-making tool to help you select and prioritise content to digitise for increased access. We're looking for examples of organisations who have used this tool so we can produce a case study. If that's you, let us know at info@digitalnz.org.

If you have decided to use the tool but are still uncertain about what is involved, then please get in touch as well. We can offer you a free workshop if in exchange we can use your experience for our case study. We'll give you and your colleagues free training and advice on how to use the Scorecard and write a selection policy. You'll also get promotion for your organisation through the case study we produce for our website. Contact us for more information at info@digitalnz.org.

DigitalNZ API HackFest - Auckland - 7 November 2009

Sat, 10 Oct 2009 12:44:00 +1300

The DigitalNZ API HackFest is making its way to Auckland, thanks to Telecom for offering a venue and WiFi!

10am-5pm, Saturday 7 November 2009
Telecom, Hereford St, Auckland

DigitalNZ is a publicly funded, not-for-profit initiative that aims to make NZ digital content easy to find, share and use. DigitalNZ Search aggregates information about NZ digital content from more than 65 content providers and makes them available via an API. You can get a glimpse of what’s inside at search.digitalnz.org

Come along and

Make nice stuff using the DigitalNZ API
Whip up some code samples to help other developers kick-start their own DigitalNZ API creations
Consume pizza, beer and sugary confections

Here are just a few examples of what you could get up to:

Image analysis of thumbnails and search by colour
Build an advanced search based on DigitalNZ facets
Example mashup with Eventfinder
Road-test the Ruby Gem
Extend the DigitalNZ Timeline app from MIT's Simile project

We're not just looking for developers. Other people we'd also love to come along are designers who can help make the apps look shiny or people with great ideas for apps.

Bring yourself and your laptop. We'll provide the WiFi. This is a low-key event - feel free to come along for however long you can, even just a couple of hours.

Numbers are limited, so please RSVP to jo.eaton AT natlib.govt.nz

Link to DigitalNZ search

Tue, 06 Oct 2009 17:57:00 +1300

Do you run a website where people come to find stuff (information, resources, images, research...) relating to New Zealand? Please consider linking to us!

We help users to find New Zealand digital content. But we also need New Zealand websites to help users find our search tools.

Some options to consider:

Link to http://search.digitalnz.org

Connect your users to 1.3 million+ unique New Zealand content items from all kinds of organisations and online sources at http://search.digitalnz.org.

View all results currently in the search system

View list of content sources indexed

You can also link to specific searches by copying and pasting a search string such as: http://search.digitalnz.org/en/search?search_text=rugby&commit=Search

Embed a DigitalNZ search widget

Add a topical or general search widget to your website or blog.

Use a simple tool to customise widget to find just content you choose

Check out this blog to see an example widget.

Integrate DigitalNZ search results into your site

Add value to your search by calling related DigitalNZ results (all, or selected) when users search your site, using our open API.

Support documentation for site developers

Examples of other organisations who have done this

If you need to find out more, just drop us a line at info@digitalnz.org or leave a comment below.

Thank you for helping us to make New Zealand digital content easier to find, share and use!

DigitalNZ get-together

Tue, 06 Oct 2009 13:32:28 +1300

We're holding a DigitalNZ get-together in Wellington, on Sunday the 22nd of November, starting at 11am. This is the day before the National Digital Forum conference and we hope that those attending the conference could arrive in Wellington the day before.

Update: Come and join us at the Lower Ground Conference Room, National Library of NZ, Corner of Molesworth & Aitken Streets, Wellington

It will be a mix of info sharing and mini working sessions focused on how we might take DigitalNZ to the next level, with a bit of socialising thrown in afterward for good measure. The idea is that we want to really start ramping things up here at DigitalNZ, but we're only six people. We need your input, your ideas and your critique.

Drop us a line at info@digitalnz.org if you're thinking of coming along. We hope to see and hear from you soon!
Andy, Dan, Jo, Lewis, Fiona and Virginia

What we're working on: DigitalNZ plans for Wave 4

Mon, 21 Sep 2009 20:27:00 +1200

In making this post I feel like DigitalNZ has reached a milestone. For the first time we are releasing our 12-week plan of work. Going forward, expect to hear a lot more detail about how we see the DigitalNZ services evolving. And take it as read that we want to hear your comment and feedback. The good, the bad, the ugly. Let us know whether you think we're hitting the mark or not. We'll keep the info fairly lightweight because... well, we don't like long documents, but please shout out if something needs clarification.

Before we jump into it though, it may be helpful to understand a little more about how we manage ourselves. The DigitalNZ work programme is focused around 12-week cycles that are referred to as Waves (don't ask me why we use Waves... I don't know!). We've just started Wave 4 which runs from September 7 – November 27 2009. DigitalNZ has two project teams that run in parallel—a business team, and a technical team. Both teams rally around the deliverables below, with the business team providing the day-to-day direction for the technical team. During the Wave planning process we set some of our deliverables as stretch goals that are hard to deliver, and we also set priority levels to stop us focusing on the easy pickings.

Wave 4 briefing

Whereas previous Waves have seen the rapid development of new services and features, the plan for Wave 4 is characterised by an emphasis on consolidation and bug fixing. With a growing base of customers, it is important we maintain a certain level of service and quality in order to sustain and build value. The general message is that we want to make a success of the current solutions before rushing off to try something new.

The Make it Digital Helpdesk was only released in Wave 3, and so the priority here is on continuing to increase user activity and engaging the community with good practice examples.

DigitalNZ Search on the other hand has been up-and-running since Wave 1, and the priority here is on reconfiguring the infrastructure to improve reliability and scalability. We’ve identified the education sector as the most effective target for utilising these services in the short-term, and that’s where we are focusing our marketing. On the content sharing side we are putting the emphasis on the quality and breadth of digital content, as opposed to pushing for large number increases in the holdings.

Support of the API data sharing service for developers and producers continues to be a priority because that is where we see the greatest opportunity for creating long-term value. Similarly, the more we see services and repositories integrating the DigitalNZ API, the more we can demonstrate our value in connecting New Zealanders to digital content.

The Shared Repository and the Hosted Search services that were newly developed during Wave 3 will be released during Wave 4, and the focus here will be on supporting new customers to make the most of these offerings; particularly in the research sector. By its very nature, DigitalNZ is supporting the accessibility of government information, and during this wave we will also investigate how we might support efforts to open up access to publicly held data.

Wave 4 Deliverables

A. Make it Digital Helpdesk

A1. Bug fixing for Make it Digital site

Priority: 1 Clear all bugs for Make it Digital at priority levels normal, high, and critical.

A2. Increase user activity on Make it Digital by 25%

Priority: 1 Build user activity on Make it Digital by running campaigns for communities of digital content creators, teachers, and relevant professional associations. Marketing activities to include direct marketing, sponsorship of the National Digital Forum event, and the building of relationships with NZ digital experts.

A3. Publish fully updated Make it Digital Guides

Priority: 2 Publish all remaining draft guides—managing digital content, discovering digital content, preserving digital content. As well as publishing any remaining sections from within existing guides.

B. Digitisation Consultancy

B1. Ensure at least one digitisation project nominated in Make it Digital is initiated

Priority: 1 Select at least one of the ideas nominated on the Make it Digital site and support the community to initiate a digitisation project. The project would follow good practice approaches identified in the Make it Digital Guides and would seek to encourage the start up of other digitisation projects.

B2. Publish case study about the Selection Scorecard

Priority: 2 Confirm use cases of the Make it Digital Selection Scorecard and publish a case study to provide a practical example of how the tool can be used in a New Zealand context.

C. Content Sharing

C1. Bug fixing for DigitalNZ Search

Priority: 1 Clear all bugs for DigitalNZ Search at priority levels normal, high, and critical.

C2. Increase user activity on DigitalNZ Search by 100%

Priority: 1 Build user activity across DigitalNZ Search by running campaigns to encourage use by teachers in the classroom. Marketing activity to include direct marketing, speaking at relevant conferences and events, and the generation of publicity. Search engine marketing, the development of new API applications and integrations (F1) and the release of the Hosted Search upgrade (E2) will also contribute to this deliverable.

C3. Consolidate infrastructure & business processes for DigitalNZ Search

Priority: 2 Improve the performance and scalability of DigitalNZ Search by reconfiguring aspects of the infrastructure. Audit business processes for improvements to harvesting workflow and update processes. Audit technical documentation for completeness.

C4. Increase number of content items and sources in DigitalNZ Search by 25%

Priority: 3 Grow the breadth of content available through DigitalNZ Search by filling gaps in categories, targeting hard to surface content, and reaching out to international sources.

D. Metadata Enhancement

D1. Enhance metadata quality of at least one facet

Priority: 1 Focus on improving the quality of location data, to support geospatial applications. Integrate placename / geotag cleanup tool from Archives New Zealand, and release functionality to allow users to make location suggestions.

E. Content Hosting

E1. Release Shared Repository

Priority: 1 Work with MoRST to release the shared research repository to CRIs and government agencies; provide training to users; and gather feedback for enhancements.

E2. Release Hosted Search upgrade

Priority: 3 Release the upgraded Hosted Search service that was developed for Matapihi, and make it available to other groups who need a fully branded search across NZ digital content. Release an instance for the NZ Housing Research consortium as another example of the service.

F. Developer Support

F1. Support 15 API applications or integrations

Priority: 1 Encourage the development of 15 applications or system integrations that use the DigitalNZ API to deliver NZ content discovery, interaction or reuse features. Market the API to content providers and NZ web development companies, and run a third HackFest.

F2. Investigate DigitalNZ support of Open Government initiatives

Priority: 1 Investigate options for supporting access to government data, and report back to the DigitalNZ Steering Group. Consultation will take place with various Government Agencies, as well as with other public initiatives.

F3. Refine functionality of APIs to support reuse

Priority: 2 Review developer feedback and make enhancements to the APIs to improve their functionality and usability.

G. Supporting activities

Marketing and community management
Technical support
Programme management

Conservation content goes live

Thu, 17 Sep 2009 19:30:00 +1200

Just in time for Conservation week (13-20 September), we are pleased to welcome the Department of Conservation as our newest DigitalNZ content provider.

DoC have started the ball rolling with around 2000 'meaty' pages of content from http://www.doc.govt.nz/ - including place, track, and species information; historic sites; Meet the Locals videos; and publications.

Try a search for conservation to see the range of content DoC has provided, along with examples from our other content providers; or hop onto the custom search builder to make a search for your own blog or website, on a conservation topic of your choice.

Weta search, anyone?

HackFest: the Christchurch edition

Tue, 15 Sep 2009 15:32:00 +1200

We decided to take the DigitalNZ API HackFest on the road to Christchurch.

We started at 10am and fueled ourselves with chocolate, chippies and cola. By lunchtime, Peter W had a working version of a DigitalNZ iPhone search app going!

After a lunch of pizza and some great conversation, the coding began again.

Late in the afternoon, we cracked open some beer and talked about DigitalNZ. By the end of the day, in addition to Peter's iPhone app, Jonathan built a DigitalNZ search Drupal module and Seth made a search plugin for FireFox.

Thanks to the lovely coders who spent a sunny Saturday indoors with us! We hope to see you all again.

I would also really like to thank the wonderful Natalia of PowerHouse/Cii for being so helpful and of course PowerHouse/Cii themselves for letting us use the venue for free.

Here's Follower's account of the day.

Beating the brittle with digital

Wed, 09 Sep 2009 12:03:00 +1200

In the darkened shelves or back rooms of many libraries sit a set of old nineteenth century law books that are seldom allowed into the public eye. Although New Zealand's paper stocks have generally been of a better quality than in the U.S., for a period through the 1880s and 1890s our law statutes were printed on low quality acidic paper. Today these are so brittle that just turning the pages of many editions causes them to crumble. They hit the headlines in 2004 after an Auckland District Law Society article led to them being nicknamed the "shattering statutes", and it wasn't long before digitisation was being promoted as a solution.

Law News article, Auckland District Law Society, February 2004

For a time it was unclear who was responsible for maintaining these law books. The National Library and Alexander Turnbull Library had dealt to some extent with the preservation issues by keeping sets in climate controlled environments, but these were difficult to access. Other libraries around the country were finding it hard to maintain undamaged copies, yet did not have the resources to digitise them. As a result, the number of available copies was rapidly dwindling. Enter the Parliamentary Counsel Office, the agency responsible for drafting and publishing New Zealand Acts of Parliament and statutory regulations.

The Parliamentary Counsel Office as far back as 2000 had established the Public Access to Legislation (PAL) project. Following the sale of Government Print in the late 1990s, PAL was intended to ensure that public access to legislation could still be maintained and improved by the government through electronic provision. The Parliamentary Counsel Office also has a responsibility for continued provision of official reprinted paper volumes, but not historical legislation. The shattering statutes were recognised as a special case and natural candidates for reprinting. But in order to reprint them digital copies had to be made as a first step. This aligned nicely with the efforts to get current legislation accessible online.

An assessment by the National Library identified that the years 1888 to 1894 were at the highest risk of loss due to acidity. These Acts include significant land legislation of interest to the Waitangi Tribunal, as well as historic laws such as the 1893 Electoral Act giving New Zealand women the vote, the 1894 Middle District of New Zealand University College Act, which established a university in Wellington, and the 1894 Destitute Persons Act, which among other things made fathers responsible for the maintenance of their legitimate and illegitimate children.

Given their brittle condition, there was no easy way to copy individual editions for reprinting. The National Library stepped in to microfilm the legislation, which enabled the creation of a preservation copy as well as a source for digital images to be made. It wasn't all straightforward though. Law library practice is to annotate law books with inserts as law changes are made, rather than reprint and rebind an entire Act each time. Finding unannotated original versions of statutes this old for copying took some effort, and some even had to have their annotations carefully removed by hand to get a clean copy.

In order to get a satisfactory printed result, missing detail on the pages was restored manually on the digital versions. Column line breaks and brackets were re-done, while the Crown crests were scanned at a higher quality greyscale in order to preserve detail. Unlike some similar efforts overseas, care was taken to preserve marginalia notes, making them useful for historical as well as legal reference. This meant page by page checking and correcting to get the quality desired for reprinting.

Digital scans of the microfilms were output to PDF for printing, with the PDFs being compressed for use on the web. Because the original aim was a reprint, the scans were not processed for OCR (Optical Character Recognition), meaning the current web versions are not searchable. However, each Act is indexed by name and year on the Knowledge Basket website where they are being hosted. It is hoped that fully accessible and searchable versions can be created as a next step.

The shattering statutes are an interesting example of the need to select content carefully when resources are scarce, one of the purposes of our Make it Digital scorecard. The Parliamentary Counsel Office clearly identified that their goal was to produce a faithful facsimile representation of the statutes sufficient to be accessed as an alternative to the originals preserved by the Alexander Turnbull Library. The project was a priority due to the scarcity of copies in libraries, the difficulty accessing the paper versions without damage, and the continued demand by lawyers and historians for access. Because of their condition, microfilming (which avoids contact with the pages) followed by scanning of the film was the best digitisation approach.

How access to the remaining body of historical legislation is dealt with is yet to be answered. In part this will be subject to the response to recommendations of the Law Commission when the government considers them later in the year. Further information about the project and links to the Law Commission's report can be found on the Parliamentary Counsel Office's website.

How does the recipe search powered by DigitalNZ work?

Wed, 19 Aug 2009 08:41:00 +1200

The DigitalNZ recipe search is made using our free search builder application. It helps you find content related to the Bake it Digital campaign.

Making a search tool is really easy - you can make one on any topic you like, and bring up results from any Digital New Zealand content sources.

Making a search

Here's how we used the tool to make the recipe search. We:

1) Decided to just include certain types of content: Images, Magazines, News and Newspapers

2) Added our keywords, using some Boolean magic.

Boolean lets you combine terms using logic such as ‘AND', ‘OR' and ‘NOT' to get just the results you want included in your search. For example:

Fire AND truck to get just fire trucks
Fire NOT truck to get just fire
Fire OR truck to get fires and trucks

Here's our current Boolean search string: recipe OR #nzbakeit OR nzbakeit OR id:1220996 OR id:1220998 OR id:1220999 OR id:1221000 OR id:1221001 OR id:1221002 OR id:1221003 OR id:1221004 NOT id:197862 OR id:197812

You can see we're deliberately filtering out some dud results (such as chicken cartoons) using NOT, and trying to include others we particularly want. Drop us a line in the comments if you want to know how to cherry pick like this.

Using OR means we get a bigger search pool - content is either described using the word 'recipe' OR has the id:1220996, for example. If we didn't add anything in the Keywords, we'd get everything from the content sources we chose.

3) Chose some particular content sources we knew had recipes - our Flickr group members, Kete friends (such as Kete Horowhenua, and people using the Kete DigitalNZ), and some other institutions.

4) Styled it up and added a logo, and then pressed save. Easy!

The result

And here's the final recipe search powered by DigitalNZ. If you click Search without anything in the search box, you can browse all the recipes currently included.

Can't find the recipe you're looking for?

There are likely two reasons:

the content doesn't exist... so you need to add it as a recipe or get one of your friends to. You might even win a prize.
the content exists but isn't described well. If you don't add good descriptive information and tags to content (otherwise known as metadata), people won't find it.

The nice thing about Flickr and the Kete is you can go in and tag other people's recipes yourself so they can be found. One day we'll let you do this inside Digital New Zealand too.

Happy searching and baking!

What's in a (file)name?

Mon, 10 Aug 2009 17:20:00 +1200

In the news recently was the revelation that researchers in Japan are working on a digital 'Rosetta stone' to back up your digital data for 1000 years. If all that content however is poorly named and organised, the nightmare won't be hardware failure but the inability to find anything - and it won't take 1000 years to be a problem. It has been predicted that by 2011 over 60 billion digital photos will be taken annually. That's a lot of content.

The moment you have more items of digital content than can be scanned with the eye in a few seconds, discovery becomes limited by how well your digital content is organised. One of the easiest practices for keeping content usable and accessible is having a consistent naming scheme for filenames and folders. Yet like many good habits, it's one of the most common practices people fail to follow.

Arguably it all started around the introduction of Windows 95. For the first time in many computer users' experience, long file names up to 255 characters meant you could let rip with what you named digital files. 'SMITHLET.DOC' could become 'Letter to J Smith re postage costs.doc'. Microsoft Office 95 even helpfully suggested you name your new file 'Document1' and your new folders 'New Folder', a pattern carried on with digital scanners and cameras.

As digital files such as MP3s grew in popularity, many software programmes took advantage of long file names and applied default names using conventions such as 'Track#_Trackname_Artist_Albumname'. On the face of it, this seems like a really good naming convention, until you get to a track such as Pink Floyd's 'Several Species of Small Furry Animals Gathered Together in a Cave and Grooving with a Pict'. Bury this a few folders into a directory tree and you will get some software, including some CD-burning software, that simply can't read the file without an error. For that software, the 255 character limit includes the folder path of the file, not just the filename.

There are three rules of thumb to ensuring you content is organised consistently. If you follow them, you will be taking a significant step to keeping your digital content discoverable over time.

Rule 1: Have a simple and unique name for every file

It can take a bit of thinking about, but devising a system that will create unique names for each file you create while keeping that name simple, will help avoid accidental deletion or overwriting. Filenames with more than 30 characters and obscure abbreviations make it hard to order the content over time, so less complexity is better. Special characters, dashes or extra dots can create compatibility problems, so stay with the basic alphanumeric characters, underscores and one three letter extension.

If date of creation is important to the content or its sequence, this can be usefully included as part of a filename. Where the date will be used to order records in sequence, starting the filename with year, then month, then day, will enable sorting chronologically and avoid confusion between day and month order. In many cases record order is important, so a serial number or lettering can be included, allowing enough leading zeros to (e.g. 001) to capture the entire series. Some other basic descriptive metadata can also be added (such as file origin, type or subject e.g.'film','scan', or 'drama'), while large projects may require an item ID. A digital photograph can be labelled 'YYYYMMDD_type_sequence' e.g. 20090801_dphoto_281.tif.

It's worth considering including folder names as part of a filename. This helps ensure files can be linked back to their origin in the directory, and can help ensure uniqueness where dates as descriptors are not useful. Pink Floyd's track could be labelled 'Track#_Artist_Foldername' or 'Track#_Foldername' where the foldername identifies the album artist and number e.g. 07_PinkFloyd_Ummagumma.wav or 07_PinkFloyd1969_01.wav.

Rule 2: Make use of folder structures to keep your content organised

Although it's not always possible, using a controlled folder structure for your file storage improves organisation and aids back-up and future migration. Like any filing system, design your folder structure to allow your content to expand over time without duplicating descriptions or nesting folders too deeply. Aim for a maximum of five folder levels from your drive letter to reduce compatibility problems and aid navigation.

The same kinds of unique naming approaches for filenames can be used for folders. If you create different versions or derivatives of files, you may need a folder system that allows you to distinguish original 'master' versions from altered versions. It may be useful to pair folders as master and copy, or keep them in completely separate structures to avoid accidental alteration. Both the master and the copy need to be backed-up.

Even if you mainly use a software library or database to navigate through your content, a good folder structure will help minimise accidental loss over time and as software systems change.

Rule 3: Embed important metadata into the file

With your filenames and folders perfected, don't rely on those names as important metadata for your digital content. Names can be changed by other users and timestamps can be altered just by copying folders or opening the file. The best way of describing your content is to embed the metadata into the file itself using an open and accepted metadata format. Many digital files, such as PDFs, TIFFs and JPGs allow the embedding of metadata in the file header, while other files such as MPEG4 and WAV can be placed in 'wrappers' that carry metadata. Embedded metadata allows you to verify the contents of a file even if the filename or folder location has changed. If you are unable to embed metadata, then it is important to have a metadata record that links your unique file and folder structure to a description of what the file contains. If you don't have software to help you do this, you can save a plain text file in each folder of content listing what the contents are.

Digital technology is accused of creating many risks, including the risk of rapid loss or obsolescence of any content stored in digital form. While this may have been a very real risk in earlier years, today we have available to us the open standards and the knowledge to manage that risk to an acceptable degree. It's up to us to apply them in a way that will work for the future, whatever technology systems may eventuate.

DigitalNZ HackFest - Christchurch

Mon, 10 Aug 2009 15:24:00 +1200

The first DigitalNZ HackFest in Christchurch is coming up soon. We're running it from Cii/powerHouse, thanks to their kind sponsorship!

10am-5pm, Saturday 12 September 2009
Canterbury Innovation Incubator/powerHouse
200 Armagh St, Christchurch

Come along and

Make nice stuff using the DigitalNZ API
Whip up some code samples to help other developers kick-start their own DigitalNZ API creations
Consume pizza, beer and sugary confections

DigitalNZ Search aggregates information about NZ digital content from more than 40 content providers and makes them available via an API. You can get a glimpse of what’s inside at search.digitalnz.org

Here are just a few examples of what you could get up to:

Image analysis of thumbnails and search by colour
Build an advanced search based on DigitalNZ facets
Example mashup with Eventfinder
Road-test the Ruby Gem
Extend the DigitalNZ Timeline app from MIT's Simile project

Bring yourself and your laptop. We'll provide the WiFi. This is a low-key event - feel free to come along for however long you can, even just a couple of hours.

RSVP to jo.eaton AT natlib.govt.nz

Help! Cultural institutions meet Wikimedians - Part II

Wed, 29 Jul 2009 09:25:00 +1200

Just over a week until GLAM-WIKI in Canberra on the 6-7 August, a workshop on how cultural institutions can work together with the Wikimedia community (and vice versa). Can you help?

In preparation for GLAM-WIKI, participants have been asked to answer a series of questions – and I thought the Galleries, Libraries, Archives and Museums within the DigitalNZ community might like to take a collaborative stab at them.

These questions are more specific than the general thoughts I asked for in my first post on cultural institutions meeting wikimedians.

So if you’re willing, please post your answers in the comments below. You don’t have to post the actual numbers, particularly to part C (you could just say “yes” or “no”) – but if you want to, go for it! Anonymity is fine.

If you don’t feel comfortable sharing this information publicly, perhaps just consider the questions within your organisation and team.

A - In preparation for attending GLAM-WIKI it would be very helpful if you could ask the web-manager at your institution what percentage of inbound traffic to your website comes from people clicking through via Wikipedia or other Wikimedia projects. What do you expect this percentage to be and are you surprised by the answer?

B - Please think of a topic that is integral to your institution (e.g. for the Australian War Memorial this might be "Australian military history") and type it into Google. Does a Wikipedia article appear in the first page of the results? Does it appear above your own institution's website? Does your institution have a mission statement that includes a phrase along the lines of "The [institution] is dedicated to helping people of all ages learn about [aforementioned topic]"? If so, has your institution previously investigated trying to fulfil this mission with Wikipedia as a one of the 'strings to its bow'?

C - Are you able to roughly calculate the financial cost to your institution of administering copyright/access to your institution's own and/or controlled and/or out-of-copyright content? Are you also able to roughly calculate the amount your institution makes in revenue as a result of this administration (for example, through content sales)? Is there a significant positive difference between these two numbers

I’m really interested to see what kind of collective picture we form. Aren’t you?

DigitalNZ Search - Te Wiki o te Reo M?ori

Mon, 27 Jul 2009 12:25:59 +1200

Just in time for M?ori Language Week, the DigitalNZ search interface is now also available in Te Reo M?ori.

This bilingual interface also extends to all customised searches - new and existing.

DigitalNZ APIs in action

Mon, 27 Jul 2009 10:03:00 +1200

One of the best ways to explain the benefits of having an open API (Application Programming Interface) to data in the DigitalNZ discovery system is to show it in use.

What’s an API? In short, it’s a way for software applications to ‘talk to’ each other, and a way for developers to ‘talk to’ applications. We use an API to share data with other applications.

The Indicommons website summarises it like this:

"Open APIs allow services and collections to become interconnected, the experience of outside developers to be engaged, and new tools and spaces to be fashioned to benefit the community at large."

Over the past couple of weeks we’ve seen two new examples that really demonstrate the joy of a ‘joined up’ web of data.

Both Te Papa and Auckland City Libraries have recently launched systems that draw links to content from other organisations into their local search experiences. Take a look below to see the DigitalNZ API in action.

Te Papa Collections Online

http://collections.tepapa.govt.nz/

Try a search for something like kiwi and check out the right hand panel. If you can’t find what you’re looking for in the Te Papa collections, maybe it’s available from another DigitalNZ content provider.

If you’ve got questions about how they did this, drop them a line on the Te Papa blog.

Chinese Digital Community

http://chinesecommunity.org.nz/

This is a really neat application of the new External Search Sources Rails Engine developed by Katipo Communications for Kete and other Ruby on Rails applications. It uses an RSS feed to draw results in from other sites, including the DigitalNZ search system via our API.

Seeing tools like these coming to light is fantastic, and we look forward to seeing more examples of developers working with the DigitalNZ API to make NZ content easier to find share and use.

Let us know if you’ve got something in the pipeline too.

Bake it Digital NZ, and be in to win

Thu, 23 Jul 2009 12:03:00 +1200

* UPDATED * Congratulations Cle0patra - winner of the prize draw! If you want to keep uploading recipes, no problem. They will still appear in our special recipe search.

****

Create a digital version of a handwritten or personally typed recipe with a New Zealand connection and be in to win!

As you may have seen from our various tweets and posts, last week DigitalNZ baked. Analogue is NOT AS HARD AS IT SEEMS. We did good.

In fact, we did better than good. We won a small prize. Now we want to give that PRIZE to one of you.

INGREDIENTS

Handwritten or type-written recipes that can be digitised
A digital camera or scanner to produce an image of the recipe
A way to upload the recipe to the DigitalNZ Kete or flickr so it appears in the recipe search powered by DigitalNZ
Optional transcription of the recipe so it can be text searched

INSTRUCTIONS

Find an old handwritten or type-written recipe. Maybe you want to join forces with your mum, dad, or nan?
Consider copyright and rights options* (we don't want any illegally copied recipes) - ask us if you're not sure about rights
Make it digital (if you need help or advice on how to do that, use our Make it Digital helpdesk and post a question)
Upload your scan or image to the DigitalNZ Kete or flickr group, license and describe it
Add the tag #nzbakeit if you want it to appear in our special recipe feed
Keep an eye on the DigitalNZ Make it Digital blog or twitter feed for new developments.

* Interestingly, only the instructions and illustrations on recipes are covered by copyright. The list of ingredients are copyright-free.

If you are already a DigitalNZ content provider and want to participate, just add the #nzbakeit tag to items in your existing feed.

COOKING TIME

Competition extended!
The ability to win a prize in Bake it Digital will be open until Friday 14 August 4 September 2009.

A prize will be awarded to one lucky digital recipe creator – randomly drawn -- and announced on Monday 17 August 7 September 2009. The winner will be contacted by email (as listed in their user sign up).

If you want to keep uploading recipes after that, no problem! They will still appear in our special DigitalNZ recipe search if they have the tag.

Cultural institutions meet Wikimedians

Tue, 21 Jul 2009 15:45:00 +1200

In a couple of weeks I’m heading across the Tasman ditch to attend GLAM-WIKI in Canberra on the 6-7 August, and I have some questions for you.

According to the latest message from the event organisers, this will be “the first time anywhere in the world where Wikimedians and the cultural sector will come together to talk with rather than about each other”. Exciting.

(For those of you not aware, Wikimedians are people involved in online collaborative wiki projects such as Wikipedia, Wikimedia Commons, and other projects provided by the Wikimedia Foundation.)

Together we’ll be heading for “a pair of guidelines - one for the Wikimedia community and one for the cultural institution - each containing specific, practical measures to enable the achievement of better online public access to cultural heritage. Whilst underlining the unique nature of each institution's role these guidelines would demonstrate a shared response to the issues raised at the workshop.”

Its certainly timely as cultural institutions worldwide follow the dispute between the National Portrait Gallery in London and Wikimedia Commons, well blogged about by Paul over at peoplepoints.co.nz.

I can’t speak on behalf of all New Zealand cultural institutions (though I’ve worked with a lot of you) but I’m definitely willing to take what you’d like to give me, and to feed back the outcomes of the workshop if you’re not attending.

If you work in the NZ ‘GLAM’ (Galleries, Libraries, Archives, Museum) sector, what are your views on why and how cultural institutions could work with Wikimedia projects?

What do you think Wikimedia needs to do to make collaboration easier and more effective?

Initiatives such as Wikipedia Loves Art or Backstage Pass at the Powerhouse Museum or the German Federal Archives crowdsourcing cataloguing via Wikimedia Commons highlight both questions and possibilities. There must be more.

More specific Q & A to follow soon [update: see questions here ], but in the meantime what are your thoughts on the above?

Image of National Library reading room on Wikimedia Commons by Dick Bos

The great bake-off

Mon, 20 Jul 2009 16:03:01 +1200

Last week, team members from DigitalNZ got together and spent a morning making and decorating gingerbread librarians.

These goodies were our soldiers in an epic battle against another team to win one of three bake-off titles. Victory was ours - DigitalNZ won the 'Best Take on a Classic' category for the gingerbread Gingerbrarians.

The whole thing has inspired something we're calling Bake it Digital.

The moon landing

Mon, 20 Jul 2009 13:24:00 +1200

The first manned moon landing was 40 years ago today.

You can search DigitalNZ for moon-related stuff here:

Search: The Moon

Here's the best thing I've found with it:

Mr J.H. Berry plotting the course of Apollo 11 carrying Neil Armstrong, Edwin Aldrin & Michael Collins. Fri. July 18, 1969 - Horowhenua Historical Society.

Whose books? Google's books.

Fri, 17 Jul 2009 16:39:00 +1200

In a little over six weeks time the doors will close on all authors and publishers wanting to opt out of the Google books settlement, the US-based class action lawsuit. The settlement, once finalised, will provide one-time compensation from Google in exchange for a licence for Google to use and sell every in-copyright book they have digitised, whether the authors are known or not.

Photo credit: Ruben Vermeersch, Flickr

Unlike the efforts for mass digitisation of public domain books by Microsoft and Yahoo in partnership with the Internet Archive, Google partnered with 20 mainly university libraries in the US and Europe to digitise both in copyright and public domain books. Google used US law to claim a ‘fair use’ right to digitise copyrighted books in their entirety and store them on their servers, making them discoverable only as ‘snippet’ views in their Google Books search. The American Authors Guild and the American Association of Publishers both started class action lawsuits against Google for breach of the US Copyright Act. After long negotiations and no admission of guilt by Google, the parties have arrived at an out of court settlement last October. The deal will effectively licence every in copyright book so far scanned to Google, to use in search results, on-licence to US public libraries and research institutions, and sell digital versions of to the public. The settlement has still to go through several stages before being final, including an anti-trust investigation by the U.S. Department of Justice.

We’ve had a look at what effect this deal might have on New Zealand books, and have come up with a few insights – the good, the bad and the ugly.

The Good

New Zealand authors and publishers are covered by the terms of settlement if they have books protected by US copyright. This extends beyond just those books sold and distributed in the US, to any book available in the US – so for instance, imported by Google’s partner libraries to become part of their collection. New Zealand authors and publishers who want to be part of the settlement and have had their books digitised can lodge a claim for compensation. Copyright Licensing Limited is New Zealand’s main agent in ensuring authors and publishers can make their claims.

The deal overall gives Google the ability to expose the content of their digitised books in search results as snippets, potentially improving digital access enormously for out of print books. This complements their current commercial deals with publishers of in print books.

The Bad

The Google books settlement only provides full book access to libraries, institutions and consumers in the United States. Google Book’s current practice is to block access to US public domain books that they consider might still be in copyright in other countries. This practice will now cover all in copyright books in the US that Google has digitised, meaning they will not be available in New Zealand for the foreseeable future. That’s why you won’t find a copy of Robin Hyde’s The Godwits Fly, or even The Story of a New Zealand River by Jane Mander in Google Books today, despite them both being out of copyright in New Zealand.

It is unlikely Google will undertake a mass digitisation programme for books in New Zealand, as New Zealand’s fair dealing provisions under our Copyright Act are not as open to interpretation as Google claims they are in the US.

The Ugly

Google’s deal has raised many concerns about its potential monopoly price-setting implications, and the precedent it sets for the treatment of out of print orphan works.

The settlement enables Google to receive the most favourable terms for the digital versions of the out of print books licensed to them. That means, for instance, that partner universities cannot provide their digital copies to competitors to licence for sale at a lower price.

The opt-out requirements of the class action effectively licences Google to sell in the US any out of print book where the author or publisher cannot be found. This can provide Google with a significant advantage as a sole publisher and digitiser of these books, given ‘ordinary’ publishers, libraries and others are prevented by the Copyright Act from re-publishing orphan works. This has big implications for digitisation in the US, as their orphan works protection lasts for 95 years for books published prior to 1978. There’s a good write up here by the Oxford University Press about the potential impact.

In New Zealand, literary works of unknown authorship are only protected for 50 years. It seems unlikely therefore that New Zealand researchers and booklovers will see any full versions of the New Zealand titles Google has digitised and that were published in the 20th century.

Calling NZ Flickr groups

Fri, 17 Jul 2009 10:52:00 +1200

** Updated 6 November 2009 **

Digital New Zealand is really interested in sharing information about photos/videos from Flickr group members through our discovery services, including http://search.digitalnz.org/ and our developer APIs.

We are a publicly funded initiative helping to make New Zealand-related digital content easier to find, share and use. The team is based in the National Library of New Zealand in Wellington.

The way it works is that we collect information (metadata) about photos/videos and other media and link back to them – so more people discover them on Flickr. We use the Flickr API to do this, and only link to public photos/videos.

Benefits include greater visibility of your work, and the ability for people to find your images alongside over a million other digital resources from New Zealand collections, including Te Papa and the National Library. You can see an example of what results look like in DigitalNZ here.

We are getting in touch with as many New Zealand groups as possible asking them to participate – starting with the ones listed in the NZ ~ Aotearoa discussion. You don’t need to do anything except say yes. If you are willing to help us bring greater awareness of New Zealand content, please post a message in your Discussion threads and let us know.

We will monitor your Discussion thread for comments and respond to any questions before starting to collect metadata in *a week* from the date the notice is posted (assuming there are no problems raised by your members). We can also take content out very easily if any issues are raised after then.

Once the metadata is in the Digital New Zealand system, it will be available through http://search.digitalnz.org/ and the developer APIs. We notify search users of the copyright status of the content they find from you in two ways:

Via terms of use accessible from every page of the search (these note that unless otherwise stated all items are protected by third-party copyright or other usage restrictions)
In search results themselves, using a license facet that appears alongside each result (as in this example)

Developers who build other websites and applications to send users to your content are bound by these Developers terms of use.

Any questions please don't hesitate to ask!

See here for more information about contributing to DigitalNZ

Digitising beyond the walls

Fri, 10 Jul 2009 14:38:00 +1200

Thinking about digitisation on any great scale tends conjure images of archives, libraries and museums, whose job it is to collect and retain the records of our culture and history. But just as there is a growth of contemporary user-generated content online, a growing number of amateur and private collectors and historians are presenting a digitised view of the past online without involving a professional curatorial eye.

A recent U.S. conference on the Digital Humanities highlighted the role of amateur endeavour in the creation of collections, and the role of digitisation in creating virtual museums and exhibitions for online display to the public. It signals that institutions are not the only hosts of worthwhile digital content, and may even get less traffic to their digitised collections than many amateur sites. There's a great write up about this conference topic here.

In New Zealand, while it can take a bit of digging, it is not hard to see the growing presence of our own amateur and private effort. Perhaps most readily discoverable are the websites dedicated to family history. For example, New Zealand Bound's free pages hosted by Rootsweb show the extent of effort put in by part-time genealogists to get New Zealand shipping lists transcribed online . A website dedicated to Dunedin's Northern Cemetery encourages contributors to add biographical details and photographs of the people buried there. My Ancestor's Story is a nicely done private effort to share New Zealand family history stories online.

Outside of genealogy is a range of content focused on diverse aspects of New Zealand's past. The New Zealand history pool on Flickr has nearly 2,000 images from our past online. The Early New Zealand Software database and the State of the Ark, an online computer collection belonging to Donovan Marshall, document New Zealand's software and hardware past.

A picnic scene from a digitised postcard in Flickr's New Zealand history pool

In the commercial arena, Fletcher Building Ltd support the Fletcher Challenge archives and an online digital collection of the past 100 years of Fletcher construction. For some time, Colonial CD Books has been focused on digitising out of copyright heritage publications from the NZ Gazette through to local histories, and making them available for sale online. Even overseas digital publishers like Lulu enable Kiwis to digitally publish family histories or other New Zealand stories for print or download, such as The Randall Family in New Zealand by Randall McMullan.

Digitisation is changing the face of our heritage, whether institutions are involved or not. What we need to learn more about is the relationship between amateur and professional effort in creating and maintaining digitised heritage. Archives and museums are already receiving digitised materials from donors in forms that are unusable or poor quality. The challenge is for institutions to create a lifelong relationship with communities and future donors so that digital content remains usable and originals being digitised are protected. Perhaps part of our future requires solutions like community-based repositories, already being demonstrated with the pioneers in this area, Kete Horowhenua.

Digital newspapers your ancestors read

Thu, 02 Jul 2009 18:28:00 +1200

Quietly launching in the last week of June was a major update to Papers Past, New Zealand's largest free online digitised resource. Papers Past, featuring newspapers from the nineteenth and early twentieth centuries, is now bigger, faster and fully text searchable thanks to Optical Character Recognition (OCR) technology.

Papers past was first launched in 2001 with quarter of a million digitised pages of New Zealand historic newspapers. It now has five times that many pages (1.3 million) covering 52 different New Zealand publications from as far back as 1839. In fact, Papers Past currently has more digitised pages online than Chronicling America, an equivalent project in the U.S. for historic newspapers.

NZ Truth is one of 52 publications now available on Papers Past

Old newspapers make for a really interesting resource to digitise, and it's perhaps not surprising that they have been one of the earliest digitisation efforts around the world.

Because of the poor quality of the paper they were printed on, newspapers are prime candidates for copying to make surrogates to access instead of fragile originals. Without microfilm and now digitisation, many old newspapers would simply not be available to view at all.

Many New Zealand newspapers have been microfilmed or are being microfilmed for preservation purposes. Digitising microfilm is a lot simpler and cheaper than dealing with the paper originals. It is only recently that directly digitising high volume large format materials like newspapers has become possible. The National Archives and Records Administration (NARA) in the United States has recently invested in 10 large format scanners at a cost of around NZ$250,000 apiece, but it's likely to be some time before that kind of technology is widely available and affordable. A lot of preparation work to sort, unbend and repair old newspapers is also required - this work has already been undertaken where the paper has been microfilmed.

From a copyright perspective, old newspapers are often less complicated than other resources, as a large proportion of the contributions of articles from the nineteenth and early twentieth centuries are unattributed. In New Zealand, where authors are unknown after reasonable enquiry, the copyright term for published works is only 50 years.

Being text-based and complex in structure, newspapers lend themselves extremely well to full text searching. OCR is the quickest way to achieve this, and while by no means perfect, it can get very good results - certainly far better than scrolling through pages of microfilm. As a way of trying to improve on OCR results, the National Library of Australia is testing out a very cool newspaper service that allows users to easily correct and tag newspaper content in a way that has search results getting better over time.

Newspapers are likely to continue to be a highly prized target for digitisation, both public domain editions and more recent in copyright ones. Engaging with newspaper publishers and encouraging them to open up their more recent back catalogue for digitisation and public access is a challenge that still lies ahead of us. In the meantime, there has already been considerable interest on our Make it Digital voting tool for a variety of newspaper editions to be digitised. We've invited the Papers Past manager to join in and post newspaper titles so you can have a say in what you think they should be digitising next. Get voting now!

The identity question

Thu, 02 Jul 2009 13:40:00 +1200

Since Make It Digital launched, we've had a lot of questions from people, wondering whether they should post on behalf of their company/agency or as an individual.

For those of us who are State servants, SSC's Principles for interaction with social media are the best place to look.

Here's an excerpt:

As an agency representative

The protocols that apply when you are acting as an official representative of your agency are the same whether you are talking to the media, speaking at a conference or using social media. Good practice is to disclose your position and that you are representing your agency. You should only disclose information, make commitments or engage in activities when you are authorised to do so. You should remember that your comments will often be permanently available and able to be reproduced in other media.

In a private capacity

State servants have the same rights of free speech as other New Zealanders, but with some additional obligations. Regardless of the media being used, you must not do anything which could harm the reputation of your agency or the State services, and you must not disclose any agency material that you are not specifically authorised to disclose. Where there may be uncertainty about the capacity in which you are acting, you should make it clear to others that your contribution is as a private individual and not as a representative of your agency. You should ensure that any comment you make on matters of government policy is appropriate to the agency role you hold, and you must respect the need to maintain politically neutral State services.

For companies with a commercial interest in digitisation or having their objects digitised, it's a matter of appropriately disclosing your commercial interest and having the authority to post on behalf of your organisation.

If you're posting as your organisation and there will be different members posting, you may want to sign each post off with your first name. If there's just one of you posting, then it doesn't matter as much. Another thing to consider is, if you also want to post about ideas that are unrelated to your work and not on the behalf of your organisation, you may wish to set up a personal account.

DigitalNZ HackTest

Thu, 02 Jul 2009 10:26:00 +1200

On Saturday the 20th of June, we ran a small DigitalNZ HackTest, a bit of a test-run before we start with the HackFests. We invited people to come along and:

Give our API a decent thrashing
Whip up some code samples to help other developers kick-start their own creations
Make nice stuff
Consume pizza, beer and sugary confections

We are really pleased with how the the day turned out.

Here are some of the things that happened:

Brenda made a Perl module
The OLPC Wellington Friends in Testing showed up, tested, learned about DigitalNZ and let us play with their cute OLPCs
Dave made a search filter
Mark started making a Flex library
Richard uncovered a security hole - now fixed. Thanks Richard! Yes, you herd rite, we do liek mudkipz.

If you would like to participate in future DigitalNZ HackFests, please get in touch.

Future of the Book conference notes

Wed, 01 Jul 2009 15:17:00 +1200

On 24-25 June, I attended the Future of the Book conference organised by the Digital Publishing Forum in Auckland.

For the sake of expediency, here are some occasional thoughts and things I took away. Hopefully the presentations will be up on the site soon so you can get a fuller picture.

Talkingtothecan (who I sat next to) has also done an excellent job of summarising some key themes. You can track back through the tweets at #nzdpf too.

General thoughts

Quote of the day for me was the suggestion that E-book readers are “a transitional device for print readers” (the true e-book has not yet been realised). Someone else then paraphrased: E-book readers are "training wheels for baby boomers".

So few NZ book publishers seem to be engaging with e-publications in New Zealand, perhaps because the commercial potential is difficult to detect, and business models are immature. This conference was a great push in the right direction...

But I think we also need to think 'outside the box' (or book) to realise the potential of new media for delivering 'reading' and literacy experiences. I have a feeling this requires a convergence of interests and expertise - libraries, publishers, web developers, readers, writers.

That said, just getting content digital and into the view of people on the move, or in front of a screen, is a great place to start.

Things I didn't really know much about previously

Stanza, free e-book reader for i-Phone mobile devices: “It’s a wireless electronic library that stays open 24/7”.

Talked to Stanza developer who referred me to the Open Publication Distribution System wrt DigitalNZ - a standard for listing online e-book content (i.e. catalogues).

EPUB (formerly Open E-Book) - standard of choice for full text e-publications (Make DigitalNZ full text compatible with EPUB standard?)

A vision we are still aspiring to 20 years after Apple ran the Knowledge Navigator promotion.

If you develop an App for the Apple store, there is no guarantee it will be accepted (all apps are reviewed) - and acceptance takes time (6 days to 11 weeks: plan launches accordingly). The presentation by Karl Von Randow of Cactuslab was a highlight.

http://buzzword.com/ Web-based word processor from Adobe. Good for collaboration, though some question about security.

http://www.nzdl.org/SmartSearch: really impressive use of Wikipedia to start implementing semantic markup from University of Waikato Digital Library team.

http://wikipedia-miner.sourceforge.net/: toolkit for navigating and making use of the structure and content of Wikipedia.

http://www.nzdl.org/Books: online book reading experience - open source PHD project from University of Waikato Digital Library team. We could use this software for something? A content exemplar of some kind?

Digital New Zealand tie-ins

We can definitely help promote E-publications and other online 'books' that publishers produce through the DigitalNZ search engine and discovery tools. Samples would be a great place to start. If you're a publisher and have readerly content online, please do get in touch.

There seemed to be support for a ‘commercial’ repository for e-publications so publishers don’t have to host them. Opportunity for DigitalNZ shared repository (a work in progress) to put up its hand here?

Synergies between the 1000 Great NZ e-books project and the DigitalNZ Make it Digital voting tool. The '1000 Great New Zealand eBooks' promotion is an industry initiative to help seed and grow the market for digital publishing in New Zealand. The Copyright Licensing Limited or the Digital Publishing Forum could use the tool to publish ideas as they come in, and gauge reader interest.

Or our community (which spans from librarians to techies to general readers to anyone online) could nominate books and publishers could pick them up.

Publishers need advice on standards and digital ways and wherefores. Again, the Make it Digital helpdesk site works nicely here. Ask a question, get an answer (not just from us, but by all sorts of voices out there).

If you were at the conference, or following it, or have any comments on the above, I'd love to hear your thoughts too.

Roadtest the Make it Digital Scorecard

Mon, 29 Jun 2009 17:13:00 +1200

As the word of our Make it Digital site launch starts to spread, it is a great opportunity to point you to a new resource available on the site which we are really proud of. The Make it Digital Scorecard has been in development and testing over the past six months, and makes its public debut for the first time on this new site. The scorecard is designed to help organisations decide and prioritise what materials they should digitise. Now it's out there, we're really keen to hear from people wanting to use it out in the real world.

Back a few years, it was fairly common for digitisation projects to start out digitising with only informal or experimental criteria such as meeting user demand, assisting special events or exhibitions, enabling delivery by email or the web, or testing techniques, workflow or feasibility. If your organisation is interested in promoting or increasing access to digital content that you hold, this ad hoc approach is no longer enough to make it happen.

In today's rich web and digital media environment we need to view our digitised materials as the basis for building digital collections or resources that users can reuse, repackage, repurpose and build services upon. If you don't fully consider your purpose, changes in access, the suitability of your technique and the value of the content you digitise, your digitisation efforts are likely to be undiscovered, underutilised and quickly obsolete. The Make it Digital Scorecard aims to help prevent that by providing a standardised set of criteria that can be applied to a wide range of materials, and a transparent scoring system that leaves room for your good judgement.

While we are excited to be able to make this home-grown resource publicly available for the first time, we are also itching to find out how well it works in a real digitisation environment. We want to document the process as a case study and make improvements for future versions. If you are thinking this might be of use in your organisation, and would be open to having us tag along to watch you use it, we would love to hear from you. In return, you would have the opportunity to tap into our knowledge of how the tool is designed and our network of expertise in digitisation, along with knowing you're helping make more New Zealand content digital! Drop us a line at advice@digitalnz.org.

Make It Digital has launched!

Thu, 25 Jun 2009 11:23:33 +1200

We're all quite excited because our new site, Make It Digital, launched today.

The site is focused on creating and sharing New Zealand digital content, which we're tackling in two ways:

1. The Guides and Ask a Question sections are designed to help people who are trying to create new digital content, or digitise their stuff. A lot of you will have questions on how to go about digitisation and this is the place where you can ask. We hope that some of you will be able to share your expertise by answering others' questions, and helping us to write and update the guides.

2. The Voting section - It's a public forum for people to share their ideas for new NZ digital content, with voting and commenting functions. There's some great ideas in there already that you can vote on, for example, School Journal, New Zealand music artwork and Appendices to the Journals of the House of Representatives.

Please jump in and have a play, ask some questions, vote or nominate something for voting.

Within 24 hours, we would love to see Make It Digital reach 100 content items nominated for voting. There will be spot prizes for interesting and useful contributions during launch day!

Interview with Paul Hagon

Tue, 23 Jun 2009 16:19:00 +1200

One of the questions we get asked most when working with New Zealand content creators is ‘what’s an API’?. Quickly followed by… ‘so, why would you do this?’

When a developer we’ve never met picks up on the DigitalNZ API to create a map-based interface for discovering New Zealand content, an answer starts to form.

In short, an API (Application Programming Interface) is a way for applications to talk to each other; and a way for developers to talk to applications.

As the Indicommons website so aptly summarises: ”Open APIs allow services and collections to become interconnected, the experience of outside developers to be engaged, and new tools and spaces to be fashioned to benefit the community at large.”

Paul Hagon’s ‘experiment’ with the DigitalNZ API brings these words to life. And DigitalNZ didn’t have to do anything except give developers and applications permission to ‘talk to’ information about and links to New Zealand digital content sources!

So, with a nod to Brooklyn Museum who looked to DigitalNZ when releasing their public API, what follows is a Q & A across the trans-tasman divide. We try to delve deeper into Paul’s creation than 140 characters of twitter permits…

You can also find out about the DigitalNZ location search on Paul's own blog, here.

How did you find out about the DigitalNZ API?

I found out about the DigitalNZ API from Seb Chan at the Powerhouse Museum when he was in New Zealand last year for the National Digital Forum. I can’t remember if it was via twitter or via the write up on their Fresher + New(er) blog. I can remember getting quite excited about it, not just from the API point of view, but from what was actually achieved in getting all the contributors involved. It’s very impressive.

Tell us about what you’ve created, thought process behind it, etc?

I've been sitting on this idea since the start of the year, but have been away on holidays for a fair bit of time and never got around to implementing it.

Typically interfaces into large amounts of data, like DigitalNZ, are based around entering something into a search box. It’s a relatively effective but rather boring interface. You need to know exactly what you want to search for and there is very little sense of discovery by accident or exploration (when was the last time you started randomly entering search terms into a search box?). What I’ve created is a map based interface into the collection that focuses more on this discovery aspect rather than getting perfect results.

The interface works by using a combination of API’s from Google and Flickr to extract the name of a location for a given latitude, longitude and accuracy on a map (based upon how far you have zoomed in to the map). This is why if you position the map over Taupo, on the zoomed out map it returns results based on Waikato, then as you zoom in further on the same spot it searches for Taupo District, then Taupo and finally Taupo Central. As each search takes place, a rough guide to the area being searched is displayed on the map using a Flickr shapefile. These shapefiles are generated from the locations people have tagged photos in Flickr.

By using an interface like this, you can see other areas on the map, that you are likely to explore. It’s easy to get results without having to do a lot of typing or thinking.

Is this it, or are there other things you want to do with it?

With the web there is never any ‘this is it’, there are always things that can be changed. It’s still a bit rough around the edges so needs a bit of cleaning up of the code it’s built on. In the short time since developing this, Yahoo have released all their geographic data (WOE data) under a Creative Commons license and Flickr have also released their shapefile data under a Creative Commons license. I’ll incorporate both of these into the service as I’m kind of using and abusing the Flickr API in its current format by not really incorporating Flickr content, which is a bad thing. By Yahoo & Flickr opening up and sharing their content under these licenses, I can use the data without feeling guilty about doing the wrong thing.

The big thing is going to be improving the accuracy of the search results. The DigitalNZ API allows me to search over data that has been given specific location information, but at the moment, not a lot of records have this data. I’ve had to set up 2 search options - one limits it to give an accurate result, and another that searches over everything and picks up place names in the title or description of records. It’s not as accurate, but provides better results. It’s a bit clunky filling up an interface with these options. I like to keep things as simple as possible.

Through the Google Group that has been set up for DigitalNZ, I’ve found that people are attempting to extract location data from the DigitalNZ records by running them through services like Open Calais and Yahoo placemaker. The community that has formed is working on filling these holes in the data and hopefully this work will be incorporated back into DigitalNZ to produce better results for both the participants and the developers that are extending DigitalNZ.

Tell us about yourself!

(I kind of know who you are because you created this extra cool Then and Now mashup… but other than that all I know is constrained to 73 x 73 pixels and a twitter profile…)

Thanks for that – I’m glad you like the mashup. I live in Canberra and I work as a web developer at the National Library of Australia. I’ve been here for 3 years and love it. I previously worked at the Australian War Memorial, so I’ve spent 10 years working with large collections on the web. I studied industrial design in the days before the web existed and have a fascination about how we interact with things. When I’m away from the computer I’m a keen photographer and a semi-retired ultra endurance triathlete.

Any final thoughts?

It’s a fantastic step forward that’s been taken with DigitalNZ. The real power of doing this type of thing comes once an ingest API is provided and we can easily build applications where corrections or further information can be added to the records. I think of it as information karma, if I use your resources to build interesting things, I also have a responsibility to add back to the service and help fix the deficiencies.

Thanks for the opportunity to participate in the interview.

Museums and the Weblog 2009

Mon, 27 Apr 2009 20:58:00 +1200

I have this theory that if I procrastinate long enough, I’ll be able to write an old style post on Museums and the Web (MW2009) that simply indexes everyone else’s blog posts.

Meanwhile, here are the things that remain in my mind. When fellow New Zealand (and other) MW2009-ers unpack their thoughts, we can join them up for a fuller picture.

Sites of interest

There were many, not all digital. Some I came across through (face-to-face) conversations, so you may not find them featured in the conference proceedings.

http://dashboard.imamuseum.org/ - transparency on the Indiana Museum of Art’s activities, using open source software – could we present DigitalNZ site statistics in a similar way? Video of keynote here.
http://www.artbabble.org/ - practical exploration of cloud computing – interesting fodder for those facing video repository or hosting solution challenges. Paper here.
http://museumpipes.wordpress.com - documentation of a project similar to DigitalNZ but using Yahoo Pipes – read from bottom up. Abstract here.
http://www.visiblepast.net/gwiki/ - collecting together geo-coded information for mobile devices & 3D visualisation. Web 3.0 is 3D. Here we come, ubiquitous computing!
http://www.seco.tkk.fi/applications/kulttuurisampo/ - impressive national-level ontology for semantic web development. Paper here.
http://conference.archimuse.com/nominee/wotr_write_record - innovative start to something that could grow much bigger. Sprinkle comments on your collections and content.
http://blogs.ukoln.ac.uk/good-apis-jisc/ - good practice for APIs.
Tate project aggregating access to collections - ome ideas for app front end to DigitalNZ API might be found here.
http://imlsdcc.grainger.uiuc.edu/about.asp - another instance, also actively researching useful applications for understanding a 'black box' of data using Patchwork Prototyping. One to follow. Abstract here.
http://www.phillyhistory.org/PhotoArchive/ - geo-coding in action. Paper here.

Papers I’m going to read (more deeply)

Presenting at a conference means you spend some amount of time either preparing for your talk, giving your talk, or running around like a headless chicken trying to find a printer to spit out your notes.

Now that I’m not doing any of those things, these papers are top of my list to return to. Click on the square paper icon to view:

Aaron Straup Cope: The Interpretation of Bias (and the bias of interpretation) - on 'reverse geocoding' among other things – turning coordinates into ‘place’ which is very often a geography of the mind or community as much as a physical one.
Eero Hyvönen: CultureSampo – Finnish Culture on the Semantic Web 2.0: Thematic perspectives for the End-user
S. Chun, R. Stein and others: Steve in Action - been tracking this social tagging project for a while, especially after Susan visited New Zealand a couple of years ago. How might a collaboration with a NZ institution might work? 88% of tags submitted by visitors were seen by museums as useful; 86% don't match existing documentation. Interesting.
Nina Simon: Going Analog. Translating Virtual Learnings into Real Institutional Change - big buzz about this mini workshop, slides here suggest why. Sadly I was presenting at the same time, so while I could hear it ;-) I couldn’t attend it.
Dan Zambonini and Mike Ellis: HoardIt: Aggregating, Displaying and Mining Object-Data Without Consent (or: Big, Hairy, Audacious Goals for Museum Collections On-line). Similar project to DigitalNZ search, only we asked content providers first.
K. von Appen and others: WeTube: Getting Physical with a Virtual Community at the Ontario Science Centre - about 1% of content created in the meet up related to science collections – the rest was social interaction in the space, or just plain social. Still worthwhile?

Here is the DigitalNZ presentation. It’s a bit abstract without someone talking alongside it. I also need to do something about the videos which were a significant component and currently show as blank slides.

UPDATE 29 April: Slides, with links to video embedded, now on slideshare. Download presentation includes brief notes.

Random thoughts from other events

While colleagues from Te Papa and Auckland Museum took care of the museum-y and visitor experience side of things, I tried to interoperate with others doing API and open data stuff.

There’s a definite movement going on, and I’m looking forward to DigitalNZ and our community being part of it.

Geo-coding data in our location fields. We have a lot of names of places, some a bit messy. But tools (and people) exist to help us start improving that – and feeding it back to our content providers if they want the data.
Persistent Identifiers. Still a hot topic, but hard to move to action. What role for DigitalNZ there? How to build on work such as this? How to extend beyond our national boundaries?
Dublin Core may be a way to achieve interoperability among APIs (if such a thing is desirable). We need to write up our data dictionary & specs for others to see, including our advice to content providers.
http://museum-api.pbwiki.com/ a Wiki for people interested in cultural heritage and APIs and open data and things – a good place to share information around open data in cultural heritage institutions.
http://microformats.org/wiki/work-of-art of interest us as we try to give consistent encoding advice for DigitalNZ Sitemap XML harvest providers.
Multi touch experiences and DigitalNZ API integration. Geo-coded data might help.
Should we be providing the DigitalNZ data in RDF XML format too?

If it’s not on Flickr it didn’t happen?

Judging by the size of the MW2009 pool, this is probably true. The whole conference kit and caboodle here.

A few of my photos here.

By the end of April I'm going to try and make a stereoscope iPod viewer with some colleagues.

Pertinent tweets (no, that is not an oxymoron)

Luckily I am of the generation that can simultaneously listen to a speaker, read a powerpoint, and follow a wall of twitter.

I’m afraid my favourite tweet (‘Yay, my panties are going to #MW2009' from @PantyGirl) disappeared too quickly for me to photograph it.

Nonetheless, reading the #mw2009 hashtag summary will give you a good sense of the zeitgeist – which trended for a while (at least until Susan Boyle sang like Elaine Page).

What you don’t get though is a sense of the real software-peopleware convergence. So many ‘question times’, for example, began with ‘yeah, so I just tweeted this, but maybe you can answer ….’

@mia_out impromptu museum API meetup during icecream bit in Cosmo at #mw2009. Come along and say hello if interested in making or using museum APIs
@briankelly Similar API buzz at #mw2009 RT @paulwalk : APIs are good; work in all kinds of contexts; help developers Get Stuff Done. 3 cheers for APIs!
@musebrarian What can you do with a semantic knowledgebase? Search for "beard fashion in Finland" across time and place. #mw2009
@georginab ninaksimon: Don't try to change visitor behavior, think about what they ALREADY do in your museum and how you can intervene. #mw2009
@mia_out would 'Star Trek as the A Team' be too frivolous an example of re-used content mashed up? http://tinyurl.com/6yxto2 #mw2009
@georginab Tate kids has re-written the standard terms and conditions for their web site into kid-friendly language - what a great idea! #mw2009 (kids.Tate.org.uk)
@anarchivist dear digitalnz: I have such a geek crush on you. please bring me out to visit and/or hire me. :) #mw2009

Finally

It was a privilege to represent the work we have all (content providers, team, developers, everyone) done on Digital New Zealand – and I hope many conversations come out of the connections Museums and the Web has afforded us.

I even got my own moment of stardom (yes, I’m proud my tweet was about metadata). OK, so a Best of the Web Award for DigitalNZ would have been nice – but then we’d have no reason to return next year, would we?

National Digital Forum 2009 Conference

Wed, 22 Apr 2009 11:18:00 +1200

The 8th annual National Digital Forum conference will be held in Wellington on Monday 23 – Tuesday 24 November 2009 at the Museum of New Zealand Te Papa Tongarewa.

The National Digital Forum (NDF) is a coalition of museums, archives, art galleries, libraries and government departments with more than 130 member organisations committed to collectively building New Zealand’s culture and heritage online.

This year’s conference theme Being online now: culture, creativity and community will explore opportunities for the creative and cultural sectors to cross traditional boundaries and collaborate on solutions to address issues facing communities being online now. The conference will explore how creating and accessing New Zealand digital content benefits all parts of society: business, education, cultural organisations, local communities.

The interactive format of the programme will encourage delegates to take part in open knowledge sharing, discussion and debate. The 2009 conference programme will include stimulating keynote speakers, discussion sessions, and practical workshops and demonstrations.

Call for Contributors / Presenters

Interested in presenting at NDF 2009?

We are looking for speakers to give brief, thought-provoking presentations that will set the scene for forum discussions with the audience. There is also the opportunity for you to share stories of your project, product, website or application with other attendees, describing what you have been up to, what you have achieved, and share the lessons you have learned, in demonstration sessions.

To submit your proposal to present, complete the NDF 2009 Presentation Submission Form or contact the conference organiser.

For full details, or to download the Presentation Submission Form for your proposal to present, visit the NDF website

If you would like more information on any aspect of this conference, please contact the Conference Organiser:

Paardekooper and Associates
Conference organisers of NDF 2009
P: +64 4 568 4576
F: +64 4 568 4563
E: ndf@paardekooper.co.nz
W: http://ndf.natlib.govt.nz/about/2009-conference.htm

NZ copyright status flowchart (photographs) - updated

Tue, 07 Apr 2009 17:34:00 +1200

Last month we posted the first version of a tool for determining the New Zealand copyright status of photographs for you to use and comment on. The updated version is now available for you to use and share:

Download the flowchart for determining the NZ copyright status of photographs

Your responses, both in the comments and by email to us, was amazing. Thank you.

We'll be turning our attention to more flowcharts in the series soon, beginning with either Sound or Video.

Any preferences?

DigitalNZ and Museums & the Web

Thu, 02 Apr 2009 09:51:00 +1300

Last year, Digital New Zealand was invited to submit a late abstract to Museums and the Web 2009 in Indianapolis. We were chuffed to have that paper accepted – and I will be heading across to present a mini-workshop and demonstration.

DigitalNZ is also up for a best of the web award – so watch this space. You can register and vote for us in the Best of the Web 2009 People's Choice.

What is Museums and the Web?

Museums and the Web aims to address the social, cultural, design, technological, economic, and organisational issues of culture, science and heritage online. So as you can see, it has a heap of relevance to the work we are doing at DigitalNZ – even if our reach extends wider than the cultural heritage sector alone.

Why Museums and the Web?

I think it’s fair to say that the Museums and the Web conference has significantly influenced the direction DigitalNZ has taken.

The DigitalNZ Memory Maker was discovered at Museums and the Web (Andy, our technical lead last year and current Programme Manager, went to the conference in 2006 before DigitalNZ was anything more than a Content Strategy, and came back brimming with excitement and ideas for content experiences).

That same year (2006) we brought over a number of speakers from Museums and the Web to the National Digital Forum (NDF) conference – such as Jim Spadacinni from Ideum; Toby Travis from the V & A; and Seb Chan from the Powerhouse Museum.

Digital New Zealand was at that NDF too, kind of. There was a session about digital content and the draft Digital Content Strategy ('Creating a Digital New Zealand'), Matapihi as a window onto New Zealand content… I think Penny Carnaby might have even talked about DigitalNZ as an abstract concept - connecting it all together...

And there was an unconference session in which Seb and Jim challenged a few of us to start thinking about opening up access to catalogue data for other people to use.

I guess if you mix a nationwide strategic approach to digital content creation; projects encouraging institutions to share metadata to locally-held content; a couple of vanguard thinkers pushing ideas like APIs, remix, reuse; and exposure to ideas and projects on the international stage, you get something not too dissimilar to this.

Not that we’re only about discovery, but it’s a pretty significant component.

What do you want to hear about?

The fact that I get to go to Museums and the Web feels completely unfair, even if it is a totally and utterly exciting professional development opportunity for me. We should all get to go – everyone on the team here, and all of you out there producing digital content as well. I hope I can do us proud.

I’ll be using this post to comment back on what jumps out at me as interesting, new, super cool and (importantly) useful. If there’s something from the programme, demonstrations or workshops you particularly want a report back on, drop me a line below.

DigitalNZ a finalist in PwC Hi-Tech Awards

Fri, 27 Mar 2009 16:21:00 +1300

We're quite chuffed about being a finalist in the PricewaterhouseCoopers Hi-Tech Awards, an awards programme focused on New Zealand's software, electronics, biotechnology, telecommunications and creative technology industries. We were nominated under the category of Outstanding Industry Initiative. The other finalists in the category are eDay and The Distiller... good luck to everyone!

Creative Commons offers new tools

Tue, 24 Mar 2009 13:17:00 +1300

With Creative Commons licences it's always been a bit tricky to give away content or declare it out of copyright. Creative Commons licences rely on provisions in copyright law that firmly fix the licence to the copyright owner. But what if you don't want to remain the copyright owner, and what if your content is not in copyright? The Creative Commons is on to it, and will offer new tools for exactly these circumstances.

Released earlier this month is a 'CC0' (CC Zero) Universal tool, which allows content creators and owners to unconditionally waive their rights on copyright for their works. A legal waiver rather than a licence, it aims to help cultivate a rich pool of freely available content for anyone to use for any purpose. Its wording is designed to be applicable worldwide for any content without having to be adapted to local laws. The waiver can be found here.

Creative Commons is also working on updating its currently U.S.-centric public domain dedication and certification. The current dedication relies on U.S. law which may not be applicable in countries such as New Zealand. The updated version will focus solely on the certification of content that is already in the public domain i.e. not covered by copyright. This should be able to be ported to a New Zealand certification, and will be useful for anyone wanting to share content that they can verify is not in copyright, and which can't be given a copyright licence.

The announcement and more details can be found on the Creative Commons blog. The New Zealand Creative Commons website is www.creativecommons.org.nz .

Would you consider using the CC0 licence for content you produce in order to give it away? Have you struck problems with putting licences on content that you know is not in copyright? Share your views with us!

NZ copyright status flowchart (photographs) - tell us what you think!

Mon, 09 Mar 2009 17:30:00 +1300

UPDATED 7 April 2009: Thanks for all the wonderful feedback, and time taken to review. We have updated the flowchart based on your comments.

Download the updated flowchart for determining the NZ copyright status of photographs (NZ copyright flowchart in RTF, 324 KB) (NZ copyright flowchart in PDF, 62 KB)

***********************************

DigitalNZ is proud to make available the first version of a tool for determining the New Zealand copyright status of photographs for you to use and comment on.

Download the flowchart for determining the NZ copyright status of photographs (RTF, 149 KB) (PDF, 120 KB)

Why provide the NZ copyright status flowchart?

One of the problems we find that our content providers face is determining the copyright status of items. If you can’t determine the copyright status of your content, you can’t even start to think about applying New Zealand Creative Commons licences or other statements.

The tool we’ve developed is intended to meet ‘80%’ or thereabouts of situations, rather than trying to provide a decision tree for every possible scenario (e.g. rights of joint photographers). These are things you need to take note of though.

Who’s the NZ copyright status flowchart for?

We think it will be useful to you if you are:

Working with photograph collections that you want to make available online
Trying to work out what the copyright status of a photograph you find online might be, if it’s not already identified.

Breaking it down by format is a great way to tackle some of the complexity in this area, so we've begun with photographs to prove the concept. If this one proves useful, we’ll extend this into a series.

Has a lawyer looked at the NZ copyright status flowchart?

Yes, and then some. We’ve worked with a number of experts and lawyers to ensure that this flowchart is a useful tool for determining the copyright status of photographs.

That said, this flowchart is of a generalised nature, for information only. As the disclaimer states, it doesn’t constitute professional advice and DigitalNZ is not responsible for any loss caused as a result of using it. You should seek professional advice from a suitably qualified professional about specific issues. Don't let that put you off using it though!

What do you think about the NZ copyright status flowchart?

Please take a look at the flowchart, try it out, show it to other people, and let us know what you think. Anything you’d like more discussion on, just let us know in the comments.

We’d also be interested to know whether anyone out there has similar tools hiding in their back rooms? We’re happy to publish them here if you’d like to share – though we can’t verify their usefulness like we have for this one.

This isn’t the end for us. We’ve got plans! From making it look nice (OK, a bit of a nice to have), through to making it an online interactive.

But for now we just want you to know we’ve taken the unglamorous approach, and can’t wait to hear from you. So, what do you reckon?

Download the flowchart for determining the NZ copyright status of photographs (RTF, 149 KB) (PDF, 120 KB)

Useful links

Information about copyright - Ministry of Economic Development website

Creative Commons Aotearoa New Zealand

Rights and permissions help - Kete Digital New Zealand

So you want to start a digital content project?

Tue, 20 Jan 2009 14:44:00 +1300

Digital technology can give easy access to many types of content that would otherwise be difficult to make available by other methods. The best technology however cannot overcome poor decision-making and implementation, which means you should carefully consider and plan for all the stages of a digital project before you spend your first dollar in bringing it to life.

If you are thinking of kickstarting the new year with a new digitisation or content creation project, here are seven tips to put you on the right path:

Tip 1: Have a clear purpose for selecting content

Tip 2: Choose appropriate formats for creating content

Tip 3: Set aside resources to describe your content

Tip 4: Work out in advance how your content will be managed

Tip 5: Structure your content for easy discovery

Tip 6: Inform potential users of what they can do with your content

Tip 7: Implement a backup and long term storage plan

You can read about these tips in full by visiting our Make it Digital Guide to Getting Started

Happy Holidays from DigitalNZ

Wed, 24 Dec 2008 15:08:23 +1300

The DigitalNZ Team wish you all a safe and happy holiday.

We'll be back in full swing from January 5 2009. We will be checking in between 25 December to the 4th of January but we might not be as responsive as usual.

Best Wishes,

DigitalNZ Team
info@digitalnz.org

Digital rights and mice

Fri, 12 Dec 2008 13:05:00 +1300

Some of you who attended the National Digital Forum (external link) in Auckland at the end of November 2008 may have caught the discussions around copyright, public domain and Creative Commons. We presented on one aspect of that, which was about DigitalNZ's experiences with rights statements, and how users have changed what it now means to make content accessible. We talked about that a bit in our case study of the Memory Maker, and if you want to check out our NDF presentation, the slideshow "Access, rights and copyright" is available on Slideshare.

At DigitalNZ, part of our focus is to encourage contributors of content to find and share NZ content that can be tagged, reused and remixed. We believe that if Kiwis can't really use digital content they will ignore it. As a country we then miss out on opportunities for our stories and knowledge of the past to be woven into the experiences and learning of the present.

So here is an opportunity through this blog to build on the rights conversation. Let's start with frightening a couple of elephants in the room to get the ball rolling, but before we do, a disclaimer: we are not lawyers here, and cannot give you legal advice. What we can do however is build an informed debate, and who knows, bring a few lawyers along with us!

First off, have you been claiming new copyright in digitised copies put on the web? Whether you are digitising images, sound or video, in almost every case your digitised copy will not have a new copyright. New copyright is only granted to original works and to new typographical arrangements (e.g. an e-text transcription). A digital scan of an image or a digital copy of a videotape, while requiring hardware and software calibration to produce a faithful copy, lacks originality in just the same way a photocopy of a novel would. That means if it's out of copyright, you can't put a copyright licence on it like Creative Commons.

Second, does your website point users to the 1994 Copyright Act for their rights of reproduction of digitised heritage content? Where copyrights granted by the earlier 1913 or 1962 New Zealand Acts had already expired, the 1994 Act did not renew them. So for instance all photographs, sound recordings and film material created before World War II are almost certainly out of copyright, as the earlier Acts only provided a copyright term of 50 years from creation (regardless of publication status) for these items.

When put together, these two practices will have the effect of misdirecting users about how copyright works. If you want to restrict out of copyright digitised content from undesirable use, maybe don’t put it on the web. Otherwise, consider developing a terms of use for those items that have simple and clear statements of what you want users to do with the content (just don’t confuse these statements with copyright licences). And at least test them out on your work colleagues – if they can’t understand what those statements mean in under 10 seconds, most of your users have no chance.

We’re really keen to hear about examples out there of good practice terms of use. We also want to know your views on whether out of copyright materials placed on the web should have any restrictions at all, and why. Perhaps we should set all this stuff free? Tell us your thoughts by leaving a comment on this post.

We're launched!

Wed, 03 Dec 2008 17:15:00 +1300

Digital New Zealand is being officially launched at an event at the National Library of New Zealand tonight (3 December 2008) at 6pm. It's been a fast and furious trip, and we wouldn't have got here without the generosity of our content providers and the hard work of a team of companies: 3Months, Boost New Media, Codec, Ideum, Aotearoa People's Network and Katipo.

If you're not with us at the launch, you can find out what's been going on in a couple of other ways:

Fiona and Andy from Digital NZ were interviewed by Simon Morton for his 'This Way Up' show on Radio New Zealand on 29 November
Seb Chan from the Powerhouse Museum, Sydney, blogged an interview with the Digital NZ team on 2 December
A media release is available on the National Library website.

And you can also see our special 'speakers' at the launch event on this video on YouTube.

Joining Flickr Commons

Fri, 28 Nov 2008 15:20:34 +1300

The National Library of New Zealand has just launched their site on Flickr Commons, and you can read more about it here. The DigitalNZ team has a couple of team members who have been working on this, and we're really happy to share what we've learnt. If you are thinking about joining the Commons get in touch and we can share the ins-and-outs of our experience. It's a great initiative and we really recommend it. You can learn more about joining the Flickr Commons here.

Memory Maker... a remix showcase

Mon, 24 Nov 2008 10:37:08 +1300

We ran the 'Coming Home' Memory Maker campaign to demonstrate what is possible when content providers ‘free up’ selected public cultural content for people to remix with permission; and used the remix editor to deliver the content to users.

The remix editor (developed by Ideum) lets people mix together and recombine video, audio, text, music, and graphic content to create their own 60 second online video that can be saved and shared (by sending a link, or embedding a link to it on another website, just like a YouTube video). We used it to help us crack open the issues around rights and licensing that prevent New Zealanders from being active participants in the creation of digital heritage experiences.

Like our first search showcase, we filled the Memory Maker with content relating to celebrations for the 90th anniversary of Armistice Day on 11 November 2008. We've been delighted to watch as schools and other web users make their own commemorative videos out of New Zealand digital content - not by stealing it, but because they know they are allowed to and we made it easy for them.

Getting content from contributors

Following a difficult search for New Zealand content that we were clearly allowed to remix (few NZ search tools letting us filter by licence, and many confusing or downright scary rights statements), we put a call out to National Digital Forum members and other organisations.

We asked these content providers to help us find content relating to the return of troops from World War I, including the devastating flu epidemic that came after. More importantly, we asked for content that could be released for people to use, adapt, and mix up with other content legally – either because it was out of copyright or in copyright but licensed for remix under Creative Commons.

“No known copyright restrictions”

A lot of content in cultural institutions is out of copyright, which means you cannot license it using Creative Commons. Also, not everyone realises that if you make a simple digital copy (like a photocopy) of an item, such as a picture or sound recording, you are unlikely to create new copyright because the copy lacks originality.

We encouraged content providers to identify content with “no known copyright restrictions” to participate in the Memory Maker campaign, a similar approach to that taken by Flickr: The Commons. You can read about what “no known copyright” means on the Kete Digital New Zealand rights and permissions page, a repository we use to put content licensed (or free) for remix into. You are welcome to borrow from this page to modify your own rights and use pages.

Creative Commons licensing

Where content providers held, or administered, copyright in digital content we encouraged them to license it under one of the New Zealand Creative Commons licences that allows people to share, remix and reuse legally:

Because users of the Memory Maker are able to crop, overprint, blend and merge the content, we stayed away from content licensed under Attribution-Non-commercial-No Derivative Works (BY-NC-ND). Sometimes we also had to modify the content to fit it into the landscape orientation of the Memory Maker and make it useful at a smaller resolution, so decided to play it safe - advice we provided to our content providers as well.

What has been learnt?

Our greatest learning was probably that we, and our content providers, have lots more to learn about digital rights, permissions, and licences. We're proud to have begun to chip away at this barrier to use of New Zealand content, but we've stil got a long way to go. Nonetheless, on the basis of our small foray, we can say that:

Creative Commons is not always the right answer, especially in the cultural heritage sector. You can’t licence what you don’t hold copyright in.
It can be difficult for content providers to be certain if an item is out of copyright - information is not always forthcoming. We advise ‘if in doubt, leave it out’ for now – but we also have a fast take down policy.
Convincing content providers to pre-approve heritage content for change is difficult when you can’t predict how the content might be used. We were glad some were willing to take a leap of faith with us.
Content providers often need advice on how to use New Zealand Creative Commons licenses appropriately before being willing to adopt them.

What's next

We were amazed at the generosity of the content providers who tested the waters with us and freed up even just a small amount of content for remix. But having to ask them to tell us what we could and couldn't do with their digital content before we used it in the Memory Maker was not ideal.

Just think how great the web would be if everyone took on the task of enabling users to know when they can (and can't) copy and re-use digital content legally. It's not hard to put this information in your metadata or content once you know how. The hard bit is deciding what licence or rights statement to use.

What do you think? Are you willing to step up to the challenge and start addressing the rights issues that make it difficult for New Zealanders to interact with digital New Zealand content? Perhaps you are doing this already and are willing to share your knowledge and expertise - either here or on a Creative Commons forum? Or maybe you need some advice?

We'd like to run more remix campaigns. Maybe one day we won't have to ask - we could just use the DigitalNZ search and browse by rights!

Coming Home... a search showcase

Mon, 24 Nov 2008 08:05:00 +1300

The Coming Home search experience went live on 11 November 2008, to celebrate the 90^th Anniversary of the World War One Armistice. This was a sneak preview of the DigitalNZ tools that allow users to build their own NZ search tool, finding hidden and buried NZ content about their topic of interest. We undertook to connect together content from a whole range of sources, and in the process learn about how to open up more of NZ's rich digital treasures.

How did we do it and what did we learn?

Getting metadata content

The Coming Home search was mainly fuelled with content contributed by members of the National Digital Forum (NDF). A key feature of the DigitalNZ search approach is that we focus on aggregating metadata content, and on collecting descriptions of content objects, so we can drive visitors to websites around NZ. We had a three-pronged approach to gathering that content:

Direct harvesting via OAI-PMH, XML sitemaps, or RSS feeds
Collaboration with aggregating sources that contributors were already part of, including Matapihi, KRIS, NZ Museums, and NZ On Screen
Providing a repository website for content that couldn't immediately get online otherwise. We call this service Kete DigitalNZ, and if you don't have a website we can host your organisation's digital treasures for you

What did we ask from content contributors?

We approached organisations and individuals to participate and asked them to let DigitalNZ do one, some, or all of the following:

Regularly harvest and cache metadata and thumbnails so the content is discoverable in search experiences powered by DigitalNZ (a bit like Google).
Establish and maintain RSS feed services so the content can be discovered through other websites and online services
Allow public API access to the harvested metadata so it can be queried, discovered, and connected with other data sources (including in third party applications).

So what does all this mean? We asked that people share their metadata with us, so that we could share it with everyone. This is not a straight-foward thing to agree to, and even now we are still working through the details. We thank everyone who gave us the benefit of the doubt when all we had were concepts and ideas.

Building a search tool

Very early on in the project we decided building another website or online exhibition around 'Coming Home' would be a wasted effort. Instead we set out to allow anyone to build a search tool that explored the NZ content that they were interested in. We created the search builder and then used it to build our very own 'Coming Home' search. The search builder uses a straight forward boolean search query to gather up all the content we we're interested in, for example here's the query for 'Coming Home':

((War OR WW? OR "great war" OR peace OR flu OR influenza OR day? OR victory OR treat* OR parade OR celebration OR armistic* OR soldier* OR army OR anzac OR wounded OR gallipoli OR commemorat* OR memorial OR monument? OR carillon OR medal* OR military OR (maori AND battalion)) AND (191? OR 192?)) OR ComingHome2008 OR (returned AND servic*) OR "R.S.A" OR RSA OR veterans

At the time of writing this post, the query pulls together 14,156 related items for the 'Coming Home' search. The result is unique New Zealand content on a chosen topic being brought together and accessed through discovery points all over the web.

What has been learnt?

There is so much more to say, but we are only at the beginning of the journey. Here are a few things we've noted:

Build solutions to fit the environment rather than expect the environment to fit the solution. More methods of harvesting give us more flexibility.
Developing a good search experience is not just about being able to harvest metadata. Good quality metadata aids discoverability of content.
Good quality metadata is not just about structure. Shared approaches to data values in dates, subject terms, and so on also aid discoverability of content.
Letting users enhance metadata might allow us to benefit both content providers and users – a test for another day.
A lot of metadata exists telling people what they can’t do with content and far less telling people what they can do. We need to make both obvious.
Explaining new generation web concepts such as ‘API’ in a vacuum is difficult without examples.

What's next?

You can of course use the search builder to create your own search experience, it's a demonstration of what's possible. What can you add? Can you become a contributor? Can you develop using the APIs? Do you know of a burning issue in this digital space that needs attention? Have something to share or comment on? We'd love to hear from you.

Digital New Zealand - just the beginning

Mon, 24 Nov 2008 07:08:00 +1300

We're delighted to welcome you to the Contributors' section of the DigitalNZ website. This project has been a whirlwind of ideas, innovation, and effort by a number of talented people dedicated to helping make New Zealand digital content easier to find, share and use. It has been only 4 months since we started the technical development. It wasn't long before then that we were working out exactly what we wanted to deliver in this, the first Wave of the DigitalNZ programme. So it has been a fun, if exhausting, ride.

Here we are in November ready to launch with over 20 different organisations and individuals contributing a massively wide range of metadata content. These content providers are helping us to show what is possible when you improve the accessibility, usability and discoverability of NZ content, and we're so pleased to be working with them.

Having so many organisations, communities, and individuals onboard with us right at the outset has been fantastic. However, this is just the beginning and we want to grow. We are really looking forward to working with an even wider range of New Zealand organisations and people. If you are interested in joining us then check out Joining DigitalNZ.

The Contributors' section provides you with guidance on how to participate in DigitalNZ as well as good practice advice on content creation and management.

The information in this section is just a start - we are going to expand it to cover many aspects of digital content creation. In fact, we'd be really interested to hear your feedback, your questions and any ideas on what will be useful to you. Simply contact us at info@digitalnz.org and we'll get back to you. We're also particulary interested in hearing from people that might be able to provide expertise and perhaps help build up the material on the site.

What do you think? Are you willing to come with us to make New Zealand content easier to find, share and use?

The DigitalNZ Team