Privacy protection as it relates to official statistics is known as “Statistical Disclosure Control”, or simply “Confidentiality”. It’s all about protecting confidential information about specific individuals while still making sure that we can maximize the usefulness and accessibility of government data.

1. This post deposes the fairly widespread view that anonymizing data by removing names, addresses and such makes it safe to publish with no threat to people’s privacy.
2. Our page and link to white paper co-authored with our partner Symbolix about safe dissemination through the use of statistical disclosure control
3. Confidentiality Information Sheets from the Australian Bureau of Statistics.
4. A comprehensive set of Government-Created Resources related to privacy put together by the organizers of Data Privacy Day.
5. The US National Institute of Standards and Technology’s Guide to Protecting the Confidentiality of Personally Identifiable Information (PII) –a fairly chunky PDF but worth a look if you are in the business of making government data available.

Happy Data Privacy Day!

• SuperTABLE Ratings - what people think of SuperTABLE and what they said
• Use of SuperTABLE Features - most popular features and segments of users
• Usability and training - what we’ll do to improve this
• SuperTABLE Online - improvements to the online version of SuperTABLE
• What’s next - accessing our survey stats online, voting and sharing ideas

  SuperTABLE Ratings

The chart below shows that we have over 70% of users who feel that SuperTABLE is “fine” or even “super”.      

We’re very interested in learning how to get everyone more satisfied with their experience and so thanks in particular to the quarter of the respondents who took the time to add additional comments with this response. Here is a summary of what those commenters had to say:

• 53% referred to usability issues: either they required some training, or felt the product was not intuitive.
• 13% said they were happy; they liked the software and found it easy to use
• 8% had installer issues or setup issues.
• 4% had issues with the content, such as inconsistencies, lack of metadata and problems finding what they needed.

Use of SuperTABLE Features

The following chart shows the level of use for each feature.    

From some follow-up analysis, we identified 3 groups:

• A third simply build a table (or use the default table) and export to Excel
• A third choose a slice of data using mostly recoding and sorting to identify the stats of interest
• The remaining third use most of the features that SuperTABLE offers

Some users commented that they were unaware of some of the features listed and may have use for them if they understood how to use the product better. We’ll look into how we can make the features more accessible.  

Usability and training

Clearly from the comments, usability and training is an area where changes would really help. What we plan to do is put up some videos and see how that helps as well as update the FAQ (if you haven’t already visited it, you may find what you’re after is already there now) and create an open space for discussion and sharing.  

SuperTABLE Online

As many of you are aware, we already have an online equivalent to SuperTABLE called SuperWEB. The most popular public deployment of this product is on the ABS website for CDATA Online or Table Builder. We’ll continue to work with ABS and other government departments this year to help make new datasets available through these interactive online tools. The results from the survey, particularly the rating of online features and the constructive comments, are very helpful in shaping our product roadmap.  

What’s next?

The survey results confirmed that we have a number of passionate and interested users. So, we would like to continue the conversation beyond the survey and invite you to participate in other ways. We’ll be creating a forum for you to submit and share your own ideas with other users and vote on what you think the best ones are.  

What else would you like to see us do? We have targeted this survey deliberately at SuperTABLE users, but perhaps if you work for a provider, you may want to comment about what is important for your organization for providing easy access to official statistics. In either case, please do feel free to share your thoughts.

Landlines

Here’s an odd one to start with. Associate Professor Warren Laffen works in the Institute for Social Science Research, which is part of the University of Queensland. He is involved in a huge range of research projects and one in particular caught my interest. It’s a study they only recently commenced comparing profiles of people who have only mobile phones to those who use only landlines. Why would you want to do that? Well, apparently some phone surveys only use landline numbers and the idea behind this research is to find out how the results of such surveys might be biased because of this. It struck me as rather obscure but at the same time, I could understand the purpose and value in running such a study. It’s the same with many stats collected – they may not be of value to us all but for some, they are very important and meaningful. I formed a picture in my mind of the “landliners”: homely elderly folk sitting around knitting and never having churned from Telstra to another provider, let alone considered moving to a mobile. I was a little surprised when I sat down for the Natstats dinner this evening and discovered that my 20-something year old neighbor and her husband, both with white collar jobs, living in Hobart, had only one landline to share at home and no mobiles.

Wikiprogress Crowd Sourcing

Philippa Lysaght from OECD introduced me to an intriguing idea for a statistical Web site in Wikiprogress. Partially funded by OECD, but also from many independent supporters, the idea behind this innovative site is to measure progress of societies from around the world. Read more about what it’s all about here. Being a wiki, it gathers statistics from any members of “the community” who choose to contribute. There are various indicators based on official stats on the site, as well as data created and shared by individual researchers, academics and the like. An obvious question statisticians would have about this is how the quality of the data is ascertained if everyone has the freedom to contribute. Well, that’s part of the challenge for Wikiprogress and something that Philippa said they are working hard to keep on top of. They certainly have plenty of officially sourced statistics that have been contributed, and working out reasonable ways of accepting and presenting statistics from related (or not) communities is something that they are managing so far. Perhaps they might help us to find a middle path that can help reconcile the gap between official and community statistics (see the earlier post about community koala data vs official sources), especially given there are so many things we want to count and only so many statistical organizations around to do the work.

Twitter

Another interesting thing to see was the level of Twitter use at the conference. Jeanette Cotterill, who was the lead person at ABS responsible for organizing the event (well done, Jeanette and team!!), explained that there had been some concerns about making Twitter an official part of Natstats communications, but the decision was taken that it was a good opportunity to engage with people and indeed, so far there has been some happy Tweeting from a number of participants, as well as from the official @Natstats2010 Twitterer. Seeing as it’s a stats conference, I’d be remiss if I didn’t note something about Twitter use in stats: there were 10 unique people on Twitter on this first day tweeting about Natstats, out of an overall attendance of around 500. Maybe someone can work out how that profile compares to the proportion of Twitter users in the population at large: are Natstats attendees more or less likely to use Twitter than other folk?

Trust

I’ll close with another statistic, this one from the ABS chief statistician, Brian Pink. He noted that 92% of the Australian public trust official statistics. Oddly enough, that stat doesn’t come from the ABS but from an independent survey that the ABS has commissioned to find out more about public opinion around official statistics. You’ll have to wait for World Statistics Day on October 20 to find out more about the results.

Well, thanks so much to the many people who attended Natstats and have made it a thoroughly enjoyable experience for myself and my colleague Mark Humphreys and no doubt many others. Please do feel free to add you own sentiments or stories about Natstats in the comments. I wish you well on day two and look forward to speaking with many of you at our booth (and if you mention this post, you’ll get an extra lolly of your choice).

First, there are problems that are common to all government data research, not just sensitive stuff. Very often, researchers come across data that may be useful to them. That leads to two challenges. First, the data is often poorly documented, which leads to guess work, assumptions, and blind alleys. Second, researchers need to perform exploratory data analysis (EDA) to find out whether the data is useful, and what kind of hypotheses might be interesting to investigate. As a minimum, this typically requires download of a dataset, some transformations, loading it up in a suitable tool and finally, checking if it there is anything of potential interest there. That can be a time-consuming process. In some cases, the data isn’t freely available, which in the best case scenario leads to emails and spreadsheets going backwards and forth, and in the worst case, leads to a dead end. And when the EDA is complete it may not be of any use after all. On to the next best guess!

So far, that’s no different from analyzing open government data. However, an additional roadblock faced by those looking at health data for example, where personal privacy is a critical issue, is getting approval to access such data. That’s an administrative headache that can dwarf the technical challenges of performing any initial exploration. For example, an Australian researcher told about how he needed to complete an ethics application which has hundreds of questions on it. Given that this is before they learn whether the data is of any use, how many times does a researcher choose to simply walk away empty handed? Many research projects these days have very fast turn-around times, so this kind of approval process is really just another way of saying that the data is not available. We know there is a cost in terms of privacy violations if sensitive information is disclosed, but what is the cost of researchers not having access to valuable data sources that may contain vital statistical information that can lead to better policy and insights on a whole range of issues such as healthcare, social security, taxation etc?

I got this feedback from a small and rather biased sample. I’d really like to learn more about what kinds of EDA people need to do and the challenges in getting it done, especially where privacy issues block people from getting the information they need. There is a lot of research containing sensitive personal information, and much of it has a very high reuse value, so this is an important issue for the successful sharing of data for research. What’s your experience? Have you had to jump through hoops, or travel to a research data laboratory to analyze sensitive government data? Could it have been simpler?

The quality of data can influence the effectiveness of government and the quality of our lives. What schools have classrooms that are substandard? What areas have the highest vaccination rates? What hospitals have the longest waiting lists? The answers can be fairly straight forward, and accurate responses can greatly improve government services. On the other hand, unfortunately for those responsible for providing the answers, the statistics can be difficult to put together, and it is only after harmonization that many statistics can become a useful foundation for good decision making.

Data harmony is what is needed in order to develop, use, and share interesting and meaningful statistics.  Before organizations reach this kind of data sharing nirvana, a few incremental improvements, such as adopting rich open data standards, can improve the quality and reliability of shared data.

Statistics are derived by collating data from multiple sources that first need to be made comparable. For example, in Germany the first population census in over 20 years will be based on administrative registers owned by regional and municipal statistical offices throughout the country. The registers are maintained independently and don’t all necessarily define citizens in the same way. The data needs to be harmonized into a coherent whole. That means that the census project needs to put together datasets from hundreds of disparate sources. Those familiar with fitting just two or three datasets together will have a feel for how painful this can be.

If integrators and providers agree on data standards up front, it avoids the huge challenge that the German census team will have to confront. Part of the SDMX (an open data standard) model are specifically dedicated to rigorously defining such agreements. Success stories such as the Joint External Debt Hub, and the pilot for the European census hub, back up the fact that it can work. These projects are built on top of agreements to adhere to certain data and metadata structure definitions (SDs), where each provider is responsible for harmonizing the data they contribute. The advantage of this scenario is that the resulting data has broader applicability, and multiple different consumers of the data are not required to perform the same harmonization work (often with varying degrees of quality). Unfortunately, getting agreement on SDs can take a very long time. It might be as little as a few days for a small, bilateral exchange, but it could also be years for a complex agreement between major organizations.

The diagram below is an attempt to sketch out a path toward more mature practices for data exchange and show some of the benefits gained along the way. Does it make sense to you? And where does your organization fit: are your data exchange practices in harmony, or more of a clanging dissonance? Either way, it would be great to hear about experiences in the world of data exchange from different perspectives, and if/how you think SDMX or other open data standards might help.

While comprehensive multilateral exchange agreements facilitated by SDMX may represent the ultimate in efficiency and data quality standards, a relatively simple data sharing exchange can still deliver benefits without requiring agreement between all parties. All that is required is for participants to adhere to the SDMX technical standard - something that is relatively easy to do with the right tools. Such simple beginnings can also help organizations prepare for exchange agreements to be worked out over time. Small steps or large, it is a journey worth taking.

1. SDMX output from SuperWEB
2. Building SDMX-driven SuperVIEW interactive presentations (with no SXV4 db required)
3. RESTful SDMX Web Services

This blog focuses on the Web Services which is arguably the most important capability.  And perhaps the other reason I’m excited by it is because it is the first time that SDMX has been introduced directly to microdata.  I’ll explain what I mean by this a bit later.

From the point of view of many data providers, the advantage of the Web Services is that it can provide their customers with just the data they need, no more and no less. This can free up staff devoted to responding to ad hoc queries.

From the customer point of view, it opens up new possibilities for consuming the data and building unique, useful services on top of it. For example, a third party application can convert user responses from a Web app into dynamic SDMX queries and then the results from this can in turn be used to determine how the Web app should behave. Without Web Services, such an app would previously have relied on potentially stale data that was downloaded and loaded into a local database. And thanks to the detailed data model of SDMX, apps can also work out what other data sources might sensibly be combined together to produce richer, more useful results.

The other thing I’ll mention before getting into some specifics about what we’ve done is that our implementation is actually that of a RESTful API, not a “traditional” Web Service. We’re glad to see this becoming so much more popular now.  SDMX orginally only had standard SOAP based Web Services defined, but we’ve based our implementation on the proposed RESTful API for SDMX version 2.1.  As developers, a RESTful API is something we find a lot easier to start using, to explore, and to scale and we we think that our customers will find the same.

What we’ve done

The SDMX API that we are focused on can be broken into three logical chunks:

1. Metadata Discovery - what data collections are available, and what concepts/classifications are used where
2. Database Metadata Discovery - What metadata (eg: concepts and code lists) are used within a particular SDMX dataset?
3. Queries - Defining and pulling back a slice of an SDMX data cube

We’ve implemented parts 2 & 3.  (Part 1 we will consider for a future version, but we are also looking at solving this gap in a different way, such as leveraging existing SDMX registries, which are used to collate and manage contents that are stored in SDMX repositories. The important thing to note here is that we don’t want SuperSTAR to be an island - many of the organisations we work with would want to reuse the same search and discovery mechanism across many different types of data and applications, so we’d like to learn more about how SDMX solutions can be part of such an environment before we proceed with this.)

Our SDMX Restful API supports access to aggregated data that is managed by SuperSTAR. This can be from several different sources:

1. SuperSTAR data cubes
2. SuperSTAR tables defined by SuperWEB users
3. SuperSTAR microdata databases

The last case is worth elaborating on, and links back to the point I mentioned earlier about introducing SDMX to microdata. Up until now, SDMX use has been limited to working with pre-aggregated data. This makes sense, especially when you consider the origins of SDMX, which is a group of organizations that deal almost solely with such aggregated statistical data and only rarely with the underlying microdata from which the statistics were derived.

From our point of view, however, and I believe from the point of view of many of our customers, dealing with microdata is very much part of the production process that they are involved in. What is useful about this is that the users are not constrained to taking slices of pre-defined cubes of data, but rather exploring and dynamically defining queries to run against the microdata. This approach can generate orders of magnitude more possible outputs and therefore relieve the provider from the burden of manually addressing many ad hoc queries that can’t be satisfied by a query against an existing cube. It does occasionally introduce other problems, namely confidentiality and performance, but these are part of our core capabilities, so our solution addresses potential drawbacks in this regard.

To make it possible to use an SDMX-based API to run tabulation queries against microdata, we’ve made some necessary innovations to the SDMX standard. Firstly, while you can query for the data structure definition (DSD) of a very large virtual cube (which is actually a SuperSTAR database), we prevent clients from requesting the full dataset for this cube - it’s simply going to be too big. What we do instead is allow for any subset of dimensions in the DSD to be combined in an SDMX query.

In addition, any tables that a user defines in SuperWEB can be accessed as SDMX datasets; both the DSD and the data from such a table can be obtained through queries against the SDMX RESTful API.

If you’ve read this whole post, you must be interested in what we are doing here. We think that the API can be very useful for many of our customers, so please leave a comment here if you have a question or something say. Or if you want to go one step further, let us know and we’ll discuss providing you with a test package that you can use to try the API against your own data.

I’m excited about our two applications because they are genuinely useful online interactive publications of complex data that everyone will benefit from.  Our Why Australians Travel application presents a dataset from Tourism Research Australia that has not been made available to the public in an interactive way before.  It also includes advanced visualization in the form of a Motion Chart (Gapminder-style) which we’re very excited by! The motion chart can tell a story with data over time that you simply don’t see in static tables or reports.

The How Safe Is Your Suburb 2.0 application provides NSW Crime data in an interactive way, allowing users to analyse relative crime rates ot absolute crime rates by suburb.  This application is supported by one of our newest features - metadata -where explanations about the data are provided to the user to help them understand the meaning of the data.

Go check our applications out and vote for us if you like them!  And if you have any feedback on our entries please don’t hesitate to make a comment on our blog here.

Said Don:
“Many people, especially Gen Y, have the view that privacy is not an issue for them and to quote Eric Schmidt, ‘If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.’ I much prefer the view of Bruce Schneier, who is pretty much the world’s leading expert in information security, who points out in an excellent essay very clearly that people espousing that view ‘… accept the premise that privacy is about hiding a wrong. It’s not. Privacy is an inherent human right, and a requirement for maintaining the human condition with dignity and respect.’”

Click here to listen to the podcast.

Using our new SuperVIEW Collaboration features, you can make comments or invite others to make comments on your visualizations using Google Friend Connect.  You can also share your customized visualisation with others using our new Share feature. The Share feature allows you to embed a link to your view in a website, blog, Facebook, Twitter or your other favorite social networking application.

Recently Craig Thomler, a well-known active participant and leader in the Australian Gov2.0 movement, wrote a blog post on the new data.gov.uk site which he considers is the world leader in open data websites.  He then goes on to make a wishlist of what we could do in Australia to the data.australia.gov.au site to make it the best in the world.  Some of what he is asking is for is delivered by SuperVIEW right now including the ability for people to embed visualizations into their own sites, and to allow every set of data to support a discussion to allow people to ask questions to clarify what the dataset contains and discuss how it could be presented in a more usable way.

View this video to see SuperVIEW Collaboration in action.

If you have any questions about SuperVIEW please contact  jo.deeker@spacetimeresearch.com

Until recently, many government agencies have not been motivated to provide data transparency. Compared with the work that directly aligns with their mission and funding being a visionary supporter of the principles of transparent government is not really high on the agenda. In fact, in many cases, the message from up high hasn’t really reached them at all (one senior US government official’s take on Gov 2.0 was “oh, that’s a subset of Web 2.0 isn’t it?”). If you add to this reluctance the quite significant disincentives such as the risks of being too transparent, inadvertent privacy breaches, and plain and simple costs, then it’s not surprising that the average department hasn’t been as enthusiastic as the Gov 2.0 activist community might like them to be. And if the ROI on the whole deal is often external, why bother?

Well, there’s nothing like a directive straight from the top to get things moving. As of December 8, U.S. federal agencies had 45 days to get three “high-value datasets” published online and available through data.gov. Wow! Having worked with national statistics agencies for many years, I have some grasp of how long they typically take to publish data and it’s often longer than this, especially when you are dealing with data that has not previously been published. Of course, the data in some cases might be basic lists of non-sensitive material, in which case perhaps it is not too much extra work to make it suitable for public access. What I’m interested in examining is what it will take for agencies that don’t have it that easy, who will need to derive statistics from their data, or reduce it in some way to make it “safe” for public consumption.

Firstly, why bother publishing statistics if the raw data is available? Isn’t the open data community interested in getting “raw data now”, so that it’s quick for the agency and promises maximum flexibility for users? The reality in many cases — and one that seems to still be ignored by some who work in Information Management — is that even after you “de-identify” data by stripping obviously identifying attributes from it such as names, addresses, SSNs, etc, it does not necessarily protect privacy. It can still be a fairly trivial exercise for an ill-meaning data analyst, or even a non-technical person in many cases, to re-identify many of the people in the list. That is why in many cases we’ll see statistics being released about the data, rather than the raw data itself.

Associate Professor of Law Paul Ohm from the University of Colorado released a paper about the “Surprising Failure of Anonymization” last year, citing some prominent cases where anonymized data was re-identified and pointing out that there are many laws and regulations that are based on the false assumption of anonymization being a panacea for data privacy protection. In one example he describes, a researcher demonstrated how 87.1% of people in the U.S. were uniquely identified by their combined ZIP code, birth date, and sex. He also covers the AOL search data scandal, where individuals were identified from vast volumes of data by their unique search habits, uncovering some embarrassing personal information along the way.

While the individual agencies may not all have a clear understanding of all the potential privacy issues related to open data, at least the federal administration does have a focus on this. The directive itself states that data can only be made available “subject to valid privacy, confidentiality ….. restrictions”. In addition, the “Concept of Operations” paper for data.gov does have privacy in its sights, stating that there will be working groups looking into privacy issues arising from how data is mashed up and/or used in applications. I would point out that these groups could make an early head start simply by reading Paul Ohm’s paper, and not wait until after this round of data has been released. It seems that for the moment at least, the idea of what constitutes adequate privacy protection for open data is really up to each agency to decide.

While the working groups deliberate how privacy issues that result from data mashups and the like should be addressed, many datasets will be posted to data.gov and despite the proven limits of the effectiveness of anonymization, the experience that my colleagues and I have gained from talking with people who work in Information Management in government is that key staff in at least some agencies are not sufficiently aware of this, and that in their view, anonymization is essentially all you need to do to make data safe for release. I’d be interested to know if this agrees with others’ observations.

My observation regarding government’s understanding of data privacy issues is based largely on anecdotal evidence collected by myself and my colleagues. Perhaps I am overstating things and agencies do have the required skills and knowledge to release data safely. It would be good to hear about how different agencies are dealing with the Open Data Directive and what you think about the challenges of releasing useful data without unduly compromising privacy.

Note: Ohm’s paper is fairly lengthy. For a very interesting summary of the paper, you can check out this post on ars technica, which sparked a lot of debate regarding the importance of privacy.

]]> http://www.spacetimeresearch.com/archives/187/feed.html http://www.spacetimeresearch.com/archives/187.html