The Improbable Blogger

New surveys reveal dynamism, challenges of open data-driven businesses in developing countries

2014-12-16T11:04:00.002-05:00

Open data for economic growth continues to create buzz in all circles. We wrote about it ourselves in this blog earlier in the year. You can barely utter the phrase without somebody mentioning the McKinsey report and the $3 trillion open data market. The Economist gave the subject credibility with its talk about a ‘new goldmine.’ Omidyar published a report a few months ago that made $13 trillion the new $3 trillion. The wonderful folks at New York University’s GovLab launched the OpenData500 to much fanfare. The World Bank Group got into the actwith this study. The Shakespeare report was among the first to bring attention to open data’s many possibilities. Furthermore, governments worldwide now routinely seem to insert economic growth in their policy recommendations about open data — and the list is long and growing.

We count ourselves among the believers (even as we roll our eyes over some of the numbers!), but what strikes us most about these reports is their northern focus ̶ the McKinsey report, for instance, is ‘global’ but concentrates almost exclusively on North America and Europe. The Shakespeare report was written for the U.K. government, and the Economist report fails to include any examples from the developing world. And the Omidyar report is broader and covers the G-20.

How are open data-driven companies doing in emerging economies?
The question we had was there a class of entrepreneurs emerging to take advantage of the economic possibilities offered by open data, were investors keen to back such companies, were governments tuned to and responsive to the demands of such companies, and what were some of the key financing challenges and opportunities in emerging markets? As we began our work on the concept of an Open Fund, we partnered with Ennovent(India), MDIF (East Asia and Latin America) and Digital Data Divide (Africa) to conduct short market surveys to answer these questions, with a focus on trying to understand whether a financing gap truly existed in these markets. The studies were fairly quick (4–6 weeks) and reached only a small number of companies (193 in India, 70 in Latin America, 63 in South East Asia, and 41 in Africa — and not everybody responded) but the findings were fairly consistent.

The time is right for an open data fund

2014-03-28T11:16:00.000-04:00

Nobody cares about open data. And they shouldn’t. What people care about are jobs, clean air, safety and security, education, health, and the like. And for open data to be relevant and meaningful, it must contribute to what people care about and need.

We wrote a few weeks ago that the private sector is increasingly using open data in ways that are not only commercially viable but also produce measurable social impact. What is missing is financing that can help catalyze the growth of data fueled businesses in emerging economies. We are developing a fund that will address this precise need.

The data revolution - has it occurred already

2014-03-24T16:37:00.002-04:00

It's quite fashionable to speak of the impending data revolution, and the academic/development community in particularly is busy getting ready for it. Quite inevitably, at least some of the discussion has centred on well-known (but still important) issues such as greater capacity building, a more central role for national statistical offices, increased standardization of data collection efforts, smarter partnerships, and ineluctably, more resources for data agencies. But in the shadow has lurked – as the report itself recognized – the specter of big data and new data techniques, the recognition that the world of data may have undergone a revolution already and that some cherished truths may no longer apply.

I found my way into a couple of conferences recently – one organized by Webdatanet and the other by UNECE, where the revolution – impending or underway depending on whom you asked - hung heavy in the air. These were gatherings of experts with authoritative reputations in the data community, and with the tools to discern that the ground was shifting beneath their feet. So while calm reigned on the surface, both the events roiled with questions such as have machines taken over the world of data already, has the traditional data expert become irrelevant, have old data coaltions become obsolete, do the standard rules of data no longer apply, what's the value of the data revolution going to be...

Some answers and more at http://blogs.lse.ac.uk/impactofsocialsciences/2014/03/05/specter-of-big-data-revolution/

From open data to development impact – the crucial role of the private sector

2014-03-03T14:56:00.002-05:00

Can open data lead to reduced energy consumption (and therefore slow down climate change)? Can open data help improve maternal health services (and thus improve facets of public delivery of services)? Can open data help farmers and crop insurers make better crop predictions (and thus lead to smarter investment decisions in agriculture)? Can open data empower citizens to fight back against police corruption (and thus help promote the rule of law)?

Is there more to open data than transparency and accountability?

The answer to all the questions above is a resounding YES – at least far as the developed economies go. The private sector is constantly building new businesses and services that use public data to provide services that are not only commercially viable but also create positive development impact. Companies such as Opower (over 3 billion kilowatt hours or about $320 million in utility bills saved), Dimagi (creating cell phone based data to help deliver maternal health services), Climate Corporation (better crop yield predictions), and many others are just some examples of a fast growing ‘new asset class’ built substantially on publicly available data (that the Economist recently likened to ‘a new goldmine’) that is likely to have a transformative impact not only economically but also in terms of development outcomes. And it’s a space that the World Bank and other development agencies have only nibbled at so far.

The question at the table is what can development organizations do to catalyze this market and hasten the associated development outcomes, especially in developing countries? How can governments and private sector companies in developing countries work together to take advantage of commercial and development opportunities in what McKinsey estimates is a $3 trillion+ open data market? How do we make sure that the growing amount of open data is truly used in new and interesting ways that early open data pioneers had hoped for?

More at http://blogs.worldbank.org/opendata/open-data-development-impact-crucial-role-private-sector

Speaking in their voice

2014-03-03T14:54:00.001-05:00

Our world is awash with increasing amounts of data, but potential audiences for this data remain under-served for the most obvious of reasons - the data just doesn’t speak their language.

This has been true for the data on the World Bank Group Finances website which has only ‘spoken’ English since it was launched. Yes, we should have done this earlier but the website, and its associated open datasets, are now available in 5 new additional languages - Chinese, French, Hindi, Russian, and Spanish . The mobile app has been available for some time in 9 languages (Arabic, Chinese, English, French, Hindi, Indonesian Bahasa, Portuguese, Russian, and Spanish) and the new release of the website is in line with the program’s quest to include new audiences and communities in the use and dissemination of open financial data.

More at http://blogs.worldbank.org/opendata/speaking-their-voice-world-bank-group-finances-goes-multilingual

Talking big data for international development

2013-05-28T16:05:00.000-04:00

The final report from the Big Data for International Development DataDive came out a few days ago and the obvious question is what's next? Here are a few ideas...

Here also is a quick list of related blogs that got us to the report (there're plenty of better ones written by other people - links within the blogs listed below).

What's next after the DC DataDive
Video report from the DC DataDive
Talking data philanthropy at a Webinar
Big data analysis interview with the BIG project
Personal data philanthropy blog with Giulio (who did most of the thinking/writing)
Big data Q&A with UNDP
On big data for operational results
Starting small with big data

Let's see where this goes next.

Open for business - open data gets serious

2013-05-21T17:31:00.000-04:00

Open data for business is suddenly the rage. The Economist calls it the new goldmine, the new open data policy released by the US government explicitly links open data with 'entrepreneurship and economic growth', a Capgemini report recently valued the impact of open data on the EU27 economy at 32 billion Euros in 2010, other estimates put the potential of open data in Europe at 180 billion a year, McKinsey valued health data alone at $350 billion annually - the numbers are eye-popping and 'no one has a clue what breakthroughs open data will allow'. The conversation around open data has definitely shifted beyond transparency, accountability, and civic engagement.

The best evidence for the potential of open data comes from weather and GPS data - open data from a time before the open data movement really took root. The recent surge in open data has only just about begun to produce commercial results - the health sector in the US seems especially ripe for open data based services. The biggest potential may however be in the developing world where mobile based Internet, open/big data, and the demand for new business services, especially from the so-called 'bottom of the pyramid' sector, may be creating the perfect storm for new businesses to emerge. There may be a genuine opportunity for entrepreneurs in developing countries to create new businesses based on open data that aren't just offshore services but that serve a new niche - as Ushahidi for instance does.

How do you kick start the open data for business movement in developing countries and create an ecosystem that entrepreneurs can tap into?...

More at https://blogs.worldbank.org/opendata/open-business-open-data-gets-serious

A rebellion in slow motion

2013-04-22T23:26:00.000-04:00

Chandan will no doubt scoff at the apparent hyperbole but Kindling of an Insurrection: Notes from Junglemahals may well be one of the more important books written about modern India. That's a tall claim to make (and one immediately vulnerable to the 'sweeping statement' slur) - especially when books about India seem to be everywhere and critics can't swoon enough about them. Sadly, almost all the writing about India is formulaic and unimaginative - mostly glorified travelogues sprinkled with a bit of second-hand history, a smattering of personal experiences, a couple of 'telling' anecdotes, and awe and wonder about the hustle and bustle of a rapidly changing country and how the ordinary Indian is coping with and flourishing amidst a remaking of everything from society to culture to politics to religion to economy to you name it. All written in gossipy-but-not prose that is fluent but interchangeable, the sort that high school teachers and newspaper style pundits love, that is glib and comforting, and that confirms our world-views by challenging it in the most predictable ways.

Chandan's book has none of the alleged strengths of modern Indian writing. There is no color, the language ebbs and ebbs (in deference to bureaucratic tradition), there are no memorable characters (or elephants, or sadhus, or taxi drivers, or fixers, and the like), and there is very little drama - not even somebody suddenly breaking out in a pithy summation of all that is wondrous about India and what ails it. There is only a foreboding - a quotidian assembly of quiet, patient discontent breaking out almost imperceptibly into a rebellion.

Chandan records the slow-moving storm through a series of notes recounting his tours as a District Magistrate in Paschim Medinipur in the early aughts - here was an itinerant officer (for that is the way the 'elite' in the Indian bureaucracy work) doing his best to implement a vast arsenal of government programs that seldom translated into reality on the ground. There's almost no issue that he encounters for which a barely functioning government program doesn't exist; the villages teem with visible signs of government investment, there are local schemes everywhere, and there are government and non-government agencies for everything. And there are people who demand little of the government, who understand that the system often works poorly for the poor, who are willing to wait and claim less than what was promised to them, who will take even fitful progress...if only...

If only.

Chandan is a meditative writer, he pauses frequently to reflect on the situation on the ground, and he writes achingly about what must be done. He describes the daily lives of the people that he meets in simple, honest, uncomplicated terms - these descriptions accrete gradually, the story of one modest despair piling into another till it becomes the grand narrative of an inevitable insurrection. And while the storm gathers, he writes in measured terms about how government works on the ground, the chasm between grandiose policy and the little actions and inactions that represent the government for the poor, the checks unsent, the ponds repaired, the village roads forsaken, the solar panels in the remotest villages, the kids lining up to sing for 'dignitaries', the truant doctors, the bicycle chases, the lists, the checklists...and he writes almost lyrically about the land, he names the trees, he describes the building materials of the houses and their roofs, he talks about the cattle in the villages, and as he stops to meet old women carrying housekeys that signify their status, he brings the real - the truly real - India alive.

Read this book.

Pandit Ravi Shankar - he was no 'expert'

2012-12-12T08:29:00.000-05:00

I am fairly comfortable talking about plenty of things I don't know a lot about. Indian classical music is not one of them. That said, the passing of Pandit Ravi Shankar is a sad, sad moment. Let others judge his music - I enjoyed it immensely, especially live - what I admired the most about him was his dissatisfaction with mastery, his constant quest to go beyond the traditional boundaries of his discipline, and his embrace, however unnerving to others, of the alien. Traits we can all learn from.

The new WBF mobile app

2012-12-05T15:37:00.001-05:00

Let’s say you are in the middle of what others may call ‘nowhere’ and need information on the Bank’s work in the vicinity before an upcoming meeting with local officials. Or you are a civil society organization rep and want to make sure that the numbers you have about a particular project are the same as what the Bank reports (and if not, you want to know why not).

Your laptop is no good because - it is the middle of nowhere after all! - and you can only rue your decision to leave your stack of papers behind.

What do you do? Well, the answer might be in your pocket...

Big data - you can start small

2012-11-12T11:21:00.002-05:00

Big data. Transformative? Possibly. But how do you test the waters? Especially if you aren't really a data driven organization in the traditional sense. Here's a recap of an event that focused on the practical side of big data.

Life in the post-transparency age

2012-09-01T20:13:00.002-04:00

Can open financial data help organizations increase development impact, can the availability of open financial data help organizations either answer new questions or answer existing development questions differently, can open financial data become a part of how organizations work (and how their partners interact with it)?

Join the conversation on September 5.

Cote d'Ivoire major contracts - community generated data view

2012-08-22T13:45:00.002-04:00

Cote d'Ivoire

GB Trust Funds at the World Bank - community generated data view

2012-08-22T13:41:00.000-04:00

GB payments to trust funds

Data standardization. Really?

2012-06-05T13:01:00.001-04:00

Everybody loves perfection. If you were perfect everybody except you would love you. If the world were perfect, you wouldn't need to do a thing and the things would be just perfect.

Take the world of data for example. Wouldn't it be perfect if perfect standards existed and people followed them perfectly. Wouldn't we then be awash with glorious new insights, wouldn't our decisions be so much smarter as data, gorgeous data told us brilliant new stories, wouldn't we always know what to do and why and know how it turned out before doing it.

If only the shirkers knew how to publish data perfectly.

Sadly, like much else in the world, only the boring and the cloistered achieve perfection. The most interesting data is often imperfectly published - sometimes because it has a personality and its seeming imperfection reflects its eccentricities, sometimes because data is a byproduct of other more pressing goals, sometimes because it is a result of how people work (successfully), sometimes because some people really are incompetent or ignorant and you can't change that, sometimes because...well this can go on!

Is that really such a bad thing? Sure, it is vital to establish and enforce data standards for mission critical tasks that depend on data (say your payroll and financial management in general), or when data standards are a crucial component of how knowledge is described and propagated and its audience is made up largely of insiders (think the academic world) but the argument loses force as data becomes more democratic and its sources expand to non-traditional creators and consumers. When everything you do becomes data, should indirect or informal creators of data be forced to adhere to standards before publishing data?

Yes, if you listen to the standards bodies and groups mushrooming everyday. This has to be heyday of the standards movement -- there is no conventional wisdom like the wisdom that all data must adhere to conventions (even though most of these conventions are either partial or hopelessly behind the times because the bureaucracy required to maintain these standards can't keep it with life which unhelpfully marches ahead most unpredictably). But no, if you listen to everyday publishers, who want to put the data out there and get on with their lives leaving it to others to use it in contexts and fashions most valuable to them. Sort of like the billions of content publishers on the Internet who don't worry about content management standards so beloved of content strategists and depend on services like Google to find and contextualize the content (based not on standards but algorithms that assume and respect the fact that people will be people). Wonder why you can never find anything on your Intranet (despite all the superb metadata you add, that experts spent months perfecting) but Internet search engines do sort of okay with content published by everyday folk who never heard of metadata? Do you really want to repeat teh same mistake with data?

If data has to ever achieve the ubiquity of other content, it must be rescued from specialists (sort of like how editors must be kept away from blogs, how filmmakers can't be allowed to dominate YouTube, and how Twitter must be shielded from communications experts). The specialist cannot see the data except in narrow, prescriptive terms - the aesthetic of the specialist, while wonderful in itself, is designed to keep people out rather than let them in.

Make data easier for people who produce it - leave it to others to create tools to use it.

A case against citizen reporting

2012-06-05T11:18:00.000-04:00

Citizen reporting is all the rage and perhaps for good reason. There is plenty of evidence from projects around the world that it is indeed possible to mobilize citizen reporting in projects and achieve positive results. The basic premise is often pretty straightforward - provide affected citizenry adequate information about development activities that impact their lives, give them them tools and incentives to participate and 'report', and use the information to make better decisions and create superior services. It sounds grand and often works.

The question is can it work often enough and is it really a good idea to create new citizen reporting channels in every potential project (some projects, that do not produce visible, on the ground results, sort of disqualify themselves at the moment)? Are new conversations really the answer? Is there a case to use citizen conversations already underway and focus on harnessing this intelligence rather than creating new, narrow (often project specific) conversations?

Most lives are not lived around single issues or themes and, while we sometimes like to imagine that it is possible and beneficial to create conversations and reporting around solitary projects or donors (teacher attendance is a favorite example!), people seldom naturally cluster around singular themes (except in extraordinary situations, such as a disaster or an especially heated political event). Obvious as it sounds, lives are complex and development is rarely just one thing or topic, and people's conversations in real life already reflect the interplay of all factors that are meaningful to them. So why not listen to conversations/reporting already underway and seek out the 'reports' you need rather than impose an arbitrary and evanescent new conversation? Why not find better tools to listen to Facebook or Twitter, and even newspapers, local leaflets, community meetings, citizen journalists, and the like? Why not go where the conversation already is? Don't ask people to send photos to you, go where they are already sharing them (just figure out how to find and use them).

None of this is to say that 'traditional' citizen reporting (as I described in the opening paragraph) doesn't have a place in the development toolkit. It is often crucial during extraordinary circumstances, it can be a powerful tool to put pressure on governments in the absence of suitable political institutions, and it is undoubtedly non-negotiable for pressure groups focused on single issues. The trick though is to walk away from the trend that sees citizen reporting as the panacea for much that ails international development and to build it as a specific layer in projects.

Do not make it an automatic reflex.

DFID contributions to WB Trust Funds

2012-05-16T16:55:00.002-04:00

UK-DFID TF contributions by Fund

Ongoing IDA projects in Moldova

2012-04-17T10:37:00.002-04:00

IDA ongoing

Interesting data visualization on World Bank Finances

2012-04-16T17:47:00.000-04:00

IBRD - Borrowers' Obligation for BRICS

From open data to public data

2011-12-12T17:39:00.000-05:00

Is open data just a glorified form of publishing or can its benefits go beyond transparency and reusability? How do you take open data beyond the realms of traditional publishers and data sources and spur people affected by the data to participate and contribute new ideas/data about development (and in effect become open data/development partners)?

What do you think? Join the conversation right here.

Taking open data to the people

2011-08-09T22:00:00.001-04:00

Is open data useful only to developers and researchers? Can 'average' users use open data to answer questions they have?

One of the (undeserved!) knocks against open data is the presumption that its core audience is technical and that the only people who can truly take advantage of open data are developers who can tap into APIs to build applications that then make sense of open data for lay audiences (unless the audience happens to be researchers in which case they probably have the necessary tools and the forbearance to troll through vast amounts of raw material). Viewed through this prism, open data is only effective via infomediaries.

That doesn't necessarily have to be true...but it is crucial to provide interactive tools that invite people to use them, that are simple, and that explicitly encourage and demonstrate social behavior. When somebody comes to the site with a question, the site must intuitively guide the person to an answer, or to a community that may have the answer, rather than just throw data at them - that kind of blunt, brute force 'transparency' can be traumatic.

Easier said than done though. Join the conversation via a another blog post I wrote about open data of the people, by the people, and for the people.

On failure

2011-08-08T14:53:00.000-04:00

If you haven't failed, you haven't tried hard enough (and if your project is unlikely to fail, it probably isn't worth doing). Failure is good. How will you ever learn to aim high if everything you touch turns to golden dross.

That said, we probably eulogize failure too much. Far too many people wear failure as badges of honor or disingenuously conceal incompetence under the guise of heroic failure. The infatuation will lessons of failure must end. Truly honorable failure always expands knowledge -- if you have failed the same old way, you have truly failed. Irredeemably so.

Open data -- what next

2011-08-04T23:53:00.000-04:00

So you've worked really, really hard and finally launched your organization's open data website. It's a success and you're tempted to rest on your laurels but can't shake off the sneaking feeling that where you are is really the beginning of the project rather than the end. The conversation has in fact only just started. But what do you do beyond launching an open data site?

Some ideas at http://blogs.worldbank.org/insidetheweb/node/568 -- join the conversation and share what you've done!

From open data model to open data website

2011-07-11T21:11:00.000-04:00

The open data model scribbled on a whiteboard finally becomes a real website. Let me know what you think and what data you'd like to see on the website. Also watch for a mobile app coming soon.

An open data model

2010-08-19T11:57:00.001-04:00

A quick and very rough model sugesting a way to describe the role/value of open data in your organization. It probably needs a lot of background talk to truly make sense but grateful for any thoughts/feedback.