It’s time to say goodbye to the open data portal. Here’s why…

by Eddie Copeland
394 views

Look at some of the most transformative operating models used by leading digital organisations such as Wikipedia, Twitter and TripAdvisor and what do you notice?

Their value comes from sourcing data from outside their organisation.

In light of this, many cities’ approach to open data can be seen to have a glaring shortfall: the flow of information is in just one direction.

Businesses, charities and citizens can receive data from government. But – with a handful of exceptions such as the Leeds Data Mill – there are few official, standardised and automated mechanisms to provide data to government.

That is a huge missed opportunity. External sources of data could be of enormous value to city authorities.

Consider, for example, how electronic payments companies know where people come from (based on the card address) and exactly where and how they spend their money – data that could provide powerful insights into how citizens and tourists interact with a city.

Consider, too, how mobile phone operators and companies like Google and Apple collect data from handsets in order to provide their services – data that could be used to create real-time maps of urban traffic or pedestrian flows.

Heat map showing pedestrian density in Kensington and Chelsea

Heat map created from aggregated and anonymised mobile phone data showing pedestrian densities at 19:20. This type of map would be used to give a very high-level view of pedestrian demand patterns across a city centre, and over a 24hr period. The image is taken from a time-sequenced video. Source: Citi Logik – see: www.citilogik.com – reproduced in Smart Devolution, a report by Policy Exchange.

And consider supermarket retailers who obsessively monitor and analyse data on their customers’ purchasing habits – data that could help reveal the economic health of a local area.

All these examples speak to a more important point: in any given city there are many different creators (or owners) and consumers of data. Just as businesses have information that could be valuable to local authorities, citizens may have information valuable to businesses or volunteer groups. Businesses may have data that could be repurposed and used by other businesses. In short, city data lies in many places and could deliver value to different people.

The problem is that there is currently no mechanism to efficiently connect those with data to those that need it. Too often data remains in its silos, its potential unrealised. So how can cities unleash the power of city data for the benefit of everyone?

I believe the answer is to replace the open data portal with a City Data Marketplace.

The City Data Marketplace

A City Data Marketplace (CDM) would be an online marketplace that connects organisations and individuals that have useful data with those that want it.

Similar to how a site like TaskRabbit.com connects those offering their skills for hire with those that need jobs done, the City Data Marketplace would bring together creators and consumers of data.

The site would have two types of listing:

Creators:

Individuals and organisations with datasets could advertise them on the site, specifying the quality, format, range and scope of the data, together with the price (if any) they would charge for providing them. (Local authorities and public sector bodies could continue to use the same marketplace to offer their open data for free.)

Consumers:

Those needing data – whether a local authority, a business, charity or individual – could advertise their data needs. They would specify whether it was for private / internal use (“shared data”), or whether they wanted to release it as open data. Creators could then see that request and publicly bid to provide the data, specifying how they would source it, its quality and so on.

This marketplace solution would offer at least eight major benefits:

1. Increase availability of previously inaccessible datasets. The marketplace could help unlock thousands of datasets that currently remain hidden within their organisations. Though some private sector organisations such as Google, Uber and Vodafone are already offering some of their data to cities, creating a mechanism to monetise data would provide an incentive for many more businesses to share their data. For the first time, people and organisations outside government would be able to see what data public authorities actually need.

2. Increase innovation. Instead of cities specifying the exact data they think they need, they could instead put a request on the CDM outlining the problem they want to solve. A variety of different individuals and organisations could then offer data that solves that problem in different ways.

Imagine a city that wanted data on traffic flow to better manage congestion. City officials may start with the assumption that they need to work with a technology vendor to install an IoT network with embedded sensors on every street. But by posting the challenge on a CDM, they may find that mobile network operators can solve the same problem at a fraction of the cost by providing aggregated and anonymised data from phone signals. Or perhaps a volunteer group could offer to create an app for free that helps citizens report congestion in their area. By focusing on ends rather than means, the marketplace would reveal the available options.

3. Competitive pricing. Some organisations could offer data for free (e.g. local authorities offering open data), others might charge, but the amount requested by consumers or listed by creators would be publicly viewable, creating competition to find the best price for any particular type of data. As a result, data from different sources would be more accurately priced. (I’d suggest that pricing should based on the value of the outcomes the data can enable rather than on a per byte basis.)

4. Review and feedback. Similar to reviews of buyers and sellers on sites like eBay, the quality of datasets and data creators would be reviewed and ranked so that others could determine whether the data provided by each organisation was to the quality specified, encouraging better practice.

5. Potential new revenue stream for city authorities. If city authorities hosted the marketplace, they could charge a small commission on any data sales conducted through the platform, helping to cover the costs of its creation and maintenance.

6. Help spread best practice. Currently there is no mechanism for local authorities or cities to see how other regions are using and paying for data to solve specific urban issues. Using the CDM, best practice could be shared as all purchases of data would be publicly viewable.

7. Highlight which open data sets cities should provide. The UK currently seems to measure its success on open data by the number of datasets it lists online. That is a poor measure. What matters is what the data is used to achieve. By offering open data alongside paid-for data from other sources, it would be more transparent to cities which data sets were of real value to their communities.

8. Policy based on hard data in place of modelling. By providing access to a wealth of new data, cities would be able to design better initiatives and policies based on actual data rather than restricted estimates or models.

What about privacy?

Clearly, the introduction of CDMs would need to be accompanied by proper protections for citizens’ data. The Information Commissioner’s Office (ICO) has a detailed Anonymisation Code of Practice. Any organisation wishing to sell or share their data via the marketplace should be obliged to certify and provide evidence that they have met that code. Where public sector organisations within cities wish to use private sector data, they would share responsibility for assuring that the data they procure has met ICO regulations.

From Open Data to City Data

The rise of the open data movement has already done much to change the way that local authorities and public sector bodies think about the value and purpose of the data they create, collect and use.

But it only represents the first step.

For innovation to thrive, cities need to move from open data to city data – unleashing data from wherever it may reside and making it available to those who can best use it.

To that end, it’s time to upgrade the humble open data portal and turn it into the data marketplace that cities really need.


For further details and casestudies on this idea, see Smart Devolution – a report I co-authored with Cameron Scott for the think tank Policy Exchange.

Find me on Twitter

You may also like