Why Governments Struggle to Innovate

by Eddie Copeland

“Why can’t governments and public sector organisations just be more like the private sector?”

This is a common question in the field of government innovation, and on the face of it it seems reasonable enough. After all, companies adopt new technologies to reshape and upheave entire industries. In recent memory, Amazon reinvented e-commerce; Facebook, the way we communicate; Google, the way we search and access information.

What’s government transformed lately?

While there are many lessons that government and public sector organisations (GPSOs) can learn from the corporate world (for example, regarding the use of technology, demand management, and user-centred design), there are also five significant dissimilarities between the two sectors that make government innovation different, and arguably harder. For unlike the private sector, government and public sector organisations:

1 – Can’t choose their customers or services. Successful companies relentlessly focus on defining and catering for the needs of specific market segments. They can choose to stop serving low-value customer groups. They can cease offering product lines that are deemed unprofitable. GPSOs can do none of these things. Instead, they have a duty to serve their entire community, including groups with the most diverse, complex and costly needs.

2 – Can’t go out of business. A key engine of the private sector’s dynamism is the creative destruction that comes about when poor performing companies are driven out of business by the competition. This corporate natural selection creates a huge incentive for companies to innovate or die. It’s hard to replicate those same incentives for GPSOs, which are created by law and are legally mandated to provide certain services no matter what the external environment.

3 – Have more numerous and diverse lines of business. City and local government bodies can be responsible for providing more than 900 different services, but with a staff numbered in the hundreds. The only parts of the corporate world where such diversity exists is in mega conglomerates, which tend to have tens of thousands of specialised staff.

4 – Must address some of society’s most intractable issues. The needs that GPSOs are tasked to fulfil are among the most complex imaginable. Providing meaningful support to improve the life chances of vulnerable children in families facing numerous factors of deprivation is a world away from providing a seamless retail experience. Innovating on a core service by adopting the startup mantra of ‘failing and learning fast’ can also feel much harder when failing can have potentially life-changing consequences.

5 – Deal in areas that are less likely to see productivity gains. Repeated studies, including Nesta’s Future of Skills – Employment in 2030 report, have made clear that many public sector activities are inherently unsuitable for productivity gains. While the cost of making widgets for, say, a television plummets as the technology gets better, human-centred services such as social care cannot make those same gains.

In short, we should be wary of over simplistic recommendations for government to copy and paste practices from the private sector.

From idea to action

Another way to understand why GPSOs struggle to innovate is to look at how good ideas typically come to life. The key steps are outlined in Nesta’s innovation spiral, shown below. It starts with seeing opportunities and challenges; sparking ideas to tackle them; shaping and testing the most promising ideas in real world experiments; before making the case to scale proven methods and shifting the way entire systems work.

Nesta Innovation Spiral

At every step of the spiral, GPSOs face barriers to innovation. Consider that:


The challenges faced by GPSOs are numerous, diverse and complex. In short, innovation is hard because the problems GPSOs have to tackle are among the most difficult to understand. Equally, there are few simple or unequivocally positive opportunities. Many potential routes to resolve issues are controversial or leave some citizens worse off.


When it comes to generating new ideas to resolve these difficult challenges, there are limitations in where ideas typically come from.

In countries where budget cuts have been severe, the reduction in staffing in many GPSOs has left many people performing roles that were once the remit of two or more people. This has inevitably resulted in less time and space to think about and implement new ways of doing things.

Criticisms have also been levelled at the lack of diversity in many of the organisations charged with coming up with new policy ideas. In a UK context, many Whitehall civil servants have similar educational backgrounds and tend to live within London’s commuter belt. And while the UK has a buoyant ecosystem of more than 120 think tanks, 80% of them are based in London, and 75% of them are led by men, pointing to a lack of geographic and gender diversity. If diverse views come from diverse people, this is problematic.

Outside the policy world, many ideas for innovation come from the technology sector. Some of those ideas can be hugely beneficial. There are two common shortcomings. First, the tech sector has a tendency to over-hype the transformative potential of new technologies. If the reality fails to live up to those inflated expectations, it can lead to disillusionment, dissuading GPSOs from trying genuinely useful innovations that could really help them. Second, those working in the tech sector are, inevitably, not necessarily those with a deep understanding of the challenges and limitations facing GPSOs. Some of the products they pitch to the public sector have a tendency to look like solutions in search of a problem.

Finally, even when genuinely good ideas are suggested, there are challenges as evidence is often undervalued. While governments publicly state their commitment to evidence-based policy making, all too often the norm is “policy-based evidence making”. Governments decide what they want to do, and then seek out the reports or evidence that back them up.


Nesta firmly believes in the value of testing new ideas in the real world. We recognise that trialling new ideas requires time, money and people. For many GPSOs, all these are lacking.

Even where resources for experimentation are available, some GPSOs find it hard to justify the use of taxpayer money on unproven ideas. If public sector leaders have to choose between delivering a core service on which many citizens depend, or funding an experimental new approach, many will feel they have no choice but to focus on the former. Testing may be also be precluded, for example where it’s legally, ethically or practically impossible to treat groups of citizens differently.

All these barriers can lead to promising ideas never getting as far as being tested. More damagingly, sometimes GPSOs skip the testing stage altogether, and go straight to rolling out new initiatives at a national scale. This can lead to the failure of otherwise worthy initiatives and potentially harm individuals in the process.


Ultimately, few ideas will have a chance to make much impact unless they can be scaled. Here, too, GPSOs face barriers.

Many find it hard to insert new innovations into their existing structures. GPSOs are often blamed for operating in organisational silos. But many of those silos exist because each organisation was established, by law, with an official remit to conduct a certain number of activities, serving a particular community in a specific way. Innovating out of that organisational straight jacket is no easy task.

A related barrier is the highly fragmented nature of the public sector. The UK has a particular challenge in this regard, having more than 420 local authorities. No fewer than 26 of the UK’s largest towns and cities fall under the remit of more than one council. London alone has 33 boroughs. Bringing about change is therefore not just a matter of shifting ways of doing things in one organisation, but numerous bodies, all of which may have different starting points.

In summary, GPSOs start their innovation journey with some formidable barriers. But innovate they must. How they can do that will be the subject of my latest report for Nesta, to be published this summer.

Image Credit: Pixabay | NeuPaddy | CC0 Creative Commons

Find me on Twitter

You may also like