ScraperWiki: solving data headaches for government

Big data specialist ScraperWiki is a key supplier for the Government Digital Service team, helping the government transform and improve its online data.

The Liverpool-headquartered company, led by CEO Francis Irving and chief marketing officer Aine McGuire, was recently asked to deliver 37 activity dashboards to provide government departments with statistics and data on how government websites are being used.

“Our task was to create performance platform dashboards that would monitor key indicators: what is the value of the site, how many people are using it, is it being successful?” explains McGuire.

“The Government Digital Service team is iterating all the time, and we work with them to populate the dashboards,” says McGuire.

“They needed a team that would give them both the expertise of data scientists and ‘software carpenters’ to pull in all the data.”

The work is part of the government’s large-scale digital transformation towards a centralised platform.

“We understand their vision of where they think government needs to go with respect to digital, and that’s important,” McGuire adds.

The impact of working on government work has been important to ScraperWiki, which has seen government business grow by 200 per cent in 12 months.

ScraperWiki recently won another government contract to help the Office for National Statistics transform how its data is published.

“It means that we can continue to hire and develop a highly-skilled pool of data scientists up here in the north west,” says Irving. ScraperWiki currently employs 10 staff, but the objective is to double the team this year.

Being a Liverpool-based business has benefited the company, he adds: “Being so close to fantastic universities in Durham, Liverpool and Manchester has allowed us to hire the best.”

There is a desire for government to work with smaller firms such as ScraperWiki, says Irving.

“There is clearly an appetite for government to engage better with SMEs. Things are evolving all the time, and this culture is spreading throughout the various departments.”

There is a technical skills shortage inside of government, adds McGuire. “Working with a small, agile organisation like ours – which has developed the right experience of working with the Government Digital Service – allows them to easily delegate work and trust it will be delivered correctly.”

As for advice to other SMEs looking to work with government departments, McGuire says the key is tenacity.

“There is an element of process to the frameworks, but you just have to stick with it and do the paperwork. It’s too easy to say that it’s too complicated; you have to look past that.

“Frameworks were clunkier in the past and are being simplified. So just be tenacious and communicate with government wherever you see a problem.”


The Government is opening up the public sector’s £187bn annual spend. The new Contracts Finder provides small and medium-sized enterprises with a single online public sector marketplace to find opportunities by price, location and sector. Businesses can register for free for contract alerts

Businesses can also get advice from buyers. Workshops to help small businesses write bids are advertised on Contracts Finder in the events section. 

Businesses can also find subcontracting opportunities and use the open data on Contracts Finder to create apps, expand services or create a new company.