Blog /

8 May 2021 | 6 min read

With the start of the pandemic, many charities lost big chunks of income basically overnight. While there were lots of individual stories, it was much harder to predict the impact for the charity sector as a whole. In the early months, a national campaign pushing for government support foresaw a £4.3bn loss in income for charities. However, this estimate was based on historic, two-year old data and a fair amount of assumptions on which income sources would be affected most. These data gaps led to the creation of a Data Collective aiming to help charities and other civil society organisations to access, analyse and share data. Others, such as Pro Bono Economics started advocating for better data on charities.

Working with charity data for more than five years now, I felt one piece of the puzzle was missing. Where do you actually find data on charities in the UK? And what are the limitations? This blog lists some of the most important data sources, largely for my own benefit, but also to help others navigate the charity data jungle. For more links and resources take a look at this list collated by the Data Collective.

Charities registered in England and Wales (169,814) account for 84% of all UK charities (201,861). The Charity Commission in England & Wales (CCEW) is one of the most important data source for the sector and holds a lot more historic data than the other two regulators. It can be accessed in three ways:

1. A website
2. A data download
3. An API

In 2020, the Charity Commission published a new public register (website) which provides information on the sector as a whole and details for individual charities. It also offers a free data download that contains all current and removed charities as well as historic financial data. Previously, the data dump came only in the form of .bcp files, a specific database format. Since March 2021, data is also available as csv files which will supersede the legacy extract. For more info on how to access the legacy data files, read this guide. In addition, the Charity Commission maintains an API that can be accessed with a developer account.

The Scottish Charity Regulator (OSCR) is the main source for data on Scottish charities. Similar to England and Wales, data can be accessed through the website or downloaded as csv files. However, the data dump does not include historic financial data but only the latest income and spending. There is a separate download for removed charities.

The Charity Commission Norther Ireland (CCNI) only started registering charities in 2013. The register is still growing and  currently contains information for 6,836 charities which can be accessed online or downloaded as csv files. Similar to the Scottish Regular, the data dump does not contain historic financial data.

In addition to the regulators, there are a number of data tools making use of the official data, for example:

• CharityBase: Free and open source database, API and web app with data on charities in England and Wales. It uses the Charity Commission data but adds in grant making data from 360Giving. Data can be accessed on individual records or downloaded in bulk.
• Find that Charity: A database of UK non-profit organisations that draws on all three charity regulators, as well as other regulatory sources. In addition to charities, it provides information on Community Interest Companies, Registered Societies, Universities, Local Authorities, Sports Clubs and NHS Bodies. Data can be accessed on individual records or downloaded in bulk.
• 360Giving: Initiative to support grant making organisations to publish their grants data in an open, standardised way. Data can be downloaded for individual grant makers or as a whole.

There are a range of other applications such as the Charity Explorer for England and Wales and the Scottish Charities App. However, they are not directly linked to the original data sources and depend on manual updating. Hence, the data can be out of date very quickly.

For anyone who is interested in the sector but doesn’t want to undertake analysis themselves, there are a couple of resources too, namely:

• UK Civil Society Almanac: An annual publication on the state of the UK voluntary sector. In addition to regulatory sources, it draws on a sample of 10,000 charities for financial details and also gives insights on the sector’s workforce and volunteers. Some of the sample data can be accessed through the UK Data Service.
• Small charities data: A digital research hub that brings together the latest and best available data on small charities in the UK, those with an income below £1m. It uses a range of data sources, including CharityBase, the UK Civil Society Almanac, 360Giving and many others.
• Foundation Giving Trends: Annual report providing key facts and figures on giving, income and assets in the top 300 UK independent charitable foundations.
• Third Sector Trends Survey: A longitudinal study of the voluntary sector in North England. It is based on surveys and draws on total population estimates from the UK Civil Society Almanac.
• TSRC: Third Sector Research Centre at the University of Birmingham undertaking various research on charities.
• VSSN: Voluntary Sector Studies Network, a community of researchers studying the voluntary sector.

Official data is patchy and limited. While all three charity regulators offer free data downloads,  data differs between them. In addition, charities have 10 months to submit their annual accounts, so official data is always slightly outdated. Smaller charities usually have to provide little information, so even fewer data is available. There is also a mismatch between data that’s available through the public register versus the data downloads.

Data does not cover all social organisations. While there is quite a lot of data on registered charities, it does not cover all civil society organisations. Data for some other types, e.g. social enterprises, community benefit societies or cooperatives, can be found on Companies House or the Mutuals register. While there is very little information on below the radar organisations.

Data is often too broad. While knowing the total number of charities and their income can be helpful, understanding the local picture or specific subsectors is often more useful. However, there is no single classification for subsectors and geographic breakdowns are always impacted by a head office effect, i.e. where charities are registered is not necessarily where they operate.

For more limitations and what’s needed, take a look at the latest report by Pro Bono Economics or this article as part of the Data Collective.