Blog /

26 July 2021 | 4 min read

As part of my job as a Data Analytics Lead I am helping my company to upskill around all things data. So a few months ago, I started sharing #ThursdayTools with my team each week and I thought these might be useful for others too. So here is a list of open source, non-code toools that I use for data viz, data for good, charity data, maps, infographics and more.

Mapping geographic data often means working with postcodes. Luckily David Kane has built Find That Postcode, a tool that lets you upload a list of postcodes and select what geo data to match, including longitude/latitue, Local Authority, MSOA, and much more. It’s such a timesaver and super easy to use.

Community Lens was developed by the Data Collective – a community for people using data in the social sector. It is another great tool to explore local needs. Upload a list of postcodes, e.g. from beneficiaries, and get an output map with your pins as well as heatmaps based on measures of deprivation you select.

The British Red Cross Covid Vulnerability Index is a fantastic resource to explore local needs. It was created similar to the Index of Multiple Deprivation but includes different dimensions (clinical, health, economic, social, socio-economic). You can download data on Ward, MSOA and Local Authority level or use the interactive map.

If you are interested in socio-economic data on cities in the UK, the Centre for Cities has built an interactive tool that lets you select from a range of indicators (eg business, population, housing, employment) for 63 cities. You can look at a point in time or a time series. Download data or as image.

Parli-N-Grams built by Guiseppe Puntofisso displays the frequency of words in UK parliament debates. You can search for words or phrases in the House of Commons or House of Lords and get a timeseries starting at 1919. Also lets you compare different terms against each other. Super interesting to look at trends over time.

360 Giving is a charity setup to help grant makers in the UK publish their data in a standardised way and make it accessible to everyone. You can use grantnav or 360Insights to explore and download grantmaking data from over 197 funders.

If you are interested in data on charities in the UK, head to Find That Charity built by David Kane. You can search for charities by using their name or charity number, upload a list of charity numbers and match data against it, get links to other official registers, etc.

Another great tool to explore charities in England and Wales is CharityBase created by Dan Kwiatkowski. You can search for charities or directly access the API. Much more user friendly than the Charity Commission register itself.

DataBasic is a suite of easy-to-use web tools that introduce concepts of working with data. The tools let you upload data and explore data in a fun and interesting way. You can use it to count words in a text, find out what’s going on in your csv file, compare differences between two texts or run a network analysis.

Turning your data into a stunning data visualisation doesn’t have to be time consuming. Krisztina Szűcs has developed Plot Parade – an amazing data art project providing a number of layouts ready to use, just pop your numbers in and export as an image.

If you got geographic data to map and need something quick and easy, Dark Horse Analytics have have built Map in Seconds which is literally that. Select a region and copy/paste your data, map done. Export map as pptx or png file. Unfortunately, because it is so easy to use, the number of regions are fairly limited/high level.

Blog /

8 May 2021 | 6 min read

With the start of the pandemic, many charities lost big chunks of income basically overnight. While there were lots of individual stories, it was much harder to predict the impact for the charity sector as a whole. In the early months, a national campaign pushing for government support foresaw a £4.3bn loss in income for charities. However, this estimate was based on historic, two-year old data and a fair amount of assumptions on which income sources would be affected most. These data gaps led to the creation of a Data Collective aiming to help charities and other civil society organisations to access, analyse and share data. Others, such as Pro Bono Economics started advocating for better data on charities.

Working with charity data for more than five years now, I felt one piece of the puzzle was missing. Where do you actually find data on charities in the UK? And what are the limitations? This blog lists some of the most important data sources, largely for my own benefit, but also to help others navigate the charity data jungle. For more links and resources take a look at this list collated by the Data Collective.

Charities registered in England and Wales (169,814) account for 84% of all UK charities (201,861). The Charity Commission in England & Wales (CCEW) is one of the most important data source for the sector and holds a lot more historic data than the other two regulators. It can be accessed in three ways:

1. A website
2. A data download
3. An API

In 2020, the Charity Commission published a new public register (website) which provides information on the sector as a whole and details for individual charities. It also offers a free data download that contains all current and removed charities as well as historic financial data. Previously, the data dump came only in the form of .bcp files, a specific database format. Since March 2021, data is also available as csv files which will supersede the legacy extract. For more info on how to access the legacy data files, read this guide. In addition, the Charity Commission maintains an API that can be accessed with a developer account.

In addition to the regulators, there are a number of data tools making use of the official data, for example:

• CharityBase: Free and open source database, API and web app with data on charities in England and Wales. It uses the Charity Commission data but adds in grant making data from 360Giving. Data can be accessed on individual records or downloaded in bulk.
• Find that Charity: A database of UK non-profit organisations that draws on all three charity regulators, as well as other regulatory sources. In addition to charities, it provides information on Community Interest Companies, Registered Societies, Universities, Local Authorities, Sports Clubs and NHS Bodies. Data can be accessed on individual records or downloaded in bulk.
• 360Giving: Initiative to support grant making organisations to publish their grants data in an open, standardised way. Data can be downloaded for individual grant makers or as a whole.

There are a range of other applications such as the Charity Explorer for England and Wales and the Scottish Charities App. However, they are not directly linked to the original data sources and depend on manual updating. Hence, the data can be out of date very quickly.

For anyone who is interested in the sector but doesn’t want to undertake analysis themselves, there are a couple of resources too, namely:

• UK Civil Society Almanac: An annual publication on the state of the UK voluntary sector. In addition to regulatory sources, it draws on a sample of 10,000 charities for financial details and also gives insights on the sector’s workforce and volunteers. Some of the sample data can be accessed through the UK Data Service.
• Small charities data: A digital research hub that brings together the latest and best available data on small charities in the UK, those with an income below £1m. It uses a range of data sources, including CharityBase, the UK Civil Society Almanac, 360Giving and many others.
• Foundation Giving Trends: Annual report providing key facts and figures on giving, income and assets in the top 300 UK independent charitable foundations.
• Third Sector Trends Survey: A longitudinal study of the voluntary sector in North England. It is based on surveys and draws on total population estimates from the UK Civil Society Almanac.
• TSRC: Third Sector Research Centre at the University of Birmingham undertaking various research on charities.
• VSSN: Voluntary Sector Studies Network, a community of researchers studying the voluntary sector.

Official data is patchy and limited. While all three charity regulators offer free data downloads,  data differs between them. In addition, charities have 10 months to submit their annual accounts, so official data is always slightly outdated. Smaller charities usually have to provide little information, so even fewer data is available. There is also a mismatch between data that’s available through the public register versus the data downloads.

Data does not cover all social organisations. While there is quite a lot of data on registered charities, it does not cover all civil society organisations. Data for some other types, e.g. social enterprises, community benefit societies or cooperatives, can be found on Companies House or the Mutuals register. While there is very little information on below the radar organisations.

Data is often too broad. While knowing the total number of charities and their income can be helpful, understanding the local picture or specific subsectors is often more useful. However, there is no single classification for subsectors and geographic breakdowns are always impacted by a head office effect, i.e. where charities are registered is not necessarily where they operate.

For more limitations and what’s needed, take a look at the latest report by Pro Bono Economics or this article as part of the Data Collective.

Blog /

18 October 2020 | 5 min read

What are charities? How big is a small charity? What do charities do? If you are working in the sector, you might think these are trivial questions. Having spent more than four years researching on charities, I can tell you that there are no easy answers.

It’s not unusual to have different understandings of social concepts like charities, so this is not a problem per se. For research, however, shared definitions are useful when comparing data sets or making assumptions about different parts of the sector. If more researchers adopted the same definitions, the evidence base on the sector would grow immensely and, with it, the ability to demonstrate impact. This blog summarises some of the key issues when defining the sector.

Voluntary sector, charity sector, non-profit sector, third sector – there are many, often interchangeable names for the sector. For example, at NCVO many of my research outputs use the term ‘voluntary organisations’. However, when talking to the media or public, I often switch to ‘charities’ as it’s more commonly understood. This isn’t perfect, but it’s the best compromise.

Similarly, it’s often unclear what kinds of organisations are included in ‘the sector’. NCVO’s annual sector report – the UK Civil Society Almanac – refers to the ‘voluntary sector’, but mostly focuses on registered charities, as that is where most data is available. And even then, only certain kinds, for example, it excludes universities with charitable registration. NCVO accepts social enterprises as being voluntary organisations and hundreds of its members are social enterprises. It’s unclear whether most people would include, or are thinking of, social enterprises when thinking about ‘voluntary organisations’ – or whether they refer mainly to registered charities and unregistered community groups. I’ve often seen research projects in or about the sector that seem to be unclear about what kinds of organisations they are referring to or aiming at.

How big is a small charity? It’s a surprisingly contested question.

Most often, income is used to determine size of organisations but even then, different groupings exist. For example, the Small Charities Coalition defines ‘small’ charities as those with an income under £1m. While the NCVO Almanac uses a more granular approach and distinguishes between six different income bands. For example, ‘micro and small’ charities are those with an income under £100,000. There are a number of different groupings used by other infrastructure bodies.

Charities work across many different sectors. When charities register with the Charity Commission in England and Wales, they are classified based on three questions.

1. What does your charity do? (its purpose, eg. health, environment)
2. Who does your charity help? (its beneficiaries, eg. children, older people)
3. How does your charity operate? (its specific activities, eg. grant-making, service delivery)

One of the main problems is that charities self-select their own categories and can pick as many as they wish. Some categories are very broad (‘general charitable purposes’), while more specific activities such as food banks or homelessness services are not included. As a result, these categories are not very helpful for exploring how different sectors are doing or how many charities are active, for example, in mental health. The Charity Commission is currently rethinking their classification system, but it will take years until new categories are available for all charities.

NCVO uses a slightly amended version of the International Classification of Non Profit Organisations (ICNPO), which puts organisations into one single category based on their main activity. This is somewhat more helpful for looking at different subsectors. But some of the categories are still very broad, eg. ‘social services’, and often do not reflect the reality of many organisations that operate in more than one sector. For example, a community centre that also runs a sports club. Official surveys will often use other classification systems, such as the Standard Industrial Classification (SIC), while many other projects apply their own categories depending on their needs. This is particularly true for funders who might be interested in specific areas and therefore use their own subsector frameworks.

For many of these issues, there is no one size fits all. However, when collecting data on the sector, researchers should be clear themselves of the definitions they are using and why. Making use of existing frameworks allows for better comparisons while custom definitions might be more appropriate for very specific audiences. The more data can be brought together and matched up, the stronger the evidence base is for charities to make their case.