Types of UBI pilot

Types of UBI pilot

The last few years have seen an upsurge in pilots of universal basic income (UBI). While all pilots aim to test the same basic concept of UBI, they vary considerably in their methods and focus. For proposers of new UBI pilots, this can lead to a bewildering array of technical characteristics to consider: sample size, target groups, sampling methods, geographical area and more. However, before deciding on the details of pilot design, there is the prior question: what is the fundamental purpose of the pilot? To help answer this question, this blog provides a summary of the basic types of UBI pilot, the type of issues they can address, and how they relate to the wider UBI debate. It draws heavily on three recent contributions by: Jain Family Institute (JFI; 2021), Guy Standing (2021) and Malcolm Torry (2021). For those interested in going more deeply into pilot design, all these publications are highly recommended, as well as an earlier Stanford Basic Income Lab (2018) toolkit (see also the list of further readings given below).

Why conduct pilots?

The first point to address is whether further UBI pilots are needed at all. Some argue that UBI is already justified on ethical grounds (Standing) or by the existing evidence base of UBI’s positive effects (JFI). However this does not mean there is no benefit from conducting new pilots. There are still unanswered questions, particularly when UBI is being considered in a new context or area, and ongoing pilots can contribute a lot to informing the debate and influencing the narrative around UBI. JFI even argue that a pilot can be worthwhile for ‘messaging’ alone, without a research component.

Types of pilot

Having decided that a UBI pilot is called for, the next question is which type of pilot should be conducted. We can distinguish three broad types:

Evidence or Research Pilot

An Evidence Pilot (called an Impact Pilot by JFI) aims to generate robust scientific evidence about the effects of UBI as well as the mechanisms behind these effects. A formal conceptual framework (probably based on social/economic theory and hypotheses) determines which questions are asked and why, and the data are collected and evaluated using scientifically validated methods. The formal structure of the research design means that ideally the results can be applied beyond the pilot itself and used to say what would happen in a fully-fledged UBI. They might also be used as part of a cost-benefit evaluation of a full UBI scheme. Even if the results are not robust enough to extrapolate beyond the pilot, they might still illuminate certain features of a UBI and add usefully to the existing evidence base.

An Evidence Pilot can be quantitative or qualitative, or a combination of the two (mixed methods). Quantitative research analyses outcomes that can be measured as numbers using standard metrics or scales - outcomes can range from time use and spending patterns to indicators of health and wellbeing. The data are usually collected using surveys of participants but some (such as earnings) may also be available from administrative (official) sources. Qualitative research usually involves the use of open-ended (‘semi-structured’) interviews to provide a richer account of people’s experiences and also explore the mechanisms behind any quantitative effects (in other words, not just what happened but why it happened). Different sample sizes are needed in each case: quantitative methods require a large and representative sample to ensure statistical reliability (and the ability to generalise the results beyond the pilot); qualitative research can be based on a much smaller ‘purposive’ sample chosen to study particular situations or hypotheses of interest. The Table below gives some typical numbers and their implications for the cost of the pilot.

Demonstration or Storytelling Pilot

The purpose of a Demonstration Pilot is to showcase UBI in a real setting and to collect stories about people’s experiences. The stories can then be used, for example, to illustrate the positive arguments for UBI or rebut various objections. A Demonstration Pilot differs from a qualitative Evidence Pilot in that the sample and data collection are not designed to explore specific research themes or hypotheses. As such the stories collected should not be seen as ‘evidence’ (or as tests of anything; they are anecdotal evidence at most) but as a way to influence the narrative around UBI. Many ‘micropilots’ (which could be defined as involving 10 participants or fewer) would fit under this heading.

Implementation Pilot

An Implementation Pilot can be seen as intermediate between the Evidence and Demonstration Pilots. It does not aim to investigate scientific hypotheses or generate stories, but instead to test some of the practicalities of administering the scheme. These can include the registration of recipients, payment methods (e.g. cash, prepayment cards or bank accounts), and level of takeup (or drop out). It could also look into the effects of changing the payment frequency. It could be the first step in setting up a ‘UBI infrastructure’.

While these are three distinct types of UBI pilot, they can of course be combined. There are definite advantages to this, for example to deal with the issue of media exposure. In an Evidence Pilot, there should be no media exposure of participants during the pilot in order to preserve experimental conditions. But this conflicts with the desire to communicate about the pilot and its progress, so a solution is to ‘ring fence’ the Evidence sample from media exposure but allow media access to a separate Storytelling sample. Thus the SEED pilot (Stockton, California) has 100 participants in the research sample and 25 in the storytelling sample. Another useful combination of pilot types would be a joint Evidence-Implementation pilot, to assess, for example, the effect of payment amount or frequency on outcomes.

The Table below summarises some key characteristics of the three pilot types, including very approximate figures for the cost of a 2 year pilot (a typical length).

Pilots versus experiments

Apart from the above distinction by type of pilot, there is another issue, which might be less obvious at first. This is the difference (emphasised by Standing and Torry) between pilots and experiments:

A Pilot is a prototype, or scaled-down version, of a full-scale UBI scheme. That implies it should operate to the same rules as the full-scale scheme, including any changes to taxes (to finance the scheme) and benefits (replaced by UBI), and be run on a representative sample of the population. This really requires full control over the tax-benefit system so that participants’ income tax and benefits can be adjusted accordingly (a convenient side-effect of this is that savings in benefits and taxes can considerably reduce the cost of the pilot). So DWP and HMRC cooperation would be needed to run a proper pilot (though that would still not allow for other taxes such as VAT or carbon taxes).
An Experiment does not replicate the full scheme but instead trials selected elements of it. Typically this might entail paying the UBI without the accompanying changes to benefits or taxes; or trialling UBI on a selected group, e.g. low-income families or particular demographics.

By these definitions, Standing and Torry both judge that most UBI ‘pilots’ are in fact experiments (a third word, trial, may apply to both, for example a randomised control trial could serve as either a pilot or an experiment) . This is most clear for the US city pilots, which tend to target very specific sub-groups, e.g. African-American mothers (in Jackson, MS). The care leavers’ pilot in Wales also comes into this category. While these experiments can produce useful evidence, it only applies to the sub-group in question and the trials are also open to the criticism that they only test the positive aspects of UBI (extra money), not the negative ones (extra taxes). It is also no doubt true that UBI will have stronger effects in these selected sub-groups than on the population as a whole - thus a true pilot probably represents a stiffer test for UBI than a targeted experiment.

Open questions and opportunities for pilots in the UK

Despite the large evidence base on the effects of cash transfers, there are many open questions about UBI and how it would operate. Unfortunately, some important questions cannot be answered by short-term pilots: these relate to economy-wide (macroeconomic) effects and long-term effects. However, other questions can be addressed, including issues around UBI delivery, richer accounts of the experience of recipients (and comparisons with existing benefits), effects among specific sub-groups, and other complementary policies to maximise the benefits of UBI. Some specific questions that could be addressed by new UK pilots might be:

Effects on a whole community, using a ‘saturation site’ pilot, in which everyone in a predefined community (for example a group of streets / tower blocks) is given a UBI. The only saturation pilots in developed countries have been the two Canadian ones (Manitoba in the 1970s and the aborted Ontario pilot in 2017-19). However, this is probably the most challenging type of pilot and by definition (to capture the community effects properly) requires a large sample size. The site needs to be chosen carefully, as well as a suitable (‘identical’) control site. The Scottish feasibility study includes a saturation site.
Impact of UBI on environmental behaviour: addressing the role UBI could play in fighting climate change.
Impact of UBI among gig economy workers.
Impact of UBI on food poverty: what role could a small UBI play in reducing food poverty / reliance on food banks?

References and further reading

Chrisp J, Smyth L, Stansfield C, Pearce N, France R & Taylor C. (2022). Basic income experiments in OECD countries: A rapid evidence review. London: EPPI Centre, UCL Social Research Institute, University College London. https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3856

Citizens’ Basic Income Feasibility Study Steering Group (2020). Assessing the Feasibility of Citizens’ Basic Income Pilots in Scotland: Final Report https://www.basicincome.scot/__data/assets/pdf_file/0024/175371/Draft-Final-CBI-Feasibility_Main-Report-June-2020.pdf

Jain Family Institute (2021). Guaranteed Income in the U.S. A toolkit of best practices, resources, and existing models of planned and ongoing research in the U.S. https://www.jainfamilyinstitute.org/assets/JFI-U.S.-Guaranteed-Income-Toolkit-May-2021.pdf

Standing, Guy (2021). Basic Income Pilots: Uses, Limitations and Design Principles, Basic Income Studies, 16(1): 75–99. https://www.degruyter.com/document/doi/10.1515/bis-2021-0021/html?lang=en

Torry, Malcolm (2021). Feasibility study: A Basic Income pilot project for Wales. A response to the report "Piloting a Basic Income in Wales”, FRIBIS Discussion Paper Series, No. 02-2021 https://www.econstor.eu/handle/10419/248429

More about the author

Mark Bryan - @M_L_Bryan

Mark Bryan is a Reader in the Department of Economics at the University of Sheffield. His research investigates people's experiences and outcomes in the labour market and their links with health and well-being.

He is particularly interested in how a UBI might change the operation of the labour market. As well as carrying out academic research, he has worked on policy-related projects for government and private-sector organisations, and is an expert on statistical techniques to assess the impacts of policy changes on the economy. He co-authored the UBI Lab Sheffield proposal for a UBI pilot.

Jonny DouglasJuly 11, 2022