Job Search and Hiring with Two-Sided Limited Information about Workseekers’ Skills

Upjohn Institute Working Papers Upjohn Research home page

6-19-2020

Job Search and Hiring with Two-Sided Limited Information about Job Search and Hiring with Two-Sided Limited Information about

Workseekers’ Skills Workseekers’ Skills

Eliana Carranza

World Bank

, [email protected]

Robert Garlick

Duke University

, [email protected]

Kate Orkin

University of Oxford

, [email protected]

Neil Rankin

University of Stellenbosch

, [email protected]

Upjohn Institute working paper ; 20-328

Citation Citation

Carranza, Eliana, Robert Garlick, Kate Orkin, and Neil Rankin. 2020. "Job Search and Hiring with Two-Sided

Limited Information about Workseekers’ Skills." Upjohn Institute Working Paper 20-328. Kalamazoo, MI:

W.E. Upjohn Institute for Employment Research. https://doi.org/10.17848/wp20-328

This title is brought to you by the Upjohn Institute. For more information, please contact [email protected].

Upjohn Institute working papers are meant to stimulate discussion and criticism among the

policy research community. Content and opinions are the sole responsibility of the author.

Job Search and Hiring with Two-sided Limited Information

about Workseekers’ Skills

Upjohn Institute Working Paper 20-328

Eliana Carranza

World Bank

ecarranza@worldbank.org

Robert Garlick

Duke University

[email protected]

Kate Orkin

University of Oxford

[email protected]

Neil Rankin

University of Stellenbosch

[email protected]

June 2020

ABSTRACT

We present field experimental evidence that limited information about workseekers’ skills distorts both firm and

workseeker behavior. Assessing workseekers’ skills, giving workseekers their assessment results, and helping them

to credibly share the results with firms increases workseekers’ employment and earnings. It also aligns their beliefs

and search strategies more closely with their skills. Giving assessment results only to workseekers has similar effects

on beliefs and search, but smaller effects on employment and earnings. Giving assessment results only to firms

increases callbacks. These patterns are consistent with two-sided information frictions, a new finding that can inform

design of information-provision mechanisms.

JEL Classification Codes: J23, J24, J31, J41, O15, O17

Key Words: Job search, hiring, two-sided limited information, worker assessment, field experiment,

employment, earnings

Acknowledgments: We are grateful for helpful comments from Martin Abel, Peter Arcidiacono, Vittorio

Bassi, Paolo Belli, Emily Breza, Rulof Burger, Stefano Caria, Jushnu Das, Giacomo di Giorgi, Taryn Dinkelman,

Rebecca Dizon-Ross, Erica Field, Simon Franklin, Doug Gollin, Marek Hanusch, Rachel Heath, Clement Imbert,

Anette John, Lisa Kahn, Melanie Morten, Simon Quinn, Imran Rasul, Duncan Thomas, Gabriel Ulysses, Chris

Woodruff, and seminar participants at the NBER Summer Institute, Barcelona Summer Forum, Duke, Georgia,

Notre Dame, NYU, Oxford, UIUC, and the World Bank. We appreciate exceptional research assistance from Wim

Louw, Wendy Trott, Shelby Carvalho, Allegra Cockburn, Goodwill Mpofu, Emmanuel Bakirdjian and Nilmini

Herath in South Africa and Lukas Hensel, Svetlana Pimkina, Laurel Wheeler, Gabriel Cunha and Rachel Sayers at

our institutions. Thanks to the staff at the Harambee Youth Employment Accelerator, the Abdul Latif Jameel Latif

Poverty Action Lab Africa Office, and our institutions. The project is conducted in collaboration with the World

Bank Jobs Group and Africa Gender Innovation Lab, and received funding from the National Science Foundation (#

1824413), Private Enterprise Development in Low Income Countries (# 3024 and # 4728), W.E. Upjohn Institute for

Employment Research, and the Global Challenges Research Fund Accelerating Adolescent Achievement Hub. This

study has been approved by the University of Cape Town Commerce Faculty Ethics in Research Committee, the

Duke Institutional Review Board for the Protection of Human Subjects in Non-Medical Research (# D0368) and the

Oxford Department of Economics Research Ethics Committee (#ECONCIA 15-055). This study is pre-registered at

https://doi.org/10.1257/ret.1631-8.0. The authors’ views do not necessarily represent the views of the World Bank or

the governments of the countries they represent.

1 Introduction

Workseekers make job search decisions and ﬁrms make hiring decisions using potentially limited in-

formation about workseekers’ skills. Limited information for ﬁrms can lead to hiring poorly-matched

workers and to wedges between wage oﬀers and productivity (Altonji and Pierret, 2001; Arcidiacono

et al., 2010; Farber and Gibbons, 1996; Kahn and Lange, 2014). These hiring distortions can reduce

both employment and average wages conditional on employment (Aigner and Cain, 1977; Pallais,

2014). Limited information for workseekers can lead them to search for jobs that poorly match

their skills or withdraw from search entirely (Belot et al., 2018; Conlon et al., 2018). These search

distortions can also lead to lower employment and lower wages conditional on employment. When

both sides of the market receive credible information on workseekers’ skills or past performance,

these workseekers’ outcomes in the labor market can improve (Abebe et al., 2020a; Abel, 2019; Bassi

and Nansamba, 2020; Pallais, 2014). These information problems may be particularly important in

settings where hiring is less formal and education provides less information about skills (Pritchett,

2013). Limited information may exacerbate other frictions in developing country labor markets,

such as high search and migration costs (Abebe et al., 2020b; Bryan et al., 2014; Franklin, 2017).

We study how providing new information about workseekers’ skills aﬀects job search and hir-

ing decisions and workseekers’ labor market outcomes. We run a series of ﬁeld experiments that

separately manipulate ﬁrms’ and workseekers’ information about workseekers’ skills. We provide

the ﬁrst evidence that both ﬁrms and workseekers adjust their behavior when they acquire new

information about workseekers’ skills, improving workseekers’ outcomes in the labor market. The

existence of two-sided information frictions is important both conceptually and for the design of

information-provision products and policies.

Many existing policies target only one-sided frictions

and may have limited returns if two-sided frictions are present. For example, ﬁrms testing job

applicants’ skills has limited returns if well-matched workseekers do not apply to these ﬁrms, while

workseeker skill assessments oﬀered in job search assistance programs have limited returns if work-

seekers cannot credibly communicate their skills to ﬁrms. Existing papers cannot identify two-sided

information frictions because they study only one side of the market or study simultaneous two-sided

information revelation.

We study ﬁrms’ and workseekers’ responses to learning workseekers’ results on standardized skill

assessments. The assessments measure non-specialist skills such as communication, numeracy, and

grit and draw on existing tools used by job placement agencies and large ﬁrms. The 6,891 assessed

workseekers are drawn from a population where information frictions may be important. They

are unemployed or underemployed black youths in urban South Africa with limited post-secondary

education, work experience, and access to referral networks. This population faces statistical dis-

We use the term ‘information friction’ to refer to distortions in behavior due to limited information about work-

seekers’ skills. Our experiments do not speak to limited information about other factors such as vacancy characteristics

or workers’ eﬀort.

crimination in this labor market (Malindi, 2017).

We demonstrate the existence of two-sided information frictions in three steps. First, we show

that giving workseekers their results from these assessments and enabling them to easily and credibly

share the results with ﬁrms improves the workseekers’ labor market outcomes. To show this, we

randomly select some workseekers for a ‘public’ certiﬁcation intervention. We give them electronic

and physical certiﬁcates describing the assessments and showing their results. The certiﬁcates show

their names and national identity numbers and are branded by the widely known agency that

conducts the assessments and the World Bank. We compare these workseekers to a control group of

workseekers who receive no certiﬁcates and do not learn their results. In the three to four months

following certiﬁcation, publicly certiﬁed workseekers use certiﬁcates in job applications, shift their

beliefs about their skills closer to their assessment results, and target their search toward jobs that

they think value their skills. Their employment rate increases by 17% (5 percentage points), weekly

earnings increase by 34%, and hourly wages increase by 20% relative to the control group. The rise

in earnings reﬂects both higher employment and higher earnings conditional on employment.

This ﬁrst step shows that information frictions exist, but does not show who has limited infor-

mation about workseekers’ skills. The treatment eﬀects are consistent with ﬁrms changing hiring

and wage oﬀer decisions as they learn about workseekers’ skills from seeing certiﬁcates, with work-

seekers changing search behavior as they learn their own assessment results, or with both types of

frictions.

In the second step of our argument, we show that workseekers face information frictions. We

randomly select some assessed workseekers for a ‘private’ certiﬁcation intervention. This inter-

vention gives them one physical certiﬁcate that shows their assessment results and describes the

assessments, but excludes identifying information and branding that might make the certiﬁcate a

credible source of information to ﬁrms. Private certiﬁcation has the same eﬀects as public certi-

ﬁcation on workseekers’ beliefs and search targeting, a positive eﬀect on earnings that is smaller

than the public certiﬁcation eﬀect, and no eﬀect on employment. These results show that giving

workseekers more information about their skills changes their behavior and outcomes in the labor

market. However, eﬀects are smaller than when they can also share the information with ﬁrms. This

pattern is consistent with both sides of the market having limited information about workseekers’

skills, but inconsistent with only one side having limited information.

In the third step of our argument, we run an audit/correspondence experiment as a further

check for ﬁrm-side information frictions. This experiment manipulates ﬁrms’ information without

scope for changes in workseeker behavior. We submit applications to real job vacancies using real

resumes from workseekers in our sample. We submit multiple applications per vacancy, randomizing

whether applications include public certiﬁcates. Applications including certiﬁcates get 11% more

interview invitations, consistent with ﬁrms having limited information about workseekers’ skills and

acquiring more information from skill certiﬁcation.

These three experiments demonstrate our main ﬁnding: both ﬁrms and workseekers have lim-

ited information about workseekers’ skills. In addition, we present four secondary ﬁndings that

further characterise how information frictions aﬀect this labor market using heterogeneity analysis

and smaller experiments. First, learning speciﬁc assessment results is important, not just learning

that workseekers have been assessed. The certiﬁcation eﬀects are not driven, for example, by ﬁrms

using workseekers’ decisions to get assessed as a signal for tenacity or proactivity, or by ﬁrms basing

hiring decisions purely on the certiﬁcates’ branding. Second, in this context, preferences for diﬀerent

skills vary across ﬁrms and relative performance in diﬀerent assessments varies across workseekers.

This pattern is more consistent with horizontal than vertical diﬀerentiation: certiﬁcation helps ﬁrms

identify which workseekers are suited for speciﬁc jobs, rather than identify a subset of workseekers

suited for all jobs.

Third, certiﬁcation has larger eﬀects on the labor market outcomes of work-

seekers who lack other ways to communicate their skills to employers, like work experience and

university education. Fourth, although we do not directly observe if certiﬁed workseekers become

employed at the expense of workseekers outside our sample, most of our results are consistent with

economic mechanisms in which certiﬁcation can increase total employment.

Our primary contribution is to show that both ﬁrms and workseekers face information frictions

and that both sides of the labor market respond to information provision. This extends existing work

documenting labour market patterns consistent with either ﬁrm-side or workseeker-side information

frictions in both developed and developing economies.

We build on this work by showing that both

ﬁrms and workseekers face limited information about workseekers’ skills. Our work is most similar

to papers that study information frictions by simultaneously revealing information to both ﬁrms

and workseekers about skill assessment results (Abebe et al., 2020a; Bassi and Nansamba, 2020) or

evaluations from workseekers’ past employers (Abel et al., 2020; Pallais, 2014). These papers show

that information revelation changes workseekers’ outcomes and interpret this as evidence of ﬁrm-side

information frictions. But they do not separately manipulate ﬁrms’ and workseekers’ information,

so cannot distinguish one-sided from two-sided frictions.

The distinction between one- and two-sided frictions is important for designing mechanisms

that private actors or government can use to address information frictions. Interventions targeted

at one side of the market are common, but may not be optimal if both sides of the market face

This ﬁnding is consistent with recent work on information frictions in matching models with multidimensional

skills (Fredriksson et al., 2018; Guvenen et al., 2020; Lise and Postel-Vinay, 2020).

Altonji and Pierret (2001), Arcidiacono et al. (2010), Farber and Gibbons (1996), and Kahn and Lange (2014)

show that wages align more closely with skills as tenure increases, consistent with limited information about skills

at the time of hiring. Wage and retention patterns for workers hired through referrals are also consistent with

information frictions (Ioannides and Loury, 2004; Heath, 2018) and some researchers ﬁnd that workers have better

labor market outcomes when they have formal educational qualiﬁcations, conditional on measured skills (Alfonsi

et al., 2017; MacLeod et al., 2017). Workseekers’ job search decisions can change when workseekers learn more about

their labor market prospects (Ahn et al., 2019; Altmann et al., 2018; Belot et al., 2018), although these papers do

not speciﬁcally examine limited information about workseekers’ skills.

Abel et al. (2020) reveal information to both sides of the market and only to ﬁrms, but do not test for workseeker-

side frictions.

frictions. On the workseeker side, some job search assistance programs oﬀer skill assessments to

workseekers (McCall et al., 2016). This can inform workseekers and improve their search targeting.

But if ﬁrms face information frictions and do not learn these assessment results, then ﬁrms’ hiring

choices and wage oﬀers will remain distorted and workseekers’ improved search will have limited

returns. On the ﬁrm side, skill assessments are sometimes used to inform ﬁrm hiring decisions

(Autor and Scarborough, 2008; Hoﬀman et al., 2018). But if workseekers face information frictions,

they might not apply for jobs that match their skills, leaving ﬁrms to assess and select from a

sub-optimal pool of applicants. Two-sided frictions may reduce the return to one-sided information

acquisition enough to deter investment in one-sided screening by ﬁrms or signalling by workseekers.

Alonso (2018) shows that introducing one-sided information acquisition in labor market matching

can reduce welfare when information frictions are two-sided.

This paper complements work on the aggregate implications of information frictions. Canonical

models show that search and matching frictions facing individual workseekers and ﬁrms can generate

aggregate unemployment (Mortensen and Pissarides, 1999). Our ﬁnding of two-sided information

frictions oﬀers an experimental foundation for general equilibrium models in which both ﬁrms and

workseekers have limited information about match productivity, leading to aggregate employment

distortions (Jovanovic, 1979; Menzio and Shi, 2011). In particular, our ﬁndings complement work

by Donovan et al. (2018), who show that models with limited information about workseekers’ skills

can explain aggregate labor market dynamics in developing countries.

Our ﬁndings on information frictions are also relevant to the design of active labor market

programs (ALMPs). We show that a skill assessment and certiﬁcation intervention, delivered during

recruitment for an ALMP, can substantially and cheaply improve participants’ employment and

earnings. The employment eﬀect is almost three times larger than the mean eﬀect size of the active

labor market programs reviewed by Card et al. (2018). The average earnings gain in the ﬁrst

three months after treatment is 5.6 times the average variable cost of this certiﬁcation intervention

alone and 2.3 times the average variable cost of assessment and certiﬁcation.

Skill assessment

and certiﬁcation may enhance the value of ALMPs to participating workseekers even when other

mechanisms for learning about workseekers’ skills exist. Importantly, certiﬁcation is available to

ﬁrst-time workseekers, unlike reference letters or performance evaluations from past employers.

Assessment results can be certiﬁed to multiple employers, while workplace performance at one

employer may be imperfectly observed by other employers (Kahn, 2013). Certiﬁcation can help

workseekers excluded from referral networks or ﬁrms who receive referrals based on factors poorly

aligned with workseekers’ skills (Beaman and Magruder, 2012; Beaman et al., 2018; Chandrasekhar

et al., 2020).

We describe the economic environment in Section 2: a simple conceptual framework, the con-

In a similar spirit, several papers show that making low-cost changes to ALMPs so they provide more information

to ﬁrms and/or workseekers improves their eﬀectiveness (Abel et al., 2020; Belot et al., 2018; Wheeler et al., 2019).

text, the sample, and the skill assessments. In Section 3, we describe the public skill certiﬁcation

experiment and the treatment eﬀects on workseekers’ labor market outcomes. In Section 4, we ana-

lyze the roles of ﬁrm- and workseeker-side frictions. In Section 5, we discuss secondary results about

what workseekers and ﬁrms learn from certiﬁcation, what this implies for the eﬀects of certiﬁcation

on diﬀerent types of workseekers, and what this might imply for certiﬁcation at a larger scale. We

conclude in Section 6 and discuss questions around markets for assessment-based certiﬁcation.

2 Economic Environment

2.1 Conceptual Framework

In this section, we sketch a simple conceptual framework with two goals. First, the framework

illustrates how either workseeker- or ﬁrm-side information frictions can lower two labor market out-

comes: the employment rate and the mean wage conditional on employment. Hence, observing

that employment and/or wages rise when both ﬁrms and workseekers acquire more information

does not show which side(s) of the market face information frictions. This highlights the impor-

tance of the separate variation we generate in ﬁrms’ and workseekers’ information sets. Second,

the framework illustrates the mechanisms that link limited information to distortions in ﬁrm and

workseeker behavior and hence to lower wages and employment. This guides our empirical tests of

these mechanisms.

Consider a stylized economy consisting of inﬁnitely many type W

and W

workseekers and

type J

and J

jobs. Workseekers choose not to search, to search for type 1 jobs, or to search

for type 2 jobs. Searching for either type of job incurs ﬁxed cost C > 0. A type i workseeker

searching for type j jobs meets a ﬁrm oﬀering such a job with probability P

i,j

. Conditional on

meeting, the match produces output with pecuniary value V

i,j

and pays wage W

i,j

≤ V

i,j

. The

workseeker receives utility U (P

i,j

· W

i,j

) − C if she searches and zero otherwise, implying that she

has a reservation wage W

(C, P ).

Non-employment is possible if search costs are high relative to

the expected utility of working (which leads to some non-participation) or if the meeting probability

i,j

is less than one for some (i, j).

We make some additional simplifying assumptions for this discussion, but none of the results

in the framework depend on these additional assumptions. First, we assume fraction p of all work-

seekers and all jobs are type 1. Second, we assume that type i workseekers are better at searching

for type i jobs, produce the most output in type i jobs, and earn the highest wages in type i jobs,

and similarly for type j workseekers and type j jobs. Under these assumptions, if type i workseekers

choose to search, they will choose to search for type i jobs rather than type j jobs, and vice versa.

Limited information about workseekers’ skills can enter this environment in two ways. First, we

consider the case where only workseekers observe their types with error. This type of friction can

For simplicity, we assume that ﬁrms post and commit to wages before workseekers make search decisions. This

implies that all workseekers who choose to search for type j jobs will accept them if oﬀered.

occur if, for example, workseekers receive limited information about their own type from education

or work experience or if they have little education or work experience. With this type of friction, each

workseeker chooses whether and where to search based on her perceived type. If a type i workseeker

‘incorrectly’ searches for a type j 6= i job, she is less likely to meet a ﬁrm and, conditional on

meeting a ﬁrm, will produce less and earn a lower wage. This type of friction reduces mean wages

conditional on employment by generating some mismatches between workseeker and job types.

This can also reduce the employment rate through two mechanisms: workseekers who search for the

wrong type of jobs are less likely to meet ﬁrms, and mismatches between workseeker and job types

may not generate enough output to oﬀer wages above the reservation wage. The former mechanism

can occur if, for example, if ﬁrms oﬀering diﬀerent types of jobs hire using diﬀerent channels, like

posting formal adverts versus hiring walk-ins. The latter mechanism can occur if, for example,

search costs and hence reservation wages are high or if there is a legal minimum wage. Belot et al.

(2018) and Falk et al. (2006) prove results of this ﬂavor formally.

Second, only ﬁrms may observe workseekers’ types with error. This type of friction can occur

if attributes observable to ﬁrms, like educational qualiﬁcations or past work experience, provide

limited information about skills. With this type of friction, workseekers search for the ‘right’ types

of jobs but ﬁrms do not know the type of the workseekers they meet. If type j ﬁrms believe that

fraction q of the workseekers they meet are type j, then the expected output from each hire is

q · V

j,j

+ (1 − q) · V

i,j

. If ﬁrms’ utility is a concave function of their output, then they will oﬀer a

wage lower than q · W

j,j

+ (1 − q) · W

i,j

. Concavity can arise from ﬁrms’ production technology or

from uninsured risks from bad hires. Possible uninsured risks include lost customers or damaged

equipment from hiring the ‘wrong’ workseekers or severance pay and dispute resolution costs when

ﬁring workseekers. This reduces mean wages conditional on employment and, if the oﬀered wage

is below the reservation wage or minimum wage, reduces the employment rate. Aigner and Cain

(1977) and Pallais (2014) prove results of this ﬂavor formally.

This simple framework shows that observing a rise in employment and/or wages when both

ﬁrms and workseekers acquire more information does not show which side(s) of the market face

information frictions. This highlights the importance of the separate variation we generate in ﬁrms’

and workseekers’ information sets. Depending on the structure of the model, frictions on both sides

of the market might interact to generate larger distortions or partly oﬀset each other. We do not

explore this in detail because our experimental design does not identify the size of any interaction

eﬀect without additional assumptions. We focus on the static case for simplicity, but recognize

that the eﬀect of information frictions may diﬀer in a dynamic framework with learning by ﬁrms or

workseekers (Conlon et al., 2018; Lange, 2007).

The framework allows either horizontal or vertical diﬀerentiation of workseekers. We deﬁne

horizontal diﬀerentiation as type i workseekers being more productive than type j workseekers in

type i jobs and vice versa. We deﬁne vertical diﬀerentiation as type i workseekers being more

productive than type j workseekers in all jobs. In both cases, either ﬁrm- or workseeker-side

information frictions can lower the employment rate and the mean wage conditional on employment.

With horizontal diﬀerentiation, frictions on either side of the market can lower wages conditional on

employment for all types of workseekers. With vertical diﬀerentiation, ﬁrm-side frictions can lower

wages for type i workseekers mistaken for type j workseekers and raise wages for type j workseekers

if they are mistaken for type i workseekers.

2.2 Context

We work in the metropolitan area of Johannesburg, South Africa’s commercial and industrial hub.

Johannesburg’s labor market has four salient features for our study. First, information frictions

are likely, as alternative sources of information on workseekers’ skills are limited. Many young

workseekers have no work experience several years after completing education, limiting the scope

to learn about or signal their skills through experience (Ingle and Mlatsheni, 2017). Grades and

grade progression in most primary and secondary schools are weakly correlated with independently

measured skills (Lam et al., 2011; Taylor et al., 2011). Workseekers who have completed secondary

school typically report their grades in the nationally standardized graduation exam in job applica-

tions. But examination grades weakly predict performance in post-secondary education and ﬁrms

report in interviews that the grades convey limited information about skills (Schöer et al., 2010).

This limits the scope for ﬁrms and workseekers to learn about workseekers’ skills from their educa-

tional attainment. Certiﬁcation is thus likely to provide both ﬁrms and workseekers with additional

information on workseekers’ skills.

Second, ‘wrong’ hires are costly to ﬁrms. Firing a worker requires a complex and lengthy

process and can be challenged by even temporary employees in courts and specialized dispute

resolution bodies.

Probationary work is permitted but regulated and probation periods cannot

exceed three months (Bhorat and Cheadle, 2009). Firms report challenges understanding labor

regulation, contributing to the perceived cost of separations.

Consistent with these factors, giving

ﬁrms free consulting on labor regulation increases hiring (Bertrand and Crépon, 2019).

Third, reservation and minimum wages exist. Minimum wage compliance in the formal sector is

high (Bhorat et al., 2016; International Labour Organization, 2016). Commuting costs are high and

likely to raise reservation wages (Kerr, 2017). The nearly universal state pension system gives work-

seekers in multi-generation households access to non-labor market income, increasing reservation

wages (Abel, 2019).

The limited information content of education qualiﬁcations is consistent with the large role of referrals in hiring:

more than half of all ﬁrms report that referrals are their preferred recruitment mechanism (Schöer et al., 2014).

Small ﬁrms report an average of two dispute resolution cases in the previous year, requiring an average of 11

days of staﬀ time per case (Rankin et al., 2012).

Only 18% of SME owners know the conditions that made a contract valid or rules governing severance pay

(Bertrand and Crépon, 2019).

Fourth, employment rates are low. In our study period, unemployment in Johannesburg was

28% for the working-age population, 51% for ages 15-24, and 32% for ages 25-34 (Statistics South

Africa, 2016).

Low employment in the presence of information frictions, costs from ‘wrong’ hires,

and reservation or minimum wages are consistent with our conceptual framework. Particularly low

employment for youths is also consistent with information frictions, as youths have less search and

work experience to reveal their types. Many other factors can contribute to low employment rates;

we merely argue that a role for information frictions is plausible.

2.3 Sample Recruitment and Data Collection

We recruit a sample of 6,891 young, active workseekers from low-income backgrounds with limited

work experience. Workseekers in our sample have limited access to traditional ways to learn about

their skills and communicate their skills to prospective employers: university education, work ex-

perience, or access to referral networks. We recruit only active workseekers, so we do not examine

the relationship between information frictions and labor market participation decisions. This is a

sample from a policy- and theory-relevant population likely to face information frictions, rather

than a population-representative sample.

To recruit the sample, we work with the Harambee Youth Employment Accelerator, a social

enterprise that assesses the skills of inexperienced workseekers and matches them to employers

looking for entry-level workseekers, amongst other activities. Harambee recruits candidates through

radio and social media advertising and door-to-door recruitment in low-income neighborhoods.

Interested candidates register online and complete a phone-based screening questionnaire.

Eligible

candidates are invited to two days of standardized skill assessments. Some candidates are invited

to further job readiness training based on their assessment results and residential location, but

only 0.2% of candidates in our sample get jobs through this training. Our sample consists of all

candidates who arrive at Harambee for the second of these two testing days, on 84 operational

days.

We conduct three surveys to measure workseekers’ labor market outcomes, job search, and beliefs

about their skills and the labor market. The baseline is a self-administered questionnaire that

candidates complete on desktop computers at Harambee under supervision. This is administered

after candidates have done skills assessments but before they receive information about their results.

We collect endline data in a 25-minute phone survey 3-4 months after treatment.

The phone survey

Throughout the paper, we use Statistics South Africa’s deﬁnition of an employed person as someone who did

any income-generating activity, for at least one hour, during the reference week. Unemployment rates exclude those

in full-time education or not in the labor force.

Candidates are eligible to work with Harambee if they are aged 18-29, have legal permission to work in South

Africa, have completed secondary school, have at most twelve months of formal work experience, have no criminal

record, and are from disadvantaged backgrounds. This information is self-reported but checked against administrative

data for some candidates.

See Garlick et al. (2019) for an experimental validation of labor market data from phone surveys in this setting.

response rate is 96%, leaving an endline sample of 6,607 respondents. The response rate is balanced

across treatment groups (Table D.5) and unrelated to most baseline covariates (Table D.6). We also

conduct a short text message survey 2-3 days after treatment. Respondents receive mobile phone

airtime payments for answering the text message and phone surveys.

2.4 Job Search and Employment in The Sample

This section describes relevant patterns around labor market outcomes and job search in our sample.

We report summary statistics for key baseline and endline variables for the 6,891 workseekers in

Tables D.1 and D.2. Respondents are 99% Black African, 62% female, and on average 24 years

old. 17% have a university degree or diploma, 21% have some other post-secondary certiﬁcate,

and 99% have completed secondary school. Malindi (2017) shows that young, black workseekers

with relatively low levels of education face discrimination in this labor market, with wage dynamics

consistent with information frictions and statistical discrimination.

38% of the sample worked in the week before the baseline and 70% had ever worked, but only 9%

had ever held a long-term job. Conditional on working, mean weekly earnings in the week before the

baseline was 565 South African Rands (94 USD in purchasing power parity terms), slightly below

the minimum wage for a full-time worker in most sectors. Wage work was eight times more common

than self-employment. Most work was relatively short-term, with median and mean tenures of 2

and 7 months respectively.

97% of the sample searched for work in the week before the baseline. In that week, they spent

on average 17 hours and 242 South African Rands (40 USD PPP) searching. The relatively high

search costs suggest that welfare gains for workseekers are possible from improved search targeting.

Workseekers submitted on average 10 applications in the preceding month and received 1.2 oﬀers,

though the medians for both measures are zero. The job search and application process is somewhat

formal: 38% of the candidates employed at endline reported that they submitted written applications

for their current job and 47% reported that they had a formal interview.

2.5 Assessments

We conduct six assessments with workseekers: communication, concept formation (similar to a

Raven’s test), focus, grit, numeracy, and planning. Firms have demonstrated interest in the results

of these assessments, though they obviously also use other information in hiring decisions. Client

ﬁrms have paid Harambee to screen roughly 160,000 prospective workers using these assessments.

Appendix A describes each assessment in detail, their psychometric properties, and how some

Harambee client ﬁrms use them in hiring.

Each assessment session is led by two or three industrial psychologists, who manage a team of

facilitators. Assessments are conducted in English and are self-administered on desktop computers.

Appendix Table D.1 shows standardized scores on all six assessments. There is a fairly even spread

of candidates over the distribution and little evidence of ceiling eﬀects.

Appendix Table A.2 shows the correlation matrix between diﬀerent skills. We interpret candi-

dates with diﬀerent assessment results as diﬀerent worker types, in the language of the conceptual

framework. Scores are weakly correlated across assessments, with pairwise correlations between 0.05

and 0.51. Hence, the assessments horizontally diﬀerentiate candidates based on their relative skills

rather than only ranking them in a single dimension of skills.

Candidates have inaccurate beliefs about their own types, suggesting a role for workseeker-side

information frictions. We ask candidates in which tercile they believe they ranked for each of the

communication, concept formation, and numeracy assessments, after taking the assessments but

before any candidates learn their results. Only 8% of candidates answer correctly for all three

assessments and 28% of candidates answer incorrectly for all three assessments. Overconﬁdence

is more common than underconﬁdence: 22% of candidates overestimate their tercile on all three

assessments and 1% underestimate their tercile all three assessments (Appendix Table D.1).

3 Labor Market Eﬀects of Certiﬁcation

3.1 Intervention

Our ﬁrst certiﬁcation intervention gives candidates information about their assessment results and

allows them to share the results with prospective employers. The eﬀects of this intervention may

reﬂect reductions in ﬁrm- or workseeker-side information frictions. In either case, the framework

predicts that certiﬁed workseekers will have higher employment and higher earnings conditional on

employment.

Candidates receive a certiﬁcate describing the assessments and their performance (Figure 1).

They receive 20 color copies printed on high-quality paper and an email version. Each certiﬁcate

brieﬂy describes Harambee and its placement and assessment work. To provide credibility to the

assessments and results, the certiﬁcate is branded with the World Bank logo and Harambee logo.

Harambee is a widely recognized brand in South African marketing surveys (Mackay, 2014).

The certiﬁcate describes the skills measured by each assessment. The certiﬁcate directs the

reader to www.assessmentreport.info for more information on Harambee and the assessments. The

website shows sample questions for each assessment and describes how psychologists have designed

and evaluated the assessments. For each skill, the certiﬁcate shows the tercile in which the can-

didate ranked on each assessment, compared to other candidates assessed by Harambee.

The

candidates assessed by Harambee are described as South African high school graduates aged 18-34

from disadvantaged backgrounds. To link candidates with certiﬁcates, each certiﬁcate shows the

candidate’s name and national identity number. National identity numbers are typically shown on

resumes and school transcripts in South Africa.

In piloting, both workseekers and ﬁrms found certiﬁcates with only rankings easier to interpret than certiﬁcates

Figure 1: Sample Public Certiﬁcate

REPORT ON CANDIDATE COMPETENCIES

name.. surname..

ID No. id..

This report provides information on assessments conducted by Harambee Youth Employment Accelerator (harambee.co.za), a South

African organisation that connects employers looking for entry-level talent to young, high-potential work-seekers with a matric or

equivalent. Harambee has conducted more than 1 million assessments and placed candidates with over 250 top companies in retail,

hospitality, financial services and other sectors. Assessments are designed by psychologists and predict candidates’ productivity and

success in the workplace. This report was designed and funded in collaboration with the World Bank. You can find more information

about this report, the assessments and contact details at www.assessmentreport.info. «name» was assessed at Harambee on 13

September, 2016.

«name» completed assessments on English Communication (listening, reading, comprehension), Numeracy, and Concept Formation:

1. The Numeracy tests measure candidates’ ability to apply numerical concepts at a National Qualifications Framework (NQF) level,

such as working with fractions, ratios, money, percentages and units, and performing calculations with time and area. This score is

an average of two numeracy tests the candidate completed.

2. The Communication test measures a candidate's grasp of the English language through listening, reading and comprehension. It

assesses at an NQF level, for example measuring the ability to recognise and recall literal and non-literal text.

3. The Concept Formation Test is a non-verbal measure that evaluates candidates’ ability to understand and solve problems. Those

with high scores are generally able to solve complex problems, while lower scores indicate an ability to solve less complex

problems.

«name» also completed tasks and questionnaires to assess their soft skills:

4. The Planning Ability Test measures how candidates plan their actions in multi-step problems. Candidates with high scores gener-

ally plan one or more steps ahead in solving complex problems.

5. The Focus Test assesses a candidate’s ability to distinguish relevant from irrelevant information in potentially confusing

environments. Candidates with high scores are generally able to focus on tasks in distracting surroundings, while candidates with

lower scores are more easily distracted by irrelevant information.

6. The Grit Scale measures whether candidates show determination when working on challenging problems. Those with high scores

generally spend more time working on challenging problems, while those with low scores choose to pursue different problems.

«name»’s results have been compared to a large benchmark group of young (age 18-34) South Africans assessed by Harambee.

All candidates have a matric certificate and are from socially disadvantaged backgrounds. The benchmark group is 5,000 for

cognitive skills and 400 for soft skills.

«name» scored in the «tercile_num» THIRD of candidates assessed by Harambee for Numeracy, «tercile_lit» THIRD for

Communication, «tercile_cft» THIRD for Concept Formation, «tercile_tol» THIRD for Planning Ability, «tercile_troop»

THIRD for Focus and «tercile_grit» THIRD for the Grit Scale.

DISCLAIMER: This is a condential assessment report for use by the person specied above. The information in the report should

only be disclosed on a “need to know basis” with the prior understanding of the candidate. Assessment results are not infallible and

may not be entirely accurate. Best practice indicates that any organisation’s career management decisions should depend on factors

in addition to these assessment results. Harambee cannot accept responsibility for decisions made based on the information

contained in this report and cannot be held liable for the consequences of those decisions.

This ﬁgure shows an example of the certiﬁcates given to candidates in the certiﬁcation treatment. Each certiﬁcate

shows some information about the assessments, the candidate’s assessment results, the candidate’s name and

national identity number, and the logo of the World Bank and the implementing agency. Each workseeker received

20 printed certiﬁcates, an email copy of the certiﬁcate, and guidelines on how to request more certiﬁcates.

Each candidate receives their certiﬁcates during a group brieﬁng with a psychologist. The

psychologist explains what each assessment measures and how to interpret the results on the cer-

tiﬁcate. They explain that workseekers can, but do not have to, attach the certiﬁcate to future job

applications and that they can request more certiﬁcates from Harambee. To ensure brieﬁngs were

standardized, the research team and Harambee psychologists jointly developed a brieﬁng script and

PowerPoint presentation. Research assistants monitored each brieﬁng to ensure psychologists used

the script.

3.2 Experimental Design

We randomly divide our workseeker sample into a certiﬁcation group, a control group, and other

groups discussed in the next section. We randomize treatment by assessment date to reduce risks of

spillovers between treated and control workseekers, assigning 2,247 workseekers assessed on 27 days

to certiﬁcation and 2,274 workseekers assessed on 27 days to the control group. Table D.1 shows

that the randomization generates balanced treatment assignments. Treated and control workseekers

diﬀer in only one way: treated workseekers receive the certiﬁcation intervention described above,

while control workseekers receive no information about their assessment results and no certiﬁcate

to enable them to share results with ﬁrms. All treated and control workseekers receive roughly one

hour of job search counselling before the assessments on how to create an email address and how to

prepare and dress for an interview. They also receive an email with a CV template, interview tips,

and job search tips.

This diﬀers from the design in Abebe et al. (2020a), where treated workseekers

receive both skill certiﬁcation and job search counselling while control workseekers receive neither.

We estimate treatment eﬀects using models of the form

= T

· ∆ + X

· Γ + S

+ 

, (1)

where Y

is the outcome for workseeker i assessed on date d, T

is a vector of treatment assign-

ments, and X

is a vector of prespeciﬁed baseline covariates. S

is a block ﬁxed eﬀect, to account

for the fact that we randomly assign days to treatment groups within blocks of 6-10 sequential

days. We use heteroskedasticity-robust standard errors clustered by assessment date, the unit of

treatment assignment. All labor market and job search measures use 7-day recall periods, except

where we specify otherwise. We apply an inverse hyperbolic sine transformation to right-skewed

variables such as earnings; the distributions of these variables in our sample allow us to interpret

these treatment eﬀects as percentage changes. We assign zeros to job characteristics for non-working

with only cardinal scores or both rankings and cardinal scores.

Harambee invites some workseekers for further training and job search assistance. These invitations depend

partly on their assessment results and may only be issued months after assessment. By the endline survey, only 1.4%

of our sample are invited for further interaction with Harambee and only 0.17% receive a job oﬀer through their

further interaction with Harambee. These outcomes are uncorrelated with treatment status and all our results are

robust to dropping these workseekers.

Table 1: Treatment Eﬀects on Labor Market Outcomes

(1) (2) (3) (4) (5)

Employed Hours

Earnings

Hourly wage

Written contract

Treatment 0.052 0.201 0.338 0.197 0.020

(0.012) (0.052) (0.074) (0.040) (0.010)

Mean outcome 0.309 8.85 159.3 9.84 0.120

Mean outcome for employed 28.85 518.3 32.28 0.392

# observations 6607 6598 6589 6574 6575

# clusters 84 84 84 84 84

Coeﬃcients are from regressing each outcome on a vector of treatment assignments, randomization block ﬁxed

eﬀects, and prespeciﬁed baseline covariates (measured skills, self-reported skills, education, age, gender, employ-

ment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering

by treatment date. Mean outcome is for the control group. All outcomes use a 7-day recall period. Outcomes

marked with

use the inverse hyperbolic sine transformation for the treatment eﬀects but the control group

means are reported in levels. All monetary ﬁgures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in

purchasing power parity terms. The sample sizes diﬀer across columns due to item non-response, mostly from

respondents reporting that they don’t know the answer.

respondents (e.g. earnings, hours) and to search measures for non-searching respondents (e.g. num-

ber of applications submitted) to avoid sample selection. We thus analyze treatment eﬀects on

realized outcomes, rather than latent outcomes that may be non-zero for the non-employed or non-

searching. We also estimate quantile treatment eﬀects on selected outcomes, which allows us to

focus on the distribution of outcomes for only employed candidates.

The estimating equations and variable deﬁnitions are prespeciﬁed. We report treatment eﬀects

on some outcomes that are not prespeciﬁed but note when we do so. In Appendix D, we show that

our results are robust to prespeciﬁed adjustments for multiple testing and omitting the prespeciﬁed

covariates X

3.3 Certiﬁcation Improves Labor Market Outcomes

The ﬁrst main eﬀect of certiﬁcation is to increase employment. Current employment rises by 5.2

percentage points from a control group mean of 30.1 percentage points (Table 1, column 1). We also

ask about employment in each calendar month between treatment and endline and show in Table

D.10 that certiﬁcation increases employment in every month between treatment and follow-up.

Certiﬁcation increases average weekly hours worked by 20% (column 2). We code hours worked

as zero for non-employed candidates. So the treatment eﬀect on hours may reﬂect two eﬀects: an

extensive margin eﬀect if treatment increases the employment rate and an intensive margin eﬀect if

treatment increases the hours that employed candidates work. We adapt a decomposition proposed

by Attanasio et al. (2011) to identify these two eﬀects (details in Appendix C). We deﬁne the ex-

tensive margin eﬀect as the treatment eﬀect on employment multiplied by mean hours worked for

employed control group candidates. Intuitively, this is the rise in hours we would see if treatment

increased employment but the marginally and inframarginally employed treated candidates worked

Table 2: Treatment Eﬀects on Labor Market Outcomes at Extensive and Intensive Margins

(1) (2) (3) (4)

Hours

Earnings

Hourly wage

Written contract

Total eﬀect 0.201 0.338 0.197 0.020

(0.052) (0.073) (0.039) (0.010)

Extensive margin 0.189 0.269 0.141 0.020

(0.042) (0.059) (0.031) (0.005)

Intensive margin 0.013 0.069 0.056 -0.000

(0.020) (0.040) (0.028) (0.008)

Treatment eﬀect conditional 0.036 0.195 0.159 -0.001

on employment (0.058) (0.113) (0.078) (0.024)

This table reports decompositions of treatment eﬀects on job characteristics into extensive and intensive margin

eﬀects. The extensive margin eﬀects are the treatment eﬀects on job characteristics due to the treatment eﬀect

on employment, evaluated at the mean job characteristics for the control group. The intensive margin eﬀects

are the diﬀerences between the treatment eﬀects and extensive margin eﬀects, which must be due to changes

in job characteristics for the employed candidates in the treatment group. The conditional eﬀect is the implied

mean change in job characteristics per employed treatment group candidate. Treatment group employment is

36%, so the conditional eﬀects on all outcomes are roughly three times larger than the corresponding intensive

margin eﬀect. Heteroskedasticity-robust standard errors are shown in parentheses, clustering by treatment date.

All outcomes use a 7-day recall period. Outcomes marked with

use the inverse hyperbolic sine transformation.

the same average hours as the inframarginally employed untreated candidates. We deﬁne the inten-

sive margin eﬀect as the diﬀerence between the total treatment eﬀect on hours and the extensive

margin eﬀect on hours. We ﬁnd that the entire eﬀect on hours is explained by the extensive margin

eﬀect (Table 2, column 1). This shows that treated candidates do not work longer hours conditional

on employment, but are simply more likely to be employed.

The second main eﬀect of certiﬁcation is to increase earnings. Weekly earnings increase by 34%

(Table 1, column 3). 27 percentage points of the 34% increase in earnings is explained by the rise

in employment (the extensive margin eﬀect, shown in Table 2, column 2). This implies an intensive

margin increase in earnings of 7 percentage points per candidate. Hourly wages, calculated by

dividing earnings by hours, also increase by 20% (Table 1, column 4). The extensive and intensive

margins account for respectively 14 and 6 percentage points of the 20% increase in wages (Table 2,

column 3). These results show that treatment increases earnings mainly by increasing employment,

but also increases earnings conditional on employment.

These results are consistent with the conceptual framework: more information about workseeker

skills (i.e. types) increases the latent value of some workseeker-job matches, leading to higher

employment and mean earnings conditional on employment. This explanation also matches the

quantile treatment eﬀects on earnings. These are positive throughout the earnings distribution,

consistent with broad increases in match value rather than an increase only in low-value, low-wage

matches (Figure 2). The earnings results are not consistent with a special case of the framework

where more information increases only job-ﬁnding rates but not the output of ﬁrm-worker matches,

which would not change earnings conditional on employment.

Figure 2: Quantile Treatment Eﬀects on Earnings

Panel A: Empirical Distributions of Earnings in Control and Public Certiﬁcation Groups

Control

Treated

.6 .7 .8 .9 1

Empirical CDF

0 2 4 6 8 10

Earnings (inverse hyperbolic sine)

Panel B: Quantile Treatment Eﬀects of Public Certiﬁcation on Earnings

−2 0 2 4 6

Treatment effect on earnings (IHS)

.6 .7 .8 .9 1

Percentile

Panel A shows the empirical distributions of earnings in the control and public certiﬁcation groups. Earnings

are the inverse hyperbolic sine transformation of earnings in South African Rands, with 1 Rand ≈ 0.167 USD

in purchasing power parity terms. Earnings are coded as zero for candidates who are not working. The vertical

axis in Panel A is truncated below at the 60

percentile because earnings below that value are zero. Panel B

shows the quantile treatment eﬀects (QTEs) of public certiﬁcation. These are unconditional QTEs, estimated

without controlling for any covariates or stratum ﬁxed eﬀects. The 95% pointwise conﬁdence intervals allow

heteroskedasticity and clustering by treatment date. The conﬁdence intervals exclude zero at all percentiles

except 73-74, 86, and 93-99.

Finally, certiﬁcation increases by 2 percentage points the probability of having a written contract,

Statistics South Africa’s deﬁnition of a formal job (Table 1, column 5). This eﬀect is entirely

explained by the higher employment rate (Table 2, column 4). Consistent with this result, 4

percentage points of the 5.2 percentage point increase in employment are in wage employment,

and only 1.2 percentage points into self employment. Wage and self employment status are not

prespeciﬁed outcomes.

The eﬀects on employment and earnings are substantial and easily exceed the cost of the pro-

gram. The employment eﬀect is almost three times larger than the mean standardized short-run

eﬀect size of active labor market programs reviewed by Card et al. (2018), larger than the eﬀect of

an intervention that helped similar South African workseekers get reference letters from past em-

ployers (Abel et al., 2020), and similar to the eﬀect of a program that subsidized ﬁrms to hire South

African workseekers from similar backgrounds (Levinsohn et al., 2013). The average earnings gain

in the ﬁrst three months after treatment is 778 South African Rands (USD 130 PPP) – 5.6 times

the average variable cost of certiﬁcation alone and 2.3 times the average variable cost of assessment

and certiﬁcation (details in Appendix B). The average weekly earnings gain is equal to 17% of the

weekly adult poverty line in South Africa (details in Appendix D.2).

4 Separating Workseeker- and Firm-side Information Frictions

Certiﬁcation may increase employment and earnings by providing information to ﬁrms, to work-

seekers, or both sides of the market. This distinction matters for modeling information frictions

and designing government or market-based remedies to information frictions. In this section, we

show that both sides of the market change behavior in response to new information. Our argument

proceeds in three steps. First, we show that public certiﬁcation changes workseekers’ beliefs and

search behavior in ways that are consistent with the existence of either ﬁrm- or workseeker-side

information frictions, or both. Second, we discuss another arm of our workseeker experiment that

reveals information only to workseekers. The results of this intervention, compared to the eﬀects of

public certiﬁcation, are consistent with both ﬁrm- and workseeker-side information frictions exist-

ing. The results are not consistent with only one-sided frictions, either for workseekers or for ﬁrms.

Third, we discuss an audit-style experiment that reveals information only to ﬁrms. The results of

this experiment are consistent with ﬁrm-side information frictions.

4.1 Certiﬁcation Changes Job Search and Beliefs

We document three patterns in the eﬀects of certiﬁcation on workseekers’ beliefs and job search

behavior. First, certiﬁcation shifts workseekers’ beliefs about their skills closer to their measured

skills. We ask candidates if they think they scored in the top, middle, or bottom third on each

of the six assessments compared to other candidates assessed by Harambee. Certiﬁcation increases

the fraction of assessments where candidates’ self-assessments match their measured results from

0.39 to 0.55 (Table 3, column 1).

In contrast, certiﬁcation has no eﬀect on candidates’ self-esteem

(column 2). This shows that their updated beliefs about the skills do not lead to more general

updating about their self-worth.

Second, certiﬁcation changes the types of jobs that candidates target. We ask candidates if the

types of jobs they are applying for most value communication, concept formation, or numeracy.

Certiﬁcation increases the fraction of candidates searching for jobs that most value the assessment

in which they scored strictly highest from 0.16 to 0.21 (column 3).

Third, candidates use certiﬁcates in job applications (columns 4-7). 70% of candidates use the

certiﬁcates with at least one job application between treatment and endline, with an unconditional

average of 6.7 applications sent per candidate.

Applications with certiﬁcates generate an average

of 0.43 interviews and 0.11 job oﬀers over the 3-4 months from treatment to endline.

The ﬁrst two patterns suggest a role for workseeker-side frictions: candidates align their beliefs

and job search more closely to their assessment results, potentially leading to better outcomes in the

labor market. The third pattern suggests a role for ﬁrm-side information frictions: candidates use

reports with job applications, potentially making the applications more informative to employers,

leading to more job interviews and oﬀers. Some combination of these patterns leads candidates to

expect 11% more oﬀers in the next month, from a control group mean of 4.2 oﬀers (column 8).

Taken together, these three patterns are consistent with certiﬁcation reducing both ﬁrm- and

workseeker-side information frictions. However, these three patterns are not suﬃcient to show

whether reducing only one of these frictions could generate the employment and earnings eﬀects

of certiﬁcation. For example, the revised search targeting might by itself increase employment and

earnings. Alternatively, the certiﬁcates attached to job applications might by themselves increase

employment and earnings. We therefore run two more experiments that separately manipulate the

information available to ﬁrms and workseekers.

Before proceeding to the next experiments, we note that certiﬁcation does not change multiple

prespeciﬁed measures of job search eﬀort in the month before the endline: the probability of doing

any search, number of applications submitted, hours spent searching, and money spent on search

(Table D.10). There are two possible explanations for this pattern. First, certiﬁcation may change

how workseekers search – targeting diﬀerent jobs and using certiﬁcates in applications – without

This question measures candidates’ beliefs about their assessment results. These may diﬀer from their beliefs

about their domain-speciﬁc skills if, for example, they believe the assessments are noisy measures of their skills. To

address this possibility, we ask candidates if their communication and numeracy skills are in the top, middle, or bottom

third of people aged 18-34, from disadvantaged backgrounds, with high school education (the population typically

assessed by Harambee). This is not a question about their result on a speciﬁc assessment. Treatment increases the

share of the two skills where candidates’ beliefs match their assessment results by 12.4 percentage points (standard

error 2.2 p.p.). This shows that candidates’ updated beliefs are not assessment-speciﬁc. We collect this measure only

for a random 50% sample of the ﬁrst 3,000 candidates to complete the survey.

This search targeting measure is not prespeciﬁed. The result is similar for the fraction of candidates searching

for jobs that most value the assessment in which they think they scored highest.

The 6.7 additional applications with reports follows from the 1.682 unit eﬀect on the inverse hyperbolic since of

the number of applications in column 5, and the fact that control workseekers send zero applications with reports

Table 3: Public and Private Certiﬁcation Eﬀects on Beliefs, Search, and Labor Market Outcomes

(1) (2) (3)

Skill belief > median Targeted

accurate self-esteem search

Public certiﬁcation 0.158 0.001 0.052

(0.008) (0.013) (0.010)

Private certiﬁcation 0.123 -0.002 0.047

(0.008) (0.014) (0.010)

p: public = private 0.000 0.806 0.698

Mean outcome 0.389 0.553 0.155

# observations 6607 6609 6609

# clusters 84 84 84

(4) (5) (6) (7) (8)

Used Applications Interviews Oﬀers Expected

report

with report

b,c

with report

oﬀers

a,c

Public certiﬁcation 0.699 1.682 0.432 0.112 0.106

(0.013) (0.040) (0.023) (0.011) (0.019)

Private certiﬁcation 0.289 0.572 0.144 0.036 0.053

(0.012) (0.033) (0.017) (0.008) (0.023)

p: public = private 0.000 0.000 0.000 0.000 0.025

Mean outcome 0.000 0.000 0.000 0.000 4.198

# observations 6609 6598 6597 6597 6531

# clusters 84 84 84 84 84

(9) (10) (11) (12) (13)

Worked Hours

Earnings

Hourly Written

wage

contract

Public certiﬁcation 0.052 0.201 0.338 0.197 0.020

(0.012) (0.052) (0.074) (0.040) (0.010)

Private certiﬁcation 0.011 0.066 0.162 0.095 0.017

(0.012) (0.048) (0.078) (0.046) (0.009)

p: public = private 0.002 0.011 0.028 0.030 0.769

Mean outcome 0.309 8.848 159.291 9.840 0.120

# observations 6607 6598 6589 6574 6575

# clusters 84 84 84 84 84

Coeﬃcients are from regressing each outcome on a vector of treatment assignments, randomization block ﬁxed

eﬀects, and prespeciﬁed baseline covariates (measured skills, self-reported skills, education, age, gender, employ-

ment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering

by treatment date. Mean outcome is for the control group. Skill belief accurate is the share of the six assessments

where the candidate’s perceived tercile matches their actual tercile. Targeted search is an indicator equal to one

if the candidate reports mainly applying for jobs that most value the skill in which the candidate scored highest.

Above-median self-esteem is an indicator equal to one if the candidate’s response on a shortened version of the

Rosenberg (1965) self-esteem scale is above the sample median. All outcomes use a 7-day recall/forecast period

unless marked with

(30-day recall/forecast period) or

(since treatment). Outcomes marked with

use the

inverse hyperbolic sine transformation for the treatment eﬀects but the control group means are reported in lev-

els. All monetary ﬁgures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity

terms. The sample sizes diﬀer across columns due to item non-response, mostly from respondents reporting that

they don’t know the answer.

changing their search eﬀort. This is consistent with a special case of the conceptual framework

where information frictions change how ﬁrms and workseekers match but do not change the share

of workseekers who choose to search. Second, certiﬁcation may temporarily change extensive or

intensive margin search eﬀort but the endline may occur too late to detect this change. Employment

already rises in the ﬁrst month after treatment (Table D.10). This suggests that any changes

in workseeker behavior that increase employment occur soon after treatment. The search eﬀort

questions use 7- or 30-day recall periods, which miss the period soon after treatment when candidates

may have increased eﬀort and found jobs. The questions on certiﬁcate use ask about the entire

period between treatment and the endline survey, which will capture any short-term changes in

search behavior.

4.2 Workseekers Face Information Frictions

In this section, we explore whether revealing information only to workseekers can replicate the

employment and earnings eﬀects of our certiﬁcation intervention. If not, then it is likely that

certiﬁcation does not work by addressing only workseeker-side information frictions.

We implement a ‘private’ certiﬁcation intervention, distinct from the ‘public’ certiﬁcation in-

tervention described above. Candidates assigned to the private certiﬁcation intervention receive an

unbranded, anonymous certiﬁcate with the assessment results rather than the branded, identiﬁable

‘public’ certiﬁcate (Figure 3). We interpret the private treatment as primarily providing information

to the workseekers about their own types.

Candidates in this group receive only one black-and-white, unbranded copy of the report, printed

on low-quality paper, and do not receive an electronic version. Candidates receive a brieﬁng from

a psychologist about the assessment results. But this brieﬁng does not encourage them to share

the certiﬁcate with ﬁrms or suggest that this is possible, unlike the brieﬁng received by candidates

in the public certiﬁcation group. Candidates in the public certiﬁcation, private certiﬁcation, and

control groups all receive the same one hour of job search counselling and email with job search

advice. We assign 2,114 candidates assessed over 27 assessment days to private certiﬁcation. We

simultaneously randomize days to public certiﬁcation, private certiﬁcation, and control. The three

groups are balanced on baseline characteristics (Table D.1).

The private and public certiﬁcation interventions have similar eﬀects on workseekers’ beliefs

and search targeting. Private certiﬁcation makes workseekers’ beliefs about their own skills more

accurate and has no eﬀect on self-esteem (Table 3, columns 1-2).

The private and public eﬀects

Consistent with this timing explanation, eﬀects on all search eﬀort measures are marginally larger for respondents

with a shorter time between treatment and endline. This result is robust to instrumenting the treatment-to-endline

time with the random order in which candidates were assigned to be surveyed.

The private certiﬁcation eﬀect on beliefs about skills is slightly smaller than the public certiﬁcation eﬀect. The

former eﬀect may be smaller because the public treatment conveys information diﬀerently (e.g. the branding makes

it more credible to workseekers) or because the information is more likely to be retained (e.g. workseekers are more

likely to keep copies of the public report or discuss it in recent job interviews). To separate these hypotheses, we

Figure 3: Sample Private Certiﬁcate

REPORT ON CANDIDATE COMPETENCIES

-Personal Copy-

This report contains results from the assessments you took at Harambee in Phase 1 and Phase 2. These results can help

you learn about some of your strengths and weaknesses and inform your job search.

You completed assessments on English Communication (listening, reading and comprehension) and Numeracy today in

Phase 2. In Phase 1, you completed a Concept Formation assessment which asked you to identify patterns.

1. The Numeracy tests measure various maths abilities. Your score is the average of the two maths tests you did

today at Harambee.

2. The Communication test measures English language ability through listening, reading and comprehension.

3. The Concept Formation test measures the ability to understand and solve problems. Candidates with high scores

can generally solve complex problems, while lower scores show an ability to solve less complex problems.

You also did some games and questionnaires to measure your soft skills:

4. The Planning Ability Test measures how you plan your actions in multi-step problems. Candidates with high

scores generally plan one or more steps ahead in solving complex problems.

5. The Focus Test looks at your ability to pick out which information is important in confusing environments.

Candidates with high scores are able to focus on tasks in distracting situations.

6. The Grit Scale measures candidates’ determination when working on difficult problems. Candidates with high

scores spend more time working on the problems rather than choosing to pursue different problems.

Your results have been compared to a large group of young South African job seekers who have a matric

certificate, are from socially disadvantaged backgrounds and have been assessed by Harambee.

You scored in the MIDDLE THIRD of candidates assessed by Harambee for Numeracy, MIDDLE THIRD for

Communication, LOWER THIRD for Concept Formation, LOWER THIRD for Planning Ability, MIDDLE

THIRD for Focus and TOP THIRD for the Grit Scale.

DISCLAIMER

Please note that this is a confidential assessment report and is intended for use by the person specified above. Assessment results are not infallible and may not be

entirely accurate.

Planning Ability

Note: This ﬁgure shows an example of the certiﬁcates given to candidates in the private treatment arm. The

certiﬁcates contain the candidate’s assessment results but no identifying information and no branding. Each

candidate received one copy of this certiﬁcate.

on search targeting are almost identical (column 3). Candidates in the private arm expect to receive

5% more oﬀers than control candidates, signiﬁcantly less than the 11% increase in expected oﬀers

in the public arm (column 8). This suggests that workseekers view the new information as useful,

but less useful than also being able to communicate their assessment results to ﬁrms.

The private certiﬁcation intervention has substantially smaller eﬀects than public certiﬁcation

on candidates’ outcomes in the labor market. Private certiﬁcation eﬀects on the probability of

employment and hours worked are positive but small, not signiﬁcantly diﬀerent from zero, and sig-

niﬁcantly smaller than the public certiﬁcation eﬀects (columns 9-10). Private certiﬁcation increases

earnings and hourly wages but both eﬀects are less than half the size of the public certiﬁcation

eﬀects and signiﬁcantly smaller (columns 11-12).

Results from both the public and private certiﬁcation interventions are consistent with quanti-

tatively important information frictions facing both workseekers and ﬁrms. Giving information to

workseekers in both the public and private certiﬁcation interventions increases the accuracy of their

beliefs about their skills, allowing better search targeting, and leading to higher earnings. When

workseekers can also credibly convey that information to ﬁrms, as in the public intervention, they

are more likely to be employed and have even higher earnings and wages.

There are two ways that these results might reﬂect only ﬁrm-side or only workseeker-side in-

formation frictions, but we view both as unlikely. First, the private certiﬁcation may deliver some

information to ﬁrms and this information, rather than changes in workseeker beliefs or search tar-

geting, may generate the positive private eﬀect on earnings. This explanation is consistent with

the fact that some candidates report using private certiﬁcates in job applications, although private

eﬀects on certiﬁcate use are on average one third as large as the public eﬀects (Table 3, columns 4-7).

We cannot conclusively rule out this interpretation but we view it as unlikely. The private reports

do not have the candidate’s name and identity number, so they cannot be linked to a particular

candidate. They have no branding, from Harambee or the World Bank. They do not explain that

Harambee has used these assessments widely to place candidates with companies or that assess-

ments predict workplace productivity and do not link to a website. None of the 15 hiring managers

interviewed during piloting reported that they would view the private reports as credible.

Second, the public certiﬁcation eﬀects on employment and earnings may be larger than the

private eﬀects because workseekers incorrectly believe that ﬁrms face information frictions. Under

this explanation, workseekers in the public certiﬁcation group believe that ﬁrms are more likely to

respond to job application submitted with certiﬁcates, hence they search more or search diﬀerently

using a method or in a time period we do not measure. These changes in search behavior may

generate higher employment and earnings, even if ﬁrms face no information frictions and do not

measure workseekers’ beliefs about their skills using a text message survey 2-3 days after treatment. The public

and private eﬀects in this survey are not diﬀerent to each other, suggesting that workseekers’ beliefs update in the

same way straight after receiving the reports and that the diﬀerence in the endline survey 3-4 months later is due to

diﬀerential retention. See Appendix D and Table D.8 for details.

respond to certiﬁcates. In the next section, we address this possibility by discussing an experiment

that directly manipulates ﬁrms’ information, without any scope for changes in workseeker behavior.

4.3 Firms Face Information Frictions

In this section, we show that revealing information only to ﬁrms changes their responses to job

applications, without allowing any potentially mediating behavior by workseekers. This is consistent

with ﬁrms facing information frictions and the employment and earnings eﬀects of certiﬁcation being

partly explained by ﬁrm-side information frictions.

We describe results from an audit-style study, with more details on the experiment in Appendix

E. We invite a random sample of assessed candidates to send us a resume that we will forward

to prospective employers on their behalf. We create a list of job vacancies by scraping online

job advertisements. We eliminate scam vacancies and vacancies that require work experience or

university education, where many candidates in our sample would be ineligible. We send resumes

from 4 randomly chosen candidates to each vacancy, each from a diﬀerent email address. We

generate two outcome variables based on the email responses from ﬁrms. ‘Interview invitations’

are invitations to interview with the ﬁrm. ‘Any responses’ are similar to ‘callbacks’ in other audit

studies and include interview invitations and requests to provide more information by email or by

visiting the ﬁrm in person.

We randomize each vacancy to receive either 1 or 3 resumes with public certiﬁcates attached. We

also randomize which of the resumes are chosen to receive public certiﬁcates. This design motivates

the estimating equation

= Certiﬁcate

· β

+ Certiﬁcate

· HighIntensity

· β

+ V

+ X

· Γ + E

+ 

, (2)

where Y

is the response to resume r sent to vacancy v, Certiﬁcate

is an indicator equal to

one if the application includes a public certiﬁcate, HighIntensity

is an indicator equal to one if the

vacancy receives 3 applications with certiﬁcates rather than 1, V

is a vector of vacancy ﬁxed eﬀects

that subsumes the main eﬀect of HighIntensity

, X

is a vector of prespeciﬁed resume covariates,

and E

is a vector of ﬁxed eﬀects for the email addresses used to submit the applications. We

cluster standard errors by resume and vacancy.

, the eﬀect of using a public certiﬁcate when other applications do not, is positive. Applications

with a public certiﬁcate are 1.6 percentage points more likely to get any response and 1 percentage

point more likely to get an interview invitation (Table 4, columns 2 and 4). These are substantial

eﬀects, both equal to 11% of the control group means, although they are only statistically signiﬁcant

at the 10% level.

Like most audit studies, we submit the same resume to multiple vacancies. Each resume includes a certiﬁcate

for half of these vacancies. Audit studies generally cluster standard errors by resume (Neumark, 2018). Abadie et al.

(2017) recommend clustering by the unit at which treatment is assigned. We therefore cluster by both vacancy and

resume. Results are very similar when clustering only by vacancy or only by resume.

Table 4: Treatment Eﬀects of Additional Information in Audit Study

(1) (2) (3) (4)

Any response Interview request

Certiﬁcate (β

) 0.015 0.016 0.009 0.010

(0.009) (0.009) (0.004) (0.006)

Certiﬁcate × HighIntensity (β

) -0.027 -0.028 -0.014 -0.017

(0.013) (0.014) (0.009) (0.010)

Mean outcome 0.130 0.130 0.088 0.088

# applications 3992 3992 3992 3992

# vacancies 998 998 998 998

# resumes 717 717 717 717

Vacancy ﬁxed eﬀects × ×

Email address ﬁxed eﬀects × ×

Resume covariates × ×

Note: Coeﬃcients are from regressing each outcome on a vector of treatment assignments and, in columns 2 and

4, vacancy ﬁxed eﬀects, email address ﬁxed eﬀects, a vector of prespeciﬁed workseeker covariates (measured skills,

education, age, gender, past employment, and the scan quality of documents they include in their application).

The vacancy-level treatment HighIntensity

is included in columns 1 and 3 but omitted in columns 2 and

4 because it is collinear with the vacancy ﬁxed eﬀects. Heteroskedasticity-robust standard errors shown in

parentheses clustered by resume and vacancy. The mean outcomes are for applications sent without public

certiﬁcates to vacancies that receive only one application with a public certiﬁcate.

These results show that more informative applications lead to higher callback and interview

invitation rates in a low-information environment. This suggests ﬁrm-side information frictions play

a role in the earnings and employment eﬀects of public certiﬁcation. Combining this result with the

observed eﬀects of the public and private certiﬁcation on workseekers’ beliefs, search behavior, and

outcomes in the labor market suggests that both ﬁrms and workseekers’ face information frictions.

, the diﬀerence between the eﬀect of being the only application with a public certiﬁcate sent

to a vacancy and the eﬀect of being one of multiple applications with public certiﬁcates sent to

a vacancy, is negative. Applications that include a public certiﬁcate are 2.8 percentage points

less likely to get a response and 1.7 percentage points less likely to get an interview invitation

when they compete against other applications with reports (Table 4, columns 2 and 4). However,

more informative applications may still be valuable in a higher-information environment for job

oﬀers, which we do not observe in the audit study, than callbacks and interviews. If ﬁrms use

callbacks and interviews to get more information, then certiﬁcates may allow them to interview

fewer candidates and still make better-matched hires. Jarosch and Pilossoph (2019) make a similar

argument, both theoretically and empirically, about audit studies that manipulate ﬁrms’ information

about applicants’ employment history.

There are, however, some caveats to the interpretation of the audit study results. This examines

only one hiring method (online applications) and one stage of that process (interview invitations).

These are standard limitations of correspondence-based audit studies. But it does mean that the

design cannot easily quantify the importance of ﬁrm-side information frictions relative to workseeker-

side information frictions on the same outcome. Furthermore, we randomly match workseekers to

vacancies in the audit study. This omits any role for search targeting, which the public and private

certiﬁcation results suggest may be important. We therefore view the audit study as less important

evidence than the arms of the workseeker-facing experiments. It simply provides additional evidence

that the diﬀerence between the public and private certiﬁcation results can be explained by ﬁrm-side

information frictions.

5 What Do Workseekers and Firms Learn From Skill Certiﬁcation?

The preceding two sections show that skill certiﬁcation provides information that improves work-

seekers’ outcomes in the labor market. In this section, we explore what workseekers and ﬁrms

learn from skill certiﬁcation, what this implies for the eﬀects of certiﬁcation on diﬀerent types of

workseekers, and what this might imply for the eﬀects of certiﬁcation at scale. This section relies

on smaller experiments and heterogeneity analysis of the main experiments, so we view these as

secondary results.

5.1 Assessment Results Matter, Not Just Being Assessed

The public certiﬁcation and audit results above are consistent with three interpretations. First, our

preferred interpretation is that ﬁrms and workseekers acquire information about workseekers’ skills

from the assessment results. Second, ﬁrms may acquire information about workseekers’ tenacity

or proactivity from their choice to get assessed, not their actual assessment results. Third, the

assessment results may provide no useful information to ﬁrms but may be visually appealing or

attention-grabbing because they are colorful, branded, and printed on high-quality paper. In this

section, we discuss two smaller experiments whose results are consistent with the ﬁrst, but not

the second or third interpretations. The ﬁrst interpretation is also consistent with the private

certiﬁcation results, while the second and third are not.

In the ﬁrst experiment, we manipulate information about workseekers’ assessment results, hold-

ing constant the information that workseekers have been assessed and any visual appeal of the

certiﬁcates. We randomly assign 254 candidates from our workseeker sample, assessed over 3 days,

to a ‘placebo’ certiﬁcation group. These candidates receive placebo certiﬁcates that are identical to

the public certiﬁcates except that the actual assessment results are omitted (Figure F.1) and the

psychologist’s brieﬁng does not discuss the assessment results.

The placebo certiﬁcation treatment has minimal eﬀects on labor market outcomes (Table F.1). It

increases an index of labor market outcomes by 0.03 standard deviations. This is not signiﬁcantly

diﬀerent to zero and is signiﬁcantly smaller than the public certiﬁcation eﬀect of 0.12 standard

deviations. This index is an inverse covariance-weighted average of the ﬁve labor market outcomes

discussed in Section 3.3: employment, hours, earnings, wages, and contract status. The placebo

certiﬁcation eﬀects on the ﬁve individual outcomes are all smaller than the public certiﬁcation

eﬀects and are on average only 26% as large. But we cannot reject equality of the public and

placebo eﬀects for some of the individual outcomes because the small size of the placebo sample

leads to large standard errors.

The second experiment measures ﬁrms’ willingness-to pay (WTP) for information on work-

seekers’ assessment results, conditional on knowing candidates have been assessed. We recruit 69

establishments located in commercial areas near the low-income residential areas in Johannesburg

where most workseekers in our sample live and are likely to work.

We conduct a survey and WTP

exercise with the person responsible for hiring decisions at each of these establishments. We show

this person a secure online database containing assessment results, contact information, and selected

resume-style information for our 6,891 candidates. This database allows users to ﬁlter and search for

candidates with speciﬁc assessment results and obtain their contact information. See Figures G.1

and G.2 for selected screenshots of the database. We use a Becker-DeGroot-Marshak mechanism to

measure WTP for access to this database relative to a placebo database with candidates’ contact

information and selected resume-style information, but no assessment results.

Firms’ WTP for access to the database with assessment results is substantial: 68% of ﬁrms

report positive WTP and the unconditional mean WTP is 1,161 South African Rands or USD 195

PPP (Figure G.3). Mean WTP is 224% of the mean weekly earnings for employed candidates in our

workseeker sample. This shows ﬁrms value information on speciﬁc assessment results, conditional

on knowing candidates have been assessed.

Both the placebo experiment and WTP measurement are consistent with the ﬁrst but not second

or third interpretations above: information about assessment results is valuable, not just information

about whether candidates have been assessed or any visual appeal of the certiﬁcates. This provides

additional support for our preferred interpretation: public certiﬁcation provides information about

workseekers’ types and facilitates more productive ﬁrm-worker matches.

5.2 Certiﬁcation Facilitates Horizontal More Than Vertical Diﬀerentiation

Our conceptual framework distinguishes two types of workseeker diﬀerentiation. Under horizon-

tal diﬀerentiation, type i workseekers are more productive than type j workseekers in type i jobs,

and vice versa. Under vertical diﬀerentiation, type i workseekers are more productive than type j

We recruit establishments by asking if they are willing to participate in a study on hiring and tell them we can

provide some useful information on hiring. We restrict the sample to establishments that have hiring responsibilities,

either single-establishment ﬁrms or branches of larger ﬁrms that hire independently. Most ﬁrms are in retail, have

multiple entry-level workers, expect to hire entry-level workers in the next year, and take on average four weeks to

ﬁll a vacancy (Table G.1).

We tell ﬁrms the database normally costs 10,000 South African Rands (USD 1,670 PPP) for three months access,

ask how much they are willing to pay for access, and then randomly oﬀer them a discount between 0 and 100%. If

their stated WTP is higher than the normal price minus the discount, we give them access to the database. If their

stated WTP is below the normal price minus the discount, we give them access to a placebo database with candidates’

contact information and selected resume-style information but no skill assessment results. We ﬁrst explain the entire

mechanism and run a practice round with a bar of chocolate.

workseekers in both type i and j jobs. The distinction is important for understanding how infor-

mation frictions aﬀect diﬀerent types of workseekers. Under horizontal diﬀerentiation, alleviating

information frictions can help both types of workseekers by matching them with jobs where they

are more productive. Under vertical diﬀerentiation, alleviating information frictions can hurt type j

workseekers by reducing their probability of being mistaken for more productive type i workseekers.

In this section, we discuss three relevant patterns in our data. Two are consistent with horizontal

diﬀerentiation and one is inconsistent with vertical diﬀerentiation, providing suggestive evidence of

horizontal diﬀerentiation in this setting.

First, there is substantial heterogeneity in ﬁrms’ relative demand for diﬀerent skills. We show

this using an incentivized choice experiment with the sample of 69 establishments described in

the previous subsection. We ask the person at each establishment responsible for hiring to rank

proﬁles of seven hypothetical candidates and tell them we will use their ranking to match them with

workseekers from the online database, following Kessler et al. (2019). Six of the proﬁles have middle

terciles for ﬁve assessments, and a top tercile for one assessment. There is substantial variation in

ﬁrms’ relative ranking of proﬁles (Table G.2). All six proﬁles’ median rank is between second and

fourth. The share of ﬁrms ranking each proﬁle highest ranges from 6 to 33%. The seventh proﬁle has

middle terciles for all six assessments and has a one-year post-secondary education certiﬁcate, while

the other six proﬁles have only completed secondary school. Only 9% of ﬁrms rank this proﬁle ﬁrst

and 76% of ﬁrms rank this last, showing that ﬁrms value the assessed skills relative to an alternative

signal of productivity in which workseekers might invest.

Second, assessment results are weakly correlated across skills within candidate. Numeracy and

concept formation are most highly correlated, with ρ ≈ 0.5. But most other pairwise correlations are

substantially lower, with ρ < 0.1 for some pairs of skills (Table A.2). As a result, most candidates’

certiﬁcates show substantial variation across skills. Table A.3 shows that 88% of the candidates

have at least one top tercile but only 24% have four or more top terciles and only 2.3% have all top

terciles. 76% of the candidates have at least one bottom tercile but only 12% have four or more

bottom terciles and only 0.7% have all bottom terciles. 64% of candidates have both top and bottom

terciles. Other studies that measure multidimensional skills also ﬁnd weak correlations across skills

within candidates (Almlund et al., 2011; Poropat, 2009).

Third, we do not ﬁnd strong evidence of vertical diﬀerentiation in the public certiﬁcation experi-

ment. To test for vertical diﬀerentiation, we construct three indices that combine the six assessment

results in diﬀerent ways: the number of top terciles minus bottom terciles, the ﬁrst principal com-

ponent of the cardinal scores, and a weighted average of the cardinal scores with weights based on

We conduct a second experiment where we ask ﬁrms to rank proﬁles with assessment results shown for some

skills and concealed for others. This assess whether ﬁrms value information about speciﬁc skills as well as the level of

the skills. The two experiments may yield diﬀerent results if, for example, ﬁrms ﬁnd skill S

most valuable but believe

the assessments of skill S

yield more new information. This second experiment also shows substantial heterogeneity

in ﬁrms’ ranking of diﬀerent proﬁles.

their association with earnings.

The ﬁrst index weights all skills equally, the second gives more

weight to skills that are highly correlated with each other, and the third gives more weight to skills

with higher associations with earnings. For each index, we construct an indicator for above-median

values of the index. We then include this indicator and its interactions with treatment assignments

in equation (1). The interaction eﬀects with public certiﬁcation on employment are smaller than 2

percentage points and not signiﬁcantly diﬀerent to zero for all indices (Table D.9) panel A.

Taken together, these results are more consistent with horizontal than vertical diﬀerentiation.

Firms and workseekers both seem to face information frictions, and certiﬁcation provides information

that facilitates more productive ﬁrm-worker matches that generate higher employment and earnings.

Employment and earnings rise for many diﬀerent types of workers, not just those with high values of a

skill index of the six assessment scores. This is consistent with models of multidimensional skill where

information frictions can lead to poor matches between workseeker skills and ﬁrm requirements

(Fredriksson et al., 2018; Guvenen et al., 2020; Lise and Postel-Vinay, 2020).

However, our experiment is not designed to provide a general test of horizontal against vertical

diﬀerentiation. Certiﬁcation may facilitate vertical rather than horizontal diﬀerentiation when it

is based on assessment of a single skill or assessments of multiple highly correlated skills; or when

certiﬁcates report a single summary measure of multiple skills. Certiﬁcation may also facilitate

vertical rather than horizontal diﬀerentiation when it covers a larger share of the workforce. Our

sample excludes workseekers with advanced degrees or less than completed secondary education,

whose skills may be more vertically diﬀerentiated from the workseekers in our sample.

5.3 Certiﬁcation Is More Eﬀective When Other Information on Workseekers’ Skills

is Limited

If certiﬁcation changes labor market outcomes by providing information about workseekers’ skills,

then it should be most eﬀective when there are limited alternative sources of information on work-

seekers’ skills. These sources might include past work experience and post-secondary education,

which allow workseekers and ﬁrms to learn about workseekers’ productivity in speciﬁc tasks. We

test this idea by augmenting equation (1) to include interactions between treatment and proxies

for alternative sources of information. Public certiﬁcation eﬀects on employment are 2.7 percentage

points smaller for candidates with post-secondary education (standard error 2.8 p.p.) and 4.3 per-

centage points smaller for candidates with prior work experience (standard error 3.2 p.p.) (Table

D.9, panel B). We also estimate the latent probability of being employed at endline as a single

The weights equal the coeﬃcients from regressing earnings on the cardinal scores using control group data. Re-

sults are similar for weighted averages based on the coeﬃcients of regressions of control group earnings on polynomial

or spline functions of the skills.

Results are similar using the continuous indices instead of binary indicators and when using alternative model

speciﬁcations: allowing nonlinear interactions between skill indices, using diﬀerent single indices, or using machine

learning methods to estimate heterogeneous treatment eﬀects simultaneously across all individual scores.

summary measure.

Candidates with above-median latent probabilities of employment have 6.9

percentage point smaller public certiﬁcation eﬀects than candidates with below-median latent prob-

abilities (standard error 2.8 p.p.). These results show that certiﬁcation can substitute for traditional

sources of information about workseekers’ skills.

This is consistent with evidence that educational

qualiﬁcations are more useful for members of groups facing statistical discrimination (Arcidiacono

et al., 2010).

5.4 Skill Certiﬁcation at Diﬀerent Scales

We show that skill certiﬁcation at a relatively small scale increases employment and earnings for

certiﬁed workseekers. In this section we discuss conditions under which eﬀects may vary with the

scale of skill certiﬁcation.

First, employment and earnings eﬀects may depend on scale if certiﬁed workseekers displace

non-certiﬁed workseekers. It is unlikely that our experimental results are due to displacement of

non-certiﬁed workseekers in the control group. We certify only 2,247 workseekers in a metropolitan

area with roughly 8 million people and 2 million employed workers (Statistics South Africa, 2016).

The probability of certiﬁed and control group workseekers applying for the same jobs by chance

is very small, and Harambee does not encourage recently-assessed workseekers to apply to speciﬁc

jobs or search for work in speciﬁc areas.

It is possible that certiﬁed workseekers displace non-certiﬁed workseekers who are not part of

the experimental sample. We cannot directly test for this, but we can evaluate the mechanisms

that might generate it. Displacement is less likely if certiﬁcation improves match quality and hence

increases the share of latent vacancies that are worth ﬁlling, as in our conceptual framework and

general equilibrium models of information frictions (Donovan et al., 2018; Jovanovic, 1979; Menzio

and Shi, 2011). Displacement is more likely if ﬁrms value certiﬁcation for some reason other than

information (e.g. visual appeal) or if certiﬁcation helps ﬁrms to identify a small set of universally-

demanded workseekers and compete for them.

Our results are more consistent with the match quality mechanism. We ﬁnd that ﬁrms’ demand

for diﬀerent skills is heterogeneous, ﬁrms value learning about workseekers’ speciﬁc skill types,

We estimate the latent probabilities following Abadie et al. (2018). We regress endline employment on baseline

demographics, education, assessment results, beliefs about assessment results, employment, earnings, and search

behavior in the control group. We use the predicted values from these regressions in all treatment groups as latent

probabilities for employment, adjusting the predicted values in the control group using leave-one-out estimation to

avoid overﬁtting.

This result is not explained by a correlation between workseekers’ skills and their education and past employment.

We regress employment on treatment assignments, a single index measure of skill from Section 5.2, a measure of

information about workseekers’ skills from this section, and a full set of interactions. The interactions between public

certiﬁcation and the single index skill measure remain close to zero, while the interactions between public certiﬁcation

and the measure of information about workseekers’ skills remain negative.

There are few existing papers that study how the eﬀects of speciﬁc active labor market policies change with

scale. For job search assistance policies speciﬁcally, Crépon et al. (2013) and Lise et al. (2004) ﬁnd larger-scale policies

generate negative spillovers on non-participants, while Blundell et al. (2004) ﬁnd no spillovers on non-participants.

and the gains from certiﬁcation are not limited to workseekers with speciﬁc skill proﬁles. All

these patterns suggest that ﬁrms and workseekers use certiﬁcation to learn about workseekers’

skills and facilitate better matches between workseekers’ skills and ﬁrms’ demand. We also ﬁnd

that certiﬁcation increases earnings and hourly wages conditional on employment, suggesting that

certiﬁed workseekers are in more productive matches. We do ﬁnd that the callback premium to

certiﬁcation drops when certiﬁed applicants compete against each other. This is consistent with some

certiﬁed workseekers displacing other certiﬁed workseekers from interviews. However, as discussed

in Section 4.3, this does not necessarily imply that certiﬁed workseekers displace other certiﬁed

workseekers from hiring. Certiﬁcation may allow ﬁrms to call back and interview fewer candidates

and still make better-matched hires, as shown by Jarosch and Pilossoph (2019).

Second, employment and earnings eﬀects may depend on scale if the extent of limited information

varies across the population of either ﬁrms or workseekers. Consider the case where the population

is divided into fraction p of uninformed workseekers, who do not know their skills and cannot convey

their skills to ﬁrms, and fraction 1−p of informed workseekers, who know their skills and can convey

this information to ﬁrms. Assessing and certifying more than p share of workseekers will have limited

returns. Our ﬁnding that certiﬁcation has larger employment and earnings eﬀects when there are

limited alternative sources of information on workseekers’ skills is consistent with this possibility.

Our experiment does not identify the population shares of workseekers or ﬁrms facing information

frictions. The share of relatively uninformed types may be higher in our sample than the population,

as we study workseekers with poor baseline labor market outcomes. But Harambee’s workseeker

recruitment does not explicitly advertize assessments or information frictions, so workseekers are

unlikely to select into the sample speciﬁcally for assessment and certiﬁcation.

Third, employment and earnings eﬀects may depend on scale if certiﬁcate (non-)use conveys

information in general equilibrium. If, for example, all workseekers get assessed and certiﬁed but

only use certiﬁcates when applying to vacancies where their match quality is high, then ﬁrms may

infer that workseekers without certiﬁcates are poor matches for these vacancies. In another example,

some ﬁrms may choose not to use assessments in hiring if assessments are costly and they believe

they can infer workseekers’ types by observing their interactions with other ﬁrms (Lockwood, 1991).

Our experiments cannot speak to these general equilibrium mechanisms. But adding either of these

two mechanisms to our conceptual framework still predicts that any non-zero use of assessment and

certiﬁcation will raise employment and earnings relative to no assessment and certiﬁcation.

Even if reducing information frictions has decreasing eﬀects on employment and earnings at

larger scales, it may still raise workseeker or ﬁrm welfare by reducing job search costs, vacancy

posting costs, and the frequency of bad hires that lead to separations. This interpretation is consis-

tent with models showing that ﬁrm- and workseeker-level search and matching frictions, including

information frictions, can lower aggregate utility through multiple mechanisms, not just through

unemployment (Donovan et al., 2018; Mortensen and Pissarides, 1999; Poschke, 2019).

6 Conclusion

Workseekers make job search decisions and ﬁrms make hiring decisions using potentially limited

information about workseekers’ skills. Providing more information about workseekers’ skills may

reduce these information frictions and hence improve workseekers’ outcomes in the labor market. We

show that assessing workseekers’ skills and communicating the assessment results to both workseek-

ers and ﬁrms increases assessed workseekers’ employment by 17% (5 percentage points), earnings

by 34%, and hourly wages by 20%. This shows that certiﬁcation gets more workseekers into work

and into higher-paying jobs. The main contribution of our paper is to show that this reﬂects both

workseeker- and ﬁrm-side information frictions. The distinction between workseeker- and ﬁrm-side

frictions is important, as it informs how economists, government, or private ﬁrms might design

information-provision products.

We study a context and sample where information frictions are likely: work experience is limited,

education-skill relationships are relatively weak, hiring mistakes are costly, and reservation and

minimum wages are relevant. However, none of these features are unique to young workseekers

in South Africa. Formal education qualiﬁcations are weakly related to measured skills in many

countries (Pritchett, 2013). Many labor markets face more regulations governing hiring, ﬁring,

and probation than in South Africa (Botero et al., 2004). Hiring mistakes may be costly even

when separations are unregulated, due to reposting and retraining costs. High rates of youth

unemployment in many countries are consistent with information frictions, as youths have less job

search and work experience that can reveal their skills to themselves or to ﬁrms (International

Labour Organization, 2017).

Our results suggest that, in similar contexts, providing information about workseekers’ skills may

be a valuable focus of government policy. Some existing job search assistance programs oﬀer skill

assessments to workseekers (McCall et al., 2016). Adding certiﬁcation to these assessments might

enhance their eﬀectiveness at low cost. We ﬁnd that adding certiﬁcation to an existing assessment

program generates earnings gains for workseekers that easily exceed the cost of both assessment and

certiﬁcation. Government involvement, through public-sector assessment programs or subsidies to

private-sector assessments, is likely to be particularly important for credit-constrained workseekers

(Abebe et al., 2020b). Better information about workseekers’ skills could also come from more

accurate assessments during formal education (MacLeod et al., 2017).

Our results suggest there may also be scope for market-based provision of information about

workseekers’ skills. We show that ﬁrms are willing to pay for access a database with information on

workseekers’ skill assessment results and contact information. We also ask workseekers in our sam-

ple how much of a hypothetical job search subsidy they would be willing to spend on certiﬁcation.

They report 17%, compared to 24% on training and 27% on transport, suggesting the possibility

of charging workseekers for assessment services. Some large ﬁrms already use in-house psycho-

metric assessments in hiring (Autor and Scarborough, 2008; Hoﬀman et al., 2018). Anecdotally,

psychometric assessments seem rarer in small ﬁrms, perhaps because in-house assessment systems

are unlikely to be cost-eﬀective when hires are infrequent. There are some third-party providers of

assessment services around the world, including Harambee, LinkedIn, and the Manpower Group.

Our results show that providing more information through certiﬁcation can be valuable even in

a labor market where some ﬁrms already use assessments, suggesting scope to grow this market.

There are important market design questions around third-party provision that might be addressed

in future work, such as which side(s) of the market will pay for assessment services, how third-party

providers can establish reputations, how precisely or coarsely information should be reported, and

under what conditions participants will opt into or out of assessment. This work might incorporate

existing models of screening and signalling when both agents and principals have limited information

(Alonso, 2018; Rosar and Schulte, 2012).

Our results also motivate future work on the interaction between diﬀerent information provision

mechanisms. For example, we ﬁnd that public certiﬁcation is most eﬀective for workseekers with

less work experience and without university education. This suggests that skill assessment and

certiﬁcation can substitute for alternative sources of information about workseekers’ skills. Future

work could examine conditions under which skill assessment and certiﬁcation are complements or

substitutes for network referrals, reference letters, or outsourcing agencies.

References

Abadie, A., S. Athey, G. Imbens, and J. Wooldridge (2017): “When Should You Adjust

Standard Errors for Clustering?” Working Paper 24003, National Bureau of Economic Research.

Abadie, A., M. Chingos, and M. West (2018): “Endogenous Stratiﬁcation in Randomized

Experiments,” Review of Economics and Statistics, 100, 567–580.

Abebe, G., S. Caria, M. Fafchamps, P. Falco, S. Franklin, and S. Quinn (2020a):

“Anonymity or Distance? Job Search and Labour Market Exclusion in a Growing African City,”

Review of Economic Studies, forthcoming.

Abebe, G., S. Caria, and E. Ortiz-Ospina (2020b): “The Selection of Talent: Experimental

and Structural Evidence from Ethiopia,” Manuscript, University of Bristol.

Abel, M. (2019): “Unintended Labor Supply Eﬀects of Cash Transfer Programs: New Evidence

from South Africa’s Pension,” Journal of African Economies, 28, 558–581.

We ﬁnd one result consistent with certiﬁcation enhancing the eﬀectiveness of referrals, potentially by helping

network links to target referrals or making their referrals more credible to employers. Public certiﬁcation slightly

increases the probability of securing a job through a formal application or interview after a referral. There is no

large or signiﬁcant treatment eﬀect on the probability of securing a job in other ways we measure: by approaching an

employer in person, dropping oﬀ an application, emailing an application, getting hired by a social contact directly,

or working at an employment broker. However, this result is only marginally statistically signiﬁcant once we account

for multiple testing across the diﬀerent ways of ﬁnding a job. Hence we view this as a suggestion for future work,

rather than a core result.

Abel, M., R. Burger, and P. Piraino (2020): “The Value of Reference Letters: Experimental

Evidence from South Africa,” American Economic Journal: Applied Economics, forthcoming.

Ahn, S., R. Dizon-Ross, and B. Feigenberg (2019): “Improving Job Matching Among Youth,”

Working Paper, Columbia University.

Aigner, D. and G. Cain (1977): “Statistical Theories of Discrimination in Labor Markets,”

Industrial and Labor Relations Review, 30, 175–187.

Alfonsi, L., O. Bandiera, V. Bassi, R. Burgess, I. Rasul, M. Sulaiman, and A. Vitali

(2017): “Tackling Youth Unemployment: Evidence from a Labor Market Experiment in Uganda,”

Manuscript, University College London.

Almlund, M., A. Duckworth, J. Heckman, and T. Kautz (2011): “Personality Psychology

and Economics,” in Handbook of the Economics of Education, ed. by E. Hanushek, S. Machin,

and L. Woessmann, Elsevier, 1–181.

Alonso, R. (2018): “Recruiting and Selecting for Fit,” Manuscript, LSE.

Altmann, S., A. Falk, S. Jäger, and F. Zimmermann (2018): “Learning about Job Search:

A Field Experiment with Job Seekers in Germany,” Journal of Public Economics, 164, 33–49.

Altonji, J. and C. Pierret (2001): “Employer Learning and Statistical Discrimination,” Quar-

terly Journal of Economics, 116, 313–335.

Anderson, M. (2008): “Multiple Inference and Gender Diﬀerences in the Eﬀects of Early Interven-

tion: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects,” Journal

of the American Statistical Association, 103, 1481–1495.

Arcidiacono, P., P. Bayer, and A. Hizmo (2010): “Beyond Signaling and Human Capital:

Education and the Revelation of Ability,” American Economic Journal: Applied Economics, 2,

76–104.

Attanasio, O., A. Kugler, and C. Meghir (2011): “Subsidizing Vocational Training for Disad-

vantaged Youth in Colombia: Evidence from a Randomized Trial,” American Economic Journal:

Applied Economics, 3, 188–220.

Autor, D. and D. Scarborough (2008): “Does Job Testing Harm Minority Workers? Evidence

from Retail Establishments,” Quarterly Journal of Economics, 123, 219–277.

Bassi, V. and A. Nansamba (2020): “Screening and Signaling Non-Cognitive Skills: Experimental

Evidence from Uganda,” Manuscript, University of Southern California.

Beaman, L., N. Keleher, and J. Magruder (2018): “Do Job Networks Disadvantage Women?

Evidence from a Recruitment Experiment in Malawi,” Journal of Labor Economics, 36, 121–153.

Beaman, L. and J. Magruder (2012): “Who Gets the Job Referral? Evidence from a Social

Networks Experiment.” American Economic Review, 102, 3574–3593.

Belot, M., P. Kircher, and P. Muller (2018): “Providing Advice to Jobseekers at Low Cost:

An Experimental Study on Online Advice,” Review of Economic Studies, 86, 1411–1447.

Benjamini, Y., A. Krieger, and D. Yekutieli (2006): “Adaptive Linear Step-Up Procedures

That Control the False Discovery Rate,” Biometrika, 93, 491–507.

Bertrand, M. and B. Crépon (2019): “Teaching Labor Laws: Evidence from a Randomized

Trial in South Africa,” Manuscript, University of Chicago and CREST.

Bhorat, H., T. Caetano, B. Jourdan, R. Kanbur, C. Rooney, B. Stanwix, I. Woolard,

L. Patel, Z. Khan, L. Graham, K. Baldry, and T. Mqehe (2016): “Investigating the Fea-

sibility of a National Minimum Wage for South Africa,” Manuscript, University of Johannesburg:

Centre for Social Development in Africa and University of Cape Town: Development Policy

Research Unit.

Bhorat, H. and H. Cheadle (2009): “Labour Reform in South Africa: Measuring Regulation

and a Synthesis of Policy Suggestions,” Development Policy Research Unit Working Paper 09/139,

University of Cape Town.

Blundell, R., M. C. Dias, C. Meghir, and J. Van Reenen (2004): “Evaluating the Employ-

ment Impact of a Mandatory Job Search Program,” Journal of the European Economic Associa-

tion, 2, 569–606.

Botero, J., S. Djankov, R. L. Porta, F. Lopez-de Silanes, and A. Shleifer (2004): “The

Regulation of Labor,” Quarterly Journal of Economics, 119, 1339–1382.

Bryan, G., S. Choudhury, and A. M. Mobarak (2014): “Under-Investment in a Proﬁtable

Technology: The Case of Seasonal Migration in Bangladesh,” Econometrica, 82, 1671–1748.

Card, D., J. Kluve, and A. Weber (2018): “What Works? A Meta-Analysis of Recent Active

Labor Market Program Evaluations,” Journal of the European Economic Association, 16, 894–931.

Chandrasekhar, A. G., M. Morten, and A. Peter (2020): “Network-Based Hiring: Local

Beneﬁts; Global Costs,” Working Paper 26806, National Bureau of Economic Research.

Conlon, J., L. Pilossoph, M. Wiswall, and B. Zafar (2018): “Labor Market Search with

Imperfect Information and Learning,” Working paper 24988, National Bureau of Economic Re-

search.

Crépon, B., E. Duflo, M. Gurgand, R. Rathelot, and P. Zamora (2013): “Do Labor

Market Policies Have Displacement Eﬀects? Evidence from a Clustered Randomized Experiment,”

Quarterly Journal of Economics, 128, 531–580.

Donovan, K., J. Lu, T. Schoellman, et al. (2018): “Labor Market Flows and Development,”

in 2018 Meeting Papers, Society for Economic Dynamics, vol. 976.

Falk, A., D. Huffman, and U. Sunde (2006): “Self-Conﬁdence and Search,” IZA Discussion

Paper 2525.

Farber, H. and R. Gibbons (1996): “Learning and Wage Dynamics,” Quarterly Journal of

Economics, 111, 1007–1047.

Franklin, S. (2017): “Location, Search Costs and Youth Unemployment: Experimental Evidence

from Transport Subsidies,” The Economic Journal, 128, 2353–2379.

Fredriksson, P., L. Hensvik, and O. N. Skans (2018): “Mismatch of Talent: Evidence on

Match Quality, Entry Wages, and Job Mobility,” American Economic Review, 108, 3303–3338.

Garlick, R., K. Orkin, and S. Quinn (2019): “Call Me Maybe: Experimental Evidence on

Frequency and Medium Eﬀects in Microenterprise Surveys,” World Bank Economic Review, forth-

coming.

Guvenen, F., B. Kuruscu, S. Tanaka, and D. Wiczer (2020): “Multidimensional Skill Mis-

match,” American Economic Journal: Macroeconomics, 12, 210–244.

Heath, R. (2018): “Why Do Firms Hire Using Referrals? Evidence from Bangladeshi Garment

Factories,” Journal of Political Economy, 126, 1691–1746.

Hoffman, M., L. Kahn, and D. Li (2018): “Discretion in Hiring,” Quarterly Journal of Eco-

nomics, 133, 1–36.

Ingle, K. and C. Mlatsheni (2017): “The Extent of Churn in the South African Youth Labour

Market: Evidence from NIDS 2008-2015,” Manuscript, University of Cape Town: Southern Africa

Labour and Development Research Unit.

International Labour Organization (2016): Enabling Environment for Sustainable Enter-

prises in South Africa, Geneva: International Labour Oﬃce.

——— (2017): Global Employment Trends for Youth 2017: Paths to a Better Working Future,

Geneva: International Labour Oﬃce.

Ioannides, Y. M. and L. D. Loury (2004): “Job Information Networks, Neighborhood Eﬀects,

and Inequality,” Journal of Economic Literature, 42, 1056–1093.

Jarosch, G. and L. Pilossoph (2019): “Statistical Discrimination and Duration Dependence in

the Job Finding Rate,” Review of Economic Studies, 86, 1631–1665.

Jovanovic, B. (1979): “Job Matching and the Theory of Turnover,” Journal of Political Economy,

87, 972–990.

Kahn, L. (2013): “Asymmetric Information between Employers,” American Economic Journal:

Applied Economics, 5, 165–205.

Kahn, L. and F. Lange (2014): “Employer Learning, Productivity, and the Earnings Distribution:

Evidence from Performance Measures,” Review of Economic Studies, 84, 1575–1613.

Kerr, A. (2017): “Tax(i)ing The Poor? Commuting Costs in South African Cities,” South African

Journal of Economics, 85, 321–340.

Kessler, J., C. Low, and C. Sullivan (2019): “Incentivized Resume Rating: Eliciting Employer

Preferences without Deception,” American Economic Review, forthcoming.

Lam, D., C. Ardington, and M. Leibbrandt (2011): “Schooling as a Lottery: Racial Diﬀer-

ences in School Advancement in Urban South Africa,” Journal of Development Economics, 95,

121–136.

Lange, F. (2007): “The Speed of Employer Learning,” Journal of Labor Economics, 25, 1–35.

Levinsohn, J., N. Rankin, G. Roberts, and V. Schoer (2013): “Wage Subsidies and Youth

Employment in South Africa: Evidence from a Randomized Control Trial,” Manuscript, Univer-

sity of Stellenbosch.

Lise, J. and F. Postel-Vinay (2020): “Multidimensional Skills, Sorting, and Human Capital

Accumulation,” American Economic Review, forthcoming.

Lise, J., S. Seitz, and J. Smith (2004): “Equilibrium Policy Experiments and the Evaluation of

Social Programs,” Working paper 10283, National Bureau of Economic Research.

Lockwood, B. (1991): “Information Externalities in the Labour Market and the Duration of

Unemployment,” Review of Economic Studies, 58, 733–753.

Mackay, A. (2014): Building Brands in a Rapidly Changing Market: Lessons for South Africa,

Yellowwood.

MacLeod, W. B., E. Riehl, J. Saavedra, and M. Urquiola (2017): “The Big Sort: College

Reputation and Labor Market Outcomes,” American Economic Journal: Applied Economics, 9,

223–261.

Malindi, K. (2017): “Imperfect Information and The Racial Wage Gap for South African Men,”

Manuscript, University of Stellenbosch.

McCall, B., J. Smith, and C. Wunsch (2016): “Government-sponsored Vocational Education

for Adults,” in Handbook of the Economics of Education, ed. by E. Hanushek, S. Machin, and

L. Woessmann, Elsevier, vol. 5, 479–652.

Menzio, G. and S. Shi (2011): “Eﬃcient Search on the Job and the Business Cycle,” Journal of

Political Economy, 119, 468–510.

Mortensen, D. and C. Pissarides (1999): “New Developments in Models of Search in the

Labor Market,” in Handbook of Labor Economics, ed. by O. Ashenfelter and D. Card, Elsevier,

2567–2627.

Neumark, D. (2018): “Experimental Research on Labor Market Discrimination,” Journal of Eco-

nomic Literature, 56, 799–866.

Pallais, A. (2014): “Ineﬃcient Hiring in Entry-Level Labor Markets,” American Economic Review,

104, 3565–3599.

Poropat, A. E. (2009): “A Meta-Analysis of the Five-Factor Model of Personality and Academic

Performance,” Psychological Bulletin, 135, 322–338.

Poschke, M. (2019): “Wage Employment, Unemployment and Self-employment across Countries,”

Manuscript, McGill University.

Pritchett, L. (2013): The Rebirth of Education: Schooling Ain’t Learning, Washington, DC:

Center for Global Development.

Rankin, N., C. Darroll, and T. Corrigan (2012): “SMEs and Employment in South Africa,”

Small Business Project, Johannesburg.

Rosar, F. and E. Schulte (2012): “Optimal Test Design under Imperfect Private Information

and Voluntary Participation,” Manuscript, University of Bonn.

Rosenberg, M. (1965): Society and the Adolescent Self-Image, Princeton University Press.

Schöer, V., M. Ntuli, N. Rankin, C. Sebastiao, and K. Hunt (2010): “A Blurred Sig-

nal? The Usefulness of National Senior Certiﬁcate (NSC) Mathematics Marks as Predictors of

Academic Performance at University Level,” Perspectives in Education, 28, 9–18.

Schöer, V., N. Rankin, and G. Roberts (2014): “Accessing the First Job in a Slack Labour

Market: Job Matching in South Africa,” Journal of International Development, 26, 1–22.

Statistics South Africa (2016): Quarterly Labor Force Survey Quarter 4 2016, Pretoria: Statis-

tics South Africa.

Taylor, S., S. Van Der Berg, V. Reddy, and D. Janse van Rensburg (2011): “How Well

Do South African Schools Convert Grade 8 Achievement Into Matric Outcomes?” Working Paper,

Stellenbosch University.

Wheeler, L., E. Johnson, R. Garlick, P. Shaw, and M. Gargano (2019): “LinkedIn(to)

Job Opportunities? Experimental Evidence from Job Readiness Training,” Working paper, Duke

University.

Online Appendices

A Assessments

We assess each workseeker’s skills in six domains. Most of the assessments are already used by

Harambee and by some large ﬁrms in South African during hiring. We do not claim that these are

the best possible assessments for predicting workplace performance. But these are assessments that

some market agents have chosen to use, have reasonable psychometric properties, and are correlated

with workplan performance in some settings.

A.1 Firms’ Use of Assessments

Harambee has used the numeracy, communication, and concept formation assessments since 2011

to select candidates for further job readiness training and recommend candidates to vacancies at

partner ﬁrms. Harambee has placed over 160,000 candidates in entry-level jobs using these as-

sessments. Table A.1 shows how 33 large client ﬁrms in retail, hospitality, logistics and corporate

services require Harambee to use assessments when recommending candidates for interviews.

All ﬁrms used at least one assessment to screen candidates and 73% of ﬁrms used all three

assessments. In contrast, only 57% required certiﬁed results on the national high school graduation

exam and only 3% required references. This shows ﬁrms ﬁnd this skill information useful relative

to other sources of information about prospective workers’ skills. Harambee also administers a set

of career aptitude measures provided by a psychometric testing ﬁrm. 67% of ﬁrms in this sample

used this assessment score to screen applicants, suggesting they value horizontal diﬀerentiation. We

could not include this assessment in the certiﬁcation because it is a proprietary instrument.

We therefore selected three alternative measures of skills which would be unlikely to be correlated

with numeracy, communication, and concept formation. To select these, we conducted interviews

with 20 hiring managers to understand which other skills they valued in successful hires. Elsewhere,

we conducted a detailed literature review of measures and selected those most overlapping with

what ﬁrms valued (Esopo et al., 2018), which were also correlated with either earnings or measures

of workplace performance in some settings.

A.2 Description of Assessments

Concept formation is very similar to the Raven’s Progressive Coloured Matrices assessment (Raven

and Raven, 2003). It is a non-verbal measure of ﬂuid intelligence, which captures the rate at

which people learn and their conceptual reasoning. It speciﬁcally assesses the ability to ignore

superﬁcial diﬀerences and see underlying commonalities across situations and to use logic in new

situations. Meta-analyses identify measures of ﬂuid intelligence as strong predictors of worker

productivity (Schmidt and Hunter, 1998; Schmidt et al., 2016). The Raven’s test is widely used

in hiring and selection (Chamorro-Premuzic and Furnham, 2010), including in recent research in

Table A.1: Firms’ Use of Psychometric Assessments in Hiring

% of ﬁrms using each piece of information to screen candidates

Assessment result for Career Criminal High school

Reference

Sector

# Communi- Concept

Numeracy

aptitude record graduation

ﬁrms cation formation proﬁle check certiﬁcate

Hospitality 11 0.82 1.00 0.91 0.64 0.91 0.64 0.00

Retail 16 0.69 0.56 0.88 0.81 0.94 0.75 0.06

Corporate 6 1.00 1.00 0.83 0.33 1.00 0.00 0.00

Total 33 0.79 0.91 0.79 0.67 0.94 0.58 0.03

Table shows use of assessment results and other information by 33 ﬁrms that have long-term recruiting rela-

tionships with Harambee. Firms are coded as using an assessment if they require candidates to reach a certain

threshold score on the assessment to be eligible for interviews or training programs. Firms are coded as using

other documents if they require these to be submitted with the candidates’ application packages. The criminal

record check is a set of checks against government records that the candidate had no criminal record or bad

credit history. We observe only what information these 33 ﬁrms request from Harambee for candidates whom

Harambee shortlists for interview, not how ﬁrms use the information.

economics (Abebe et al., 2020b; Beaman et al., 2018). Scores on this assessment are correlated with

interview ratings, technical scores and supervisor ratings in several South African ﬁrms (De Kock

and Schlechter, 2009; Lopes et al., 2001; Taylor, 2013).

Numeracy focuses on practical arithmetic and pattern recognition. We calculate a single nu-

meracy score using the inverse variance-weighted average of two numeracy assessment scores. The

more advanced assessment is developed by a large retail chain and used in their applicant screening

process, as they believe it identiﬁes some of the skills needed by cashiers. The simpler assessment

was developed by a South African adult education provider (www.mediaworks.co.za) and assesses

proﬁciency in arithmetic used in high school: comparing diﬀerent types of numbers; working with

fractions, ratios, money, percentages and units; and performing calculations with time and area.

Communication captures English language listening, reading and comprehension skills. The as-

sessment was developed by a South African adult education provider (www.mediaworks.co.za) and

is designed to assess English proﬁciency for high school students. It evaluates both listening and

written comprehension. It focuses on ability to identify and recall the main message of a text or

passage, infer meaning of vocabulary through context clues, and infer meaning when information is

not directly stated. Both numeracy and communication skills are correlated with educational attain-

ment and wages in OECD countries (Heckman et al., 2006; Heckman and Kautz, 2012; Hanushek

et al., 2015). There are also correlations between wages and numeracy (du Rand et al., 2011) and

wages and English communication skills (Casale and Posel, 2011) in South Africa, conditional on

education.

Grit is a self-reported measure of a candidate’s inclination to work on diﬃcult tasks until they

are ﬁnished and whether they show perseverance to achieve long-term goals. This assessment is

a validated self-reported 8-item psychological scale (Duckworth et al., 2007). Grit correlates with

academic performance and workplace retention in the US (Eskreis-Winkler et al., 2014).

The assessment labelled Focus on certiﬁcates captures inhibitory control, the ability to distin-

guish relevant from irrelevant information, control one’s attention to focus on what is needed for a

task (Diamond, 2013) and guide thought and action in accordance with a goal (Posner and DiGiro-

lamo, 1998). The assessment is a computerized version of the widely-used Stroop Test, using colors

(Stroop, 1935). Similar measures are correlated with employment status (Kalechstein et al., 2003)

and moderate the negative eﬀects of workplace related stress, such as burnout and absenteeism, in

service sector jobs (Schmidt et al., 2007).

Planning measures how candidates behave when faced with complex, multi-step problems. The

assessment is adapted from the Hit 15 lab task (Gneezy et al., 2010). The computer and the subject

take turns adding either one, two or three points to the points basket. The goal is to be the ﬁrst

player to reach 15 points. It captures ability to search for relevant information and anticipate the

consequences of actions. High planning scores predict retention rates among truckers in the US,

conditional on cognitive skills (Burks et al., 2009). Similar measures of complex planning skills are

correlated with wages in South Africa, controlling for ﬂuid intelligence and education (Ederer et al.,

2015).

For the ﬁrst 17 of the 84 assessment days, covering 26% of candidates, computer problems meant

that we used two self-reported psychological scales, labelled Control and Flexibility on the reports

instead of focus and planning. We used two subscales of the Personal Problem-Solving Inventory

(Hepner and Petersen, 1982). The Personal Control scale (control) captures whether candidates

take a systematic or impulsive and erratic approach when faced with new, challenging problems.

The Approach Avoidance (ﬂexibility) scale captures whether candidates actively consider several

approaches to solving a problem or whether they pursue their ﬁrst idea without thinking about

alternatives. These are not exact analogues of the tasks: they capture self-perceptions as well as

behaviors (Heppner, 1988). But scores are correlated in other samples: for example, the PSI is

correlated with the Stroop task (Rath et al., 2004). None of the main results in the paper are

substantially diﬀerent between the sample using the focus and planning assessments and the sample

using the control and ﬂexibility assessments.

We use the assessment scores in the paper in three ways. First, we use assessment scores as a

prespeciﬁed conditioning variable when estimating treatment eﬀects. We use the concept formation,

communication, grit, and numeracy scores individually for this purpose. We combine the remaining

scores into a single measure by taking the ﬁrst principal component of control and ﬂexibility and

standardizing it, taking the ﬁrst principal component of focus and planning and standardizing it,

and then appending the two principal components together. Second, we use assessment scores in the

heterogeneity analysis described in Section 5.2. We use only the scores observed for all candidates

(concept formation, communication, grit, and numeracy) for this analysis. Results are similar when

we restrict to the 74% of candidates who took the focus and planning assessments and use all six

assessments. Third, we use assessments in the ﬁrm-facing experiments described in Sections 5.1 and

5.2. The online platform reports all eight assessment results and explains that each candidate took

Table A.2: Correlations of Assessment Results

Panel A: Correlations In First 17 Days of Assessment (1,615 workseekers)

Concept formation Grit Numeracy Control Flexibility

(Personal (Approach

Control) Avoidance)

Communication 0.337 0.127 0.386 0.237 0.126

Concept formation 0.108 0.489 0.174 0.098

Grit 0.163 0.507 0.334

Numeracy 0.212 0.107

Control 0.173

Panel B: Correlations In Remaining 67 Days of Assessment (5,276 workseekers)

Concept formation Grit Numeracy Focus Planning

(Stroop) (Hit 15)

Communication 0.346 0.088 0.393 0.171 0.258

Concept formation 0.094 0.519 0.225 0.292

Grit 0.129 0.049 0.106

Numeracy 0.162 0.325

Focus 0.181

Table shows pairwise correlation coeﬃcients between assessment results. The sample is split because two of the

assessments changed after the ﬁrst 17 days of assessment, from the control and ﬂexibility scales to the focus

and planning tasks. None of the pairwise correlations between the four assessments used for the entire period

(communication, concept formation, grit, and numeracy) are substantively or statistically signiﬁcantly diﬀerent

between the two periods.

Table A.3: Distribution of Top, Middle, and Bottom Terciles Shown on Candidates’ Reports

Fraction with _ top terciles

Total

0 1 2 3 4 5 6

Fraction with _

bottom terciles

0 0.001 0.007 0.025 0.032 0.029 0.018 0.007 0.119

1 0.009 0.036 0.059 0.064 0.037 0.011 - 0.215

2 0.027 0.077 0.079 0.040 0.011 - - 0.235

3 0.054 0.076 0.048 0.009 - - - 0.187

4 0.070 0.059 0.009 - - - - 0.138

5 0.060 0.024 - - - - - 0.084

6 0.023 - - - - - - 0.023

Total 0.243 0.279 0.220 0.146 0.076 0.029 0.007

Table shows the share of the sample with i top terciles and j top terciles on their

reports for each i, j ∈ {0, 6}. The number of middle terciles equals 6 − i − j.

only six of the eight assessments. The proﬁle-ranking exercise does not use the control or ﬂexibility

scales.

A.3 Administration of Assessments

All assessments are conducted in English, the same language used for all Harambee interaction with

candidates. All assessments are conducted on desktop computers, so the assessment results may be

sensitive to candidates’ computer skills. To minimize this sensitivity, all candidates do some practice

computer exercises before the assessments and all assessments are designed to be completable within

the available time limit. Before starting assessments, candidates consent to their assessment results

being shared with Harambee, the research team, and external ﬁrms.

Registered industrial psychologists employed or contracted by Harambee oversaw administration

of all assessments. They also delivered brieﬁngs to candidates to interpret results. Finally, the lead

psychologist at Harambee approved the language on certiﬁcates. This ensures compliance with

South African law on psychometric testing in workplace settings.

A.4 Validation of Self-Reported Psychological Scales and Tasks

We use four self-reported psychological scales in the paper: grit, control and ﬂexibility are used as

skills measures, while self-esteem is used as an outcome measure. We followed standard procedures

in psychology to ensure the self-reported scales were well-understood and valid as measures. See

Esopo et al. (2018) for a full discussion of the process followed. We use the same seven-point Likert

scale for all scales.

The Problem-Solving Inventory had already been validated in South Africa with young black

African students of a very similar demographic proﬁle to our sample and we used this item wording

(Pretorius, 1993; Heppner et al., 2002). For grit and self-esteem, we ensured language used was well-

understood by conducting cognitive debrieﬁngs with 20 Harambee candidates. Cognitive debrieﬁng

captures the underlying cognitive processes that respondents use to answer questions to detect

and solve problems in questionnaires (Tourangeau, 2003; Willis, 2008, 1999). For example, the

interviewer asks for speciﬁc information relevant to the question or the answer given. Examples of

probes used are “What does the term mean to you?”, “Can you repeat this question to me in your

own words?” and “What made you answer the way that you did?” We simpliﬁed the wording of

some items and altered some culturally speciﬁc idioms in response to the cognitive debrieﬁngs.

Second, we estimated the extent to which diﬀerent items in each scale move together, using

Cronbach’s alpha (Cronbach, 1951). All assessments have α > 0.65. Third, we administered the

scales twice for 150 candidates, ten days apart. We estimated Lin’s Concordance Correlation Coef-

ﬁcient (Lawrence and Lin, 1989) between the two administrations. All assessments have ρ

> 0.62.

Fourth, we check if any items on the scales have very low variation across candidates using maxi-

mum endorsement frequencies. No items meet the threshold for being dropped due to insuﬃcient

variation from Bowling (2014).

The terciles shown on the assessment results are based on assessment results from candidates

assessed before the study started: 5,000 workseekers for communication, numeracy and concept

formation test, and 500 workseekers for the other skills. Tercile assignments are largely unchanged

if we retrospectively construct them using our full sample of assessed workseekekers.

Table A.2 shows the correlation of assessment results for the diﬀerent skills. Numeracy, concept

formation and communication have pairwise correlations of 0.34 to 0.52. Numeracy and commu-

nication assessments capture acquired knowledge, often from schooling, which is often positively

correlated with ﬂuid intelligence. This is potentially because learning at a higher rate improves ac-

quisition of knowledge (Heckman and Kautz, 2012; Nisbett, 2009; Roberts et al., 2000). However, as

we intended, these are less strong correlations between the other tasks (focus and planning) and the

scales (grit, ﬂexibility, and planning). These suggest the certiﬁcates will horizontally diﬀerentiate

workseekers from one another.

B Implementation Costs

This appendix reports the costs of the public certiﬁcation intervention and compares these to gains

experienced by treated workseekers, showing that the latter easily exceed the former. We measure

costs from the Harambee and J-PAL Africa ﬁnancial statements. All costs are reported in 2016/7

PPP USD terms and are averaged over the 2,247 candidates who received the public certiﬁcation

intervention. The cost ﬁgures in nominal USD are 42% of the cost ﬁgures in PPP USD, though this

does not aﬀect the cost-beneﬁt comparisons. We report average variable costs and, where these are

possible, total and average ﬁxed costs. The average variable costs may change with scale but we do

not attempt to project scale eﬀects on costs.

The average variable cost of adding certiﬁcation to Harambee’s existing assessment operation

was USD 23.10. This included report printing, software license fees, website hosting fees, the time

of J-PAL and Harambee staﬀ used to prepare the reports, and the time of Harambee psychologists

used to conduct brieﬁngs. This also included a USD 10.32 transport subsidy to each participant

to cover the cost of travel to the Harambee oﬃce, which is arguably not a necessary cost of the

intervention. These cost calculations exclude the private and placebo certiﬁcations, audit study,

and ﬁrm-facing experiments.

The average variable cost of certiﬁcation and assessment was USD 57.27 per participation. This

included all certiﬁcation-only costs, facility rental, computer rental, data and internet costs, and

the time of Harambee staﬀ who administered the assessments. Facility and computer rental costs

were the largest line items for the assessment cost, jointly accounting for USD 23.43.

The average variable costs exclude ﬁxed costs such as licenses for the assessment tools, market re-

search into ﬁrm preferences over assessments, and senior management fees. For these costs we either

cannot calculate a meaningful average ﬁxed cost or cannot reliably separate Harambee’s total ﬁxed

costs for developing the assessment program from its costs of other activities. J-PAL Africa’s ﬁxed

cost for developing the certiﬁcation program on top of the assessment program was approximately

USD 17,685 or USD 7.87 per candidate who received the public certiﬁcation intervention. This

covered J-PAL Africa staﬀ costs during development and all costs of piloting the certiﬁcates with

ﬁrms and workseekers. This includes the cost of developing and piloting the private and placebo

certiﬁcations, which we cannot easily separate from the public certiﬁcation, but excludes the costs

of developing and piloting the audit study and ﬁrm-facing experiments.

We compare these average costs to the average beneﬁt per participant who received the public

certiﬁcation intervention over the ﬁrst three months after the intervention. Public certiﬁcation

increases average earnings by USD 9.05 in the week before the endline survey and the endline

survey occurred on average 14.4 weeks after treatment. Multiplying these together gives an average

eﬀect on earnings since treatment of USD 130.2: 5.6 times higher than the average variable cost of

certiﬁcation, 2.3 times higher than the average variable cost of assessment and certiﬁcation, and 2.0

times higher than the average variable cost of assessment and average variable and ﬁxed costs of

certiﬁcation. The gains to treated workseekers over just three and a half months easily exceed the

cost of public certiﬁcation and assessment.

The preceding calculation assumes that the treatment eﬀect on weekly earnings does not vary

through time from treatment to the endline. The public certiﬁcation eﬀect on earnings does not

substantially vary with the time period from treatment to endline. But the treatment eﬀects on

recalled employment in the ﬁrst and second months after treatment are not identical, suggesting

a possible time trend (Table D.10). To account for this, we convert the weekly earnings eﬀect

into monthly terms and multiply this by the sum of the employment eﬀect in the ﬁrst month after

treatment, the second month after treatment, and the week before the endline. This gives an average

on earnings since treatment of USD 110.1, which also easily exceeds the cost of public certiﬁcation

and assessment.

C Labor Market Eﬀects at the Extensive and Intensive Margins

Treatment eﬀects on labor market outcomes such as earnings and hours can occur at the extensive

margin – due to treatment eﬀects on employment – and at the intensive margin – due to treatment

eﬀects on job characteristics conditional on employment. This distinction is important, as intensive

margin eﬀects indicate that treatment is changing the type of jobs candidates secure. The intensive

margin eﬀects are not identiﬁed from regressions of labor market outcomes on treatment indicators

for employed candidates, as the set of employed candidates may be selected based on treatment

assignment.

We adapt a method from Attanasio et al. (2011) to decompose of labor market eﬀects into

extensive and intensive margins. We describe the decomposition here for earnings, but the same

idea applies to any labor market outcome that is observed only for the employed. We use the term

“treatment” to refer to the public certiﬁcation. Using the law of iterated expectations and the fact

that observed earnings are zero for non-employed candidates, we can write the average treatment

eﬀect on earnings as:

E[Earn|T reat = 1] − E[Earn|T reat = 0]

| {z }

ATE for earnings

(3)

= (E[Earn|T reat = 1, W ork = 1] − E[Earn|T reat = 0, W ork = 1])

| {z }

ATE for earnings | employment

· P r[W ork = 1|T reat = 1]

| {z }

Treated employment rate

+ E[Earn|T reat = 0, W ork = 1]

| {z }

Control earnings | employment

· (P r[W ork = 1|T reat = 1] − P r[W ork = 1|T reat = 0])

| {z }

ATE for employment

We deﬁne the second line on the right-hand of the regression as the extensive margin eﬀect. Intu-

itively, this is the average treatment eﬀect on employment ‘priced’ at the mean earnings value in

the control group. If treatment has no eﬀect on the employment rate, then this expression is zero.

We deﬁne the ﬁrst line on the right-hand side of the regression as the intensive margin eﬀect. If

treatment only changes the employment rate but has no eﬀect on earnings for employed candidates,

then this term is zero.

All terms in equation (3) except the average treatment eﬀect on earnings conditional on employ-

ment are identiﬁed by the experiment and can be consistently estimated using sample analogues.

Hence, we can consistently estimate the remaining term using the formula in (3). We obtain stan-

dard errors by estimating all quantities as a system and using the Delta method.

This decomposition applies to realized earnings, which are zero by deﬁnition for non-employed

candidates. This decomposition does not apply to latent earnings, which may be non-zero for

non-employed candidates. Alternative methods are available for studying latent earnings. One

set of approaches point identiﬁes the average treatment eﬀect on latent earnings by modeling the

selection process into employment and adjusting observed earnings for selection (e.g. Gronau, 1974

and Heckman, 1974). Another set of approaches bounds the average treatment eﬀect on latent

earnings by assuming that the earnings for the non-employed fall in some region of the observed

earnings distribution (e.g. Lee, 2009 and Manski, 1989). Neither approach is ideal in our setting:

the former methods require an instrument for selection into employment that we do not have and the

latter methods will yield wide bounds given the large eﬀect of public certiﬁcation on employment.

Another set of approaches point identiﬁes quantile treatment eﬀects on latent earnings by assuming

that the earnings for the non-employed fall in some region of the observed earnings distribution

(e.g. Powell, 1984). Our analysis of quantile treatment eﬀects has a similar ﬂavor to this approach,

Attanasio et al. (2011) show that the intensive margin eﬀect can be further decomposed into two terms: the

treatment eﬀect on earnings conditional on candidates’ baseline characteristics, and the diﬀerence in baseline charac-

teristics between employed candidates in the treatment and control groups. However, neither of these terms is point

identiﬁed. Separating these eﬀects is not important in our application. Our conceptual framework is consistent with

certiﬁcation either increasing the same workseekers’ latent treated wages conditional on employment, or increasing

mean wages conditional on employment by helping workseekers with higher latent treated wages get employed.

Figure C.1: Density of Earnings in Control and Public Certiﬁcation Groups

Control

Treated

0 .1 .2 .3

Density (scaled by relative employment rate)

0 5 10

Earnings (inverse hyperbolic sine)

This ﬁgure shows the densities of earnings in the control and public certiﬁcation groups. To account for

the positive treatment eﬀect on employment, the treatment density is scaled by the ratio of employment

in the treatment group to employment in the control group. Hence the vertical diﬀerence between the

densities at each earnings level E represents the treatment eﬀect on the share of all candidates earning

E, not on the share of employed candidates earning E. The density is estimated only for the employed,

so candidates with zero earnings are excluded.

though we do not directly interpret these as eﬀects on latent earnings.

As discussed in Section 3.3, this decomposition shows that the earnings and wage eﬀects of

public certiﬁcation occur at both the extensive and intensive margins. The hours and contract type

eﬀects occur only at the extensive margin.

The intensive-margin eﬀect on earnings is also visible in the distributions and densities of earn-

ings for the public certiﬁcation and control groups. Figure 2 (in the main text) shows the distribu-

tions of earnings for each group and the quantile treatment eﬀects of public certiﬁcation. Figure C.1

shows the densities of earnings for employed candidates in the control and treatment groups. We

rescale the latter density by the ratio of treatment group to control group employment. Hence, the

vertical diﬀerence between the densities at each earnings level E represents the treatment eﬀect on

the share of all candidates earning E, not on the share of employed candidates earning E. The treat-

ment eﬀect on the earnings density is almost entirely above median earnings for employed control

group candidates. This shows that either the marginal candidates employed only when treated earn

more than most inframarginal control candidates, or treatment increases earnings for inframarginal

candidates, or both.

D Additional Results about Workseeker Experiments

D.1 Summary Statistics and Balance Tests

This section reports summary statistics for the baseline workseeker sample (Table D.1) and endline

workseeker sample (Table D.2). Balance tests for equal means of baseline measures are also re-

ported in the ﬁnal column of Table D.1. Table D.3 compares our workseeker sample to the broader

population of the country and of Gauteng province, where the study took place.

Table D.1: Summary Statistics for Baseline Variables

Variable # obs Mean Std 10

p:balance

dev. pctile pctile

Panel A: Demographic Measures

Age 6891 23.6 3.3 19.8 28.3 0.583

Male 6891 0.382 0.486 0.267

University degree / diploma 6891 0.167 0.373 0.889

Any other post-secondary qualiﬁcation 6891 0.212 0.409 0.642

Completed secondary education only 6891 0.610 0.488 0.794

Panel B: Assessment Results

Numeracy score 6891 0.052 0.988 -1.187 1.411 0.523

Communication score 6891 0.050 0.992 -1.093 1.694 0.206

Concept formation score 6891 0.047 0.991 -1.516 1.260 0.764

Grit score 6891 0.031 0.992 -1.313 1.279 0.089

Other scores 6891 -0.002 1.070 -1.305 1.318 0.859

Panel C: Labor Market Measures

Employed 6891 0.378 0.485 0.468

Earnings 2116 565 740 100 1400 0.083

Ever worked 6877 0.704 0.457 0.418

Ever held a long-term job 6877 0.090 0.286 0.696

Panel D: Job Search Measures

Searched 6891 0.968 0.175 0.058

Applications submitted

6815 9.9 18.6 2.0 20.0 0.809

Search cost 6147 242 1520 30 400 0.276

Search hours 6699 17.0 20.8 2.0 48.0 0.231

Oﬀers received

6810 1.20 7.20 0.00 2.00 0.280

Panel E: Belief Measures

Planned applications

6840 48.9 1629.9 4.0 36.0 0.252

Correct about all assessment results 6891 0.082 0.274 0.960

Incorrect about all assessment results 6891 0.290 0.454 0.961

Overconﬁdent about all assessment results 6891 0.219 0.413 0.732

Underconﬁdent about all assessment results 6891 0.010 0.100 0.783

Table shows summary statistics for selected baseline variables. Percentiles are omitted for binary variables. All

monetary ﬁgures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms.

In this table, intensive-margin labor market measures (e.g. earnings) are set to missing for non-workers, and

intensive-margin search measures (e.g. search cost) are set to missing for non-searchers. All assessment results

are standardized to have mean zero and standard deviation one in the control group. Missing values reﬂect

item non-response, mostly due to respondents reporting that they don’t know the answer. All period-speciﬁc

outcomes use a 7-day recall/forecast period unless marked with

(30-day recall/forecast period). The ﬁnal

column reports the p-value for testing equality of means of the baseline variables across all treatment groups,

using heteroskedasticity-robust standard errors clustered by treatment date.

Table D.2: Summary Statistics for Endline Variables

Variable # obs Mean Std dev. 10

pctile 90

pctile

Panel A: Labor Market Measures

Employed 6607 0.323 0.468

Earnings 2112 623 1183 2 1500

Hours worked 2121 28.5 21.6 4.0 56.0

Hourly wage 2097 33.1 72.3 0.1 77.8

Wage employment 2102 0.885 0.319

Self employment 2102 0.114 0.318

Panel B: Job Search Measures

Any search 6608 0.692 0.462

Applications submitted

6577 12.8 21.5 1.0 27.0

Hours searched 6601 9.9 14.2 0.0 25.0

Search cost 6599 116 167 0 300

Responses

6593 0.861 2.147 0.000 2.000

Oﬀers

6592 0.207 0.680 0.000 1.000

Panel C: Belief Measures

Fraction of assessments overconﬁdent 6607 0.345 0.237

Fraction of assessments underconﬁdent 6607 0.176 0.166

Targeted search 6891 0.175 0.380

Planned applications

6591 16.1 29.7 3.0 30.0

Expected oﬀers

6531 4.49 5.70 1.00 10.00

Table shows summary statistics for selected endline variables. Percentiles are omitted for binary variables. All

monetary ﬁgures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms.

Intensive-margin labor market measures (e.g. earnings) are set to missing for non-workers. Intensive-margin

search measures (e.g. search cost) are set to zero for non-searchers. Missing values reﬂect item non-response,

mostly due to respondents reporting that they don’t know the answer. All period-speciﬁc outcomes use a 7-day

recall/forecast period unless marked with

(30-day recall/forecast period).

Table D.3: Comparison between the Workseeker Sample and External Populations

(1) (2) (3) (4)

South Africa

Gauteng Gauteng Study sample

province age 18-29 of workseekers

Age 28.914 31.461 23.776 23.646

(19.606) (19.099) (3.326) (3.299)

Male 0.489 0.504 0.512 0.382

Currently Employed 0.290 0.375 0.346 0.378

Currently Searching 0.102 0.151 0.302 0.984

< complete secondary school 0.737 0.612 0.429 0.011

Complete secondary school 0.184 0.255 0.449 0.610

> complete secondary school 0.072 0.120 0.116 0.379

Table compares the sample of workseekers in this study (column 4) to several external benchmarks: the country

(column 1), the province of Gauteng where the study takes (column 2), and people in Gauteng in the eligible

age range for the study (column 3). National and provincial statistics are calculated from the Quarterly Labour

Force Surveys (QLFS), averaging over all 2016 waves and using post-stratiﬁcation weights supplied by Statistics

South Africa. QLFS data are not available by city but the greater Johannesburg metropolitan area where the

study is conducted accounts for over half the population of the Gauteng province. Standard deviations are shown

in parentheses for all continuous variables.

D.2 Benchmarking The Magnitude of The Earnings Eﬀects

In this section we show that the earnings eﬀects are substantial relative to two local benchmarks.

Minimum wage: During our study period, minimum wages in South Africa varied by sector

and location. Sector- and location-speciﬁc minimum wages were either set by the Ministry of Labour

or in bargaining councils, where large ﬁrms and unions agreed minimum wages that applied to all

ﬁrms (Budlender et al., 2015; Isaacs, 2016). Table D.4 shows minimum wages for urban areas at

the time of the study for several sectors relevant to workseekers in our sample.

Poverty Lines: South African poverty research often uses poverty lines based on the cost of

purchasing 2100 calories plus the average amount spent on non-food items by households whose

food expenditure equals the food poverty line (Budlender et al., 2015; Leibbrandt et al., 2012).

Using this deﬁnition, the adult monthly poverty line just before the study period was 1,386 South

African Rands or USD 232 in purchasing power parity terms (Isaacs, 2016, p.22).

The average treatment eﬀect on earnings is equal to 17% of the adult monthly poverty line or

7-9% of the monthly minimum wage at the time of the study.

Table D.4: Benchmarking Earnings Figures to Minimum Wage and Poverty Lines

Panel A: South African poverty lines and minimum wages at baseline

Monthly Weekly aaa

Date ZAR USD ZAR USD

Poverty line

Adult Early 2016 1386 232 320 54

Household (4 people) Early 2016 5544 927 1279 214

Minimum wage

Domestic work 2015-2016 2550 427 588 98

Hospitality 2015-2016 2750 460 634 106

Wholesale and retail 2015-2016 3250 544 750 125

Private security/contract cleaning 2015-2016 3500 585 808 135

Panel B: Benchmarking sample earnings and certiﬁcation treatment eﬀects on earnings

Weekly As % of poverty line As % of min. wage

Date ZAR USD Adult Household Hospitality Retail

Baseline mean earnings if employed Late 2016 562 94.1 1.76 0.44 0.89 0.75

Endline mean earnings Early 2017 159 26.6 0.50 0.12 0.25 0.21

Endline mean earnings if employed Early 2017 518 86.7 1.62 0.41 0.82 0.69

Treatment eﬀect Early 2017 54.1 9.05 0.17 0.04 0.09 0.07

Calculations assume 1 Rand ≈ 0.167 USD in purchasing power parity terms; 4.33 weeks per month. Household

poverty lines assume households of four people with only one earner. Control group respondents work 29 hours

per week conditional on being employed; earnings for those in full time work will be higher than mean earnings

here. Poverty lines are from Isaacs (2016, p.22) and minimum wages are from the Department of Labor for

2015. Minimum wages are for large urban areas (Area A). They are for hospitality businesses with less than 10

employees and shop assistants in the wholesale and retail sector.

D.3 Non-response

The phone survey after 3-4 months is our main source of endline data. We use a text message survey

after 2-3 days only to measure beliefs about numeracy and self-esteem. The response rates for the

text message and phone surveys are respectively 83 and 96%. Non-response does not diﬀer by

treatment arm (Table D.5). Non-response does not diﬀer over most baseline characteristics (Table

D.6). Men are less likely to respond in both surveys. Higher numeracy and concept formation scores

predict higher response rates in the text message survey. Higher grit predicts lower response rates

in the endline survey.

Table D.5: Non-response by Treatment Group in Each Post-Treatment Survey Round

(1) (2)

Text Message Survey Endline Phone Survey

Control 0.170 0.040

(0.013) (0.006)

Public 0.177 0.039

(0.011) (0.004)

Private 0.182 0.044

(0.010) (0.004)

Placebo 0.142 0.047

(0.032) (0.026)

p: Control = Pvt. 0.481 0.632

p: Control = Pub. 0.670 0.855

p: Pvt. = Pub. 0.785 0.388

p: Control = Pvt. = Pub. 0.778 0.681

p: Control = Plc. 0.414 0.787

p: Pvt. = Plc. 0.238 0.888

p: Pub. = Plc. 0.297 0.746

p: Control = Pvt. = Pub. = Plc. 0.641 0.841

# observations 6891 6891

# clusters 84 84

Coeﬃcients show the fraction of each treatment group that does not complete each follow-up survey round.

Heteroskedasticity-robust standard errors clustered by treatment date are shown in parentheses.

Table D.6: Non-response by Baseline Covariates Group in Each Post-Treatment Survey Round

(1) (2)

Text Message Survey Endline Phone Survey

Completed at most high school -0.008 -0.003

(0.012) (0.005)

Numeracy score -0.029 0.003

(0.006) (0.003)

Communication score 0.008 0.003

(0.006) (0.003)

Concept formation score -0.019 0.002

(0.006) (0.003)

Grit score -0.001 -0.007

(0.005) (0.003)

Other scores 0.001 -0.002

(0.004) (0.003)

Perceived numeracy score -0.000 -0.000

(0.000) (0.000)

Perceived literacy score 0.014 -0.003

(0.010) (0.005)

Perceived concept formation score 0.010 -0.003

(0.009) (0.004)

Self-esteem index 0.006 0.002

(0.004) (0.002)

Age -0.002 0.001

(0.001) (0.001)

Male 0.049 0.014

(0.010) (0.005)

Employed -0.005 -0.001

(0.008) (0.005)

Above median discount factor 0.012 0.007

(0.009) (0.005)

Respondent is present-biased 0.016 0.007

(0.011) (0.006)

Above median risk aversion -0.007 0.001

(0.008) (0.005)

p: All coeﬃcients jointly zero 0.000 0.041

Mean outcome 0.170 0.040

# observations 6891 6891

# clusters 84 84

Coeﬃcients are from regressions of round-speciﬁc attrition on the list of baseline covariates displayed here.

All assessment scores are standardized to have mean zero and standard deviation one in the control group.

Heteroskedasticity-robust standard errors clustered by treatment date are shown in parentheses.

D.4 Additional Treatment Eﬀects

Table D.7 shows the public certiﬁcation eﬀects of our main labor market outcomes without condi-

tioning on the prespeciﬁed covariates. The results are very similar with or without the covariates.

Table D.8 shows public and private certiﬁcation eﬀects at two points in time: in the text

message survey conducted 2-3 days after treatment and the endline phone survey conducted 3-4

months after treatment. This table shows four patterns, which expand on the discussion in footnote

19 of the paper. First, both treatments make candidates more likely to report that their assessment

result matches their actual assessment result immediately after treatment. Second, both treatment

eﬀects decline over the following 3-4 months, although the diﬀerent survey methods mean the time

comparison should be interpreted cautiously. Third, the public treatment eﬀect on self-beliefs is

signiﬁcantly larger than the private eﬀect after 3-4 months but not after 2-3 days. This suggests

that the larger public treatment eﬀect at 3-4 months does not occur because the information it

conveys is immediately more credible or easier to understand than the private treatment. Instead,

it may be larger because the information is more memorable or the public treatment generates other

eﬀects, such as more job interviews or employment that provide more opportunities to learn about

skills. Fourth, neither treatment aﬀects self-esteem at either point in time.

Table D.9 shows how treatment eﬀects on employment vary by single index summary measures

of candidates’ skills (Panel A) and baseline candidate characteristics that might provide alternative

measures of candidates’ skills (Panel B). We discuss these treatment eﬀects in Sections 5.2 and 5.3

of the paper.

Table D.10 reports public and private certiﬁcation eﬀects on all prespeciﬁed workseeker-level

job search and labor market outcomes. These are organized into families of conceptually similar

outcomes, which we use for multiple testing adjustments. First, we report q-values that control

the false discovery rate across outcomes within each family (Benjamini et al., 2006). None of

the q-values in this table are substantively diﬀerent to the corresponding p-values reported in the

main paper. Second, we estimate treatment eﬀects on inverse covariance-weighted averages of

the outcomes within each family (Anderson, 2008). This provides a single summary test of the

information contained across all outcomes in the same family. None of the the treatment eﬀects

on these averages provide substantively diﬀerent information to the treatment eﬀects on individual

outcomes.

We omit some prespeciﬁed outcomes related to beliefs from this paper and analyze them in

separate work. The search targeting measure discussed in Section 4 is not prespeciﬁed. We did not

prespecify an analysis plan for the smaller extension experiments discussed in Section 5.

Table D.7: Treatment Eﬀects on Labor Market Outcomes Without Covariates

(1) (2) (3) (4) (5)

Employed Hours

Earnings

Hourly wage

Written contract

Treatment 0.046 0.175 0.336 0.206 0.018

(0.013) (0.058) (0.076) (0.041) (0.010)

Mean outcome 0.309 8.848 159.291 9.840 0.120

Mean outcome for employed 28.847 518.291 32.283 0.392

# observations 6607 6598 6589 6574 6575

# clusters 84 84 84 84 84

Coeﬃcients are from regressing each outcome on a vector of treatment assignments and randomization block ﬁxed

eﬀects without any other covariates. Heteroskedasticity-robust standard errors shown in parentheses, clustering

by treatment date. Mean outcome is for the control group. All outcomes use a 7-day recall period. Outcomes

marked with

use the inverse hyperbolic sine transformation. The sample sizes diﬀer across columns due to item

non-response, mostly from respondents reporting that they don’t know the answer.

Table D.8: Treatment Eﬀects on Self-Beliefs through Time

(1) (2) (3) (4)

Perceived numeracy tercile correct Above-median self-esteem

Public 0.233 0.315 0.001 -0.001

(0.013) (0.015) (0.013) (0.015)

Private 0.200 0.333 -0.002 0.016

(0.015) (0.016) (0.014) (0.015)

p: public = private 0.010 0.240 0.806 0.239

Mean outcome 0.396 0.399 0.553 0.479

# observations 6601 5297 6609 5027

# clusters 84 84 84 84

Coeﬃcients are from regressing each outcome on a vector of treatment assignments, randomization block ﬁxed

eﬀects, and prespeciﬁed baseline covariates (measured skills, self-reported skills, education, age, gender, employ-

ment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering

by treatment date. Mean outcome is for the control group. Above-median self-esteem is an indicator equal to

one if the candidate’s response on a shortened version of the Rosenberg (1965) self-esteem scale is above the

sample median. Columns (1) and (3) report results from the main phone follow-up survey. Columns (2) and (4)

report results from the text message survey conducted 2-3 days after treatment. The sample sizes diﬀer across

columns due to item non-response, mostly from respondents reporting that they don’t know the answer.

Table D.9: Heterogeneous Treatment Eﬀects on Employment

(1) (2) (3)

Panel A: Heterogeneous Eﬀects by Single Index Skill Measures

Public treatment 0.052 0.052 0.054

(0.011) (0.011) (0.012)

× Share top - share bottom terciles 0.019

(0.028)

× PC

(Scores) 0.004

(0.025)

× Earnings-weighted average of scores -0.007

(0.029)

# observations 6607 6607 6603

# clusters 84 84 84

Panel B: Heterogeneous Eﬀects by Alternative Information Sources

Public treatment 0.052 0.052 0.051

(0.011) (0.012) (0.012)

× post-secondary education -0.027

(0.028)

× employed at baseline -0.043

(0.032)

Pr(Employed at endline | X) -0.069

(0.028)

# observations 6607 6607 6607

# clusters 84 84 84

Coeﬃcients are from regressing each outcome on a vector of treatment assignments, displayed interaction terms,

randomization block ﬁxed eﬀects, and prespeciﬁed baseline covariates (measured skills, self-reported skills, edu-

cation, age, gender, employment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown

in parentheses, clustering by treatment date. The measures used for interactions in Panel A and column 3 of

Panel B are indicators for above-median values of the underlying indices. All measures in panels A and B are

demeaned before being interacted with treatment, so the coeﬃcient on the treatment indicator equals the average

treatment eﬀect.

Pr(employed at endline | X) is estimated by regressing endline control group employment sta-

tus on the baseline covariates listed above and predicting employment for all candidates. Prediction for control

group candidates uses leave-one-out-estimation to avoid overﬁtting. PC

(Scores) is the ﬁrst principal component

of the skills. The earnings-weighted average of scores is the weighted average of the assessment results, with

weights derived from a regression of control group earnings on assessment results.

Table D.10: Treatment Eﬀects on Prespeciﬁed Outcomes with Multiple Testing Adjustments

(1) (2) (3) (4) (5)

Index Any search Applications

a,c

Search hours

Search cost

Public -0.012 -0.020 0.018 -0.035 -0.092

(0.032) (0.014) (0.042) (0.048) (0.081)

Private 0.006 -0.006 0.036 -0.035 -0.031

(0.032) (0.014) (0.038) (0.049) (0.088)

q: Public eﬀect = 0 1.000 1.000 1.000 1.000

q: Private eﬀect = 0 1.000 1.000 1.000 1.000

q: Public = private eﬀect 1.000 1.000 1.000 1.000

Mean outcome 0.001 0.695 12.356 9.791 112.684

# observations 6608 6608 6577 6601 6599

(1) (2) (3) (4) (5)

Index Responses

a,c

Oﬀers

a,c

Responses per Oﬀers per

application

Public 0.016 0.023 0.006 0.000 -0.000

(0.029) (0.024) (0.013) (0.004) (0.003)

Private 0.019 0.016 0.013 -0.005 0.001

(0.026) (0.022) (0.013) (0.004) (0.004)

q: Public eﬀect = 0 1.000 1.000 1.000 1.000

q: Private eﬀect = 0 1.000 1.000 1.000 1.000

q: Public = private eﬀect 1.000 1.000 1.000 1.000

Mean outcome -0.023 0.871 0.195 0.099 0.030

# observations 6593 6593 6592 5944 5943

(1) (2) (3) (4) (5)

Index

Used report

Applications Interviews Oﬀers

with report

b,c

with report

b,c

with report

b,c

Public NA 0.699 1.682 0.432 0.112

(0.013) (0.040) (0.023) (0.011)

Private NA 0.289 0.572 0.144 0.036

(0.012) (0.033) (0.017) (0.008)

q: Public eﬀect = 0 0.001 0.001 0.001 0.001

q: Private eﬀect = 0 0.001 0.001 0.001 0.001

q: Public = private eﬀect 0.001 0.001 0.001 0.001

Mean outcome 0.000 0.000 0.000 0.000

# observations 6609 6598 6597 6597

(1) (2) (3) (4) (5)

Index

Employed Employed Employed

Hours

in last week in month 1 in month 2

Public 0.137 0.052 0.036 0.058 0.201

(0.025) (0.012) (0.011) (0.014) (0.052)

Private 0.049 0.011 0.028 0.009 0.066

(0.028) (0.012) (0.013) (0.015) (0.048)

q: Public eﬀect = 0 0.001 0.001 0.001 0.001

q: Private eﬀect = 0 0.504 0.142 0.504 0.336

q: Public = private eﬀect 0.003 0.132 0.002 0.008

Mean outcome 0.001 0.309 0.465 0.437 8.848

# observations 6609 6607 6604 6607 6598

(1) (2) (3) (4)

Index Earnings

Hourly Written

wage

contract

Public 0.106 0.338 0.197 0.020

(0.028) (0.074) (0.040) (0.010)

Private 0.069 0.162 0.095 0.017

(0.030) (0.078) (0.046) (0.009)

q: Public eﬀect = 0 0.001 0.001 0.019

q: Private eﬀect = 0 0.066 0.066 0.066

q: Public = private eﬀect 0.047 0.047 0.345

Mean outcome 0.006 159.291 9.840 0.120

# observations 6609 6589 6574 6575

Coeﬃcients are from regressing each outcome on a vector of treatment assignments and randomization block ﬁxed eﬀects.

Heteroskedasticity-robust standard errors shown in parentheses, clustering by the 84 treatment dates. Sharpened q-values

control the false discovery rate across outcomes in each panel, following Benjamini et al. (2006). The ﬁrst column of each

panel shows inverse covariance-weighted averages of outcomes in each panel, following Anderson (2008). The index is

omitted for the report use variables because these are zero for all control group candidates, so the covariance cannot be

estimated. Mean outcome is for the control group. All outcomes use a 7-day recall period unless marked with

(30-day

recall period) or

(since treatment). Outcomes marked with

use the inverse hyperbolic sine transformation. The sample

sizes diﬀer across columns due to item non-response, mostly from respondents reporting that they don’t know the answer.

E Audit Study

We conduct an audit study to identify the eﬀect of information provision on ﬁrm decisions, without

any scope for mediating behavior by workseekers. We submit real workseekers’ applications to

entry-level job vacancies and randomly vary the information ﬁrms see about workseekers’ skills.

This appendix reports more information about the process and sample to help interpret the results

reported in Section 4.3.

We implement the audit study in nine sequential rounds. In each round, we invite candidates

by text message to submit application materials to us, within 7 days, for an undisclosed job oppor-

tunity.

We do not explicitly indicate our aﬃliations or link the message to Harambee. We send

one reminder text message to all candidates 1-3 days after the initial invitation.

We invite 2,220 candidates to send CVs over the nine rounds. We randomly sample candidates

from those who had already completed the workseeker survey. 717 candidates (28%) submit CVs

within the one week period. Most CVs include some information about proxies for candidates’ skills:

91% include a reference letter or contact information for referees and 55% include their secondary

school graduation results (Table E.1, panel A). The 717 responders are similar to the full workseeker

sample on all baseline covariates except gender, where deliberately oversampled men for an even

gender split. Candidates in the private treatment group are slightly more likely to respond to the

invitation (Panel B). All treatment eﬀects are robust to reweighting the responders to have the

same distribution of treatment assignments and baseline covariates as the full workseeker sample.

For each application received, we record information on when the application was received,

where it was sent from, what documents are included, and an indicator for scan quality of in-

cluded documents (e.g. photographs versus high-quality scans). We also send the candidate an

acknowledgement of receipt.

Simultaneously, we compile job vacancies from several online job posting sites. We selected only

vacancies suitable for entry-level workers, so that all candidates in our sample are eligible to apply.

We exclude jobs that look suspicious or are discriminatory, for example: jobs that ask for payments

of any kind to apply, promise unrealistic salaries or beneﬁts, or discriminate based on appearance,

race, or gender. This generates a sample of 1,068 vacancies over the nine rounds, though we exclude

70 vacancies for reasons discussed below. 48% of the vacancies are for sales jobs, with the remaining

vacancies spread over clerical, call center, factory, restaurant and retail jobs.

We submit 4 applications to each vacancy, each “from” a diﬀerent candidate using a diﬀerent

email address. We do not represent ourselves as the candidate. Instead, we use a generic email

address designed to look like the application was scanned at a copy/printing shop, a generic subject

We send each individual a text message: “Dear <name>, we have identiﬁed a job opportunity for you. We are a

group of researchers trying to help young people ﬁnd jobs. If you are interested, email your CV to <email address>

or fax your CV to <fax number>. Find more info at <website>. Please send your CV within 7 days.” A CV in

South Africa is generally understood to include all materials relevant to job applications.

Table E.1: Comparison Between Audit and Workseeker Study Samples

Workseekers in audit sample All workseekers

Mean Std Dev. Obs Mean Std Dev. Obs

Panel A: Characteristics of applications received from workseekers

Includes references or a reference letter 0.91 0.29 713 - - -

Includes a copy of ID document 0.47 0.50 714 - - -

Includes information about secondary school completion 0.55 0.50 714 - - -

Panel B: Characteristics of workseekers

Public treatment 0.30 0.46 717 0.33 0.47 6891

Private treatment 0.37 0.48 717 0.31 0.46 6891

Age 23.2 3.12 717 23.6 3.30 6891

Male 0.48 0.50 717 0.38 0.49 6891

University degree / diploma 0.18 0.38 717 0.17 0.37 6891

Any other post-secondary qualiﬁcation 0.24 0.42 717 0.21 0.41 6891

Completed secondary education only 0.58 0.49 717 0.61 0.49 6891

Numeracy assessment score (z-score) 0.06 0.96 717 0.05 0.99 6891

Literacy/communications assessment score (z-score) 0.02 0.94 717 0.05 0.99 6891

Concept formation assessment score (z-score) 0.11 0.93 717 0.05 0.99 6891

Grit assessment score (z-score) 0.10 0.99 717 0.03 0.99 6891

Worked in the last 7 days (endline) 0.40 0.49 717 0.38 0.48 6891

line, and generic email message.

We send most applications within 2 weeks of compiling the

vacancy list.

We use a three-stage randomization process. First, we generate multiple applications per can-

didate and randomly assign half of these to treatment status and half to control status. Treatment

applications are sent with a public certiﬁcate and control applications without any certiﬁcate. In

all other respects, treatment and control applications are identical. This randomization is inde-

pendent of workseekers’ treatment status in the workseekers’ study. This generates within- and

between-candidate variation in the information content of their applications. Second, we randomize

vacancies to receive either one or three applications with reports. This generates within-vacancy

variation in the information content of the applications received and between-vacancy variation in

the overall information environment. Third, we randomly match applications to vacancies, subject

to the target number of treated and control applications and the constraint that no candidate’s

application is sent to the same vacancy more than once. The realized distribution of treatment

assignments shown in Table E.2, Panel A matches the intended design: half of the applications are

sent with certiﬁcates and, mechanically, applications sent with reports are three times more likely

to be sent to vacancies that receive three applications with reports.

We monitor and record responses for two weeks after sending the applications. We classify each

response into one of these categories: (1) interview invitation, (2) request to send more information

or visit the establishment in person, (3) email bounce, (4) scam, and (5) other - mostly personalized

We cross-randomize the subject lines “Application for <vacancy>” and “Application for <candidate name>”

with the email messages “Please ﬁnd attached the application for <vacancy> as recently advertised online” and

“Please ﬁnd the application for <candidate name> for <vacancy>, as recently advertised online.”

Table E.2: Descriptive Statistics for Application-Level Attributes

Mean Std Dev. # Obs

Panel A: Characteristics of applications submitted

Had one report in a vacancy with one report 0.12 0.33 3992

Had one report in a vacancy with three reports 0.38 0.48 3992

Had no report in a vacancy with one report 0.37 0.48 3992

Had no report in a vacancy with three reports 0.13 0.33 3992

Panel B: Responses to applications submitted

Any response received 0.15 0.35 3992

Interview request received 0.09 0.29 3992

acknowledgements of receipt. If any application sent to a vacancy receives a type (3) or (4) response,

we drop the vacancy from the sample. We deﬁne two outcome variables for analysis. First, any

application that receives a type (1) response is coded as an ‘interview invitation.’ Second, any

response that receives a type (1), (2), or (5) response is classiﬁed as ‘any response’. We forward all

responses to the relevant candidate so they can contact the ﬁrm.

The ﬁnal sample consists of 3,992 applications sent to 998 vacancies, after dropping 70 vacancies

with bounce or scam responses. 15% of these applications receive any response, including 9% that

receive interview invitations (Table E.2, panel B).

F Placebo Certiﬁcation Experiment: Sample Report and Treatment Eﬀects

Figure F.1: Sample Placebo Certiﬁcate

REPORT ON ASSESSMENT PROCESS

name.. surname..

ID No. id..

This report provides information on assessments conducted by Harambee Youth Employment Accelerator

(harambee.co.za), a South African organisation that connects employers looking for entry-level talent to young, high-

potential work-seekers with a matric or equivalent. Harambee has conducted more than 1 million assessments and placed

candidates with over 250 top companies in retail, hospitality, financial services and other sectors. Assessments are

and funded in collaboration with the World Bank. You can find more information about this report, the assessments and

contact details at www.assessmentreport.info. «name» was assessed at Harambee on «date».

«name» completed assessments on English Communication (listening, reading, comprehension), Numeracy, and Concept

Formation:

1. The Numeracy tests measure candidates’ ability to apply numerical concepts at a National Qualifications Framework

(NQF) level, such as working with fractions, ratios, money, percentages and units, and performing calculations with

time and area. This score is an average of two numeracy tests the candidate completed.

2. The Communication test measures a candidate's grasp of the English language through listening, reading and

comprehension. It assesses at an NQF level, for example measuring the ability to recognise and recall literal and non-

literal text.

3. The Concept Formation Test is a non-verbal measure that evaluates candidates’ ability to understand and solve

problems. Those with high scores are generally able to solve complex problems, while lower scores indicate an ability

to solve less complex problems.

«name» also completed tasks and questionnaires to assess their soft skills:

4. The Planning Ability Test measures how candidates plan their actions in multi-step problems. Candidates with high

scores generally plan one or more steps ahead in solving complex problems.

5. The Focus Test assesses a candidate’s ability to distinguish relevant from irrelevant information in potentially

confusing environments. Candidates with high scores are generally able to focus on tasks in distracting surroundings,

while candidates with lower scores are more easily distracted by irrelevant information.

6. The Grit Scale measures whether candidates show determination when working on challenging problems. Those with

high scores generally spend more time working on challenging problems, while those with low scores choose to

pursue different problems.

DISCLAIMER: This is a condential assessment report for use by the person specied above. The information in the report should

only be disclosed on a “need to know basis” with the prior understanding of the candidate. Harambee cannot accept responsibility for

decisions made based on the information contained in this report and cannot be held liable for the consequences of those decisions.

This ﬁgure shows an example of the certiﬁcates given to candidates in the placebo treatment group. The

certiﬁcates contain the candidate’s name and national identity number, and the logo of the World Bank and the

implementing agency. Each work seeker received 20 of these certiﬁcates, an email certiﬁcate, and guidelines on

how to request more certiﬁcates.

Table F.1: Public and Placebo Certiﬁcation Eﬀects

(1) (2) (3) (4) (5) (6)

Index Employed Hours

Earnings

Hourly Written

wage

contract

Public 0.120 0.052 0.201 0.338 0.197 0.020

(0.027) (0.012) (0.052) (0.074) (0.040) (0.010)

Placebo 0.027 0.020 0.039 0.069 0.054 0.005

(0.043) (0.027) (0.075) (0.184) (0.129) (0.021)

p: public = placebo 0.041 0.245 0.045 0.147 0.266 0.471

Placebo / public ratio 0.222 0.379 0.196 0.204 0.275 0.238

# observations 6609 6607 6598 6589 6574 6575

# clusters 84 84 84 84 84 84

Coeﬃcients are from regressing each outcome on a vector of treatment assignments, randomization block ﬁxed

eﬀects, and prespeciﬁed baseline covariates (measured skills, self-reported skills, education, age, gender, employ-

ment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering

by treatment date. Mean outcome is for the control group. All outcomes use a 7-day recall period. Outcomes

marked with

use the inverse hyperbolic sine transformation. The index in the ﬁrst column shows the inverse

covariance-weighted averages of the 5 labor market outcomes, following Anderson (2008). The mean ratio of

placebo to public eﬀects is 0.258 for all 5 non-index outcomes. The sample sizes diﬀer across columns due to

item non-response, mostly from respondents reporting that they don’t know the answer.

G Experiments with Firms: Willingness to Pay and Skill Ranking

This appendix provides more information about the ﬁrm-facing experiments described in Sections

5.1 and 5.2. We recruit a sample of 69 ﬁrms located in commercial areas near the low-income

residential areas in Johannesburg where most workseekers in our sample live. We survey them

about their hiring practices, measure their willingness-to-pay for a database containing information

about assessment results for workseekers in our sample, and measure their preferences for diﬀerent

types of skills using an incentivized resume-ranking exercise.

Table G.1 reports summary statistics for this sample. Table G.2 shows summary statistics on

ﬁrms’ preferences for diﬀerent types of skills. Figures G.1 and G.2 shows screenshots of the platform

marketed to ﬁrms. Figure G.3 shows the distribution of willingness-to-pay.

Table G.1: Summary Statistics for Firm Sample

Variable # obs Mean Std dev. 10

pctile 90

pctile

Wholesale & retail trade 69 0.623 0.488

Transport, storage & communication 69 0.014 0.120

Restaurant & hospitality 69 0.188 0.394

Agriculture 69 0.014 0.120

Financial & insurance 69 0.087 0.284

Community & social services 69 0.014 0.120

Hiring decisions made exclusively at

69 0.754 0.434

location interviewed

Uses external recruiting services 69 0.25 0.43

# employees 69 15.0 29.6 3.0 32.0

# entry-level employees 67 7.24 14.94 0.00 14.00

# vacancies for entry-level employees 59 1.42 3.70 0.00 4.00

# entry-level hires expected in

58 3.95 5.43 0.00 10.00

next 12 months

# applications received for last

56 16.2 21.2 2.0 30.0

entry-level vacancy posted

# weeks required to ﬁll last

58 4.17 6.47 1.00 8.00

entry-level vacancy posted

Mean monthly compensation for

58 8,447 16,273 2,500 9,000

employees in last ﬁnancial year

Total payroll costs in last

31 1.277 2.766 0.078 3.200

ﬁnancial year (millions)

Table shows summary statistics for selected ﬁrm attributes variables. Percentiles are omitted for binary variables.

First six rows are indicators for sectors. All monetary ﬁgures are reported in South Africa Rands. 1 Rand ≈

USD 0.167 in purchasing power parity terms. # observations varies due to item non-response. Missing values

for the ﬁnal variables are more common because the survey was completed by the person responsible for hiring

decisions, who did not always have access to ﬁnancial records.

Table G.2: Firm Ranking of Proﬁles with Diﬀerent Assessment Results and Education

(1) (2) (3)

Proﬁle content Share of ﬁrms ranking proﬁle: Median

Top tercile Highest education aaaa First Last ranking

Communication Complete secondary school aaaa 0.119 0.015 3

Concept formation Complete secondary school aaaa 0.075 0.030 4

Focus Complete secondary school aaaa 0.328 0.060 3

Grit Complete secondary school aaaa 0.134 0.045 4

Numeracy Complete secondary school aaaa 0.060 0.090 2

Planning Complete secondary school aaaa 0.194 0.000 4

None One-year post-secondary diploma aaaa 0.090 0.761 7

Table shows summary statistics from ﬁrms’ ranking of proﬁles with diﬀerent skill assessment results and diﬀerent

level of education. All proﬁles have middle terciles for skills except that listed in the ﬁrst column.

Figure G.1: Screenshots of Login Page and Filtering Page

Figure G.2: Screenshot of Individual Candidate Proﬁle on Platform

Figure G.3: Willingness-to-pay for Database of Workseekers’ Assessment Results

0 10 20 30

Frequency

0 2000 4000 6000 8000 10000

Willingness-to-pay (South Africa Rands)

Notes: This ﬁgure shows the distribution of willingness-to-pay for access to the database of assessment results

described in Section 5.1 and shown in Figures G.1 and G.2. Values are in South African Rands, with 1 Rand ≈

USD 0.167 in purchasing power parity terms. The maximum possible bid is 10,000 South African Rands.

Appendix References

Abebe, G., S. Caria, and E. Ortiz-Ospina (2020): “The Selection of Talent: Experimental and

Structural Evidence from Ethiopia,” Manuscript, University of Bristol.

Anderson, M. (2008): “Multiple Inference and Gender Diﬀerences in the Eﬀects of Early Interven-

tion: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects,” Journal

of the American Statistical Association, 103, 1481–1495.

Attanasio, O., A. Kugler, and C. Meghir (2011): “Subsidizing Vocational Training for Disad-

vantaged Youth in Colombia: Evidence from a Randomized Trial,” American Economic Journal:

Applied Economics, 3, 188–220.

Beaman, L., N. Keleher, and J. Magruder (2018): “Do Job Networks Disadvantage Women?

Evidence from a Recruitment Experiment in Malawi,” Journal of Labor Economics, 36, 121–153.

Benjamini, Y., A. Krieger, and D. Yekutieli (2006): “Adaptive Linear Step-Up Procedures

That Control the False Discovery Rate,” Biometrika, 93, 491–507.

Bowling, A. (2014): Research Methods in Health: Investigating Health and Health Services, Maid-

enhead, GB: McGraw Hill; Open University Press, 4 ed.

Budlender, J., M. Leibbrandt, and I. Woolard (2015): “South African Poverty Lines:

A Review and Two New Money-Metric Thresholds,” Manuscript, Southern Africa Labour and

Development Research Unit, University of Cape Town.

Burks, S., J. Carpenter, L. Goette, and A. Rustichini (2009): “Cognitive Skills Aﬀect

Economic Preferences, Strategic Behavior, and Job Attachment,” Proceedings of the National

Academy of Sciences, 106, 7745–7750.

Casale, D. and D. Posel (2011): “English Language Proﬁciency and Earnings in a Developing

Country: The Case of South Africa,” Journal of Socio-Economics, 40, 385–393.

Chamorro-Premuzic, T. and A. Furnham (2010): The Psychology of Personnel Selection,

Cambridge University Press.

Cronbach, L. J. (1951): “Coeﬃcient Alpha and the Internal Structure of Tests,” Psychometrika,

16, 297–334.

De Kock, F. and A. Schlechter (2009): “Fluid Intellingence and Spatial Reasoning as Pre-

dictors of Pilot Training Performance in the South African Air Force (SAAF),” SA Journal of

Industrial Psychology, 35, 31–38.

Diamond, A. (2013): “Executive Functions,” Annual Review of Psychology, 64, 135–168.

du Rand, G., H. van Broekhuizen, and D. von Fintel (2011): “Numeric Competence,

Conﬁdence and School Quality in the South African Wage Function,” Manuscript, Stellenbosch

University.

Duckworth, A., C. Peterson, M. Matthews, and D. Kelly (2007): “Grit: Perseverance

and Passion for Long-term Goals,” Journal of Personality and Social Psychology, 92, 1087–1101.

Ederer, P., L. Nedelkoska, A. Patt, and S. Castellazzi (2015): “What Do Employers Pay

for Employees’ Complex Problem Solving Skills?” International Journal of Lifelong Education,

34, 430–447.

Eskreis-Winkler, L., A. Duckworth, E. Shulman, and S. Beal (2014): “The Grit Ef-

fect: Predicting Retention in the Military, the Workplace, School and Marriage,” Frontiers in

Psychology, 5, 1–12.

Esopo, K., D. Mellow, C. Thomas, H. Uckat, J. Abraham, P. Jain, C. Jang, N. Otis,

M. Riis-Vestergaard, A. Starcev, K. Orkin, and J. Haushofer (2018): “Measuring

Self-Eﬃcacy, Executive Function, and Temporal Discounting in Kenya,” Behaviour Research and

Therapy, 101, 30–45.

Gneezy, U., A. Rustichini, and A. Vostroknutov (2010): “Experience and Insight in The

Race Game,” Journal of Economic Behavior and Organization, 75, 144–155.

Gronau, R. (1974): “Wage Comparisons – A Selectivity Bias,” Journal of Political Economy, 82,

1119–1143.

Hanushek, E., G. Schwerdt, S. Wiederhold, and L. Woessmann (2015): “Returns to Skills

Around the World: Evidence from PIAAC,” European Economic Review, 73, 103–130.

Heckman, J. (1974): “Shadow Prices, Market Wages, and Labor Supply,” Econometrica, 42, 679–

694.

Heckman, J. and T. Kautz (2012): “Hard Evidence on Soft Skills,” Labour Economics, 19,

451–464.

Heckman, J., J. Stixrud, and S. Urzua (2006): “The Eﬀects of Cognitive and Non-Cognitive

Abilities on Labour Market Outcomes and Social Behaviour,” Journal of Labour Economics, 24,

411–482.

Hepner, P. and C. Petersen (1982): “The Development and Implications of a Personal Problem-

Solving Inventory,” Journal of Counseling Psychology, 29, 66–75.

Heppner, P. (1988): The Problem Solving Inventory (PSI): Manual, Palo Alto: Consulting Psy-

chologists.

Heppner, P., T. Pretorius, M. Wei, D. Lee, and Y. Wang (2002): “Examining the Generaliz-

ability of Problem-Solving Appraisal in Black South Africans,” Journal of Counseling Psychology,

49, 484–498.

Isaacs, G. (2016): “A National Minimum Wage for South Africa,” Manuscript, University of the

Witwatersrand: National Minimum Wage Research Initiative.

Kalechstein, A., T. Newton, and W. van Gorp (2003): “Neurocognitive Functioning is As-

sociated with Employment Status: A Quantitative Review,” Journal of Clinical and Experimental

Neuropsychology, 25, 1186–1191.

Lawrence, I. and K. Lin (1989): “A Concordance Correlation Coeﬃcient to Evaluate Repro-

ducibility,” Biometrics, 45, 255–268.

Lee, D. (2009): “Trimming, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment

Eﬀects,” Review of Economic Studies, 76, 1071–1102.

Leibbrandt, M., A. Finn, and I. Woolard (2012): “Describing and Decomposing Post-

Apartheid Income Inequality in South Africa,” Development Southern Africa, 29, 19–34.

Lopes, A., G. Roodt, and R. Mauer (2001): “The Predictive Validity of the APIL-B in a

Financial Institution,” SA Journal of Industrial Psychology, 27, 61–69.

Manski, C. (1989): “Anatomy of the Selection Problem,” Journal of Human Resources, 24, 343–

360.

Nisbett, R. (2009): Intelligence and How to Get It: Why Schools and Cultures Count, New York:

W. W. Norton and Company.

Posner, M. and G. DiGirolamo (1998): “Executive Attention: Conﬂict, Target Detection, and

Cognitive Control,” in The Attentive Brain, ed. by R. Parasuraman, MIT Press, 401–423.

Powell, J. (1984): “Least Absolute Deviations Estimation for the Censored Regression Model,”

Journal of Econometrics, 25, 303–325.

Pretorius, T. (1993): “Assessing the Problem-Solving Appraisal of Black South African Students,”

International Journal of Psychology, 28, 861–870.

Rath, J., D. Langenbahn, D. Simon, R. L. Sherr, J. Fletcher, and L. Diller (2004):

“The Construct of Problem Solving in Higher Level Neuropsychological Assessment and Rehabil-

itation,” Archives of Clinical Neuropsychology, 19, 613–635.

Raven, J. and J. Raven (2003): “Raven Progressive Matrices,” in Handbook of Nonverbal Assess-

ment, ed. by R. McCallum, Boston: Springer, 223–237.

Roberts, R., G. Goff, F. Anjoul, P. Kyllonen, G. Pallier, and L. Stankov (2000):

“Armed Services Vocational Aptitude Battery (ASVAB): Little More than Acculturated Learn-

ing?” Learning and Individual Diﬀerences, 12, 81–103.

Schmidt, F. L. and J. E. Hunter (1998): “The Validity and Utility of Selection Methods in

Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,”

Psychological Bulletin, 124, 135–168.

Schmidt, F. L., I.-S. Oh, and J. Shaffer (2016): “The Validity and Utility of Selection Meth-

ods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research

Findings,” Manuscript, University of Iowa.

Schmidt, K., B. Neubach, and H. Heuer (2007): “Self-Control Demands, Cognitive Control

Deﬁcits, and Burnout,” Work and Stress, 21, 142–154.

Stroop, J. R. (1935): “Studies of Interference in Serial Verbal Reactions,” Journal of Experimental

Psychology, 18, 643–662.

Taylor, T. (2013): “APIL and TRAM Learning Potential Assessment Instruments,” in Psycho-

logical Assessment in South Africa, ed. by S. Laher and K. Cockroft, Wits University Press,

158–168.

Tourangeau, R. (2003): “Cognitive Aspects of Survey Measurement and Mismeasurement,” In-

ternational Journal of Public Opinion Research, 15, 3–7.

Willis, G. B. (1999): “Reducing Survey Error through Research on the Cognitive and Decision

Processes in Surveys,” Short Course Presented at the 1999 Meeting of the American Statistical

Association.

——— (2008): “Cognitive Aspects of Survey Methodology (CASM),” in Sage Encyclopedia of Survey

Research Methods, ed. by P. J. Lavrakas, Thousand Oaks, California: Sage Publications, 104–106.