Do certification schemes improve worker wellbeing?
19th May 2017
3Keel senior consultant Catherine McCosker worked with a team at SOAS (University of London) under Dr Carlos Oya to conduct a systematic review of agricultural certification schemes (CS). The review set out to assess the effectiveness of CS in improving socio-economic outcomes for farmers and agricultural workers in low- to middle- income countries.
Systematic reviews examine the effectiveness of interventions on specific outcomes by aggregating, summarising and analysing the findings of existing studies on the topic. By using robust search processes as well as specific quality and relevance criteria to screen potential studies, systematic reviews assess the highest quality evidence available on a topic, to provide “gold standard” evidence on impact.
This systematic review included studies that assessed second- and third-party certified CS but excluded studies on proprietary and own-company standards. The review asked two main questions to assess the effect of CS on socioeconomic outcomes for farmers and agricultural workers. The first looked at experimental studies to assess the effectiveness of CS on quantitative socioeconomic outcomes such as yield, price, wages, total household income, illness, and education. The second addressed qualitative factors, such as barriers, facilitators, and contextual factors for CS effectiveness.
43 studies were assessed in the effectiveness review and 136 in the qualitative review. In general, the quality of the studies was mixed, including a relatively large proportion of studies with a high risk of bias, especially in the quantitative synthesis. The majority of the included studies were based in Latin America, with coffee and fruit the most common agricultural commodities. Fairtrade was the most heavily represented CS and only 12 CS overall were represented in the review, a reflection of the fact that only a small section of the overall CS population attracts significant research on impact.
The first part of the review found that impacts of CS were mixed, with both positive and negative impacts, and many outcomes showed weak or not statistically significant effects. Statistically significant results were found for outcomes of prices for certified producers (14% higher than for non-certified producers), income from certified production (11% higher than non-certified production), wages (13% lower compared to workers engaged in non-certified production), and education (children in households of certified producers had 6% more schooling). Where possible, effects were disaggregated by CS, but these findings did not yield any more conclusive results.
The variability makes it difficult to draw meaningful conclusions on the effectiveness of CS. The variability found is partially due to limited high quality research in this area, restricting the pool of quantitative evidence from which to draw conclusions. The other key issue is the importance of context in the implementation and effectiveness of CS, as found in and elaborated by the qualitative synthesis. The barriers, facilitators, and contextual factors found by the qualitative review were numerous and wide-ranging, and these factors (such as cultural practices, long existing gender dynamics, and regional/national historical systems of agricultural production and marketing) can and do mediate both the establishment and implementation of CS as well as their impact on socioeconomic outcomes.
It is not unusual for systematic reviews to be unable to draw clear and strong conclusions on effect; after all, the conclusions a systematic review reaches can only be as strong as the evidence it is based on. However an inability to draw a strong conclusion, and the reasons for this, are an important finding in their own right. The findings of this review are a clear exhortation that more and better research on the effectiveness of CS is needed. Indeed, the mixed effectiveness speaks especially to the necessity of gaining a better understanding of CS effectiveness, and that positive effects of CS cannot be assumed. Additionally the findings on the clear impact of contextual factors on CS effectiveness can be used by practitioners to better understand and mitigate the effects context will have on the implementation of CS.
Read the review here.
