Reflection on challenges and criticism
From the start of the implementation of the ARP approach, scholars have followed this intervention with interest. Some have said they felt it could complement legal and holistic activities to protect girls (Boyden 2012), while others regarded it as a ‘social engineering experiment’ (Hughes 2018), or criticised the bold claims about its success (Droy et al. 2018) and effectiveness (Matanda et al. 2023). From a theoretical perspective it has been argued that evidence of its effectiveness has not been solid, and methodologies describing its effectiveness are weak or inadequately described and lack a comparison group (Droy et al. 2018). These external and independent views of scholars on ARP and CLARP, however painfully they were experienced at the time by practitioners who work on the issue every day with a lot of passion, provided an essential helicopter view. Interdisciplinary discussions and debates between academics and implementers have taken place ever since. This section will discuss the dialogue between researchers and practitioners of CLARP and how it has influenced the current implementation of CLARP by Amref.
An important criticism and eye-opener for reflectiveness on CLARP came with Mepukori’s qualitative study (2016) that aimed to evaluate the effectiveness of CLARP in a Samburu intervention site, where the NGO had worked since 2011. This study alarmed the organisation and resulted in a rather defensive management response, showing it was not used to receiving criticism from academics. It became an essential turning point for growth. Among Mepukori’s conclusions was this important point: ‘To the extent that ARP interventions are only successful when the majority rallies against female circumcision, there remains insufficient community buy-in for the intervention to succeed in the Samburu community’ (ibid.:64). Mepukori stressed social norms theory (e.g. Mackie and LeJeune 2009) and the challenges faced in order to reach the tipping point.
1 In social norms theory, Mackie and LeJeune (2009) describe the tipping-point concept as a critical threshold in changing practices like FGM/C. The tipping point is reached when enough people in a community decide to abandon FGM/C, creating a shift in norms where not cutting becomes acceptable, reducing the social pressure to conform to the old practice. For more information on social norms theory, see the Introduction to this book. The scaling-up of CLARP has proved to be a challenge for many reasons, including limited resources (funds, time, capacity) and sustainability, as change takes time and it is not easy to leave a CLARP intervention area. Sustainability is therefore a valid point of critique and worry when implementing CLARPs, as contextual changes such as the COVID-19 pandemic and climate change, which threaten the livelihoods of the Maasai people, can cause setbacks in communities that are progressing toward eliminating FGM/C (Esho
et al. 2022).
Another important point of criticism that Mepukori raised was on the monitoring and evaluation of CLARP, which had not been consistent, therefore running the risk of ‘open[ing] up an already vulnerable intervention to a host of unintended consequences including severe backlash and the possibility of driving female circumcision underground’ (Mepukori 2016:64). It was a criticism that Amref took very seriously, and from that point it invested heavily in monitoring through developing and tracking a theory of change model. It also developed the monitoring tool Tracking the Girls (as described above).
It might be the lack of support networks and the isolation of their community that has caused Maasai girls to continue to succumb to FGM/C. Or is it related to the fact, often argued by scholars, that CLARPs (and ARPs in general) lack the cultural depth and authenticity of the traditional rites they are intended to replace? Even when communities are involved in the design of community-led ARPs, deciding which elements of initiation should stay and which should be abandoned, Graamans et al. (2019:6) write that ‘girls who participate in CLARPs do not feel like women, neither are they considered as such by the community’. It is a problem commonly heard among young people in Africa who struggle to attain recognised social adulthood (Johnson 2018), also conceptualised as waithood (Honwana 2012). Johnson argues that traditional rites provide adults with the ability to actively support youth (especially girls) in shaping their futures and acknowledging the tensions of the current era (marriage preparation versus urging marital delay), while symbolising continuity with past generations. What is needed for Maasai girls to perceive themselves and be perceived as women is something they will need to negotiate themselves. The question is whether the prolongation of their youth, combined with education and empowerment, will strengthen girls’ voices and agency with regard to FGM/C.
Like Graamans
et al. (2018), the review of existing evidence that Matanda
et al. (2023) conducted concluded that the complexity of FGM/C requires a holistic approach. Matanda
et al. (ibid.:13) mention that certain elements of ‘health education, community dialogues with parents and religious leaders, the use of media and social marketing efforts, and formal education for women and girls’ show sufficient evidence and therefore can be seen as appropriate. It is acknowledged, also by practitioners, that education can provide an alternative path for youth, although girls’ ambitions might become unrealistic, as Graamans
et al. (2018) argue: ‘inflated promises about what education brings may lead to disappointment. And this sequential disappointment can lead to a setback in progress being made in the work against FGM/C or the adoption of ARP.’
2 Seen online, no page numbers. Additionally, it would be good to mainstream sexual reproductive health and rights and FGM/C (where relevant to the context) education in the school curriculum, but this seems to have had little political support and currently Kenya (along with many other African countries) is moving towards more conservative ideas on sexual education for primary school students.
Furthermore, there is a long-heard argument that the effectiveness of CLARP can only be shown through baseline versus endline comparisons, a view that Amref has fully embraced. In addition to such evaluations (Mbogo et al. 2022), other methodologies like the stepped-wedge cluster randomised controlled trial (Esho et al. 2023), and the difference-in-difference method (Muhula et al. 2021), have been used to strengthen evidence through quantitative research. Quantitative evidence (more than the different qualitative studies that have already been conducted) of CLARP’s effectiveness could really boost the political, advocacy, and funding agenda (ibid.).
The Muhula
et al. (2021) study was the first in Africa to apply a quasi-experimental approach to examine the impact of the CLARP intervention. The study used the difference-in-difference approach
3 The difference-in-difference approach is a widely-used method in impact evaluation, especially in the development sector, to estimate the causal effects of interventions or policies. It is particularly useful in situations where randomised controlled trials are not feasible, but pre- and post-intervention data are available for both treated and comparison groups. to quantify the average impact of the CLARP intervention, focusing on Kajiado as the intervention county, with Mandera, Marsabit, and Wajir as control counties. To empirically investigate and respond to whether CLARP has had any impacts on the social and educational outcomes of the beneficiaries, the study employed the Kenya Demographic and Health Survey (KDHS) data from 2003, 2008–9, and 2014. KDHS data are not only statistically representative but also detailed, with various useful variables on demographic, health, and socio-economic indicators. The study demonstrated that CLARP had contributed to the decline in FGM/C prevalence, child marriage, and teenage pregnancies among girls in Kajiado by 24.2 per cent, 4.9 per cent, and 6.3 per cent respectively. The study also showed that the intervention had contributed to an increase in girls’ schooling by two-and-a-half years, thereby keeping girls in school longer and reducing their chances of being circumcised. This paper elicited much discussion among scholars in the FGM/C space, criticising the use of secondary KDHS data, rather than the authors’ own collected data. This, together with the questions raised around the use of difference-in-difference as a quasi-experimental design, points to the lack of understanding of how econometric models can be exploited to address the evaluation challenges in health programmes. The difference-in-difference approach calculates the average treatment effects by differencing the average outcomes before and after programme implementation across control (without intervention) and treated (with interventions) groups. In the context of the CLARP programme, the difference-in-difference approach measures the average impacts of the CLARP roll-out on social and educational outcomes in Kajiado County, where Amref introduced the CLARP model. The results were then compared with Mandera, Marsabit, and Wajir – counties with no CLARP model (control groups) – before and after the model was rolled out in 2009.
The KDHS are nationally representative household surveys that provide data for a wide range of monitoring and impact evaluation indicators in the areas of population, health, and nutrition, and have been implemented in over ninety countries. It should be noted that the KDHS asks limited questions about FGM/C. However, we consider it to be the most reliable data to support impact evaluation of programmes such as CLARP with the confirmation of the availability of necessary variables of interest in the database.
However, admittedly, measuring the attribution of the CLARP and CLARP-related interventions indeed has limitations, especially because FGM/C is a long-term and complex change process in which it is extremely difficult to isolate the attribution of one intervention to reduced FGM/C prevalence. Should that stop us from trying to measure its effectiveness? We should not be shy to look for ways to measure behaviour change around FGM/C, although it might be difficult to achieve academic rigour. If we are supported and can continue the dialogue with scholars, we might get more insight into which systems and aspects need to be further developed and addressed to improve effectiveness measurements. It takes interdisciplinary collaboration, allowing the perspectives of local actors, stakeholders, and structures to give us the necessary insights. But what is also key is consistent and good quality documentation.
Another limitation that NGOs face is fundraising. While striving to improve models of intervention, our transparency about lessons learned can impede the flow of funding that is essential to continue the work. This dichotomy between learning and accountability/fundraising causes dilemmas. As an organisation, we need to raise funds to continue supporting and protecting more girls and reach the sustainable development goal to end FGM/C.