By Diego da Silva Rodrigues
To monitor and evaluate public and social policies is a key element of the development agenda. More and more, governments and international organisations tend to make their choices based on evidence, in order to identify the best and most cost-efficient options among a massive amount of programmes which are all aiming to improve people’s lives.
From an academic perspective, monitoring and evaluation (M&E) of public and social policies is challenging and exciting: theories of chance, field works, data collection, determination of treatment and control groups, generation of indicators and the use of sophisticated quantitative methods have become popular in the development literature – and interesting papers with great methodological contributions and rigorous outcomes have been published in academic journals.
On the practical side, the demand for actually applying M&E is high and even increasing, particularly in small non-governmental organisations (NGOs). To define NGOs is not straightforward – even regarding their sizes. According to my own experience, I define “small NGOs” as those who often work with small teams, mostly volunteers, on a very local level, without constant sources of funding, and who perform specific social activities focused on narrow groups of beneficiaries – children, the elderly, women, etc. On a small scale, these NGOs are responsible for a large range of activities to improve the social standards of their communities, thereby generating outcomes which are potentially transformative and impactful.
Despite this, many of these small NGOs are still far from academics, monitors, evaluators and their sophisticated techniques. Not only are their actions rarely based on evidence, but also the outcomes of their countless social interventions often remain unknown in academia and among policymakers. Unfortunately, this hinders the comprehension of those actions, their effectiveness and, consequently, their eventual replication.
Do small NGOs simply not care?
Working with small NGOs, I have realized they do not care much about M&E indeed. Although this is regrettable, it has become quite reasonable for me after a while: small NGOs simply don’t have resources, staff and skills for M&E. Besides, being faced with so many more urgent demands, it is almost a prerequisite for their existence and operation not to make M&E a priority.
The point is that M&E is commonly expensive. It requires funding, resources, staff and time – resources small NGOs usually struggle with. The quest for funding sources, the logistics of the NGO, the meeting of its basic needs, the administrative (if not bureaucratic) work, its own operation… All these fully consume the energy of the NGOs and their staff who are very often unpaid volunteers. M&E remains the last point of their to do lists, to be made when there is time and money left, which means almost never.
In addition, many governments and international organisations do not seem to have much interest in supporting small NGOs, including their M&E activities. This task would be too segmented, and on such a small scale that it might not be worth the effort and expenses for evaluating all those NGOs one after another. Thus, governments and international organisations focus their M&E efforts on large social programs and interventions, leaving the work of small NGOs aside.
Beyond the M&E methods
Apart from this general M&E debate, many small NGOs are not able even to develop a kind of “culture” of M&E. Although the members of the small NGOs I have worked with are usually motivated and passionate about their causes and work, they are sometimes not even aware of the necessity to monitor and evaluate their actions.
The consequence is that these NGOs, although sometimes organised and professionalised, often lack the structure from which to begin a process of M&E – even the most basic one. In my work with some of them, for instance, I have found NGOs which simply do not register their beneficiaries; others do not collect basic information about their families, residence or social conditions; some do not present criteria to select their beneficiaries, others do not follow up on them after they leave the NGO. In sum, my work with small NGOs usually consists of raising awareness about the importance of these simple but extremely relevant procedures – a job very different from the econometric techniques for impact evaluation, but certainly closer to the reality of these NGOs.
Last but not least, small NGOs frequently present a certain “resistance” against the M&E of their actions – which is particularly intense when it is suggested by an external evaluator. They commonly perceive M&E as a type of alien intervention in their activities, as if an outsider would “teach” them how to do their work. Furthermore, the staff of those NGOs are usually so emotionally involved that they become afraid of coming across eventual disappointing results shown by an evaluation. In the end, however, all this just reinforces the lack of awareness of the role of M&E for the better functioning of the NGO.
Empowering for M&E
Governments and international organisations have their reasons to focus their M&E efforts on large social programs. Indeed, to spend resources and effort on the M&E of small NGOs would be laborious and probably not lead to significant returns – both economically and academically. Yet, the small NGOs cannot just be set aside: when taken together, their activities are socially huge and important.
If to monitor and evaluate each of them is not possible, to enable and empower them to perform M&E is. It means breaking the “resistance” some of these NGOs have against M&E, making them aware of its importance and, to the extent possible, to generate the conditions for them to evaluate themselves.
It means also to break the “resistance” of some evaluators, who tend to perceive themselves as alien interveners who “teach” NGOs how to work better. What I’ve learned with the small NGOs is: nobody knows more about them, their weaknesses and strengths, than their own stakeholders. And: nobody is more interested in the M&E of their NGOs than themselves – if they are aware of and prepared for it.
Therefore, I increasingly believe that raising awareness for the importance of M&E is the main job of any academic or policymaker interested in M&E.
Diego da Silva Rodrigues is a PhD candidate in Economics at the University of Kent, UK. Currently based in Brazil, he has worked as M&E consultant for the third sector and as lecturer in Social Policy at Ibmec-DF.
Image: The International Livestock Research Institute (ILRI) under a Creative Commons Licence