Over the past few weeks I've been writing up an evaluation report. Part of the evaluation involved a review of the M&E arranagements employed by four prominent international NGOs. It was encouraging to find that all four agencies were committed (at least intellectually) to learning and improving aid effectiveness. And it was encouraging that all had taken positive steps towards implementing M&E processes. However, it was also disappointing to find that the efforts to implement an 'M&E system' seemed to post hoc. They seemed to lack an overall 'information architecture' to give the M&E processes some coherence and purpose.
I may have developed an inadequate appreciation for the situation, but on reflection it seems that the focus of M&E remains very much on 'data' rathern than on 'knowledge'.
As noted by Henri Poincaré, the great French ...
I've been periodically engaged by AusAID since 2005, working on the development of a quality framework for M&E. In other words, a set of criteria to articulate what AusAID considers to be the basis for good quality M&E arrangements. This framework and the associated good practice guide are now at final draft status within AusAID.
One of the 6 broad criteria for good quality M&E arrangements (and arguably the most challenging) requires that M&E practitioners ensure that M&E arrangements are both comprehensive and efficient.
On the surface, this seems perfectly reasonable. But as with many things in life, a dig slightly below the surface reveals a surprising amount of complexity.
I'm just in the throws of writing up a report for AusAID; an evaluation of emergency response interventions in Pakistan following the earthquake in October 2005.
The earthquake was 7.6 on the Richter scale and affected an area the size of Belgium. An estimated 75,000 people were killed and over 3.5 million were rendered homeless...to say nothing of the widespread destruction of public infrastructure and services.
To visit the affected area is incredibly disturbing.
Part of the terms of reference of the evaluation was to examine the M&E of implementing agencies. This has led me to reflect deeply on what is appropriate for an emergency.
Much of what was done by way of M&E during the emergency response appears to be ad hoc or informal/unstructured. There appears to have been a focus on accountability (i.e. reporting to donors and powerful ...
There has been further discussion about my appraisal comments regarding the Cambodia rural development program discussed in my last blog. The following is drawn from my email response, seeking to clarify the issues from my perspective...
The issues that you have raised in your feedback are a fractal of a broader debate taking placed within the field of Design, Monitoring and Evaluation. I wholly agree with everything you have articulated in your last email, including the limitations of linear causality and the rigid ‘blue print’ (Fowler) approach to the design of social change interventions (for which the logframe is one prominent tool). I agree with much of the critique of the logframe, including Chambers and others (e.g.Gasper, Smilie, Smutylo, Roche, Kaplan, den Heyer, Lavergne). However, while agreeing completely with your line of argument, my conclusion is slightly different.
The heart of th ...
I've just completed an appraisal of a project design for an AusAID NGO Cooperation Agreement. I found the design document to be well written, and in some ways innovative.
The design embraced the complexity and systemic nature of development challenges in rural Cambodia, and proposed an iterative/emergent approach to tackling these issues based on a form of action-learning cycle (GTZ's Participatory Extension Approach (PEA)).
Although I appreciated the gist of the thinking behind the design, I raised the following questions:
The tension between Aristotle's functionalist ('classical') view and Plato's interpretist ('romantic') view of M&E that I discussed in my last blog is everywhere.
I'm engaged as an M&E Advisor to a major bilateral aid program in South East Asia. Recently a senior government official asked a colleague of mine, "why are you guys bothering with all this 'operational' M&E data? We're ultimately interested in the impact or outcomes of the program, so why not just focus on that kind of information?".
My colleague responded "well if you just want the head of the dog we can do that. But in order to really learn and improve our effectiveness, we need to know about the body and legs of the dog as well".
The idea of focussing exclusively on 'higher order' performance is particularly vogue at the moment. This is probably a reaction to the bad old days when per ...
So why 'Zen and the art of monitoring and evaluation'??
The truth is I've stolen the title from Robert Pirsig's classic book, 'Zen and the art of motorcycle maintenance'.
Pirsig's book is one of my favourites. At a superficial level it tells the story of his trip across the United States on a motorcycle with his young son on the back. But at a deeper level it paints a holisit philosophy of life...a 'Zen' philosophy.
While riding for hour-upon-hour, Pirsig muses the tension between very strong oposing (polemical) views of the world that can be traced to Aristotle and Plato respectively. Aristotle's view he calls 'classical', Plato's view he calls 'romantic'.
A classical worldview is all about function. It's the view that underpins scientific and engineering thought processes. It seeks to understand a 'wall' by ex ...