Risk management involves 'identifying risks' (i.e. detecting an event of significance), and then 'mitigating the risk' (i.e. responding appropriately).
Risk management has increasingly become recognised as an integral part of good practice M&E. But in the international aid industry we still have a lot to learn about risk management. The realisation of risks can be seen as a 'failure', but it represents a clear opportunity to learn. One of my favourite quotes is from Gharajedaghi, J. (1999):
"Learning results from being surprised: detecting a mismatch between what was expected to happen and what actually did happen. If one understands why the mismatch occurred (diagnosis) and is able to do things in a way that avoids a mismatch in the future (prescription), one has learned."
One of the problems I frequently observe with risk management in the aid industry is that i ...
Recently Rick Davies eloquently wrote in his blog ('Rick on the Road': http://mandenews.blogspot.com/) about different ways that people articulate program logic in logframes.
This is an issue that I've also battled, and have written about elsewhere (http://www.aid-it.com/Portals/0/Documents/070105_Aristotle%20and%20Plato%20at%20it%20again.pdf).
Many users of logframes seem to lose sight of the fact that the 'vertical logic' is supposed to describe the temporal sequence of change. That is, how a particular project is anticipated to contribute to social change through time. Instead, some folk use the vertical logic to systematically disaggregate the problem...in what Rick succinctly decribes as a "hirerearchy of inclusion". An analogy of t ...
Yes, yes...I know I haven't been a good boy in the blogging department! And I've promised Santa that I'll be better this year. No excuse really except for the familiar cries of too busy, too much international travel, a new baby...
Anyway, enough with excuses.
An issue that has come up several times recently is the role of information systems in support of M&E; specifically, the relative merit of technology-based systems.
As one of the founders of Aid-IT Solutions, I obviously have a view that IT systems can help to improve M&E outcomes. But it is clear that IT is no 'magic bullet'.The world learned a very harsh lesson about being unrealistic in this regard during the Dot.Com mania of the new millennium!
A technology-supported M&E system does not change the fundamental problem facing M&E practitioners...that measuring amorphous social change and atributing this ...
Outputs are universally defined as the tangible/quantifiable deliverables for which an implementation team can be held responsible. There are two things that seem to commonly confound the way we define 'outputs' for M&E purposes.
In either case, what we end up with ...
In recent years 'Outputs' have become the pariah of M&E. There's good reason for this of course...because in the bad old days, people tended to get stuck on only reporting the widgets that projects produced, and in so doing, lost sight of the bigger picture of the human changes that were expected to result from these widgets...the outcomes and impact.
But, I think that we've somehow thrown the baby out with the bathwater. There is a sense the M&E of outputs is only concerned with the boring 'quantitative' data about projects, and so tends to be downplayed by 'enlightened' M&E practitioners. But in fact there can be a lot of qualitative value in ensuring the 'outputs' remain one element within our M&E arrangements. This can be a source of valuable information for learning about the quality of our work...and what works or doesn't work in terms of fostering the desired outcomes.
Let me use an example from my per ...
I occasionally come across debate about how to distinguish 'monitoring' (M) from 'evaluation' (E). I have to admit that I have also engaged in this debate in the past, but in the end I've found most defintions unhelpful for practical purposes. As a result I tend to side-step the issue by avoiding a distinction and simply using the combined label 'M&E' rather than distinguishing 'M' from 'E'. Whichever way someone chooses to define 'M' v 'E', for practical purposes, both processes end up involving the identifcation --> capture --> analysis --> dissemination --> utilisation of information for accountability and for learning.
Most people attempt to differentiate 'M' from 'E' in one of four ways:
The raison d'etre of M&E is to enable accountability ('to prove') and learning ('to improve'). This can only be done if M&E data is used...in other words, it is supplied to interested stakeholders in a accurate, relevant and timely form.
But who are these "interested stakeholders"?
It has occured to me that M&E information is required by stakeholders in four 'directions':
I've been doing work for two different clients who would like to implement a standardised or generic evaluation method or instrument across portfolios of projects implemented by unrelated NGOs. Both clients are eager to have a consistent way to conduct comparative analysis of individual projects, and sets of projects. They want to know if individual projects have achieved what was anticipated; and if the portfolio of projects as a whole has fostered desireable outcomes.
One client is predominantly focussed on gathering (to the extent possible) quantitative performance data; the other is predominantly focussed on qualitative perceptions gathered from a range of stakeholders. But both assume that it is possible to obtain ‘scalable’ performance information. In other words, information that will be meaningful at any scale: individuals, communities, projects and programs. The other complication is that this ...
According to the State of Texas, Office of the State Auditor, (A Guide to Measure the Performance of Information Systems, April 1991) the following items are critical to successful implementation of a performance management system (similar issues apply to implementing a M&E system):
M&E is not an end in itself. It should serve a means...a means to learning and being accountable.
So, unless M&E processes/tools render information that is actually used by someone to further these ends, they are an unethical waste of resources.
So what does 'utilisable' M&E information actually mean?
I suspect that there are at least fourcontributing factors: