SSM 4 M&E Using Soft Systems Methodology to Define M&E Requirements for International Aid Projects

It is widely acknowledged within the international aid industry that monitoring and evaluation
(M&E) processes are the way by which information for accountability and learning can be
obtained. A challenge facing M&E practitioners is that social changes fostered by aid are
complex, and hence the development of M&E systems to judge these changes can be
conceptually and practically complex. This paper identifies soft systems methodology (SSM)
as one way to grapple with the complexity frequently associated with developing M&E
information systems (MEIS). We describe how SSM can be applied in MEIS development by
reporting a detailed case study demonstrating its use within an Australian Government
(AusAID) funded community development program in the Philippines. We find SSM to give
methodological structure to the development of M&E systems while accommodating the illdefined
and emergent nature of the development context.

Aristotle and Plato At It Again

International aid projects are broadly concerned with fostering change. Frequently, the 'theory of change' within an aid project is communicated using Logical Framework Analysis, or the 'logframe'. The logframe may be viewed from at least two philosophical perspectives-functionalist and interpretist. Functionalism is found to be useful for problem analysis and project design since it enables a deconstruction of the goal into functional components. Interpretivism is found to assist project monitoring and evaluation since it draws attention to the role of human actors within the social change process, thereby clarifying the social research plan. A bilateral aid program in the Philippines is described to illustrate the practical differences arising from the divergent philosophies.

The Yin and Yang of M&E

A brief outline of a conceptual model is provided to help identify what data is required to enable the monitoring and evaluation of international aid.  The model identifies the interdependence of 'performance information' (the 'Yang') and 'risk information' (the 'Yin') so that a full picture of intervention performance can be accured (i.e. a 'systems perspective').

Project Appraisal Framework

A logic-based conceptual framework to rigorously appraise proposed project designs.  The framework identifies three hypotheses of change implicit within any development 'theory of change', and articulates key risks likely to erode each stage of change.  A series of questions are posed to guide reflection by project appraisers concerning the efficiency, efficacy and effectiveness of the project design.

AIDING AID: A Monitoring and Evaluation Framework to Enhance International Aid Effectiveness

A doctoral thesis providing a coherent theoretical framework for developing monitoring and evaluation (M&E) information systems within international aid agencies.

Using soft systems methodologies (SSM), the thesis develops the theoretical basis for M&E through a transdisciplinary review of three academic fields: information systems, organisational effectiveness and project management. It is argued that inadequacies in the operationalisation of M&E systems arise from divergent epistemological and ontological assumptions about the nature of information and its role within organisations that are concerned with effecting social change. The M&E framework proposed seeks to resolve the dilemma posed by these divergent assumptions. This involves a M&E information system (i.e. MEIS) that is novel in terms of its scope, purpose and application.

Firstly, the scope of the proposed MEIS takes in the entire aid organisation, going beyond the dominant, conventional approach, which is project-centric. This enables alignment of project strategies with organisational mission. Further, it aims to promote the institutionalisation of lessons learned within projects (conceived as ‘social experiments’) for organisational learning, thereby enabling informed debate about the effectiveness of the organisation in fostering sustainable development.

Secondly, the purpose of the proposed MEIS is concerned with promoting organisational success. The critical success factors of learning and accountability are identified, and the role of M&E in encouraging responsive management decision-making and critical inquiry and reflection is described.

Thirdly, the application of the proposed MEIS involves a modified logframe. The ‘3D-Logframe’ serves as a conceptual basis to address limitations found with the conventional two-dimensional logframe matrix when employed for M&E purposes.

The proposed M&E framework was developed out of iterations of action in the field and reflection.  Further research will involve applying the framework in its entirety.

PACAP M&E Plan

A monitoring and evaluation (M&E) plan developed for the Philippines-Australia Community Assistance Scheme (PACAP) which is funded by AusAID and implemented by Hassall & Associates International.  The M&E plan applies Soft Systems Methodologies (SSM) to define information requirements and utilizes the Aid-IT eM&E system to manage the M&E processes.