Considerations for Developing Monitoring and Evaluation Approaches for Climate Change Adaptation

Introduction

Jennifer Frankel-Reed, formerly of the Bureau for Development Policy, UNDP and currently with the German Technical Cooperation (GTZ) writes,

“Based on good practices in monitoring and evaluation, an inclusive definition of adaptation, and UNDP’s experience to date, seven considerations are suggested for establishing adaptation monitoring and evaluation approaches. The considerations apply to a range of contexts and challenges in designing and implementing ‘climate-resilient’ development (i.e. via mainstreaming) and/or addressing priority climate change threats (i.e. via specific initiatives), and are intended to simplify the seemingly daunting task of tracking adaptation in a structured way.”

The views expressed in this essay/article are those of the authors and do not necessarily reflect the official policy or position of the RMIT Global Cities Institute. Readers should note that RMIT Global Cities Institute seeks a diversity of views and opinions on contentious topics in order to identify common ground.

Essay

Considerations for Developing Monitoring and Evaluation Approaches for Climate Change Adaptation by Jennifer Frankel-Reed.

* The author recognizes co-author Nick Brooks for his role in developing UNDP's M&E Framework for Adaptation.

Monitoring and evaluation (M&E) is becoming a headline issue as climate change adaptation gains recognition in development policies and programmes and garners increasing financial support. Beyond simply providing means for accountability and tracking delivery of results, monitoring and evaluation tools offer promising avenues for accelerating knowledge. Based on good practices in monitoring and evaluation, an inclusive definition of adaptation, and UNDP’s experience to date, seven considerations are suggested for establishing adaptation monitoring and evaluation approaches. The considerations apply to a range of contexts and challenges in designing and implementing ‘climate-resilient’ development (i.e. via mainstreaming) and/or addressing priority climate change threats (i.e. via specific initiatives), and are intended to simplify the seemingly daunting task of tracking adaptation in a structured way. The example of UNDP’s monitoring and evaluation framework for adaptation illustrates the application of these considerations at the portfolio or programme scale.

Background

An understanding of the differences, synergies, and scope of climate change adaptation and ‘business as usual’ in the context of development is fundamental to designing monitoring and evaluation guidance. The starting point for developing an M&E approach is naturally, ‘what are we trying to achieve?’, and the second question likely, ‘how are we trying to achieve it?’ In the context of development, both the similarities and differences between adaptation and status quo policies and practices are important in order to take advantage of effective entry points and implement the necessary innovations to improve resilience.

So what is known about the similarities and differences? Early adaptation literature focused on the differences between adapting to climate change and business-as-usual development and environmental management. This was a helpful phase for a few reasons:

• Examples were gained of adaptation needs that extend beyond the scope of ongoing development priorities – for example, securing coastal infrastructure and settlements against sea level rise, reducing the risks of glacial lake outburst flooding, combating new health risks, and addressing saline intrusion into aquifers, to name just a few. These examples suggest that simply redoubling the efforts of ‘business-as-usual’ development could be insufficient in addressing many of the risks of climate change.

• National vulnerability and adaptation (V&A) assessments were launched that began to link climate change risks to national development objectives and to identify priorities for doing things differently. While National Adaptation Programmes of Action (NAPAs) and National Communication V&A assessments are not national adaptation frameworks, they begin to lay the analytical and capacity foundations for drawing linkages and setting priorities.

• The utility and limitations of incorporating climate change scenarios and impact models in management and decision processes were explored. While action can often be taken with imperfect climate change information, some decisions will hinge on greater certainty. ‘Soft’ approaches and adaptive capacity have emerged as priorities in many cases.

More recently, conceptual and practical synergies have become clearer. In part, this may be driven by increasingly apparent impacts, more convincing economics, and growing awareness within disaster risk management circles and other climate-sensitive areas such as food security, water resources and public health. Knowledge from within a wide range of fields is being applied to climate change challenges, enriching the discussion with practical ways to enhance resilience. Notably, during the Adaptation Financing Discussion at Climate and Development Days during Bali’s UNFCCC COP13, representatives from Uganda and the Gambia underscored advances being made to address climate change risks through national policy and budget processes in their countries, rather than focusing on new mechanisms or funding. Many opportunities for adaptation are already apparent at the national level, although additional financing is clearly needed, and these representatives also stressed the need for enhanced national capacity.[1]

Based on this emerging understanding of differences and synergies, the seven considerations for developing adaptation M&E approaches recognize that:

• M&E for adaptation can fit within ongoing development M&E (mainstreaming approach), and/or can support efforts focused particularly on addressing climate change risks (climate risk reduction approach).

• Regardless of the design and execution, both specific adaptation actions (measures) and systemic adaptive capacity (including vulnerability reduction) are considered essential dual components of adaptation.

Challenges

A number of challenges face the development of monitoring and evaluation frameworks for adaptation, including both conceptual and practical issues. At the conceptual level, there has been much disagreement regarding the definition and scope of adaptation, for example, differing views on how adaptation contrasts with ‘business-as-usual’ development, or disaster risk management, and how adaptation should be designed and implemented relative to ongoing development planning and financing (see, for example, Schipper 2007 and Eriksen et al. 2007).

Practical challenges facing M&E for adaptation have been discussed elsewhere (see UNDP 2007 and GEF 2008), but include:

• the nature of adaptive capacity and factors contributing to vulnerability vary greatly across circumstances;

• ‘calibration’ is a challenge given constantly changing climatic conditions that provide the backdrop for adaptation;

• adaptation is a long term issue, for which M&E would ideally extend over long periods and account for multiple vulnerability and risk factors beyond climate.

Proposed Solutions

Monitoring and evaluation frameworks for adaptation can be developed that manage these challenges. At the conceptual level, the considerations in the following section suggest clearly stating synergies and differences between adaptation and ongoing development. At the practical level, a structured approach recognizes the varied elements and types of adaptation.

Considerations

The following considerations can aid in developing a monitoring and evaluation approach that enhances results and learning in a wide range of contexts. These considerations could be applied in designing a portfolio-scale or project-scale monitoring and evaluation framework.

1. Drawing development-climate change linkages;

2. Focusing the scope of adaptation on key sectors, themes, or issues;

3. Identifying target processes, institutions, and capacities to strengthen system-wide adaptive capacity;

4. Identifying adaptation practices and behaviours related to development outcomes;

5. Identifying adaptation measures necessary to reduce climate-related risks;

6. Incorporating climate hazard and vulnerability/capacity factors; and

7. Balancing quantitative, qualitative, survey-based, and narrative M&E tools.

The results of applying these steps at the portfolio scale are illustrated by UNDP’s M&E Framework in the next section.

An Example: UNDP’s climate change adaptation M&E framework

Defining adaptation to structure M&E

A review of UNDP’s climate change-related risks and opportunities to support climate resilience was the first step in shaping the formulation of an M&E framework for UNDP’s Adaptation Programme. Building on the findings of this review, adaptation was defined in both strategic and practical terms that were consistent with the agency’s human development mandate and operational emphasis on capacity development, support to national development policies and programmes, and UN system coordination.

The overarching goal for the adaptation programme is to ‘ensure progress toward the Millennium Development Goals (MDGs) under a changing climate’ However, to guide programming at a practical level, the M&E framework sets more specific thematic objectives in line with climate-sensitive development topics, aligned with the MDGs (considerations 1, 2). The framework recognizes that objectives and practices for adaptation vary significantly depending on the themes or sectors involved. Specific objectives set for each of six ‘Thematic Areas’ (themes/sectors), such as improved food security and reduced climate-related disaster losses.[2] UNDP encourages cross-sectoral adaptation approaches, but recognizes that themes help to ‘zoom in’ and determine actors, roles, responsibilities, technologies, methods, and results for adaptation:

Thematic Area 1: Agriculture/food security

Thematic Area 2: Water resources and quality

Thematic Area 3: Public health

Thematic Area 4: Disaster risk management

Thematic Area 5: Coastal zone development

Thematic Area 6: Natural resources management

Next, a set of adaptation practices were identified, which, based on preliminary experience,[3] when addressed in combination, are expected to reduce vulnerability, enhance adaptive capacity, and ameliorate priority climate change hazards (considerations 3, 4, 5):

1. Policymaking and planning (including budget and regulatory processes);

2. Capacity development and awareness raising;

3. Information management (including EWS, monitoring and analysis processes);

4. Design and decision-making for investments; and

5. Risk reduction practices/livelihood activities and/or resource management.

Having identified these five components of adaptation programming, and different themes across and within which they apply, UNDP’s M&E framework proposed four categories of indicators to be tracked using various techniques, including the quantitative indicators, stakeholder perceptions, and narrative (consideration 7). The indicator categories are:

1. Coverage (quantitative)

2. Impact (quantitative, qualitative, survey-based, narrative)

3. Sustainability (quantitative, qualitative, survey-based, narrative)

4. Replicability (quantitative)

Figure 1 reflects the main elements of the structure of UNDP’s M&E framework.

Vulnerability reduction assessment (VRA) is an evaluation tool that allows stakeholders to rate behaviours, vulnerabilities, capacities or practices through surveys or interviews on a scale from 1-10 and to provide reasons alongside their scores. Survey questions structured around VRA build on vulnerability and hazard factors identified by stakeholders and other assessments undertaken during project development phases (consideration 6). Monitored throughout a project’s lifetime, VRA scores should reveal changes in conditions and the reasons for these changes, as seen through the eyes of stakeholders. Reasons provided in these evaluations can also feed into adaptive programme or project management and help to describe the role of project activities in progress observed. UNDP’s framework proposes the use of VRA surveys for some of the ‘impact’ indicators related to behaviour, capacity, and perceived vulnerability described below.

UNDP’s Standard Indicators for adapting to relevant Thematic Areas

Based on this structure, a standard set of indicators covering the range of adaptation processes, applicable across different themes, and employing a variety of methods was developed. The indicators in Table 1 provide options for adaptation projects and programmes to select from and adapt as appropriate to different contexts. More detailed guidance and thematic illustrations are currently under preparation.

I. Coverage

1. Number of policies, plans or programmes introduced or adjusted to incorporate climate change risks.

2. Number of stakeholders (e.g. communities, households, agencies, decisionmakers) engaged in capacity development activities for vulnerability reduction or improved adaptive capacity.

3. Number of stakeholders served by new or expanded climate information management systems (e.g. early warning systems, forecasting, etc.)

4. Number of investment decisions revised or made to incorporate climate change risks).

5. Number of risk-reducing practices/measures implemented to support adaptation of livelihoods and/or resource management.

II. Impact

1. Percent change in stakeholders’ behaviours utilizing adjusted processes, practices or methods for managing climate change risks, assessed via QBS or other evidence (relevant across processes i-v).

2. Percent change in stakeholders’ capacities to manage climate change (e.g. communicate climate change risks, disseminate information, or make decisions based on high quality information), as relevant, assessed via qualitative survey (e.g. H form).

3. Percent change in use of/performance of information management systems, for example, early warning response times.

4. Percent change in stakeholder perceptions of vulnerability to (or capacity to adapt to) a recurrence of primary climate change-related threat(s), assessed via qualitative survey.

5. Improvement in the relevant quantitative development outcome (food security, water resources, health outcomes, etc.) in relation to average historic data or to years marked by extreme conditions.

III. Sustainability

1. Number of project beneficiaries involved in capacity development for implementation of specific adaptation measures or decision-support tools.

2. Availability of skills and resources necessary to continue adaptation after conclusion of project (at relevant scale), assessed via qualitative survey.

3. Stakeholder perceptions of adaptation sustainability, assessed via qualitative survey.

IV. Replicability

1. Number of ‘lessons learned’ codified.

2. Number of relevant networks or communities with which lessons learned are disseminated.

Table 1. Standard Indicators applicable across themes or sectors

Conclusion

Given the confusion surrounding adaptation to climate change, and the inherent complexity of adaptation challenges, a structured approach to monitoring and evaluation can help convey both the conceptual foundations and the practical means by which climate resilience will be achieved. Standard development and environment indicators fall short in the adaptation context. They do not reflect the nature of adaptation – which is about capacity, behaviour, and risk-reducing measures for the advancement of development outcomes. UNDP's monitoring and evaluation framework is a work in progress and is currently being tested by a number of projects. The practical implications of stakeholder surveys and the utility of using development indicators to help validate adaptation progress will be reviewed by projects focused both at the national scale as well as a number of community-scale projects. Given the high demand for knowledge and exchange on this topic, an interface for sharing monitoring and evaluation experiences and good practices could be explored through the Adaptation Learning Mechanism (ALM) or other platform for structured exchange on the subject.

References

E. Lisa F. Schipper, 2007. Climate Change Adaptation and Development: Exploring the Linkages. Tyndall Center Working Paper No.107.

Global Environment Facility Evaluation Office, 2007. Elements for an M&E Framework for Climate Change Adaptation Projects: Lessons from GEF Climate Change Adaptation projects

Siri E.H. Eriksen, Richard J.T. Klein, Kirsten Ulsrud, Lars Otto Næss, Karen O’Brien, 2007. Climate Change Adaptation and Poverty Reduction: Key interactions and critical measures, Report prepared for the Norwegian Agency for Development Cooperation (Norad).

Siri E.H. Eriksen and P. Mick Kelly, 2006. Developing credible vulnerability indicators for policy assessment. Mitigation and Adaptation Strategies for Global Change, Mitigation and Adaptation Strategies for Global Change (2007) 12: 495–524.

UNDP, 2007. A Proposed Monitoring and Evaluation Framework for Adaptation to Climate Change.

UNICEF, 2006. New Trends in Development Evaluation.

UNICEF, 2008. Bridging the gap: The role of monitoring and evaluation in Evidence-based policy making.

End notes

Information about the author

Jennifer Frankel-Reed has worked on a range of climate change adaptation topics, including monitoring and evaluation, guidelines for adaptation project design, and systems for knowledge sharing with the Bureau for Development Policy, United Nations Development Programme (UNDP) in New York. Recently, Jennifer assumed the role of climate change advisor with GTZ (German Technical Cooperation) and plans to be based in India. She holds a Masters degree in global change science and policy from the Yale School of Forestry and Environmental Studies. E-mail: jfrankelreed@gmail.com

Global Cities Institute - RMIT University, Australia invites your response

AdaptNet invites your responses to this article. Please send responses to the editor AdaptNet, Saleem Janjua: adaptnet@rmit.edu.au

Responses will be considered for redistribution to the network only if they include the author's name, affiliation, and explicit consent.

AdaptNet subscription information

AdaptNet is a free weekly report produced by RMIT University Global Cities Institute's Climate Change Adaptation Working Group, Melbourne, Australia.

• Subscribe or unsubscribe.
• View the archive: 2006, 2007, 2008.

Translations

• Terjemahan dalam Bahasa Indonesia: 2007, 2008.
  
• AdaptNet in Vietnamese: 2007, 2008.
  

For further information, please contact the editor, Saleem Janjua.