DFID Research: Sustainable development: A review of monitoring initiatives in agriculture
A new review examines 20 years of monitoring initiatives in sustainable agriculture. It provides insights and tools to help stakeholders prioritise investments and manage competing development goals.
Farming Gliricidia and Maize. Picture: World Agroforestry Centre, ICRAF
In an increasingly crowded planet, the need to combat the challenges of food security, nutrition, poverty and environmental degradation in a coordinated manner has never been greater. At Rio+20 this need was articulated by the support toward the formation of Sustainable Development Goals (SDGs) designed to complement and accelerate progress in the Millennium Development Goals (MDGs).
A new report has just been released on the Review of the Evidence on Indicators, Metrics and Monitoring Systems. Led by the World Agroforestry Centre (ICRAF) under the auspices of the CGIAR Research Program on Water, Land and Ecosystem (WLE), the review examined monitoring initiatives related to the sustainable intensification of agriculture. Designed to inform future DFID research investments, the review assessed both biophysical and socioeconomic related monitoring efforts.
With the aim of generating insights to improve such systems, the report focuses upon key questions facing stakeholders today:
- How to evaluate alternative research and development strategies in terms of their potential impact on productivity, environmental services and welfare goals, including trade-offs among these goals?
- How to cost-effectively measure and monitor actual effectiveness of interventions and general progress towards achieving sustainable development objectives?
The first step in the review process was to identify key initiatives in data monitoring systems relating to agriculture, paying particular attention to those that also acknowledge the impact on ecosystem health, and/or poverty and well-being. A total of 103 monitoring initiatives were identified.
The second step was to review the identified initiatives with respect to their degree of achievement in meeting a set of 34 criteria that had been established from a general literature review. All initiatives were evaluated with respect to their conceptual framework and a subset of 24 initiatives was screened against the full set of criteria. Based on this information a gap-analysis of the systems, indicators and metrics was conducted identifying strengths and weaknesses in methodology and use. Experience with monitoring in other fields, including public health surveillance, systems thinking in industry and public services, and decision sciences was also reviewed. Insights, lessons and recommendations were then drawn.
An over-riding lesson, outlined in the report, was the surprising lack of evidence for the impact of monitoring initiatives on decision-making and management. Thus, there are important opportunities for increasing the returns on these investments by better integrating monitoring systems with development decision processes and thereby increasing impacts on development outcomes. The report outlines a set of recommendations for good practice in monitoring initiatives, key among which are:
- The need of a clear conceptual framework to demonstrate an understanding of the system under study. In particular theories of change on how the monitoring results would affect behavior and explicit linkage to specific decisions are weak or lacking.
- Clear definition of the target inference space (geography, population) and how that is sampled. This is critical for making sound inferences from the monitoring results in terms of their wider applicability.
- Well-defined sample units or strata. It should be clear how units represent a sample of a larger area for which inference is desired.
- Consistent and well-documented measurement protocols, so that there is opportunity for aggregation and meta-analysis of results towards the development of generalizable knowledge and provision of a reliable picture of state and trends.
- Build scale hierarchy explicitly into the sampling design and statistical analysis methods, which is particularly critical for decision-making on sustainable agricultural intensification. Tools for doing this, for example through multilevel sampling, and use of mixed effects statistical models, have recently become more easily accessible.
- Determined efforts to integrate biophysical and socio-economic indicators both conceptually and in sampling frames. A particular challenge is how to link sampling units used in biophysical monitoring initiatives (e.g. fixed area sampling or watershed delineations) with units commonly used in socio-economic monitoring (e.g. households, villages).
- Designs that allow attribution of impacts of interventions. Use statistically sound study designs where possible. Disaggregate indicators across different levels of important conditioning variables (e.g. by gender, income group). Monitor variables along the impact pathway to accumulate evidence of intervention impacts.
- Link choice of variables and indicators to objectives, value of additional information, sample units, and measurement methods. Provide guidelines for interpreting indicators for management or policy decisions.
- Represent uncertainty, both conceptually and in communicating results. Make trade-offs among objectives explicit and separate material from preference trade-offs.
- Make data and information generated by research and government institutions accessible and reduce costs associated with access.
- Develop and put in place mutually acceptable agreements with traditional knowledge holders, whereby they agree to share important information related to resources, environment, ecosystems, on terms that are acceptable to them.
- Put in place active mechanisms for dissemination of results to target audiences, beyond web-based dissemination. *Collect relevant data to be able evaluate the impact and cost-effectiveness of monitoring initiatives, to help make a better case for sustaining initiatives.
The report ends with a series of clear steps and opportunities on how to transform these recommendations into reality to help make more informed decisions on agricultural intensification that reduces hunger and poverty sustainably for generations to come.
DFID welcomes the publication of this review. The complexity of the challenges which face decision makers aiming to enhance global food security is such that evidence (i.e. metrics) of what is working and what is not is essential. This review highlights an apparent disconnection between what is measured and what is required by decision-makers. It also identifies opportunities for a way forward. Progress will require global co-operation to ensure that relevant data are collected and made easily accessible. DFID is currently working with G8 colleagues on the planning for an international conference on Open Data to be held in Washington DC from 28th to 30th April 2013. The topline goal for the initiative is to obtain commitment and action from nations and relevant stakeholders to promote policies and invest in projects that open access to publicly funded global agriculturally relevant data streams, making such data readily accessible to users in Africa and world-wide, and ultimately supporting a sustainable increase in food security in developed and developing countries. Examples of the innovative use of data which is already easily available will be presented, as well as more in-depth talks and discussion on data availability, demand for data from Africa and on technical issues. Data in this context ranges from the level of the genome through the level of yields on farm to data on global food systems.