Corporate report

Fleming Fund annual review: January to December 2019

Published 24 September 2020

This was published under the 2019 to 2022 Johnson Conservative government

Clearance checklist

	Name	Date
Quality Assurance	Head of Fleming Fund	19 Feb 2020
External Assurance – Independent body	DFID Health Adviser	13 Feb 2020
Project Board	Fleming Fund Project Board	28 Feb 2020
Global Health Security (GHS) Programme Board	GHS Programme Board	16 Mar 2020

Abbreviations and acronyms

Abbreviation or acronym	What it means
AMC/U	antimicrobial consumption and use
AMR	antimicrobial resistance
AMRCC	antimicrobial resistance coordinating committee
AMS	antimicrobial stewardship
ATLASS	FAO assessment tool for laboratories and antimicrobial resistance
DEFRA	UK Department of Environment, Food and Rural Affairs
DFID	UK Department for International Development
DHSC	UK Department of Health and Social Care
ES	evaluation supplier
FAO	United Nations Food and Agriculture Organization
FCO	UK Foreign and Commonwealth Office
GBD	global burden of disease
GHS	Global Health Security
GHSA	global health security agenda
GLASS	World Health Organization global antimicrobial resistance surveillance system
GRAM	global research on antimicrobial resistance
HMG	Her Majesty’s Government (UK)
IATA	International Aid Transparency Initiative
IHME	Institute for Health Metrics and Evaluation
KPI	key performance indicator
LMICs	low and middle-income countries
M&E	monitoring and evaluation
MA	management agent
MOU	memorandum of understanding
NAP	National Action Plan
ODA	Official Development Assistance
ODI	Overseas Development Institute
OIE	World Organisation for Animal Health
OU	Open University
OUCRU	Oxford University Clinical Research Unit
PHE	Public Health England
Q1	quarter 1
QD1	quarterly deliverable 1
RFP	request for proposals
SA/CSA	sustainability analysis/comprehensive stakeholder analysis
SDG	Sustainable Development Goal
TAG	technical advisory group
TOC	Theory of Change
UN	United Nations
VfM	value for money
WHO	World Health Organization

Introduction

Outline of programme

In the 2015 spending review, the Global Health Security (GHS) team was given £477 million of UK Official Development Assistance (ODA) funding to develop projects in and for low and middle-income countries (LMICs), with the aim of contributing to a ‘world safe and secure from infectious disease threats and promotion of Global Health as an international security priority’. This accounts for 34% of total Department of Health and Social Care (DHSC) ODA funding. The programme is made up of 5 projects:

• Fleming Fund
• Global Antimicrobial Resistance Innovation Fund (GAMRIF)
• UK Public Health Rapid Support Team
• International Health Regulations Strengthening project
• UK Vaccine Network project

Through delivery of each of these projects the programme aims to support ODA-eligible countries to:

• prevent and reduce the likelihood of public emergencies such as disease outbreaks and antimicrobial resistance (AMR)
• detect health threats early to save lives
• provide rapid and effective response to health threats.

Outline of project

The Fleming Fund is an investment by the UK government of up to £265 million ODA to tackle the growing global threat of AMR through the DHSC. AMR if left unchecked will result in a pronounced increase in extreme poverty. Of the additional 28.3 million people falling into extreme poverty in 2050 in a high-impact AMR scenario, the vast majority (26.2 million) would live in low-income countries. Currently, the world is broadly on track to eliminate extreme poverty (at $1.90/day) by 2030, reaching close to the target of less than 3% of people living in extreme poverty. AMR risks putting this target out of reach.

There will be an additional and substantial impact on global ability to meet the third Sustainable Development Goal (SDG) to ensure healthy lives and promote well-being for all if current trends in the development of AMR continue. Global AMR-related deaths are predicted to rise to 10 million by 2050, with 89% of all AMR deaths occurring in Africa and Asia. In particular, common infections will become complex and expensive to treat, with consequences for the functioning of health systems. This burden on health systems and services for care and prevention threatens in particular the achievement of SDG 3 indicator 8 on Universal Health Coverage, of which ensuring access to effective antimicrobials at the appropriate dose is a critical part.

By 2021, the Fleming Fund will have helped up to 24 LMICs to establish the foundations of sustainable surveillance systems for AMR and antimicrobial usage (AMU) through a portfolio of country grants, regional grants, and fellowships. We also provide support to a significant number of additional LMICs through global grants.

Expected outcomes are an increase in relevant high-quality data that is shared nationally and globally and can provide the basis for changes in policy and practice to increase the rationale use of antimicrobial medicines and reduce the number of drug resistant infections.

We aim to improve laboratory capacity and diagnosis as well as data and surveillance of AMR at a country level through a ‘One Health’ approach, covering human health, animal health and agriculture. By supporting countries in South and South-East Asia and sub-Saharan Africa to develop One Health AMR National Action Plans (NAPs) and implement the surveillance aspects of these, we support delivery of objectives from the 2015 World Health Assembly Global Action Plan, the O’Neill Review and the Inter-Agency Coordination Group on AMR recommendations as well as the UK’s own NAP. Contributions to these key international objectives and outcomes are captured in the Theory of Change (ToC), but in particular Fleming Fund outputs contribute to the following Global Action Plan outcomes:

• improved awareness and understanding of AMR
• strengthened knowledge through surveillance and research
• ensuring sustainable investment in countering AMR
• optimising the use of antibiotics

In addition, the Fleming Fund also makes an important contribution to broader work on health systems strengthening in LMICs, particularly by supporting improvements to diagnosis and data which contribute to improved health information systems and help to ensure that essential antimicrobial medicines are safe and effective. The ToC includes an outcome around strong, resilient and integrated health systems with antimicrobial stewardship (AMS) practices embedded.

Outline summary of programme

The table below shows the ‘red, amber, green’ (RAG) ratings for the programme in 2018 and 2019.

	RAG rating 2018	RAG rating 2019
Project management	A/G	A/G
Finance	A	A/R
Theory of Change	A/G	A/G
External engagement	A/G	A/G
Overall delivery confidence	A/G	A

Summarised key recommendations from the previous review

The following recommendations were made and accepted by the programme board at the last annual review:

Recommendations 1

RAG rating: A/G

Develop clear medium- and longer-term objectives for the Fleming Fund and a strategy to achieve these.

Be clear about the Fleming Fund’s comparative advantage and focus and how its contribution will complement wider efforts on AMR.

Extend the current timeframe for the Fleming Fund to 2030 and develop a longer-term vision and a supporting Business Case.

Develop a revised TOC that better reflects project objectives, strategy, and assumptions, including what needs to change and who needs to be influenced.

Develop a Fleming Fund logical framework for the current project timeframe, to enable the DHSC to monitor overall progress

Share the strategy, TOC, and logical framework with Fleming Fund implementing partners to ensure there is a common understanding and that partners see where their contribution fits into the bigger picture.

Actions taken to date to address recommendation 1

A 10-year strategy has been developed to set out a longer-term vision for the Fleming Fund under a second phase. This responds directly to the recommendation from the 2018 Annual Review recognising that there is a need for a longer-term programme, based on evidence of the time required to embed sustainable surveillance systems and in order to realise the full benefits of investments to date and have a more sustainable impact on the management of AMR in the countries supported. Alongside the 10-year strategy a 3 to 5-year Business Case is being developed with the aim of securing further funding from 2021/22.

The Fleming Fund ToC was revised in mid-2019 to better reflect project objectives, strategy, assumptions and pathways. The revised version was discussed with delivery partners in a webinar and again in more detail at the annual meeting in November 2019 to support partners to understand where their project activity contributes to the wider programme. Delivery partners had the opportunity to challenge the ToC assumptions and pathways and support further revisions. The next step will be to develop a ToC narrative to further understand the assumptions and pathways for change, and whether any of these are changing over time. This process is being supported by the Evaluation Supplier (ES).

Recommendation 2

RAG rating: A

Develop clear medium- and longer-term objectives for the Fleming Fund and a strategy to achieve these.

Be clear about the Fleming Fund’s comparative advantage and focus and how its contribution will complement wider efforts on AMR.

Extend the current timeframe for the Fleming Fund to 2030 and develop a longer-term vision and a supporting Business Case.

Develop a revised TOC that better reflects project objectives, strategy, and assumptions, including what needs to change and who needs to be influenced.

Develop a Fleming Fund logical framework for the current project timeframe, to enable the DHSC to monitor overall progress

Share the strategy, TOC, and logical framework with Fleming Fund implementing partners to ensure there is a common understanding and that partners see where their contribution fits into the bigger picture.

Actions taken to date to address recommendation 2

The following activities, meetings and initiatives took place over 2019 to strengthen coordination across grantees, Delivery Partners, Cross-HMG and Development Partners.

Grantees achieve these.

Ensure coordination of communities of practice established by fellowship host institutions and with other initiatives, e.g. WHO’s online community of practice.

Monthly meetings established between the Regional Grants "Round One" and the Global Research on AMR (GRAM) project to support greater synergies and efficiencies in data collection activities across Asia and Africa.

Six monthly HMG AMR focal point teleconferences held with posts in South East Asia, South Asia, West Africa and East and Southern Africa.

Fleming Fund one pagers developed for all 24 countries setting out the full range of Fleming Fund investments, allowing partners a view on country-wide investments.

Delivery partners

A Delivery Partners portal has been established to support information sharing, coordination and collaboration between all Delivery Partners with accompanying monthly seminars.

Strengthening coordination was a major theme of the 2019 Delivery Partners Event – including the tripartite, HMG and country grantees/beneficiaries.

A comprehensive review of country coordination was conducted drawing on findings from ITAD, the Management Agent (MA) and Department for International Development (DFID)/Foreign and Commonwealth Office (FCO). Plans are in place to implement the findings and recommendations of the review in February 2020.

Development Partners Key donor coordination meetings attended by the Fleming Fund Team including:

• Association of Southeast Asian Nations (ASEAN) – antimicrobial stewardship seminar
• Donor Coordination Meeting - Bangkok
• Second Ministerial Meeting on AMR - Netherlands

Recommendation 3

RAG rating: A

Strengthen Fleming Fund coordination and alignment.

Assess the scope for greater synergies between regional and country grants, and between other Fleming Fund investments, e.g. support for the tripartite organisations, and country grants.

Clarify responsibility for coordination of Fleming Fund partners and activities at country level, within the Fund, with other Her Majesty’s Government UK (HMG) investments, and with wider development partner support.

Ensure coordination of communities of practice established by fellowship host institutions and with other initiatives, e.g. WHO’s online community of practice.

Consider how the Fund can complement the work of other DHSC and HMG actors to strengthen coordination with other global and bilateral actors in the AMR arena.

Actions taken to date to address recommendation 3

Following delays in the first 2 years of implementation and the recommendations from the 2018 Annual Review and first formative evaluation report, DHSC and the MA agreed to make changes to the design of the country grant round 2 process. Where grantees are performing well under the first country grants there will be an opportunity to extend the grant, or to directly award a new grant rather than going through open competition. This approach recognises that the market has already been tested and grantees identified as the most suitable delivery partner within a country or region, which will maximise the time available for implementation. Second country grants will focus explicitly on improving the sustainability of investments.

The professional fellowships programme has grown significantly during 2019 after initial delays in implementation. The approach and objectives of the initial workshop at the start of fellowships has supported a common understanding between fellows, host institutions, beneficiary institutions and country grantees as well as improved alignment between fellowship and country grant objectives. Roll-out of policy fellows was paused over 2019 to prioritise delivery of professional fellows. This will now commence in 2020 and will focus on supporting policy and advocacy in-country and supporting fellows to think about how to use the data being generated through country grants.

The grants to the tripartite organisations (United Nations Food and Agriculture Organisation (FAO), World Organisation for Animal Health, OIE and World Health Organisation, WHO) have been extended beyond 2019 in recognition of the continued need to support the development of global guidance and protocols on AMR and AMU/C surveillance. These grants also support countries to develop and implement NAPs and establish AMR governance structures which directly contribute to creating the country enabling environment for country grants as set out in the updated ToC.

A specific grant to strengthen research into the economic costs of AMR globally has not been taken forward but the Fleming Fund is represented on the expert panel for WHO work to develop a tool to support countries to cost and budget for AMR NAPs and the importance of this work has been reflected in the 10 year strategic plan and 3 year business case.

Recommendation 4

RAG rating: A/G

Improve the efficiency and impact of the grant portfolio.

Agree a more efficient grant process for country grants (e.g. one longer grant, follow-on grants, direct awards) to maximise the time available for implementation.

Strengthen the focus of country grants on integrated approaches to surveillance and planning for sustainability and ensure that the sustainability strategy is implemented.

Promote country networking, to enable countries with weaker commitment and structures to learn from countries that have made greater progress; opportunities to do this through Round 2 of the regional grants should be explored.

Ensure there is a common understanding of the objectives of the fellowship scheme and that fellowships are aligned with country needs.

Consider how the fellowships can complement other areas of Fleming Fund activity, including country grants and policy and advocacy work, and specifically, how the economic fellowships can strengthen the evidence base and the case for investment in action to tackle AMR.

Consider an economic grant to strengthen research into the economic costs of AMR globally, regionally, and at country level, or engage with the World Bank or other partners that have a comparative advantage in this area.

Extend funding for the tripartite organisations to consolidate gains to date and provide future support for the implementation of NAPs and for the use of data, with funding linked to clear deliverables.

Actions taken to date to address recommendation 4

2019 has seen significant communication activity and the dissemination of learning across the Fleming Fund. A partner portal has been established to enable implementing partners to network and share information directly with each other. This has promoted greater transparency and openness and has supported improved partner alignment. DHSC and the MA have also developed several documents and tools, including project and country one pagers, to further aid a shared understanding of the programme and support opportunities for collaboration.

The programme has a comprehensive communications strategy covering internal and external communications and has worked with the MA to develop core messages. These core messages and the programme’s communication approach has been shared with partners through the partner portal and through sessions at the Delivery Partners event. A series of webinars with delivery partners will be arranged in 2020 including sessions on communication recognising that the communications strategy and key messages will need revising as the programme evolves.

Itad, as both the evaluation supplier and learning partner of the Fleming Fund have supported DHSC to develop a dissemination plan to ensure that learning from the programme is shared with partners and other AMR actors. There was a dedicated workshop session at the delivery partners meeting to support partners to think about how they can better capture and share learning.

The first grant to the Open University ended in 2019. A second grant proposal was shared in late 2019 and has now been approved by DHSC. This will be closely managed to ensure that it aligns with other global AMR learning initiatives.

In 2019 elements of the website were updated following user feedback and changes to the programme. Now that the majority of country grants are in place, the website will be more comprehensively restructured in early 2020 to better reflect the wider programme portfolio and to ensure that it remains fit for purpose throughout the lifecycle of the programme.

The programme twitter account and publications like the Petridish are being used to improve communication to international and national stakeholders. Country grant launches have also been used to effectively communicate messages about country grant activities to national stakeholders as seen with the Pakistan launch in September 2019.

Key successes

Communications

Strengthened visibility of the Fleming Fund and UK leadership on AMR through high profile grant launches at the United Nations General Assembly 2019, a media launch in Pakistan and Delivery Partners Event in Laos. These complemented ongoing strategic communications comprising the Fund’s newsletters, bulletins,website and social media. In early 2020 we will see further grant launches in Uganda and Nigeria.

Country grants

• 18 country grants active at end December 2019, up from 4 in January 2019
• 14 Fleming Fund countries enrolled in Global AMR Surveillance System (GLASS) with 7 countries reporting data
• 250 laboratories assessed and 132 supported

Fellowships

• 82 professional fellows selected, and 13 workshops completed
• 45% of scientific fellows working in human or animal health are women
• 3 health economists placed for cohort one of the Overseas Development Institute (ODI) fellowships in Thailand and Nigeria
• 2 of 3 ODI health economist fellows for cohort one are women

Regional grants

• 9 regional grants active
• 26 countries targeted by round one regional grants focusing on historical AMR data collection, analysis and publication

Tripartite Grants (WHO, FAO, OIE)

• tailored support given to over 75 LMICs in Asia and Sub-Saharan Africa to develop and implement AMR NAPs
• 12 countries given additional support to develop One Health NAPS
• WHO’s Tricycle One Health AMR surveillance protocol developed and piloted in 4 countries, with successful results and plans for wider roll out
• WHO AWaRe classification for antibiotics released in June 2019. It includes details of 180 antibiotics classified as Access, Watch or Reserve and has had a high level of uptake.
• quality of medicines surveys completed in Sierra Leone, Ghana, Nigeria and Uganda
• quality of medicines smartphone reporting app piloted in Tanzania and Indonesia
• FAO Assessment Tool for Laboratory and Antimicrobial Resistance (ATLASS) assessments provided recommendations to strengthen AMR surveillance systems in 11 countries
• OIE third annual report on antimicrobial agents intended for use in animals published February 2019. 155 countries responded with 118 providing data, significantly more than the first and second reports.

Commonwealth Partnerships for Antimicrobial Stewardship (CWPAMS)

• 12 partnerships active between UK NHS Trusts and hospitals in Zambia, Uganda, Ghana and Tanzania, with 16 Global Health Pharmacy fellows additionally being supported. Major learning events planned for Spring in four countries to explore and share findings from the projects
• MicroGuide App launched to provide easy access to information that is vital to use antimicrobials appropriately. It includes national treatment guidelines for Ghana, Tanzania, Uganda and Zambia, and uses the AWaRE categorisation

FAO Reference Centre for AMR

• the UK-based International AMR Reference Centre for Animal Health and Agriculture received designation as an FAO centre in April 2019
• missions to support in-country capacity undertaken in Bangladesh, Ghana, Laos, Nigeria and Vietnam

Foundation for Innovative New Diagnostics (FIND)

• landscape analysis underway in 15 countries to assess an in-country tracking and reporting mechanism for substandard and falsified medicines across Africa and Asia
• paper published on the field evaluation of the MedSnap medication authentication smartphone application in Laos

Global Research on AMR Project – Institute for Health Metrics and Evaluation

• 6 million isolates identified across 145 countries ready for incorporation into the Global Burden of AMR analysis
• 47 data sharing agreements finalised or agreed in principle

South Centre – civil society engagement

• regional AMR event in Nairobi attended by AMR focal points and representatives from 27 countries to share learning between countries on development and implementation of NAPs
• strengthened AMR focal points knowledge and collaboration in Africa and Asia through regional workshops and the sharing of national action plan implementation experiences
• strengthened civil society engagement with global AMR process through the support and participation in the activities of the Antibiotic Resistance Coalition (ARC)
• supported awareness and participation of developing country delegations on global health governance processes related to AMR at the WHO and the UN

Project management

Delivery assessment for reporting year

RAG rating for this reporting year: Amber/Green

Changed since last year (Yes/No): No

1. Evidence of managing the delivery of project

Quarter 1	Quarter 2	Quarter 3	Quarter 4
A/G	A/G	A/G	A/G

Overall delivery RAG rating over the reporting period: A/G

The Fleming Fund project team commenced the year with monthly reviews of delivery performance for each project in the portfolio until April 2019. At this point a review was carried out by the team to divide the delivery category into quality and timeliness to better distinguish drivers of performance and challenge and better drive improvements. This approach was intended to provide more detail on delivery and was approved by the Project Board in June 2019. It was also agreed that the team would report every 2 months as the Board were content with the consistent standard of reporting and recognised the significant time and resource required by the team to complete monthly reporting. All delivery partners continue to report into DHSC monthly, quarterly or 6 monthly according to their governance agreements.

An overall amber green picture for delivery (quality and timeliness) for all Fleming Fund projects reflects the fact that most projects are performing well, with some small fluctuations in individual project performance over the year resulting in mitigation to bring scores back to amber green. The Mott Macdonald portfolio has consistently scored more poorly on timeliness. However, these delays reflect the complexity of setting up new the country grants and fellowships and ensuring country ownership and sustainability; ensuring investments made as part of the Fleming Fund programme are maintained beyond programme closure.

2. Evidence that the project is meeting the agreed milestones and deliverables

The following milestones were delivered by Fleming Fund for this reporting year. For each output, we will summarise whether the activities were completed as planned and indicate if expected results were achieved.

If an agreed milestone/deliverable has not been achieved, provide a brief explanation as to why, and provide details of current status or actions that are still required. (e.g. what is the new deadline, what has been done to resolve the issue and are there any critical dependencies or issues that you should flag for the attention of programme board).

A 5-point score (A++ to C) has been used. Further explanation of this scoring system can be found in Annex 1.

Overall Fleming Fund Programme score A

Output Indicator	Milestones / deliverables	Current status	5-point scores
1.1	Number of country grant request for proposals (RFP) published cumulatively Year 2 Milestone: 20	22	A+
1.2	Number of country grants active[footnote 1] Year 2 Milestone: 16	16	A
1.3	Fleming Fund projects achieving green or amber green rag rating for quality, timeliness and finance on average across the year Year 2 Milestone: 70%	56%	B
1.4	Number of Fleming Fund supported Human Health surveillance sites showing progress through the London School of Hygiene and Tropical Medicine (LSHTM) roadmap functions and stages.[footnote 2] Year 2 Milestone: 60%	70% (23/33)	A+
1.5	Number of regional grant planning/’kick off’ workshops undertaken Year 2 Milestone: 9	9	A
1.6	Number of countries supported to undertake ATLASS assessments by the Fleming Fund Year 2 Milestone: 12	12	A

Output 1 key points

On track with publishing RFPs and the majority of country grants are now up and running. The MA have experienced some challenges with identifying grantees and placing grants in West Africa and South Asia but are looking at alternative approaches to address this.

At 56% the number of Fleming Fund projects achieving green of amber green RAG rating for delivery on average across the year is under the target of 70%. The ratings for timeliness were particularly low bringing down the average of the quality and finance ratings. This in part reflects the early stages of many of the complementary grants which were only established in 2018/19. Efforts need to be made in 2020 to improve the timeliness and finance rag ratings of Fleming Fund grants.

Output 2: Theory of change output area: Standardisation of data/quality of surveillance/quality improvement Score A+

The primary output of the Fleming Fund is the building and improvement of One Health AMR laboratory capacity and surveillance systems. This includes establishing a National AMR reference centre and developing protocols for sharing and disseminating AMR data across a nationwide network and then ensuring that these protocols are used to improve the quality of the surveillance data generated.

Output Indicator	Milestones/deliverables	Current status	5-point score
2.1	Number of Fleming Fund supported protocols and guidance implemented in countries. Year 2 Milestone: 5	5	A
2.2	% of LMIC hospitals rolling out protocols and guidance for antimicrobial stewardship (AMS) practice (rolling out meaning available for use). Year 2 Milestone: AMS policies rolled out in 80% of target hospitals	13 AMS protocols, policies or guidelines developed across a total of 14 institutions (93%)	A++
2.3	Number of countries implementing Tricycle using Fleming Fund funding. Year 2 Milestone: 4	4	A

Output 2 key points

There has been progress in the standardisation of data/quality of surveillance/quality improvement with 5 protocols being supported since the start of the programme. These include: LSHTM Roadmap, the Tricycle Protocol, OIE questionnaire-based protocol for collection of national animal health AMU data, WHO protocol for collection of national human health AMU data, and the WHO point prevalence protocol for collection of data on antimicrobial consumption in hospitals.

The programme has supported the implementation of the Tricycle protocol in 4 countries: Ghana, Pakistan, Malaysia and Indonesia. A number of other countries are now implementing tricycle funded by other donors, some countries may look to implement tricycle through Fleming Fund country grant activity.

Output 3: Theory of change output area: Strengthening capacity and workforce on AMR Score B

Alongside the development of laboratories, the Fleming Fund considers the professional development of in-country staff a key requirement in achieving intended outcomes. Technical capacity comes in many forms including: microbiologists, veterinarians, pharmacists, clinicians, nurses and health economists. By upskilling in-country staff and providing them with the right training, equipment and systems, countries will be able to gather, analyse and share AMR data.

Output Indicator	Milestones / deliverables	Current status	5-point score
3.1	Number of ODI Fellows placed Year 2 Milestone: 3	3	A
3.2	Number of professional Fellows selected[footnote 3] Year 2 Milestone: 106 professional fellows selected	82 professional fellows selected	C
3.3	Number of countries with active[footnote 4] Professional Fellows in place Year 2 Milestone: 12	4	C
3.4	Number of online Open University modules available Year 2 Milestone: Piloting Phase 1 report submitted to inform Phase 2 [Interim milestone]	Not applicable until 2020	N/A
3.5	Number of clinical staff with improved AMS knowledge Year 2 Milestone: 240 clinical staff	626 clinical staff	A++
3.6	Number of partner LMIC institutions visited by NHS staff LMIC institutions Year 2 Milestone: All UK institutions visited LMIC partner with NHS staff	All institutions have now been visited by NHS volunteers	A

Output 3 key points

There are notable delays in delivery of the professional fellowships evident in the failure to meet 2019 targets. 82 fellows have been selected and 13 workshops have been held and this is below the target of 106 fellows and 17 fellowship workshops. There has also been far fewer workplans finalised with 24 workplans finalised across 4 countries rather than 80 in 12 countries. Delays to workshops have in many cases been due to security issues in-country or challenges in scheduling caused by the need to identify times that work for fellows, beneficiary institutions and host institution mentors. The professional fellowships model is designed to provide bespoke, tailored mentoring and respects country ownership. With the recruitment of additional regional staff to increase MM capacity to deliver fellows there have been improvements in the rate of delivery of this element of the programme. With cohort 2 professional fellows and policy fellows being rolled out in 2020 it will be important for MM and DHSC to monitor progress closely and ensure contingency plans are in place to mitigate further delays.

All UK institutions have visited LMIC partners as part of the CWPAMS project. This indicator will be updated to track actual number of NHS volunteer days next year.

Output 4: Theory of change output area: Lab equipment and assessment Score A+

The primary output of the Fleming Fund is the building and development of one-health AMR laboratory capacity and surveillance systems. Alongside the strengthening the AMR workforce, this also requires appropriate laboratory equipment to be in place and for labs to be assessed and supported. The Fleming Fund is supporting countries to establish and strengthen National AMR reference centres and the surveillance sites that report data into these centres. We note performance against milestone 4.1 significantly exceeds the target. We expect future annual reviews will have a more granular and potentially stretching target in relation to the support afforded laboratories

Output Indicator	Milestones / deliverables	Current status	5-point score
4.1	Number of labs supported Year 2 Milestone: 67	132	A++
4.2	Number of labs assessed Year 2 Milestone: 250	250	A

Output 4 key points

The programme has surpassed the 2019 lab equipment and assessment milestone targets set for this reporting year, demonstrating the significant progress that has been made in this area. Double the number of laboratories are being supported than predicted with 132 sites already receiving active support through country grants to date. 2019 targets were based on estimates and an assumption that country grantees would not start work in all sites at the beginning of a country grant, whereas in practice grantees have begun some support immediately in all sites. Next year’s milestone will reflect this.

Output 5: Theory of change output area: Governance Score C

The development and sustainability of AMR surveillance networks in-country is dependent on strong AMR governance and leadership. In most instances, this takes the form of an AMR Coordinating Committee (AMRCC) which is chaired by a senior leader in the Ministry of Health (MoH) with representatives across all AMR relevant sectors. AMRCCs often also have technical working groups leading on specific objectives within the Global Action Plan on AMR. The Fleming Fund aims to engage both the AMRCC and the surveillance technical working group in-country to ensure country ownership and sustainability of Fleming funded activity. In establishing these governance and leadership mechanisms, countries are demonstrating their commitment to tackling AMR.

Output Indicator	Milestones / deliverables	Current status	5-point score
5.1	Number of countries with functioning surveillance technical working groups meeting at least once over the 2019 calendar year Year 2 Milestone: 16	7	C

Output 5 key points

It was hoped that as many countries with active country grants would also have active surveillance technical working groups, but this milestone was over-ambitious. Many countries which were approved towards the end of the reporting period did not have working groups in place, so it has been a challenge for the MA to seek agreement on attendance and follow national processes to support the creation of new working groups. The Fleming Fund are considering further support to AMRCC as part of the second round of country grants, which may help increase performance against this milestone.

Output 6: Theory of change output area: Antimicrobial Consumption (AMC)/Antimicrobial Use (AMU) Data Score A

The Fleming Fund’s main aim is to support the generation of high-quality data across human health and animal health sectors, this includes antimicrobial use (AMU) and consumption (AMC) data. The generation of AMU and AMC data refers to the monitoring and recording of drug production, import and prescription, and consumption patterns. The Fleming Fund aims to collect this data through surveillance networks across both human and animal health sectors. Standardised protocols such as Point Prevalence Survey and the OIE database on antimicrobial agents intended for use animals will be used to collect this data.

Output Indicator	Milestones / deliverables	Current status	5-point score
6.1	Number of countries contributing to OIE database Year 2 Milestone: 155	155	A
6.2	Number of countries reporting quantitative data to OIE database Year 2 Milestone: 118 (countries submit data with quantities of anti-micro agents to OIE)	118	A
6.3	WHO produce global report on AMR/AMU consumption using country data Year 2 Milestone: WHO produce first AMR/AMU report [Interim milestone]	Yes	A

Output 6 key points

There has been good progress with programme supported activity on the generation and sharing of AMC/AMU data across both the human and animal health sectors. The OIE has been publishing data on antimicrobial agents intended for use in animals since 2016 but with the Fleming Fund support the number of countries providing data and the quality of the data shared has significantly improved. The Fleming Fund is now supporting the OIE to update the database to an automated rather than manual system which will better enable countries to analysis and interpret data and use this to inform policy and practice changes to tackle AMR. The WHO published their first global report on AMR/AMU consumption with Fleming Fund support and have continued to train national representatives to collect and share this data locally, national and globally.

Output 7: Theory of change output area: Substandard and Falsified (SF) data Score B

The Fleming Fund’s main aim is to support the generation of high-quality data across human health and animal health sectors, this includes data on Substandard and Falsified (SF) medical products. A very direct relationship exists between AMR surveillance and data on quality of medicines. With improved treatment outcomes, a treatment failure should be a signal for investigation both for resistance, and for substandard or falsified medicines. Data on SF medicines will enable countries to better understand the quality of medicines in their markets and will inform an improved understanding of the prevalence of AMR and how it can best be tackled.

Output Indicator	Milestones / deliverables	Current status	5-point score
7.1	Number of countries undertaking quality surveys for SF medicines supported by Fleming Fund Year 2 Milestone: 4	4	A
7.2	Evaluation report of field screening technologies completed Year 2 Milestone: Authentic and falsified Library database established [Interim milestone]	No	C

Output 7 key points

Progress in programme supported activity on substandard and falsified (SF) medicines has been as expected under the WHO grant where Sierra Leone, Ghana, Nigeria, Uganda have been supported to undertake quality surveys for SF medicines to capture the level of SF antibiotics in these countries and contributing towards a better understanding of the impact this is having on AMR. Funding in this area has been leveraged with the same study also conducted in Togo and Benin using other funding.

There have been delays in the delivery of activity in the FIND SF project where an authentic and falsified library database has been established for one of the two antibiotics. This is due to contracting and recruitment delays.

Output 8: Theory of change output area: AMR^{[footnote 5]} data Score A+

The Fleming Fund’s main aim is to support the generation of high-quality data across human health and animal health sectors, this includes antimicrobial resistance (AMR) data. The generation of AMR data refers to the testing and recording of samples using standardised protocols for sample collection and using best practice testing processes.

Output Indicator	Milestones / deliverables	Current status	5-point score
8.1	Number of Fleming Fund countries submitting data into WHO Global Antimicrobial Resistance Surveillance System (GLASS) Year 2 Milestone: 7	7	A
8.2	% of Fleming Fund supported countries submitting improved data into GLASS[footnote 6] Year 2 Milestone: 40%	80%	A++

Output 8 key points

There has been good progress with programme supported activity on the generation and sharing of AMR data. The number of Fleming Fund supported countries reporting data has increased and the quality of this data is also improving. Much of this is a direct result of the country grant activity which has resulted in a number of countries sharing data to GLASS for the first time. Supporting countries to generate and share this data nationally and globally will continue to be a priority for the Fleming Fund.

Output 9: Theory of change output area: Burden data Score A

AMR burden data is vital to understanding the human cost and economic impact of AMR. This is particularly important when trying to develop the case for investment in AMR. The quantity and quality of AMR burden data is currently inadequate, as are the analytical frameworks available to analyse the burden of AMR.

Output Indicator	Milestones / deliverables	Current status	5-point score
9.1	Increase in historical data on AMR burden published Year 2 Milestone: 4 publications submitted for publication in 2019	4	A

Output 9 key points

The GRAM project is the core source of burden data for the Fleming Fund. There has been good progress over 2019 in the finalisation of an analytical model to calculate the global burden of AMR. A large number of collaboration agreements have been established and data from multiple sources/geographies is starting to flow into the project, increasing the likelihood of a reliable estimate of burden. Four publications were submitted to major journals, laying the methodology and groundwork for the future AMR burden publication.

Output 10: Theory of change output area: Awareness and advocacy Score A

Given the complexity of AMR, with a number of sectors and factors both contributing to the threat and to the solution, there is still a lot of uncertainty around the problem, its severity and the most appropriate interventions to reduce the acceleration of resistance. The Fleming Fund recognise improving awareness and understanding of the threat is crucial to ensure countries are convinced of the need for action.

Output Indicator	Milestones / deliverables	Current status	5-point score
10.1	Number of news stories published on website Year 2 Milestone: 20	12	C
10.2	Number of articles published through the South Centre Year 2 Milestone: 4	8	A++
10.3	Number of Monitoring & Evaluation (M&E) meetings held between South Centre and WHO Year 2 Milestone: 2	3	A+

Output 10 key points

The broad range of activities, articles and news stories undertaken by the South Centre this year have contributed to the advancement of the global AMR agenda and helped to shape global processes aimed at tackling AMR in LMICs. The South Centre have been active in raising awareness from key perspectives including developing countries and Civil Society Organisations and ensuring their views are taken into account on the issue of AMR.

3. Evidence of risk management

The following risks were the top 2 risks identified by the Fleming Fund during this reporting year as part of the Fleming Fund portfolio Risk Register. This is monitored regularly and reviewed every 2 months by the Fleming Fund Project Board. The Fleming Fund also has a joint Risk Register with the MA and a country risk register, which is also shared with the Project Board.

	Risk	Mitigation Actions	RAG rating (Residual)	Current Status / Update
1	Underspend on project forecasts and HMT profile in 19/20 Please note this risk covers the Financial Year April 2019 to March 2020	Monthly financial and forecasting meetings with the MA More realistic and conservative April baseline forecast which takes into account overall likely underspend within the existing lifespan of the MA contract Close work with the rest of GHS to plan for any underspend being absorbed by other programmes Planning for Spending Review Managing risk of underspend by risk-adjustment of forecasts and managing this across GHS with other programmes forecasting overspends	A/R	This risk has been rolled into wider finance risk for the GHS programme board. There was a £7m reduction in December forecasts (which was largely anticipated for as part of the risk adjustments). There was then a further £9.6m reduction in January forecasts following receipt of the grantee quarterly reports, which indicated much slower spend than MA anticipated. However, the latter point is outside of this Annual Review period.
2	Roll out of Fleming Fund country and regional grants and fellowships are delayed beyond sequence outlined in their work plan, compromising achievement of results set out in the MM Implementation Plan	Weekly management meetings with MA to support prioritisation and problem shooting On-going work to press for improvements to grant making times including starting EY grant assurance earlier and streamlining approvals for fellowships Increased engagement including Fleming Fund country visits and/or international calls to design solutions in countries including Myanmar and India DHSC signing Memorandum of Understandings (MoUs) and LoEs with select countries where this will expedite project progress (Laos, Nepal, Sri Lanka, Pakistan, Indonesia, Senegal) Agreement on non-competitive approach to Country Grant 2s where this is the quicker, most effective and value for money (VfM) option More realistic and conservative workplan revised to reflect changes to grant timings, and some countries only having 1 instead of 2 country grants	A/R	This risk was also reported as an issue. Fleming Fund now active in all 24 priority countries, but 4 country grants are more complicated than predicted and are still pending. We are looking at different approaches in all these countries but can no longer expect them to be agreed this financial year. Discussion on a no cost extension has confirmed that MA can use savings and underspend on management costs for 5 months of additional delivery at no extra cost.

Please summarise any risks during the reporting year that have materialised as issues, provide a brief explanation as to why, and provide details of current status or actions that are still required.

	Risk	Mitigation Actions	RAG rating (Residual)	Current Status / Update
5	An insufficient number of eligible grantees are identified (for country and regional grants) that meet the rigorous selection criteria for inclusion.	MA encouraged to share RFP documents as they go live so that DHSC can help promote MA encouraged to consider appropriate mechanisms of outreach and to improve quality of applications by considering longer lead in times and improved market shaping work DHSC agreement to consideration of direct awards where competitive process has failed to mitigate impact on results and time Consideration of targeting existing surveillance projects where increased VfM/impacts would justify Fleming Fund. investment.	A/G	Previously rated as an issue. DHSC now seeing a healthy level of applications for many of the country grants but the recent regional grant round 2 resulted in only 3/8 awards being made and still low numbers of fellowship grants. We have yet to see a country grant where the mitigations did not afford a way forward.

Risks 1 and 2 in the table above were highlighted as issues during 2019. Both risks are reviewed regularly by the Fleming Fund team and the mitigations in place resulted in a reduction from a Red rating to an Amber/Red rating. Risk 1 was rolled into the wider GHS programme board financial risk register for further monitoring. Risk 5 was also raised to an issue for a couple of months, until mitigation actions addressed this issue and it was de-escalated as a potential risk.

4. Safeguarding

Please detail and highlight any changes or improvements you have made in the past year to ensure safeguarding policies and processes are in place in your project and your downstream partners.

All new contracts and grants issued this year have included the latest HMG safeguarding clauses. Letters were sent to all Fleming Fund Delivery Partners in August 2019, highlighting the importance of safeguarding processes and policies and that these would be considered as part of the Annual Review process and during due diligence checks for any new contracts or grants. The Fleming Fund team gave a presentation on Safeguarding followed by Q&A at the Delivery Partners Event in Laos in November. Further information was shared via the Partners Portal. As part of the Annual Review safeguarding due diligence checks, 9 out of 11 Delivery Partners submitted their self-assessments on time.

Please summarise any safeguarding risks that have arisen during the reporting year and provide details of current status or actions that are still required.

A general safeguarding risk is monitored monthly on the Fleming Fund portfolio risk register and individual risks are highlighted more broadly in the Fleming Fund country risk register.

Summary of risk management recommendations for improvement

Recommendation: The Fleming Fund portfolio risk register currently uses a 5 by 5 risk matrix for likelihood and impact. The GHS Programme Board have received approval for their revised risk strategy which uses a 4 by 4 risk matrix. The Fleming Fund will review the difference and consider aligning.

Finance

Delivery confidence assessment for reporting year

RAG rating for this reporting year: Amber/Red

Changed since last year (Yes/No): Yes

5. How is the funding being used?

Annual summary

Total annual budget for this reporting year (2019): £66,000,000

Total annual spend for reporting year: £41,865,571

Did you meet your budget this year? If no, please provide a short summary of why your budget has not been met in the space below.

The Fleming Fund budget is allocated by HM Treasury on a financial year basis, so the annual budget for 2019 of £66m is an estimated amount consisting of £12.75m for Q4 2018/19 and £53.25m for Q1 to Q3 2019/20. The Fleming Fund project realised an underspend against 2018/19 budgets of £33.8m and is under-profiled for 2019/20 in both risk-adjusted (-£17.4m) and non-risk-adjusted (-£12.4m) forecasted positions. Annual spend for Q4 2018/19 and Q1-Q3 2019/20 activities (on an accruals basis) was £41m.

There has been slower than anticipated implementation of country grants and overly optimistic forecasts provided by grantees under the MA portfolio, which has resulted in a large underspend now being expected against 2019/20 budgets. In terms of the country grant portfolio to date, there has been very little in the way of data to analyse. However, now that there are 18 country grants in progress and grantees are starting to submit their quarterly progress reports, this should enable both the MA and the Fleming Fund team to more accurately scrutinise and risk-adjust grantee spending profiles, and to anticipate delays in negotiating agreements to ensure more realistic start dates are profiled.

There have also been underspends realised elsewhere across the Fleming Fund portfolio – primarily with the International Reference Centre, which has experienced delays so spend has shifted into the latter part of the financial year, and expenditure under the Institute for Health Metrics and Evaluation (IHME)/GRAM grant is similarly not accelerating as quickly as initially anticipated.

If there are any changes to the financial and/or spending plans that were in place at the start of the reporting period, please explain below.

There has been a significant reduction in the forecasted position provided by the MA at the beginning of 2019/20 due to slower than anticipated implementation of country grants. Baseline forecasts at the beginning of 2019/20 were £61m, and the current outturn position is now £42m.

There has also been sizeable underspend by the IHME/GRAM activity against their 2019/20 grant award. Unattainable forecasts were provided at the start of the year, which has subsequently resulted in underspends being realised. As a result of this, the Fleming Fund team have reverted to paying this grantee based on their actual expenditure, rather than on a set spending profile as originally agreed. This ensures that ODA payment in advance of need rules are adhered to.

The International AMR Reference Centre received additional non-Fleming Fund funding in 2019. This was used to deliver planned and additional activities in 2019. This led to lower than anticipated spend of FF funding which has meant that the payment of just over £0.5m slipped from 2019, and is now expected to be made in March 2020.

6. Evidence of ability to administer ODA funding

Outline any process changes to finance reporting and monitoring to assure ODA eligibility

The ODA eligibility of all expenditure is established at the outset of the programme.

Country selection: All beneficiary countries of the Fleming Fund are included on the OECD DAC list of ODA eligible countries; the majority of Fleming Fund partner countries are lower income countries.

Aid Purpose: Fleming Fund activities are reviewed and confirmed prior to funding being undertaken as having “the promotion of the economic development and welfare of developing countries as its main objective^{[footnote 7]}”

ODA Reporting: The Fleming Fund complies with the requirements to report ODA funding to the OECD in line with OECD DAC Directives, via the DHSC Departmental parent and DFID.

7. Evidence of activities undertaken to meet IATI transparency standards

Self-assessed score against the IATI transparency standards

Our self-assessed score against the IATI transparency standards was 80 to 100% (very good).

Summarise what steps have been taken to ensure transparency of activities

Over this reporting period the Fleming Fund has provided descriptive data for all activities alongside key supporting documentation such as commercial agreements and one-page summaries. This data has supported the Department to score 82.1% in the recent Publish What You Fund (PWYF) assessment against the Aid Transparency Index – resulting in DHSC achieving ‘Very Good’ rating, second only to DFID, so this is a huge achievement for DHSC over such a short space of time.

8. Evidence of Value for Money (VfM)

The Fleming Fund, in line with HMT and DFID SMART guidance, recognise that VfM has 4 constituent components, known as the 4 E’s:

• Economy - Are we buying inputs of the appropriate quality at the right price?
• Efficiency - How well are we converting inputs into outputs? (‘spending well’)
• Effectiveness - are outputs produced by an intervention having the intended effect? (‘spending wisely’)
• Equity - How fairly are the benefits distributed? (‘spending fairly’)

These are reflected in the MA VfM strategy and workplan and the guidance disseminated to downstream grantees which states that:

VFM is considered as part of country selection, fund allocation and ensuring the grants are designed to be effective in the country and regional context. Also critical are management processes that deliver economy and efficiency in use of funds.

This report focuses on these 4 VfM components as evidenced in the Fleming Fund’s core country grant, fellowship and regional grant programmes. Some of the recommendations may be applicable to other Fleming Fund workstreams.

Economy

The Fleming Fund’s strategic approach to delivering economy throughout the design of the Fleming Fund is detailed in the Business Case, Implementation Plan and Mott MacDonald VfM report and workplan. In particular, there is evidence that in 2019 where goods and services were purchased, that the structures and review processes in place ensured they were purchased at the right and best price while maintaining quality.

Procurement also takes into account core development considerations such as supporting local supply chains to enable stationarity of the investment^{[footnote 8]}

Services

• Travel – The MA and DHSC have applied a moratorium on business class flights, combined with the routine interrogation of grantee flights and hotel costs to ensure VfM.
• Events – DHSC brought the event management of the Fleming Fund Delivery Partners’ Event in-house to minimise costs; flight costs were proactively managed and procured in bundles to secure savings; investment in diplomatic engagement through the Foreign Office secured visa exemptions (saving £2000) and significant cost reductions in hosting of key events (saving £2100)
• Expenses and per diems - these are routinely reviewed by DHSC (at both the budget submission and payment stage) and benchmarked locally to ensure practice is consistent with other Development Partners.

Goods

There is good evidence of the systematic use of a procurement expertise to guide the purchase of high value equipment and consumables, ensuring cost are reduced while quality is maintained. The MA and DHSC adopted an ambitious strategy to maximise VfM of purchase of laboratory equipment, through 3 approaches:

• central procurement
• government to government agreement (MoU)
• consolidating delivery

The total central procurement saving is estimated at over €4m.

This is made up of: central procurement of automated blood culture instruments, average cost saving of €2,289 per blood culture instrument, overall estimated saving of €223,810 across 24 countries, central procurement of mass spectrometry instruments, general benefits of mass spectrometry, sustainability – lower reagent cost, reduced need for highly trained workforce, reduced reliance on complex reagent supply chains, improved quality of results - rapid improvement in reference laboratory performance, reduction in production of biohazardous waste, cost savings from central procurement of instruments, supplier has offered almost 50% discounted price due to large scale purchase and opening-up of new market to supplier ((€86,000 less than competitor per instrument: equates to over €3m saving for 36 instruments purchased through this supplier), bundling of mass spectrometry instruments leading to additional average saving of €17,010 per bundle which equates to approximately €816,480 for purchase of 48 bundles.

Efficiency

There is good evidence that the Fleming Fund has robust systems in place that are supporting efficiencies in delivery of the programme, at the DHSC, MA and grantee levels.

The ratio of programme to administration costs is an indicator of efficiency. It is expected that there will be a significant range of administration costs between different grants, reflecting different geographies organisational structure, activities and environmental challenges and risks.

Administration costs associated with the Fleming Fund programme can be broken down into 3 areas:

• Portfolio Management – DHSC – 1.55%
• Programme Management - Mott MacDonald the MA – 18.7% 2019 against a 12.9% target
• Grantee Management and Overhead Costs and Indirect Costs – 25.33%

Portfolio Management - DHSC – Fleming Fund Team Management

For 2019 the Fleming Fund’s total administration costs were approximately £0.65m. This is c1.55% of the total Fleming Fund expenditure for the 2019 calendar year.

The Fleming Fund’s MA regional hub structure combined with the leveraging of HMG’s existing global network (FCO, DFID and Public Health England, PHE) to support in-country activity, significantly reduces the resources involved in administration of the programme.

Grant Management - Overhead Costs and Indirect Costs

There is good evidence that costs are being actively managed at the mobilisation stage, through the grant review/ award process. Management and Overhead costs (M&OH)^{[footnote 9]} and Indirect Costs (IDC)^{[footnote 10]} are closely monitored and in the case of M&OH they are manged to target levels through Key Performance Indicator 6: KPI 6 is "grantee management and overheads expenses no more than 12.5% of grant costs". The use of a KPI has successfully contained management and overhead costs of country grants to around 2% below the target level ^{[footnote 11]}.

In addition, quarterly reporting is in place with all grantees, which includes review and discussion on budget and actual variances. Where activities are not completed as planned, discussions are held between regional teams and implementing partners to reschedule activities.

An example of the MA achieving economy and efficiencies through a country grantee is Ghana, where partners rates were challenged during contract negotiations, achieving a 23% cost reduction on trainings and staff. This was used to fund additional resources in a revised budget that was agreed prior to implementation. During implementation, savings have been made through negotiating discounts with hotels and also through combining activities. In the second quarter of the grant, economy and efficiency savings were reported to be 15% of the budget for the quarter, as reported in ITAD’s second evaluation deliverable.

Recommendation: There are clear mechanisms in place to identify and review high M&OH costs. Indirect costs should also be closely reviewed as part of the rollout of the second-round country grants – with a particular focus on those grantees which are outliers and have multiple grants where economies of scale can possibly be sought.

Equity

Poverty reduction

An equitable investment directs resources toward those groups where there is greatest need, an equitable approach therefore would support a reduction in poverty that arises out of health inequalities.

Fleming fund country selection recognises the need to focus on lower income countries. The majority (13) of the Fleming Fund’s partner counties are considered least developed or low income ^{[footnote 12]} and where investment is in middle income countries discussion are raised with the MA on opportunities to poorer provinces/states.

However, without disaggregation of data and increased patient data, the extent to which Fleming Fund investments in building reference lab capacity and sentinel surveillance contribute to equity is not yet measured at the beneficiary level. This limitation can be considered justifiable when recognising that the first phase of implementation is focused on strengthening surveillance system architecture and has not yet translated into outcomes that impact on beneficiaries directly.

Gender

The Fleming Fund currently monitors gender ratios in 2 ways:

• gender breakdown of fellows (applicants and those selected), are routinely analysed
• review of gender of grantee and MA staff who are funded by programme annually

Recommendation: Consideration should be given as to the mechanisms available for gathering metadata, that will help identify the extent to which surveillance data is being appropriately collected from all groups (socio-economic, geographic and gender), reflective of the burden of AMR.

Recommendation: A clear statement on equity in the Fleming Fund should be developed, this would be underpinned by overarching objectives, to which grantees can work towards. It would help in setting out how benefits from investments can be equitably distributed during phase 2, which is when the programme will be much closer toward delivering health outcomes.

Effectiveness

Finding: It is not possible, at this stage, to fully assess if the Fleming Fund is managing for effectiveness at this early stage. There are opportunities to strengthen this through the next planning cycle.

In order to measure effectiveness, robust monitors of effectiveness must be in place. However, the challenges in measuring this were identified within the MA’s VfM Report:

At this early stage, the programme is not expected to demonstrate systematically outcomes or impact in terms of changing policy and practice, reducing resistance or reducing mortality related to AMR (although examples will be gathered where possible). This limits the capacity to judge effectiveness or analyse cost/effectiveness, so the VfM focus is on processes that are conducive to maximising impact and outputs, as well as minimising costs.

The independent evaluator notes that evidence of use of data and therefore managing for effectiveness is largely absent. However, there is some ad hoc evidence of Fleming Fund investments starting to catalyse policy discussions at this early stage of investment in the second evaluation deliverable including:

• human health: updating of National Treatment Guidelines, strengthening of regulatory frameworks and facility level stewardship.
• animal health: some evidence of the use of surveillance data to inform the updating of legislation in Bhutan, and the veterinary medicine regulation in Nepal and Uganda.

Recommendation: The Fleming Fund team should consider potential mechanisms available for capturing the extent of the Fund’s contributions toward outcomes (policy changes/health outcomes) in future funding cycles/investments beyond the current phase.

External engagement

Delivery confidence assessment for reporting year

Activity areas	RAG rating	Changed since last year (Y/N)
Website	Amber	n/a
Social Media	Green	n/a
Delivery Partners Event	Green	n/a
Country Launches	Green	n/a
Partner engagement (e.g. partners portal, webinars)	Amber/Green	n/a

Overall RAG rating: Amber/Green

9. Evidence of use and success of the communication strategy

i) Brief summary to the communications strategy or policy for each element of the project.

ii) Overview of the planned communications activities (activities outlined or agreed in the relevant communication strategy or policy) in the current reporting period.

The Fleming Fund has developed a detailed external communications strategy that identifies the messages relevant to each target audience, and the platform used to communicate with them. The messages are grouped under a revised core purpose, then split into aims, outcomes and activities. This is supported by a communications plan and content calendar that is currently populated to March 2020.

The programme has moved to a campaigns-based communications plan to provide a greater coherence and thematic approach to the messaging produced. The campaigns used over 2020 will reflect the messaging strategy’s aims including ‘building partnerships’, intended outcomes such as ‘make AMR a policy priority’, and activities including ‘establish lab capacity and surveillance systems’. These campaigns will then determine how we frame our content on social media and the website.

The team have also developed communications guidance that has been shared with delivery partners and updated with the new strategy to encourage them to actively promote the programme’s activities. This updated guidance is available to partners on the Partners Portal and was communicated to Country and Regional Grantees and Fellows in a webinar lead by Mott MacDonald’s Knowledge and Communications Manager. Another communications webinar for DHSC direct grantees is planned for January 2020. A protocol has been developed alongside this guidance to formalise the approach and responsibilities of DHSC, Mott MacDonald, country grantees and FCO/DFID in-country grant launches.

The Fleming Fund website is being restructured to better reflect the wider programme portfolio and to ensure that it remains fit for purpose throughout the lifecycle of the programme. The developer, Softwire, is currently contracted to carry out this work, and has already updated the Grants pages. Plans are in place to upgrade the Country pages. The Fleming Fund team is in the process of transferring the website hosting to Amazon Web Services (AWS) so that hosting costs are invoiced directly to DHSC, who will then retain ownership of the website after the contract with Softwire has expired. In 2020, the website will be linked to a Google Analytics page so that the team can monitor use of the website to inform future updates, content and restructuring.

Mott MacDonald has recruited a full time Knowledge and Communications Manager to support with the creation and distribution of content to delivery partners and the public. This role includes creating content for the Fleming Fund website and social media channels, supporting with country launches, developing resources for delivery partners and developing the updated messaging strategy.

Increased collaboration between the Fleming Fund’s communications lead, the Global Health Security communications manager and the Mott MacDonald Knowledge and Communications Manager has led to an increase in productivity and coherence across communications channels. A joint content planning call is held every month between the 3 communications leads to maintain a consistent flow of content through all channels in line with the campaign’s strategy and the wider AMR communications landscape. A shared portal has also been established to bank stories and case studies to be used in future campaigns.

The Fleming Fund team maintains a twitter account (@FlemingFund) that has accumulated a total of 1500 followers, and 214,707 impressions and 834 mentions over the last reporting period. Going forward, the team will monitor engagement on a monthly basis to measure the success of the new communications and campaigns strategy.

Before the end of the next reporting period, the Fleming Fund intends to formalise and streamline the distribution of work between the communication leads at DHSC and Mott MacDonald. This will ensure a more consistent output of content across all channels and audiences, greater adherence to the communications strategy and avoid a bottle neck where content is held with the Fleming Fund lead for approval. This will also provide the Fleming Fund with greater resource to continue to maintain other areas of communications outside of content creation and sharing, such as the core script and website management.

10. Evidence of external engagement (other)

Please provide an overview of additional engagement with relevant partners (include, where appropriate, reference to engagements with in or out-country organisations, research and industry stakeholders and public audiences.

Over the last reporting period, the Fleming Fund team refreshed the cross-government communications plan to improve engagement with FCO and DFID country offices and to increase understanding and visibility of Fleming Fund activities in-country. This has been supported by the development of country one-pagers that outline all Fleming Fund activity in each country. These have been distributed to Mott MacDonald regional offices and will be edited so that they can be shared with governments and key stakeholders in-country.

The Fleming Fund also produces several bulletins throughout the year for a variety of audiences. The Quarterly DHSC Bulletin is shared with key stakeholders and partners, including country offices and the Science and Innovation Network in the FCO, the Technical Bulletin is sent monthly on behalf of the AMR Special Envoy to partners and the Technical Advisory Group, and the Petri dish is published monthly and focuses on developments in the Country, Regional and Fellowships grant programmes. A bulletin sign-up link will also be included on the website to expand the audience beyond direct partners.

Following feedback from the Delivery Partners Event last year and also recommendations from the last Annual Review, the team have worked to improve communication and co-ordination across all partners in the programme. Following a consultation exercise, a partner’s portal was created and provides a platform for partners to access key resources and information such as communications guidance, summary one-pagers, evaluation reviews and reports and a travel calendar. In addition to the portal, the team has started hosting monthly partners webinars, where all partners are invited to a deep-dive on a cross-cutting area of work. In 2019, the team hosted a webinar on the Theory of Change and has further sessions planned for 2020 on the new communications guidance and how to strengthen country co-ordination.

The Fleming Fund Delivery Partners Event was held in November 2019 in Vientiane, Laos, and provided an invaluable opportunity for all partners to come together and share learning. By collecting feedback after each day of activities, the team was able to identify the sessions considered most valuable by the attendees. The Strategic Direction session was most highly scored by partners as they appreciated being consulted on where the Fleming Fund should focus if awarded further funding. The Fleming Fund team noted a high level of engagement throughout this session, demonstrating partners’ enthusiasm at being involved in high-level decision-making. The Improving Co-ordination and Alignment session was also well-received and set the groundwork for the improvements the team is planning to make in this area throughout the next reporting period.

Country Grant launches have also been a good opportunity to engage with political stakeholders in-country and demonstrate the breadth of Fleming Fund activity in the area. The Nigeria country launch is set to be the most ambitious one yet, with the Vice President and other high-profile politicians set to attend. In 2019, the team increased the visibility of the Fleming Fund and UK leadership on AMR through high profile grant launches at the and a high profile media launch in Pakistan that was covered by all of Pakistan’s national news outlets. DHSC also signed a Memorandum of Understanding on AMR with the Government of Nepal in September, that lead to a spike in interaction on social media with Fleming Fund tweets reaching 31,200 impressions over that month. Four more MoU signings are in development for 2020.

The Fleming Fund has actively contributed to promoting global AMR campaigns over the calendar year including: the publication of the OIE annual report, the launch of the WHO AWaRe campaign, and World Antibiotics Awareness Week. The team have also been present at and contributed to several high-profile AMR events including the Dutch Ministerial Event on AMR, the Global Antibiotic Research & Development Partnerships (GARDP) access and stewardship of antibiotics workshop, the South-East Asia Regional Health Security Donor Co-ordination meeting and the ASEAN Antimicrobial Stewardship seminar. When Fleming Fund team members from DHSC are unavailable, representatives from Mott MacDonald attend to represent the programme. In the last reporting period, Mott MacDonald representatives attended the WHO NAP Costing Expert Group and the Surveillance and Epidemiology of Drug Resistant Infections Consortium (SEDRIC), Wellcome Trust Annual Global Meeting, among others.

A comprehensive stakeholder mapping and global coordination effort has been completed by the FCO’s Health Attaché for AMR based at the UK Mission to the UN Geneva, who is funded by and seconded to the Fleming Fund. The primary purpose of this activity is to identify key stakeholders that are actively working in the 24 Fleming Fund target countries across Africa and Asia. This is set out in a one-page geographical mapping table showing which countries these stakeholders, and their specific programmes relevant to AMR including surveillance, are actively being conducted in. Secondly, to provide a detailed analysis of each key stakeholder within depth detail on the type of activities they are conducting in the 24 Fleming Fund target countries. This will ensure alignment of activities and avoid duplication of efforts, as well as better coordination of donor funding to maximise impact at the country and regional level.

VfM and External Engagement - A large proportion of external engagement activities are conducted through existing government mechanisms / platforms to ensure costs incurred represent VfM. Where costs are incurred (e.g. through the appointment of a communication manager / website contractor), the benefit is considered by DHSC before purchase to ensure economies, efficiencies and effectiveness. DHSC and the MA are working to streamline communications, and integrate communication activity across the different workstreams to increase efficiency

Summary of external engagement recommendations to improve the effectiveness of stakeholder and delivery partner engagement.

Recommendation: Commission Softwire to update the Fleming Fund website and fix programming issues to ensure more programming issues to ensure more information is available to key country stakeholders including the original request for proposals, country one-pagers, and map of sites supported.

Recommendation: Host 2020 Delivery Partners Event in Africa, building on the feedback from partners at the Laos event.

Theory of change

11. Evidence to show if the Theory of Change (Toc) assumptions remains accurate?

Please include a link to your ToC in the annex. In the space below please summarise any major changes to your ToC in the past year:

The programme undertook development of a new Theory of Change over this reporting period for the whole portfolio, replacing the previous Theory of Change which encompassed only the MA part of the portfolio and was developed during inception phase before activities had been fully defined. The Theory of Change was adapted through a workshop with Itad and then a reflection session at the Delivery Partners Event, where all partners were encouraged to provide comments and additions to the version.

One of the main changes from the original Theory of Change to the revised version has been the inclusion of a "country enabling environment" section which identifies key factors which need to be in place for AMR data to be gathered, analysed and shared and which Fleming Fund projects have specific outputs to strengthen. It is recognised that different projects from across the Fleming Fund will contribute to different sections of the Theory of Change and impact the change pathways at different stages and levels from output to outcome.

In addition to this change, longer-term outcomes have been included to reflect the relevant objectives of the Global Action Plan on AMR and our alignment with the global response as well as our contribution to health system strengthening. A timeline has been added to situate Fleming Fund activity over a longer timeline and highlight that the changes we are hoping to effect will only become evident over an extended period beyond the original length of the Fleming Fund (2016-2021). A key learning has been realism about the length of time it will take to embed truly sustainable One Health surveillance systems at country level, and for the necessary and sufficient conditions to be in place for the data that is produced to be used to change policy and practice.

The programme has developed a monitoring matrix, which has been used to inform section 2 of this review. The monitoring matrix takes indicators from each of the Fleming Fund projects to provide an objective, annual performance-based measurement on how projects are progressing in terms of delivery of expected outputs and for the portfolio as a whole.

Describe where the project is on track to contribute to the expected outcomes and impact. Please state what action is planned as a result in the year ahead:

Findings from the second evaluation deliverable, based on country-level evidence drawn from the 5 countries which began implementation first, is very useful in supporting an assessment of Fleming Fund progress towards outputs and outcomes.

Overall, country stakeholders anticipate important progress by end 2021 in terms of strengthening AMR surveillance in both human and animal health sectors. This is also measured through a key performance indicator for the MA and is showing that countries are making solid progress in strengthening lab capacity, governance and other areas necessary to reach a core competence which allows for the generation and sharing of robust quality data.

We also expect to see a significant increase in both a) the number of sites providing credible reporting to GLASS; and b) the number of tested patients (samples) in labs providing evidence for GLASS, from a baseline of 2018 to programme end in 2021. There are also visible improvements to specimen types reported and clear improvements to animal health AMU data reported to OIE.

However, in terms of contribution to outcomes, it was found that we are unlikely to see evidence of the use of AMR analysis/data at a country level before 2021. This is largely as expected, as Phase 1 of the Fleming Fund is focused on getting the foundations of surveillance systems in place. However, there are some specific examples of intended use of data to change policy, regulation and practice in both human and animal health sectors. In human health, opportunities to address stewardship could be identified through linking into broader health system strengthening initiatives. In animal health, opportunities appear fewer, reflecting both a lesser government role in the sector in most countries, and less capacity within governments in this sector.

Whilst the Fleming Fund works on a principle of adaptive management, it is important to note that at this stage there is limited scope to fundamentally change the programme in response to some identified challenges, given there is only around 24 months of implementation remaining. We have prioritised those improvements which will have the greatest impact on results and most likelihood of success, including improving coordination in-country within our control, such as formal mechanisms to coordinate better between Fleming Fund delivery partners. We are also focusing on developing the second round of country grants to better tackle country level sustainability priorities and gaps identified in first round country grants, whilst managing expectations on what can realistically be achieved by end of Phase 1.

Summary of changes recommended to the theory of change

Recommendation: The Fleming Fund has considered developing a logframe that can track progress against the revised Theory of Change across the portfolio, with a particular focus on contribution to outcomes. This would also need to link into the updated GHS logframe still under revision. However, due to the complexity of the programme of work and constraints of existing governance and monitoring arrangements, this is likely to be an action for a future phase of the programme.

Monitoring, evaluation and learning

12. Evidence of evaluation

Overview of any evaluation activities that have taken place throughout the review period.

The Fleming Fund country, regional and fellowship grants programme is independently evaluated by Itad. In January 2019 the team received the first formative learning report from Itad which noted a number of suggestions to improve the management and delivery of these projects. These suggestions, alongside the recommendations from last year’s annual review were included in an adaptive management plan. The plan collected the different learnings into thematic areas and identified actions that were to be taken forward to improve delivery of the programme and support the achievement of outputs and objectives.

The team have worked to this plan over this reporting period and Itad have run facilitated reflection sessions with DHSC and Mott MacDonald to discuss additional learning and identify where further action needs to be taken. Itad provide country debriefs following each of their evaluation visits. This are a further opportunity to capture learning from the programme to date and to identify actions that should be taken to improve delivery. This learning is also captured in the adaptive management plan.

There is provision within grant agreements with other Fleming funded delivery partners to conduct end of project evaluations. No evaluations were due in this reporting period but there will be a number of grants coming to an end in 2020 and the team will work with partners to ensure that these evaluations capture learning and feed into the adaptive management plan and support improvements to the programme.

Please summarise any recommendations and key issues that have been raised in your evaluation.

The headline recommendations in the second formative evaluation report, in addition to the key positive results on improvements to surveillance system capacity and generation of data in-country in Section 11 above – are:

Recommendation: Whilst challenges are consistent with DFID experience, as identified in ICAI reporting, there are concerns about both a) managing for effectiveness under Value for Money (VfM) and b) the sustainability of this phase of the grants programme. DHSC and the MA need to agree what makes sense to address in this current phase.

Recommendation: Coherence is challenging given the range of Fleming Fund work and other investments by the UK government. The MA and country grantees do promote coordination and coherence, focused mainly around avoiding duplication, but probably do not have the mandate to deliver more. Again, experience is similar to that of DFID in terms of coordination.

Given limited time available (c.24 months), and substantial set of decisions planned for 2020, DHSC and the MA should focus on what gets prioritised for this phase of the Fleming Fund and what can be better addressed in a next phase (2022 onwards). Key considerations include:

1. Whether the data/analysis that will be produced is the right data/analysis to influence outcomes of interest. Is data of sufficient quality, representativeness, relevance for clinicians?

2. Whether the expected status of the AMR surveillance system is sufficient to generate the data that is needed? If not, where is more effort needed? If so, where to focus next?

3. Do DHSC and the MA have the right capacity and processes in the right places to ensure smooth decision making, contracting and delivery on cross-cutting issues? Do all processes add value? Can some be deprioritised?

4. How can country-level experience and views be factored into the design of a second phase?

5. How can coordination be improved with the right capacities and mandates established to do this?

13. Evidence of monitoring

Summarise any monitoring activities that have taken place throughout the review period.

The Fleming Fund monitors all projects through a quarterly or 6-monthly review process. All but 3 of the projects have logframes which are updated at the review point to show progress against plans. Where logframes have not been developed deliverables and key performance indicators are used instead.

Over this review period, the Fleming Fund refreshed the performance dashboard that is updated for the Fleming Fund Project Board. The dashboard now provides RAG ratings for quality, timelines, finance and overall performance. This breakdown has allowed the team to better monitor performance and be more transparent as to where delivery is lacking. This is supporting any performance issues to be more directly and swiftly addressed.

Deliverables, service levels and key performance indicators (KPIs) for the Fleming Fund contracts with Mott MacDonald and Itad were updated in 2019. The team negotiated changes to the KPIs for the Mott MacDonald contract which provide an indication of whether project activities are supporting outcomes.

Please summarise any major changes to your logframe in the past year.

As mentioned all but 3 of the projects have logframes in place. Given the number of Fleming funded projects and the complexity of the programme there is no single overarching logframe that covers the full programme of activity. Over this reporting period the team has developed a monitoring matrix that contains indicators from projects across the programme. These indicators and the annual milestone targets link to the updated ToC and provide an indication of whether the programme is on track to deliver the intended outputs and to support the achievement of the longer-term outcomes. The milestone targets for 2020 will be set in the first quarter and be reported against in the next annual review.

14. Evidence of learning

Summarise any learning activities that have taken place throughout the review period.

The Itad contract was extended in early 2019 to include provision for more activity on learning and dissemination. This was intended to support adaptive management and course correction within the programme and for early findings and results to be more effectively shared with international stakeholders.

Itad facilitated a workshop in January 2019 linked to the findings and suggestions from the first formative evaluation report. The workshop provided an opportunity for DHSC to work collaboratively with the MA to make changes to the design and implementation of the country, regional and fellowship grants programme to reflect learning to date. The suggestions and findings from the first formative report, alongside the recommendations from last year’s annual review were included in an adaptive management plan that is being actively managed and revised by DHSC with the support of Itad, the MA and other delivery partners.

The key learning areas from the adaptive management plan are:

Learning Area: Communications

The first formative evaluation and the 2018 Annual Review found that the objectives of the Fleming Fund and how its contribution will compliment wider efforts on AMR needed to be more clearly communicated. It was also flagged that communications to national stakeholders about the programme needed to be improved.

Action taken to date: Communications were discussed in depth at the 2019 Delivery Partners Event. Input from these discussions was used to develop updated key messages that were agreed by DHSC and the MA. This messaging has been used to update the core Fleming Fund narrative and communications materials which will be shared with partners through webinars scheduled in January 2020.

Learning Area: Implementation Plan

The first formative evaluation report and the 2018 Annual review found that DHSC and the MA needed to address the causes of delays in grant making and agree a more efficient grant making process going forward.

Action taken to date: DHSC and the MA have agreed improvements to the recommendation and approval process for country and fellowship grants to maximise time for grant delivery.

Learning Area: Monitoring results

The first formative evaluation report found that the approach to monitoring and oversight of the Fleming Fund grants programme focused mainly on efficient delivery and not enough on whether the grant programme is on track to deliver the right things.

Action taken to date: Improvements have been made in the availability of data and reporting from partners through the quarterly reporting process. DHSC has worked with the MA to agree increasingly stretching key performance indicators (KPIs) which provide a better sense of whether the country, regional and fellowships grants programme is on track to support the achievement of intended outcomes. Monitoring Matrix information included in this 2019 report also attempts to provide on indication of whether projects are on track to achieve intended outputs and outcomes rather than just demonstrating the efficient delivery of processes.

Learning Area: Fellowship coordination

Learning on fellowships was focused on the need for improved understanding of the fellowship model, and improved coordination with other Fleming Fund activity. Feedback from Itad evaluation country visits found that some Fellowship Host Institutions (HIs) and mentors are still unclear about the fellowship model and core objectives. Ensuring that there is a common understanding of the objectives of the fellowship scheme between fellows, HIs, national governments, and country grantees has been a priority this year, as well as making sure that these objectives are aligned with country needs. It was also noted that opportunities for fellows complimenting other areas of Fleming Fund activity should be identified.

Action taken to date: An infographic on the fellowship model and regular dissemination of the Petri Dish aim to support improved understanding of the objectives and approach of the fellowship scheme. The addition of the regional fellowship coordinator role has increased the capacity of the MA to support communication, the sharing of documents and information, and co-ordination between fellows, Host Institutions, national governments, and country grantees. This should ensure that fellowship activities are better aligned with other areas of the programme. There is still further work to be done with plans to implement regular webinars for fellows and webinars between HIs to support the building of communities of practice.

Learning Area: Partnership coordination and alignment

There is a need to assess the scope for greater synergies between regional and country grants, and between other Fleming Fund investments. In some countries there are multiple projects and delivery partners being funded by the Fleming Fund. National governments and other key stakeholders were not always clear about these workstreams and the role of various partners in-country. This leads to confusion and missed opportunities for alignment and collaboration. The first formative report identified a need to clarify responsibility for the coordination of Fleming Fund partners and activities at the country level. This includes coordination across the Fleming Fund, with other HMG investments, and with related wider development partner activity.

Action taken to date: This was a topic covered in the 2018 Fleming Fund Delivery Partners Event. Partners were consulted on how to address this issue and support increased collaboration, alignment and coherence across the programme. Country one pagers have been developed setting out Fleming Funded activity and the partners involved to inform national governments and other stakeholders. Currently exploring setting up quarterly coordination meetings in-country, led by the Fleming Fund country grantee but involving all other FF partners and relevant stakeholders.

Other key learning activity includes:

• Reflections session in October

• Delivery partners meeting learning and disseminations sessions

• Learning and dissemination working group and strategy

Summary of recommendations for evaluation, monitoring and learning activities

Recommendation: Guidance to support quality and consistency in end of project evaluations

Recommendation: Updates to monitoring matrix for 2020. Reconsider logframe a portfolio wide logframe.

Recommendation: Regular learning/reflection workshops with MA and other key partners

Recommendation: Focus on dissemination, blogs, communications on learning to date (Review L&D plan)

Diversity and sustainability

Please summarise any activities that have taken place to ensure everyone is treated fairly, regardless of gender, gender identity, disability, ethnic origin, religion or belief, sexual orientation, marital status, transgender status, age and nationality, in your project.

The Fleming Fund respects and supports the principles of inclusion and equality and has worked with the MA to share these principles with countries, country grantees and fellowships to prevent discrimination on the grounds of gender, gender identity (including transgender status), disability, ethnic origin, religion or belief, sexual orientation, marital status, age and nationality. As the Fleming Fund is primarily a data and surveillance programme, it is unclear whether it will be possible to ensure complete equality between genders when collecting data due to the existing social structures in countries. Within these limitations, the Fleming Fund has endeavoured to ensure equal gender representation in recruitment and the Fellowship scheme, but we do not currently collect accurate disaggregated data so cannot reliably determine the gender balance with the data produced. Discussions are underway with key partners to explore how gender can be considered throughout the programme and where it was already considered in the design and implementation. In the next reporting period, the gender lead will develop a paper setting out the findings from these discussions and Fleming Fund’s future approach.

Sustainability has been considered from the inception phase of the Fleming Fund portfolio. We have worked to improve the approach to sustainability using best practise and suggestions from ITAD. To date we have received 2 Sustainability Analysis/Comprehensive Stakeholder Analysis (SA/CSAs) for 2 Fleming Fund countries. These SA/CSAs will help us to understand the situation in-country, to agree decision point papers for Country Grant 2s and to highlight what more can be done towards sustainability in Country Grant 2s themselves.

Please summarise any activities that have taken place to minimise carbon emissions and impact on the environment in your project.

The Fleming Fund recognises the importance of consideration of the impact of our programme to the environment. The MA has developed a comprehensive environmental policy statement with the intentions to be carbon neutral by 2021. The Fleming Fund are considering a policy of carbon offsetting for essential travel which the Global Health Security team review wider environmental policies for the team.

Overall project delivery and recommendations

Overall assessment RAG rating

Activity areas	RAG rating	Has RAG rating change since last annual review?
Project Management	A/G	No
Finance	A/R	Yes (A)
Theory of Change	A/G	No
External Engagement	A/G	No
Overall Delivery Confidence rating:	Amber

List of Recommendations:

Project Management

Agree stretching year 3 milestones for the monitoring matrix by end March 2020

Finance

Agree set of mitigation measures to improve Mott Macdonald forecasting and financial performance.

Risk Management

The Fleming Fund portfolio risk register currently uses a 5 by 5 risk matrix for likelihood and impact. The GHS Programme Board have just received approval for their revised risk strategy which uses a 4 by 4 risk matrix. The Fleming Fund will review the difference and consider aligning.

Theory of Change

Consider developing an overarching Fleming Fund logframe that can track progress against the revised Theory of Change, the revised GHS Theory of Change (pending), and which can crucially measure contribution to outcomes. This may have to be something that is developed for a future Phase II of the programme.

External Engagement

Commission Softwire to update the Fleming Fund website and fix programming issues to ensure more information is available to key country stakeholders including the original request for proposals, country one-pagers, and map of sites supported.

Host 2020 Delivery Partners Meeting in Africa, building on the feedback from partners at the Laos event.

Value for Money

There are clear mechanisms in place to identify and review high M&OH costs. Indirect costs should also be closely reviewed as part of the rollout of the second-round country grants – with a particular focus on those grantees which are outliers and have multiple grants where economies of scale can possibly be sought.

Consideration should be given as to the mechanisms available for gathering metadata, that will help identify the extent to which surveillance data is being appropriately collected from all groups (socio-economic, geographic and gender), reflective of the burden of AMR.

That clear statement on Equity in the Fleming Fund is developed, this would be underpinned by overarching objectives, to which grantees can work towards. It would help in setting out how benefits from investments can be equitably distributed during phase 2, which is when the programme will be much closer toward delivering health outcomes.

The Fleming Fund team should consider potential mechanisms available for capturing the extent of the Fund’s contributions toward outcomes (policy changes/health outcomes) in future Funding cycles/investments beyond the current phase.

Annex 1: 5-point score ratings

This is the scoring system we have applied to the output milestones/deliverables:

Score	Output Description	Outcome Description
A++	Outputs substantially exceeded expectation	Outcome substantially exceeded expectation
A+	Outputs moderately exceeded expectation	Outcome moderately exceeded expectation
A	Outputs met expectation	Outcome met expectation
B	Outputs moderately did not meet expectation	Outcome moderately did not meet expectation
C	Outputs substantially did not meet expectation	Outcome substantially did not meet expectation

1. “Active” defined as: period post grant signature ↩

2. "Supported" defined as: of effective date of Grant Agreement, sites are deemed to be supported. "Progress" defined as: Annex E of LSHTM roadmap – movement in 2 or more subcomponents in more than 1 component after 9 months or more of support, against the baseline. The baseline will be the grantee completion of the verification of the laboratory needs assessment undertaken by the Supplier, or the baseline assessment by the grantee if not already done by the Supplier. "Surveillance sites" defined as: human health surveillance sites and national reference laboratories detailed in the country’s Request For Proposals. This excludes animal health surveillance sites because these are not covered by the LSHTM roadmap. ↩

3. "Selected" defined as: Mott MacDonald selecting Fellow and confirming the fellowship with the beneficiary and host institutions. ↩

4. "Active" defined as: Fellows with approved workplans ↩

5. Current indictors on AMR data focus on Human Health. Indictors for Animal Health will be identified for 2020 milestones. ↩

6. "Fleming Fund supported countries" defined as: Fleming Fund countries with a live Country Grant 9 months or more into implementation. ↩

7. Official development assistance – definition and coverage ↩

8. Tanzania procurement of laboratory equipment ↩

9. M&OH costs are attributable to a project but not to specific activities detailed in the workplan. These are typically project finance and admin staff, project office costs, etc. ↩

10. IDC, or Non-Project Attributable costs (NPAC), are the costs incurred that cannot be directly attributed to a specific project. These include the running costs of the organisation (HR and finance functions, Head Office and governance costs. ↩

11. A review of country grants found M&OH costs to be on average 11.55% ↩

12. As defined in the OECD DAC list of ODA eligible countries. ↩