Policy paper

AI Safety Summit 2023: Roundtable Chairs' Summaries, 1 November

Published 1 November 2023

This was published under the 2022 to 2024 Sunak Conservative government

The discussions that took place at the AI Safety Summit at Bletchley Park on 1-2 November 2023 brought together international stakeholders from governments, leading AI companies, civil society and academia.

The following points, discussed with the Chairs of each roundtable, summarise the key discussion points, themes and emerging issues from the discussions on the first day of the Summit.

The UK is publishing this as Chair of the Summit, not as a UK government policy document.

Roundtable 1: Risks to Global Safety from Frontier AI Misuse

Discussion of the safety risks posed by recent and next generation frontier AI models, including risks to biosecurity and cybersecurity.

Chair: François-Philippe Champagne, Minister for Innovation, Science and Industry, Government of Canada

• The latest frontier AI systems (GPT4 and equivalents) make it slightly easier for less sophisticated bad actors to carry out attacks, including cyberattacks, designing biological or chemical weapons.

• Frontier AI systems are very likely to become more capable and accurate, as well as more prevalent and accessible to bad actors, so these risks will grow.

• Frontier AI companies have started to put some safeguards around their models, but this needs to be complemented by government action. There is a need to work together across governments, industry and experts, especially on testing.

• The risks these AI systems pose to the public are significant.  It is urgent that we both research and discover ways to ensure current models and future models do not enable bad actors to cause harm.

• We are only at the early stages of understanding how these models work, the risks they pose, and therefore how to develop adequate safeguards. - We need global and decisive action now to acknowledge and act on these risks. Frontier AI companies, governments, and academic and civil society researchers need to rapidly work on this together.

Roundtable 2: Risks from Unpredictable Advances in Frontier AI Capability

Discussion of risks from unpredictable ‘leaps’ in frontier AI capability as models are rapidly scaled, emerging forecasting methods, and implications for future AI development, including open-source.

Chair: Yi Zeng, Chinese Academy of Sciences

• The current abilities of frontier AI systems are far beyond what many predicted only a few years ago.

• As investment increases, it is very likely we will continue to be surprised by what future AI systems can do, in ways that are not necessarily predicted or intended by their creators.

• These models will also be able to connect to other systems and enhance their capabilities – the number of possible permutations means it is hard to anticipate the potential outcomes before release.

• Advanced AI capabilities are likely to bring huge benefits by solving currently unsolved challenges in health, education, environmental, scientific and other fields. But the very same properties of AI systems that create these benefits also create significant risks.

• New frontier AI models must be developed and tested rigorously, in secure conditions. The promise of potential benefits should not be a reason to skip or rush safety testing or other evaluation.

• While open access models have some benefits like transparency and enabling research, it is impossible to withdraw an open access model with dangerous capabilities once released. This merits particular concern around the potential of open access models to enable AI misuse, though an open discussion is needed to balance the risks and benefits.

• It is good to share evaluation tools but this does not mean we are risk-free. We need to keep a continuous eye on emerging risks.

Roundtable 3: Risks from Loss of Control over Frontier AI

Discussion of whether and how very advanced AI could in the future lead to loss of human control and oversight, risks this would pose, and tools to monitor and prevent these scenarios.

Chair: Josephine Teo, Minister for Communications and Information, Government of Singapore

• Current AI systems are relatively easily controlled - they require human prompting, generally fail when asked to plan over time towards a goal, and have limited ability to take actions in the real world. However, future models are likely to improve on these dimensions, having significant consequences.

• Even when AI systems appear to display high cognitive abilities, we cannot be sure that they will behave like, or take the same decisions, as humans. For example, future systems might, without adequate supervision, consider actions that their developers would not anticipate or intend.

• Current models do not present an existential risk and it is unclear whether we could ever develop systems that would substantially evade human oversight and control. There is currently insufficient evidence to rule out that future frontier AI, if misaligned, misused or inadequately controlled, could pose an existential threat.  This question is an active discussion among AI researchers.

• It may be suitable to take more substantive action in the near term to mitigate this risk. This may include greater restrictions upon, or potentially even a pause in, some aspects of frontier AI development, in order to enjoy the existing benefits of AI whilst work continues to understand safety.

• This may be implemented with the support of new incentives and levers. The suitability and impact of such measures will depend on compliance and therefore the ability of governments to ensure ‘bad actors’ are not able to circumvent them.

• We can take concrete actions now to prevent these scenarios. There are decisions that should not be handed to an AI system, and so working through how as a society we avoid overreliance. We also need to rigorously test models in secure environments and do further work to understand how loss of control could come about.

Roundtable 4: Risks from the Integration of Frontier AI into Society

Discussion of risks from the integration of frontier AI into society include impacts on crime and online safety, election disruption, and exacerbating global inequalities. Discussion will include measures countries are already taking to address these risks.

Chair: Marietje Schaake, Stanford Cyber Policy Institute

• Known frontier AI poses societal risks that are an existential threat to democracy, human rights, civil rights, fairness, and equality (e.g. economic opportunities, healthcare, and development).

• We need to make better use of the tools we already have to address these risks. We need to clarify how existing rules should be applied to address things such as privacy, liability and intellectual property. We should apply and build on known solutions in the fields of trust and safety.

• We need more comprehensive and better quality technical evaluations of AI models which include societal metrics and recognise the context of their application in the real world. Evaluations need to be continuous and handle workflows, not just static datasets.

• We should invest in basic research, including in governments’ own systems. Public procurement is an opportunity to put into practice how we will evaluate and use technology.

• We must not miss out on the opportunity to use AI to solve global problems, including strengthening democracy, overcoming the climate crisis, and addressing societal bias.

• We need to involve citizens in how AI is used: governments should actively seek to include a wide cross section of citizens, including young people, in addition to technical experts, civil society and others for example in government advisory boards.

Roundtable 5: What should frontier AI developers do to scale capability responsibly?

Discussion of Responsible Capability Scaling policies at frontier AI developers including defining risk thresholds, effective model risk assessments, pre-commitments to specific risk mitigations, robust governance and accountability mechanisms, and model development choices.

Chair: Michelle Donelan, Secretary of State for Science, Innovation and Technology, Government of the United Kingdom

• There is a continuing debate about how much we should aim to scale capabilities, or whether capability scaling is inevitable, but agreement that we must be prepared for the risks.

• Frontier AI companies are making significant progress on AI Safety policies, including responsible, risk-informed, capability scaling policies. However, these company AI safety policies need to evolve further. This work is urgent, and must be put in place in months, not years.

• Company policies are just the baseline and don’t replace the need for governments to set standards and regulate. In particular, standardised benchmarks will be required from trusted external third parties such as the recently announced UK and US AI Safety Institutes.

• Enhanced cybersecurity, including secure by design principles, is a foundational measure for all frontier AI developers, among others.

• Frontier AI developers also have a unique responsibility to support and enable efforts to understand AI capability and risk, including cooperation in AI safety research, and sharing data on how their systems are used.

• These developer practices are one part of the overall risk mitigation approach. Other processes may be required to increase societal resilience for a world in which proliferation of capable AI systems is the norm.

Roundtable 6: What should national policymakers do in relation to the risk and opportunities of AI?

Discussion of different policies to manage frontier AI risks in all countries including monitoring, accountability mechanisms, licensing, and approaches to open-source AI models, as well as lessons learned from measures already being taken.

Chair: Rebecca Finlay, Partnership of AI

• National governments, from all corners of the world, have an interest in ensuring that the benefits of AI can be realised to their fullest, and risks properly managed.

• It is important to address existing as well as emerging risks, including misuse, loss of control, and wider societal harms, while ensuring proportionate risk-based approaches that can fully realise the unique opportunities brought by frontier AI.

• Managing this balance between risks and opportunities has been challenging given the rapid pace of AI development and successfully navigating this will require regulation and innovation to go hand in hand. They do not sit at opposite ends of the spectrum, and regulation can drive innovation, including through policies such product safety laws and sandboxes.

• To support this, there is a need to build a deeper understanding, including of frontier capabilities, the application of AI across different domains, and the potential role of AI safety institutes. Noting the announcements by the UK and US, further consideration is needed to understand what work such institutes will take on.

• Governments recognise the borderless nature of AI. We will be better positioned to overcome these global challenges by working together, including where national circumstances differ. There is a particular need for building capacity across governments, which may include shared resources and standards to enable interoperability, collaboration, and a shared understanding of AI risks and mitigations.

• There is a need for governance to be rapid, agile and innovative. However, recognising the unique role of governments to act on behalf of their citizens, action must also be taken to address the digital divide, including through education and skills development, to build and maintain public trust.

Roundtable 7: What should the International Community do in relation to the risks and opportunities of AI?

Discussion of where international collaboration is most needed to both manage risks and realise opportunities from frontier AI, including areas for international research collaborations.

Chair: Tino Cuéllar, Carnegie Endowment

• Concerted action is needed. An international approach should be underpinned by shared values and by implementable and realistic actions. The approach should be innovative, collaborative and inclusive, as well as an awareness of risk.

• AI systems are inherently international. Systems created in one country can be easily and quickly deployed in another. The number of countries hosting frontier AI systems will rapidly increase in the coming years, including open-source models which can quickly spread.

• Globally, we also face common challenges in responding to AI advances. We are only at early stages of understanding the capabilities of frontier AI systems, risks and opportunities they present, as well as policies and tools to safeguard systems.

• It is both necessary and in our mutual interest to work together, including where political systems and national frameworks differ, to coordinate responses and collaborate on shared challenges. The global AI Safety Summit, and other multilateral fora, shows that the world is already choosing this path.

• Over the next 12 months our assessment of the priorities for international collaboration with respect to frontier AI are:

• to develop a shared understanding of frontier AI capabilities, and the risks these systems present to global safety and human wellbeing

• to develop a coordinated approach to safety research and model evaluations of frontier AI systems, including how these are applied

• to develop international collaborations and partnerships aimed at ensuring the benefits of AI are shared by all, in ways that narrow global inequalities rather than widen them

• These priorities can be taken forward in a range of multilateral fora. We should work together to ensure different initiatives are complementary and focussed.

Roundtable 8: What should the Scientific Community do in relation to the risks and opportunities of AI?

Discussion of the current state of technical solutions for frontier AI safety, the most urgent areas of research, and where promising solutions are emerging.

Chair: Dame Angela McLean, Government Chief Scientific Adviser, Government of the United Kingdom

• Current models are not the answer. We need better ones. We need lots of research on new architectures, which are engineered to be safe by design. We have a lot to learn from safety engineering. We need to add non-removable off switches. We need to discuss open and closed release but not too heatedly, and model size matters in that discussion. Epistemic modesty is crucial, we have lots of uncertainty.

• We need to understand existing risks of current models. The number of actors involved in designing AI, and evaluating throughout the lifecycle, is tiny. The burden of proof on safety should remain with the vendors of AI. The scientific community’s role is to design the tests to demonstrate safety for policymakers.

• We need a list of open research questions which we will gather together. We should draw on multiple methodologies, including social science, as this is a sociotechnical challenge. We need to identify and focus on the most important questions; speed is of the essence. We both need to focus, but balance this with spreading our bets given the uncertainty. Scientists must work hand in glove with policymakers, bringing the scientific method to the heart of these risk assessments.

• We need to beware of the concentration of power we have seen with the internet. We need geographical and linguistic inclusion. We need to hear from the public and there are many voices that need to be heard and amplified.

See a summary of the discussions from day 2 of the Summit.