AI Compliance, Inc.: The AI Audit Bot
The AI Audit Bot is an interactive tool that allows creators of new and existing AI applications to check their products against existing and proposed regulations in a range of jurisdictions.
Background & Description
The AI Audit Bot is an interactive tool that allows creators of new and existing AI applications to check their products against existing and proposed regulations in a range of jurisdictions. Through this, the AI Audit Bot enables creators to determine whether any special requirements and considerations might apply to their products, including information on whether their application might be at risk of being banned in certain jurisdictions. As part of this, the AI Audit Bot also supports creators to identify and clarify any risks that might be associated with different types of AI.
The audit this tool provides is delivered using a user-friendly interactive “Chat” interface. The AI Audit Bot then creates a PDF report to provide a detailed summary of the relevant information.
The tool can be used to conduct audits at any stage of the life cycle of an AI project, from ideation to reviewing possible problems for a system that is already deployed and in use.
Currently the AI Audit Bot is able to support organisations to explore regulations and requirements in the UK, EU and USA, including state and city regulations. This will expand in the future to include more jurisdictions as regulations are adopted in more countries to provide creators with a wide view of requirements and standards that may need to considered.
How this technique applies to the AI White Paper Regulatory Principles
More information on the AI White Paper Regulatory Principles
Safety, Security & Robustness
This tool supports organisations to better understand and address relevant safety and security considerations by clarifying the regulatory considerations that may apply to their technology. It helps organisations to quantify and assess the risks associated with their AI systems by calculating a risk score based on the technologies that are used.
In addition to this, to further support users to enhance the safety, security and robustness of their systems, the AI Audit Bot offers amelioration recommendations for organisations to take in order to mitigate risks.
Jurisdictions vary in the approaches they adopt towards setting and maintaining safety, security and robustness standards. The Audit Bot helps organisations to navigate this to ensure their systems meet the required standards for different jurisdictions. Where specific applications are prohibited or regulated this is explained. Where jurisdictions are concerned with general performance, suggestions are made as to technology and methodology to mitigate this.
Appropriate Transparency & Explainability
Transparency and explainability are factored into the criteria of the assessments the AI Audit Bot conducts. Through this approach, technologies that align well with these principles are highlighted as so, and as a result, will be marked as lower risk. Explainability and transparency are not features of current generative AI solutions, but other AI tools can be used in some applications to help to provide these capacities. Additionally, the technology that is used to generate the reports is transparent and full explainability can be given for any generated results.
Why we took this approach
There are a variety of regulatory approaches that are currently in use across the world, and this may prove challenging for organisations to navigate. However, despite their variations, different regulatory frameworks and approaches lend themselves to systematisation. Understanding these differing approaches is important for organisations that are seeking to deploy or maintain AI systems; it is important in determining whether a given AI application will be likely to require possibly costly compliance work, or whether an application may even face an outright ban.
Using an intelligent Knowledge Graph approach, we have created an underlying knowledge representation that can be used both interactively to create a chat interface available on a website, and that can also be used within a Teams app. Additionally, this approach can be easily updated by non-specialists to generate documentation and training material.
This approach dramatically simplifies the process of navigating increasingly complicated requirements for all AI creators from start-ups to in house developers and managers in established firms.
Benefits to the organisation using the technique
A key benefit that the AI Audit Bot offers to organisations is that this approach, using Intelligent Knowledge Graph technology, offers a repeatable and auditable result. Unlike Generative AI solutions, the results are testable and explainable.
The Knowledge Graph will also be updated as regulations change and develop. We offer a service that also alerts the owner if a change in regulations will affect their technology later on. If more information is required by new regulations, the system will alert the user by email, and request the necessary information to keep the model up to date via our online portal.
Limitations of the approach
A limitation of this approach is that its effectiveness is very much dependent on information that is supplied by a user, and following the assessment, effectiveness also depends on the actions users choose to implement to manage potential risks. The AI Audit Bot is unable to test the application itself. The application will point the user to the appropriate sections of regulations, but following this, it will be the user’s ultimate responsibility to determine and implement the measures that should be taken to ensure compliance .
Further Links
Further AI Assurance Information
- For more information about other techniques visit the Portfolio of AI Assurance Tools
- For more information on relevant standards visit the AI Standards Hub