FairNow: Conversational AI and Chatbot Bias Assessment
FairNow's chatbot bias assessment provides a way for chatbot deployers to test for bias.
Background & Description
More organisations are starting to use chatbots for many purposes, including interacting with individuals in ways that could result in harm from differential treatment in terms of the user’s demographic status. FairNow’s chatbot bias assessment provides a way for chatbot deployers to test for bias. This bias evaluation methodology relies on the generation of prompts (messages sent to the chatbot) that are realistic and representative of how individuals interact with the chatbot.
In order to test for bias in a chatbot, FairNow’s platform populates a suite of relevant prompts with information that associates the prompt with an individual’s race or gender. The evaluation analyses differences in responses between demographic groups to understand if the chatbot treats members of a different group more or less favorably. The evaluation varies by prompt type in terms of the specific content being assessed. Where customers have logs of previous chatbot interactions and are able to share them, FairNow leverages those logs as context to ensure the bias evaluation reflects user queries and engagement in terms of content, style, and tone.
How this technique applies to the AI White Paper Regulatory Principles
More information on the AI White Paper Regulatory Principles
Safety, Security & Robustness
FairNow’s bias evaluation methodology allows chatbot deployers to test their application for bias. The evaluation can be applied before the chatbot is released, ensuring that the risk of bias is assessed before being placed in front of individuals. It can also be applied when changes are planned to ensure updated versions of the chatbot are not biased.
FairNow’s bias evaluation methodology is not designed to test for safety or security.
FairNow’s bias evaluation methodology can be used to evaluate a chatbot for robustness. By evaluating the quality of responses when the subject is inferred to belong to different demographic groups, FairNow’s evaluation can ensure the chatbot is robust in this way. The input prompts include a level of variety in style and word choice to further test that the chatbot responds in a consistent manner to the same message.
Fairness
This methodology includes an evaluation of chatbot responses to subjects of different races and genders. FairNow’s methodology applies various techniques to measure the favorability of responses in order to measure differences in responses by demographic group.
Accountability & Governance
Bias evaluation results enable the organisation to take accountability for ensuring their chatbots are safe. The results can also be tied to the laws and standards the organisation adheres to in order to demonstrate compliance.
Why we took this approach
The evaluation of bias in chatbots and large language models is a new and evolving space. Companies looking to deploy chatbots in a way that doesn’t favor individuals in certain demographic groups need a way to understand the risks their applications pose and the magnitude of potential issues. FairNow’s chatbot assessment methodology enables users to evaluate their models before they deploy and as part of ongoing monitoring.
Benefits to the organisation using the technique
Organisations attain high-fidelity bias evaluations of their models that reflect the ways in which their customers use the chatbot. Compared with existing chatbot bias benchmarks – which are often not specific enough to reflect actual usages – FairNow’s chatbot bias assessment methodology enables organisations to pinpoint specific issues with bias in relation to the chatbot’s intended and realized use.
The evaluation can be run at any point and does not require the organisation to share any protected data from customers or employees since the prompts are synthetically generated.
Limitations of the approach
The field of chatbot and LLM evaluations is emergent, and we’re committed to ongoing research and development to stay at the forefront of LLM bias testing. First, the field doesn’t yet fully understand the sensitivity of evaluation results to changes in testing procedures. Research shows that evaluation outcomes can change unexpectedly due to slight changes in the wording or style of the input prompt. Second, this evaluation is not comprehensive of all the different ways that a chatbot could display bias. The evaluation currently tests for bias by gender and race, and does not yet test for bias in terms of other relevant factors like age. We’re committed to following the latest scientific literature on this topic and applying our own testing to reduce these limitations. Lastly, this bias assessment is focused on bias (and robustness of responses to individuals of different demographic groups), and is not designed to measure a chatbot’s safety or security posture.
Further links:
- AI Governance Tools: Automated AI Bias Evaluations
- Overview Of FairNow’s AI Governance Platform
- AI Governance Case Study: Bias Evaluation For Recruiting Chatbot
Further AI Assurance Information
- For more information about other techniques visit the Portfolio of AI Assurance Tools
- For more information on relevant standards visit the AI Standards Hub