Barnet Council: Ami Chatbot
Implementing an automated webchat functionality ('chatbot') to the Barnet Council website to provide a better customer experience.
Tier 1 Information
###Name
Ami Chatbot
Description
Integration of an AI-powered chatbot into Barnet Council’s website to provide a more seamless resident experience across the council’s services.
Website URL
N/A
Contact email
insightandintelligencehub@barnet.gov.uk
Tier 2 - Owner and Responsibility
###1.1 - Organisation or department
Barnet Council
1.2 - Team
Resident Experience and Digital
1.3 - Senior responsible owner
Assistant Director of Resident Experience and Digital
1.4 - External supplier involvement
Yes
1.4.1 - External supplier
Capita PLC
Conversations By Ami Ltd (Ami Ltd)
1.4.2 - Companies House Number
Capita PLC Company number 02081330
Conversations By Ami Ltd (Ami Ltd) Company number 11961044
1.4.3 - External supplier role
Capita plc (‘Capita’) provides back-office services to Barnet Council under contract. These services include IT and customer services. This contract focuses on supporting Barnet Council by enhancing its digital capabilities and providing services that residents and businesses interact with every day. In line with Barnet Council’s Resident Experience and Digital Strategy and Capita’s Advance & Evolve function, the Ami chat bot has been deployed by Capita in to the AWS Contact Centre managed by Capita to provide a better customer experience by signposting customers 24/7 to the information or service they need on Barnet Council’s website without requiring staff interaction.
Ami Ltd supplies conversational AI under contract to Capita to fulfil their contractual agreements to provide customer services to Barnet Council.
1.4.4 - Procurement procedure type
Competitive procedure with negotiation
1.4.5 - Data access terms
Capita plc provides customer services to Barnet Council and has full data access in order to fulfil the service requirements.
Ami Ltd will implement the Ami software onto the Barnet Council website. Ami staff as well as the Barnet/Capita project team will have access to the data being collected via the Ami dashboard. There will only be a window of time where this data might contain personal data (i.e., IP address, or where customer volunteered unsolicited personal data) prior to the redaction process, which at its longest would be 90 days. Handling and processing of personal data by the Ami development team is restricted in the live production infrastructure. No personal data is extracted or used outside of the AWS production setup. As such, all personal data is protected by the physical controls in place at AWS data centres.
Tier 2 - Description and Rationale
###2.1 - Detailed description
The processing involves AI (artificial intelligence) chatbot technology running on the Ami platform. Ami is a cloud-based application which resides in a Virtual Private Cloud (VPC) hosted by Amazon Web Services (AWS) Ireland. Ami does not currently use any third-party API’s or AWS services to achieve its AI capability. Ami’s AI uses in-house algorithmic models utilising decision lists and sets which enable Ami’s AI to analyse the designated web content and any other supplied text or documents, along with already-supplied general knowledge, creating a unique model for near instant querying.
Ami utilises a ‘tag’ of JavaScript code which is applied to designated Barnet Council web pages to read content from the page and/or display the chat widget so that it is available for the customer to engage. (Barnet Council has full control over which pages Ami ‘reads’ and which pages display the Ami widget for a customer to use.)
2.2 - Scope
Ami will be integrated into Barnet Council’s website and will provide a better customer experience by signposting customers 24/7 to the information or service they need on the website without requiring staff interaction. Ami will also be able to assist with supporting more vulnerable customers by identifying when they need further support and suggesting the best way for them to get this. During business hours when chat is staffed, the chatbot will be able to connect them directly to an agent in the contact centre. This pilot project will focus on a few key content areas (Council Tax, Housing Benefits, Waste and Recycling, Schools, Pest Control) and will link to Barnet Council’s customer contact centre AWS Connect system for agent-supported chat.
2.3 - Benefit
The provision of a chatbot on the Barnet website aims to provide greater choice for how residents communicate with the council, in line with Barnet’s Resident Experience and Digital strategy. It will provide a more seamless resident experience across the council’s services. Information and insight from the pilot will also be used to inform the ongoing provision of Barnet webchat.
2.4 - Previous process
Prior to the deployment of the tool, a resident would source the information either directly from Barnet Council’s website by finding the relevant document through standard internet searchers or by phoning or emailing Barnet’s contact centre (or relevant service) to be provided with the information required.
2.5 - Alternatives considered
Capita plc provides back-office services to Barnet Council, including IT and customer services. Capita plc have had previous experience working with Ami Ltd and the Ami Ltd conversational AI tool. This is a pilot project to evaluate the impact of an AI tool on improving the resident experience. Given Capita’s prior experience, no other alternatives were considered for this pilot.
Tier 2 - Decision making Process
###3.1 - Process integration
The Ami chatbot will be integrated into the Barnet Council website and for the purposes of this pilot limited to providing customers with information on: council tax, housing benefits, waste and recycling, schools and pest control. Customer engagement with the Ami chatbot will be entirely voluntary and throughout any engagement with the chatbot, customers will have the option to stop using the chatbot or to be directed to a human agent.
No formal decisions about a customer will be made based on their interaction with the Ami chatbot. The ‘decisions’ made will be the AI using the information provided by the customer to guide them to the correct Barnet Council webpage or content within a Barnet Council webpage. Where a customer is transferred to a human agent, the chat transcript will be shared so as to provide background and context to the customer’s request so that the human agent can quickly support the customer and provide a response to their query.
3.2 - Provided information
There will be no formal decisions made about a customer based on their interaction with the Ami chatbot. The ‘decisions’ made will be the AI using the information the customer has provided to guide them to the correct Barnet Council webpage or content within a Barnet Council webpage, to inform or answer their query. Ami will have the ability to automatically move a customer directly to that page or content.
Ami will be able to assist more vulnerable customers by identifying when they need further support and suggesting the best way for them to get this. This may include the chatbot directly connecting the customer with a human agent in the contact centre.
The Ami chatbot will be limited in terms of which webpages it can pull content from and it will not be able to pull content from outside of the Barnet Council website. At all times, a customer will be able to stop using the Ami chatbot or request to be connected to a human agent in the contact centre (during standard operating hours).
3.3 - Frequency and scale of usage
It is expected that the Ami chatbot will be used every day or multiple times a week. The pilot project anticipates 30,000 chats over the six month pilot period.
3.4 - Human decisions and review
The Ami chatbot will not make any formal decisions about a customer. The ‘decisions’ made will be the AI using the information provided by the customer to guide them to the correct Barnet Council webpage or content within the Barnet Council website, to inform or answer their query. Interaction with the Ami chatbot is entirely voluntary and the Ami chatbot will also have the ability to connect a customer to a human agent in the contact centre either based on a specific request made by the customer or if the Ami chatbot identifies any vulnerabilities.
The Ami chatbot will only have the ability to direct users to specific Barnet Council webpages and no option to direct users away from the Barnet Council website. Customers will have the option to contact a human agent or not to use the Ami chatbot. Where a customer is transferred the chat transcript will be available to the human agent to provide contextual information and to help the human agent respond to the customer’s requirements. Customers will not be prevented from exercising any rights or using any services, nor will they gain or lose access to any benefits or services.
3.5 - Required training
Implementation of the Ami chatbot into Barnet Council’s website will be conducted by trained staff from Ami Ltd and Capita Plc. Training will be required for contact centre staff to understand the hand-off process between Ami and any human agent.
3.6 - Appeals and review
N/A
Tier 2 - Tool Specification
###4.1.1 - System architecture
Ami chatbot is a scalable application which resides within a secure VPC which itself contains auto scaling and where possible self-healing infrastructure, hosted by AWS Ireland, a multi-location data centre. Ami operates based on tags (JavaScript code) applied to specific pages on Barnet’s website.
Data is transmitted and sent from Ami’s tags into the cloud using end-to-end encryption, and then processed in two auto scaling AWS EC2 (Elastic Compute Cloud) groups: the main application layer and the real-time application layer. The real-time application is used for live chat and agent dashboards. The main application is where the core processing is achieved. A caching layer is used to reduce load into the storage layer. The storage layer contains different types of storage in both auto scaling groups and using serverless storage.
The environment is protected using AWS firewall, WAF, Shield and other AWS products to ensure the highest level of security.
AWS Relational Database Service hosts the storage of Barnet data on physical storage shared with other customers, but in a gated area and clearly delineated.
4.1.2 - Phase
Beta/Pilot
4.1.3 - Maintenance
24 hour system monitoring sends alerts and notifications to a central development team inbox, where they are assigned to specific individuals or teams to be addressed. Current status and resolution information is documented in Ami’s central project management system.
Logs are analysed as part of Ami’s monthly review and maintenance cycle - during which any errors are assigned for action, and potential areas for improvement and enhancement identified. Third parties, customers and other stakeholders are formally notified about incidents within 72 hours if their information or systems are impacted. Maintenance releases are at the discretion of Ami’s team, and main releases are completed weekly or monthly as new functionality and improvements are available. Release notes are circulated at least 48 hours prior.
For Barnet’s implementation, a hypercare plan is in place for 6 weeks post go-live. Performance of the chatbot will be closely monitored during daily reviews of key metrics and regularly scheduled calls, daily for the first week, 3 times a week thereafter. Once moved into BAU, there will be two points of contact for all issues related to the chatbot: the solution product owner for Capita and the operational lead for Barnet’s contact centre. Performance review meetings will take place monthly and a continuous improvement approach taken, in line with other Barnet customer contact centre solutions and processes.
4.1.4 - Models
The Ami AI product is able to use both Prescriptive models and Generative AI models to achieve a balance between the expansive capabilities of Generative AI, while maintaining the precision and reliability that comes from a more structured, prescriptive system. However, for the implementation at Barnet, Ami AI is only utilising prescriptive models as the requirement is for precise and reliable responses. In-house algorithmic models utilising decision lists and sets enable Ami’s AI to analyse the designated web content and any other supplied text or documents, along with already-supplied general knowledge, creating a unique model for near instant querying.
Tier 2 - Model Specification
###4.2.1 - Model name
Ami Chat, Prescriptive R2 (for Barnet)
4.2.2 - Model version
Version 1
4.2.3 - Model task
The Ami model is a conversational AI tool that supports client to end-user interactions through a chat widget interface.
4.2.4 - Model input
User defined conversational enquiry in text format.
4.2.5 - Model output
Conversational response to user defined enquiry in text format.
4.2.6 - Model architecture
Ami does not use generative AI to create messages. Various AI models are used to manage the use of client-approved conversational content - identifying the appropriate approved responses and controlling the overall direction of the conversation.
Inputs: Defining the conversation The Ami linguistics team is responsible for teaching and defining the conversational content for each client instance. As part of the setup process, they define the key conversational content and the algorithms for how and when it should be used. For example: - Prescribe responses to customer questions - Defining goals and outcomes within the conversation - Defining algorithms for how and when these differing conversational elements should be used within conversations.
All the content that is taught and labelled by the Ami linguistics team is based upon agreed and publicly available content. There is no customer personally identifiable information used to setup the client on Ami.
Ami’s intelligence layer: Ami’s intelligence layer is built to interpret the content and goals that have been defined by the Ami team: - Identifying appropriate times to surface specific content - Intervening to proactively guide chats to client defined outcomes - Ensuring conversations are escalated to human agents at appropriate times
The intelligence layer for the Barnet tool is based on algorithmic (prescriptive) models.
Outputs: Response, transcript, review and supervised learning
Following conversations, the Ami linguistics team works with the conversational outputs to analyse performance and support supervised learning. This includes reviewing overall conversational content to identify key trends and patterns in customer behaviour, and conversational events, as well as identifying specific areas of weakness where the overall customer experience can be improved. The outputs of this analysis are fed back into the Ami instance setup, helping to refine the customer experience and continually improve. No specific end user data or personally identifiable information is fed back into the training models or used to generate content.
4.2.7 - Model performance
On average, 90% of queries that the Ami chatbot has content for end up being answered correctly. The model is incapable of hallucination and only provides client-approved, human-assessed outputs.
4.2.8 - Datasets
Tabular data supporting Ami’s understanding of user defined queries and text data defining Ami’s output. Primary knowledge base for this specific implementation of Ami chatbot is all content published on Barnet website (barnet.gov.uk) URLs covering services in scope of pilot: council tax, housing benefits, school admissions, waste & recycling, and pest control. At times this includes specific PDFs as hosted on the website.
4.2.9 - Dataset purposes
Tabular and text data is used to support the Ami model in understanding text prompts. Open data on Barnet’s website is used as Ami’s Barnet specific knowledge base.
Tier 2 - Data Specification
###4.3.1 - Source data name
Barnet Council website data for in-scope services
4.3.2 - Data modality
Text
4.3.3 - Data description
Primary knowledge base for this specific implementation of Ami chatbot is all content published on Barnet website (barnet.gov.uk) URLs covering services in scope of pilot: council tax, housing benefits, school admissions, waste & recycling, and pest control. At times this includes specific PDFs as hosted on the website.
4.3.4 - Data quantities
Ami is reading content from 159 webpages across council tax, housing benefits, pest control, schools and street scene service pages.
There are a total of 1,659 web pages across all of the Barnet Council’s website. Ami is therefore reading content from ~10% of Barnet’s website.
Percentages presented below are percentages of all webpages.
-
Council tax - 28 pages - 1.69%
-
Adult Social Care - 1 page - 0.06%
-
Revenues & Benefits - 20 pages - 1.21%
-
Regulatory Services - 15 pages - 0.90%
-
Schools - 46 pages - 2.77%
-
Recycling and waste - 49 pages - 2.95%
4.3.5 - Sensitive attributes
None
4.3.6 - Data completeness and representativeness
Barnet website data contains content as relevant for Barnet council, in order to explain their services, provide customers avenues to self-serve, contact information and to answer customer queries. The Ami chatbot will highlight where customers are asking Qs which are not addressed by the web content, and thus contribute over time to the improvement of the website content’s ability to address what site visitors want to know.
4.3.7 - Source data URL
All website URLs starting with (parent and child pages):
www.barnet.gov.uk/council-tax;
barnet.gov.uk/council-tax-arrears-and-debt-recovery;
barnet.gov.uk/benefits-grants-and-financial-advice/housing-benefit;
barnet.gov.uk/adult-social-care/money-and-legal-matters/claiming-benefits;
barnet.gov.uk/environmental-problems/pest-control;
barnet.gov.uk/schools-and-education;
barnet.gov.uk/recycling-and-waste;
barnet.gov.uk/gardenwaste: barnet.gov.uk/bulkywaste;
barnet.gov.uk/repair and barnet.gov.uk/reuse
4.3.8 - Data collection
N/A - existing web content for the Council so not developed through data collection.
4.3.9 - Data cleaning
N/A
4.3.10 - Data sharing agreements
N/A - this is council’s existing web content publicly available over the internet
4.3.11 - Data access and storage
N/A - this is council’s existing web content publicly available over the internet. Ami is not separately storing this content.
Tier 2 - Risks, Mitigations and Impact Assessments
###5.1 - Impact assessment
Data Protection Impact Assessment - completed 07/05/2024
5.2 - Risks and mitigations
- Environmental/physical catastrophes which damage or affect the AWS data centre where Ami data is held
- Risk of external attack on AWS data centre (hacking, phishing, etc)
- Risk of internal attack or unauthorised access of Ami data within AWS data centre
- Risk of internal attack or unauthorised access of Ami data by Barnet Council or Capita staff
- Risk of customer inputting information about someone in imminent risk of harm, such as child or elder abuse, domestic violence, suicide, hate crime, dangerous buildings or trees
- Risk of customer disclosing personal or sensitive information about themselves or about other people, including payment details
- Risk of customer utilising the chatbot to make threats against the council or engaging in abusive language toward the council
- Risk of customer disclosing an issue with the belief they have made an official complaint and expect someone from the council to contact them, or otherwise disclose contact details unrelated to a complaint with the expectation of someone from the council contacting them
- Risk of Ami chatbot surfacing incorrect or unrelated content or sending the customer to the wrong page, or not understanding the customer’s query and not being able to address
- Risk of redaction process not fully removing all personal data. Names for example, do not follow any pattern and thus would not be removed.