MOD: Textio
An AI-powered writing assistant tool that enhances job adverts by analysing and optimising language for inclusivity, engagement and effectiveness.
Tier 1 Information
Name
Textio
Description
Textio is an AI-powered writing assistant tool that enhances job adverts by analysing and optimising language for inclusivity, engagement and effectiveness. It provides real-time feedback and predictive analytics, suggesting alternative phrases, to eliminate bias and improve readability. This tool was introduced to improve the quality of job adverts, minimising issues such as unexplained acronyms and long and difficult to understand content blocks. Along with other complementary recruitment process improvements, Textio ensures that our adverts provide content on career opportunities, that we are clear on the skills we are looking for, and effectively promote the range of opportunities available in Defence.
Website URL
Contact email
People-CivHR-Recruitment@mod.gov.uk
Tier 2 - Owner and Responsibility
1.1 - Organisation or department
Ministry of Defence, Defence People Team
1.2 - Team
Defence People Civ HR- Workforce Planning and Resourcing
1.3 - Senior responsible owner
Deputy Director, Workforce planning and resourcing
1.4 - External supplier involvement
Yes
1.4.1 - External supplier
Textio, Amazon AWS, Amazon Guard duty
1.4.2 - Companies House Number
US company, not registered on companies house
1.4.3 - External supplier role
Textio is responsible for the engineering of its tool by designing, developing and maintaining the software and algorithms that power its language optimisation features. Amazon AWS and Amazon Guard Duty are considered as part of Textio’s infrastructure. AWS provides the cloud computing resources necessary for processing large datasets and running machine learning models. Amazon Guard Duty offers security monitoring to protect data and ensure compliance with security standards.
1.4.4 - Procurement procedure type
Head Office Corporate Services (HOCS) commercial outlined using the Crown Commercial Framework to deliver this requirement, this was the preferred option. However, following review there was not a suitable framework in place at the time which fully met the requirement. The requirement was openly tendered enabling CIV HR to assess and select the most economically advantageous supplier against the statement of requirement, this was unsuccessful and no contract was awarded. Textio was the only remaining option, all avenues were explored. As it is actively used by other government departments, Textio was then awarded a direct contract which was processed by HOCS commercial to ensure that MOD terms and conditions are adhered to.
1.4.5 - Data access terms
MOD grants Textio a non-exclusive, worldwide, royalty-free license to use, reproduce, modify and make derivative works based upon the Data solely in connection with use of the Services and Textio’s provision of the Services to Company. MOD represents and warrants that MOD or MOD’s licensors own all right, title and interest in and to the Data, and that Company has all rights in DocuSign Envelope ID: 9D0C21A8-C63C-4A46-BBFB-AAD2A3C936AD- 4. Data that are necessary and sufficient to use the in connection with MOD’s Account on the Services.
Tier 2 - Description and Rationale
2.1 - Detailed description
Textio is a writing platform that optimises job postings, emails, and other recruiting materials to draw in diverse applicants with the skills we need in Defence. It analyses language patterns using machine learning and offers ideas in real-time to improve the efficacy of job advertising. Textio uses Artificial Intelligence and continuously learns from the content uploaded onto the platform.
2.2 - Scope
Textio is a web accessed online platform which assesses how effective your job advert is and suggests improvements. It is designed to improve the number, quality and diversity of candidates to help get the best person in post, first time around. It also has access to the MOD’s online advert library, and can save time by choosing to adapt an existing advert. Textio was introduced as a tool for MOD Main only (i.e. excluding arm’s-length bodies) due to the different grading, and job roles in these parts of the organisation. It is also designed for Civilian roles only.
2.3 - Benefit
The tool was introduced to improve the inclusiveness and effectiveness of adverts, enhancing the Department’s ability to attract and retain a more diverse workforce. The tool also helps us to simplify language in job adverts, improve understanding of what we are looking for in our organisation and this is one part of the jigsaw of continuous improvement to our ability to attract and retain staff.
2.4 - Previous process
Prior to the introduction of Textio, recruiting managers were solely responsible for the creation of the job adverts, this process was supported by internal guidance provided by the CIV HR Recruitment policy team and quality assurance was administered by Defence Business Services (DBS). Introduction of this tool is part of a suite of activity to improve our ability to attract and retain staff.
2.5 - Alternatives considered
Success in recruitment activity is down to a number of factors throughout the recruitment process, from our social media presence, job adverts and the end-to-end recruitment process, therefore more than one intervention is required to optimise our ability to attract and retain the right people with the right skills at the right time. Other initiatives were implemented to strengthen the quality of job adverts, improve applicant engagement and diversity of successful candidates. These include:
- Development of the Recruitment & Resourcing Toolkits to guide managers on the end-to-end process including attraction
- Development of additional guidance e.g. How To Write A Better Job Advert, How To Assess Using CVs and Recruiting Line Manager checklist
- Policy bites (video content hosted on our internal intranet (Defnet)) “How To Write A Better Job Advert and Recruitment”
- Implementation of the new CS Jobs system, VX (offering a simpler and user-friendly system) and DBS How to guides on using VX (e.g. creating an advert)
- LinkedIn advertising for external roles, enabling adverts us to reach a wider audience
- Removal of Minimum Entry Qualifications (no longer stipulating certain qualifications depending on grades across all adverts)
- Monthly Network working group sessions (comprising of reps across MOD) to share best practice when recruiting and filling vacancies
Tier 2 - Decision making Process
3.1 - Process integration
Recruiting managers copy text from an advert created on the CS jobs platform and populate it within Textio. By providing managers with this tool to optimise job adverts for language inclusivity and effectiveness, it ensures that we are able to reach a broad range of potential applicants. Textio helps managers to craft engaging, unbiased adverts, aligning with recruitment efforts and MOD’s diversity and inclusion goal whilst enhancing overall candidate engagement.
3.2 - Provided information
Textio provides decision makers with detailed insights, offering overall text quality scores, tone and inclusivity analysis and word and phrase suggestions based on content uploaded onto the software by the employee. It also includes data-driven insights with performance metrics, predictive analytics, context-specific recommendations tailored to industries and audiences, real time feedback, comparative analysis through benchmarking and historical data insights helping the user to refine their writing.
3.3 - Frequency and scale of usage
Since the start of using the platform, 2556 internal users (recruiting line managers) have registered to use Textio. The quantity of data being processed will also depend on the volume of content and the frequency of usage. The software is mandated for use for all recruiting line managers, as awareness grows across the organisation in regards to the tool usage will increase.
3.4 - Human decisions and review
Textio utilises artificial intelligence and predictive analysis to analyse language patterns and provide real-time feedback. The user is then given a score out of 100 which determines readability and the level of inclusive language, a score between 80-100 is deemed to be more effective in terms of potential candidate engagement, users have the ability to add or subtract words to improve the scoring. They also have the ability to override suggestions that may not align with the MOD brand. The Employee/User makes the final decision regarding the content of the job advert.
3.5 - Required training
Textio: Employees are trained in data protection principles, data handling, data regulation and best practice.
MOD employees: Intermittent training is provided to staff to ensure best practice and to enhance knowledge around the software.
MOD account manager for Textio: Cybersecurity awareness & training procedures for privileged users.
3.6 - Appeals and review
N/A
Tier 2 - Tool Specification
4.1.1 - System architecture
Textio’s architecture combines data ingestion from various sources, Natural Language Processing (NLP) to analyse job advert content to predict advert effectiveness. Providing real time feedback through a user-friendly interface, it leverages cloud infrastructure (AWS) for scalability and data processing. Security and compliance measures are integrated to protect user data and ensure it is a robust and reliable tool for optimising job postings.
4.1.2 - Phase
Beta/Pilot
4.1.3 - Maintenance
Textio employs several security measures to safeguard user data, including the encryption for secure transmission and storage, strict access controls limiting system, and database access to authorised personnel. Textio conduct regular security audits to identify and address vulnerabilities and also it has established incident response protocol for swift detection and mitigation of security incidents. Textio ensures that it complies with data protection regulation (GDPR) and DEFCON 532b.
4.1.4 - Models
- Rule-based model: identifies and flags biased and non-inclusive language.
- Machine learning: uses extensive datasets to predict the effectiveness of different language choices and its impact on candidate engagement.
- Statistical: analyses trends and patterns in job postings providing benchmarking against industry norms.
- Natural language processing (NLP): analyses and understands text to offer real-time suggestions for improvement.
Tier 2 - Model Specification
4.2.1 - Model name
Textio uses a wide range of AI techniques and tools, including regularized linear and logistic regression, topic modelling (LDA), conditional random fields (CRFs), convolutional neural networks (CNNs), GLoVe, ELMo, BERT, DistilBERT, word2vec, and tok2vec. Textio uses prompt engineering over GPT-3.5 for generative AI features and an internal, Textio-proprietary scoring system to post-filter generated text.
4.2.2 - Model version
For AI models, GPT-3.5 turbo is the version. Since Textio’s product is comprised of many different ML models, these are kept up-to-date to the latest versions.
4.2.3 - Model task
The AI model designed by Textio is focused on assisting users in writing various documents like job postings and recruiting emails.
4.2.4 - Model input
The AI model’s input for Textio includes various types of data sources used for training and operation, such as job postings, recruiting emails, public sources, survey responses, and expert linguistic annotations. The input text from writers is masked to exclude personal information, and the training data is segmented to focus on relevant text data collected through the Textio Data Exchange Program. Customers must explicitly opt into the Textio’s Data Exchange program.
4.2.5 - Model output
AI-Generated Text: Scored internally for quality, with low-quality output flagged and corrected. Model Predictions: Combined to provide user-understandable scores and guidance. Continuous Retraining: Models are continuously retrained with new data to maintain effectiveness.
4.2.6 - Model architecture
Textio uses a mix of Natural Language Processing (NLP), Machine Learning (ML), and heuristic approaches in its AI models. The AI model also utilizes deep learning techniques such as ELMo and BERT. Textio models are composed of multiple layers of sub-models to enable introspection and user-facing guidance. All documents are featurised using an NLP data pipeline. Heuristic models layer reasoning over these NLP features. Models that use ML are trained against proprietary datasets, where models learn to predict effectiveness of writing based on observed outcomes.
A vast majority of these are traditional ML algorithms such as SVMs, Random Forest, LDA-based topic modelling, and regression models. These include supervised (19) and unsupervised (1) traditional models. We have 1 transformer-based LLM/AI model based on GPT-3.5 turbo.
4.2.7 - Model performance
To ensure the models are accurate and performant for a given task, Textio employs linguist analysts to hand-label new data to test our ML features against on a regular basis. We also have continuous monitoring of our services, including ongoing, regular unit testing and automatic evaluations of our models to check for model decay or drift.
Textio’s team tests the predictions of every new model version against a held-out set of the most recent known hiring and survey outcomes, sampled from across representative industries and data sources. To ensure that custom-built models are accurate and performant for a given task, Textio’s team tests the predictions of every new model version against a held-out set of the most recent known hiring and performance feedback outcomes. Textio does not share its results.
4.2.8 - Datasets
The data sets used to develop the AI model at Textio include: - Job postings with time to fill, applicant numbers, and demographics - Recruiting emails and responses - Data from public sources - Proprietary licensed data sets - Internally labelled data - Customer data per service agreements - Survey responses - Expert linguistic annotations
4.2.9 - Dataset purposes
The data used for training Textio’s ML systems is hand-labelled and validated by humans. Textio double-annotate data and only accept judgments that have been validated by several points of view before using it to train Textio’s models. This gold standard data is split and used for training, validation, and testing before being released to the production model.
Tier 2 - Data Specification
4.3.1 - Source data name
Textio
4.3.2 - Data modality
Text
4.3.3 - Data description
Job postings, emails, reports, marketing content and any other form of written communication.
4.3.4 - Data quantities
N/A
4.3.5 - Sensitive attributes
Textio captures user/employee information such as name, email and recruiting managers job title.
4.3.6 - Data completeness and representativeness
To ensure high quality text data for Textio language models, their processes include normalising text, tokenising the text, removing duplicates and irrelevant content, handling missing data, correcting linguistic errors and adding annotation for specific tasks. In addition they conduct manual reviews, these steps help to ensure accurate and reliable datasets.
4.3.7 - Source data URL
N/A
4.3.8 - Data collection
- All data is stored with Amazon - AWS data and is stored in the US.
- In regards to the use of meta data, this option has not been activated for MOD as we do not have SSO (Single Sign On). This means information on IP addresses is not being captured.
- MOD is not a ‘data exchange partner’ - a service highlighted in the contract whereby the organisation shares information from their ATS (Applicant Tracking Software) to measure the performance of Textio job adverts.
4.3.9 - Data cleaning
To ensure high quality text data for Textio language models, their processes include normalising text, tokenising the text, removing duplicates and irrelevant content, handling missing data, correcting linguistic errors and adding annotation for specific tasks. In addition they conduct manual reviews, these steps help to ensure accurate and reliable datasets.
4.3.10 - Data sharing agreements
All data is stored with Amazon - AWS data is stored in the US.
4.3.11 - Data access and storage
Data retention and deletion of personal data: Textio provides a mechanism that allow staff to request the deletion of data. The user can contact privacy@textio.com, this is also referenced in the Textio privacy policy. Textio employs several security measures to safeguard user data, including the encryption for secure transmission and storage, strict access controls limiting system and database access to authorised personnel. Textio conduct regular security audits to identify and address vulnerabilities and also it has established incident response protocol for swift detection and mitigation of security incidents. Textio ensures that it complies with data protection regulation (GDPR) and DEFCON 532b.
Tier 2 - Risks, Mitigations and Impact Assessments
5.1 - Impact assessment
Textio employs several security measures to safeguard user data, including the encryption for secure transmission and storage, strict access controls limiting system and database access to authorised personnel. Textio conduct regular security audits to identify and address vulnerabilities and also it has established incident response protocol for swift detection and mitigation of security incidents. Textio ensures that it complies with data protection regulation (GDPR) and DEFCON 532b.
5.2 - Risks and mitigations
Due to MOD employee personal data being stored in overseas territory (MOD staff names, role and email), a data breach may have concerning consequences, i.e. identification of defence personnel. Due to the minimal storage of sensitive data and robust safeguards put in place by the supplier, this was deemed a low level risk according to MOD’s Secure By Design process. The suppliers service agreement, risk assessment and treatment process incident response process and information security policy have been reviewed and approved by subject matter experts within MOD.