Student Loans Company: Postgraduate Assessment Rules
Performs an automatic assessment of applications from applicants wishing to obtain contribution to cost funding for postgraduate study.
Tier 1 Information
1 - Name
Postgraduate Assessment Rules
2 - Description
The postgraduate assessment rules engine can perform an automatic assessment of applications from applicants wishing to obtain contribution to cost funding for postgraduate study.
The system uses the information provided by applicants and course information from higher education providers to automatically determine eligibility and entitlement to funds in accordance with government policy (English and Welsh devolved administrations) on any given supported academic year.
The system allows the SLC to provide the service to the public at scale in a logical and consistent manner.
3 - Website URL
https://www.gov.uk/funding-for-postgraduate-study
4 - Contact email
Tier 2 - Owner and Responsibility
1.1 - Organisation or department
Student Loans Company, Technology Group
1.2 - Team
Assess Platform
1.3 - Senior responsible owner
Chief Information Officer Head of Software Delivery
1.4 - External supplier involvement
Yes
1.4.1 - External supplier
Thoughtworks Mastek Atos / Eviden
1.4.2 - Companies House Number
Thoughtworks - Company number 04091535 Mastek- Company number FC025645 Atos/Eviden - contracted with Syntel Europe Limited - Company number 03227061
1.4.3 - External supplier role
Delivery of postgraduate student finance products began in 2015 since then SLC has engaged multiple third party partner organisations to help develop, evolve and maintain the post-graduate (PG) rules engine.
Thoughtworks were involved in the early inception of the service, providing the SLC with additional software development staff to develop the initial core functions of the service to deliver the PG Masters and then PG Doctoral products.
Mastek provided software development staff to supplement SLC and Thoughtworks as regular annual updates were implemented and the new PG Doctoral product was launched.
Atos / Eviden replaced the previous partners and provided continued involvement alongside SLC development staff implementing annual changes from updated government policy.
1.4.4 - Procurement procedure type
SLC - Awarded via Framework - Technology Services 2 Agreement - RM3804
The original supplier involved in the procurement is a legacy supplier that are no longer involved with the product which is developed within SLC.
1.4.5 - Data access terms
The terms of access to (Government) data granted to the supplier are provided within the Framework Terms and Conditions of the Technology Service 2 Framework Agreement - RM3804. In specific clause 22.7 discusses the license granted by the Customer to the Supplier as follows: The Customer hereby grants to the Supplier a royalty-free, non- exclusive, non-transferable licence during the Call Off Contract Period to use the Customer Software, the Customer Background IPR and the Customer Data solely to the extent necessary for providing the Services in accordance with this Call Off Contract.
Tier 2 - Description and Rationale
2.1 - Detailed description
Postgraduate Rules make automated decisions considering the information provided by students (e.g. identity and residency information), higher education providers (e.g. course information) and other government sources (e.g. HMRC for income information). It determines any outstanding information and/or evidence needed to help guide students towards successful completion of their applications, as well as automatically deciding eligibility and entitlement amounts if possible. The tool supports manual assessment and manual overrides which allows student finance officers to manually validate and update evidence and eligibility information, this takes place when there is missing information or evidence e.g. proof of residency history for eligibility. The rules involved are individually straightforward but collectively complex. This complexity is tackled partly using a third party open source rules engine named Clara. This is not an AI tool - there is no “scoring” or analysis of data using complex statistical or machine learning algorithms - it is instead a forward-inferencing rules engine (based on the RETE algorithm). Its usefulness derives from its support for order-independent authoring, composition and application of the large and overlapping rulesets required by the Department for Education and the Welsh Government each academic year.
2.2 - Scope
The post-graduate assessment tool and its rules are used solely for assessing student finance applications for post-graduate student funding for England and Wales. This process can be initiated in two ways: either by submitting an application via the SLC post-graduate application portal, or via a paper based application which, when received, is data entered by SLC operational staff into the tool. Both routes result in automated processing of the application via the rules engine.
2.3 - Benefit
The Student Loan Company receives a large volume of applications each year, with varying policy rules applied based on the cohort, domicile, product and academic year. The volumes received across the business cannot be manually processed in a timely, consistent, or cost effective manner. The automated processing of applications supports the large volumes of applications and results in the rules for each application being applied in a consistent manner. Where exceptions occur the system updates the application state accordingly to indicate to student finance officers that manual intervention is required. The automation of the majority of applications allows operational staff have the capability to deal with each case and provide more care and attention to individual customers with specific needs.
2.4 - Previous process
Prior to the introduction of this assessment functionality, SLC provided no financial support for post-graduate study. The introduction of government policy to provide student finance to post-graduate students necessitated the need to introduce a solution to assess all the extra tens of thousands of applications on top of the volume of applications SLC already received for other SLC services. A project was instigated to build a new system, with a decision made at senior levels of the organisation to implement the solution using modern development standards, tools, technologies and processes.
2.5 - Alternatives considered
SLC considered other implementation options, including using existing assessment systems, to deliver the postgraduate assessment rules. The decision was taken to use more modern technologies and technical architecture to deliver the postgraduate service.
Tier 2 - Decision making Process
3.1 - Process integration
Post-graduate Rules provide decision making within a wider postgraduate assessment platform handling processing of student finance applications. Applications are either entered online or submitted on paper, when these applications are received by the postgraduate assessment platform they are automatically and immediately assessed to determine their status. The post-graduate rules engine is the sole decision making tool for post graduate student finance application for England and Wales. The platform gathers and persists the information used by, and calculated by, the rules and presents it where needed to customer, assessor and advisor portals. If the rules determine that insufficient information or evidence has been provided that will be the outcome of the assessment flagged to student finance officers for manual intervention. The platform also orchestrates automated correspondence to students, including chasers, and instructs SLC’s downstream processes to set up payments to students (maintenance loans, grants and bursaries), universities (fees) and others (e.g. child care providers for those receiving child care support) that total up to their calculated entitlement amounts on registering with their university. From there repayment processes are ultimately set up on completion of their course.
3.2 - Provided information
The information considered by post graduate rules includes:
- the data provided by the customer on their application for student finance (including their address)
- course details supplied by their university
- any supporting evidence submitted
- other reference government data (e.g. assessed household income (where applicable)
- other applications for student support (e.g. customers cannot have concurrent undergraduate and post graduate applications)
The post-graduate assessment rules engine will use this information to return an outcome based on the postgraduate assessment rules. The outcome will typically comprise: - identified missing information, if any - identified missing evidence requirements, if any, including a breakdown of validated evidence supplied that may meet these requirements - a list of eligibility criteria and whether they have been met or need manual assessment - entitlement amounts per product (loans, grants etc) - an overall application status (e.g. pre-approval - missing evidence, pre-approval - missing information, approved, ineligible, etc)
3.3 - Frequency and scale of usage
The following provides approximate numbers for loan applications processed by the post-graduate rules engine: Over the last 5 years, the algorithm has processed an average of 115,000 applications per annum across England and Wales. Applications from England make up 106,000 of these applications, while Wales makes up the remaining 8,000. In England, 100,500 applications come from students studying Masters degrees and 5,500 from Doctoral students. In Wales, these numbers are roughly 7,500 for Masters students and 500 for Doctoral students. Approximately 3-4% of applications in England are submitted as paper based applications. In Wales, this number increases to 5-6% of applications. All applications are processed by the same rules engine.
3.4 - Human decisions and review
A typical application for a post-graduate loan is fully automated, adhering to the business rules developed from government policy. Only when there is missing information (paper-based applications only), missing evidence or evidence that fails to meet requirements (online and paper-based), will the rules result in fallback to manual intervention. These exceptions are manually processed by staff trained to deal with these circumstances. Applications that do not meet the eligibility criteria (e.g. the applicant is over the age limit at the start of the course) will be automatically assessed as ineligible.
3.5 - Required training
Development team staff are provided an overview of the entire system and an introduction to the individual services that encapsulate the assessment process. Development staff are provided a checklist of information that they are required to learn about the system and are given reference documentation on the development, deployment, support and maintenance of the system. Staff also work with more experienced members of the team for a period of time to get real world knowledge of the system and companywide processes to support the overall system.
Operational Staff undertake training provided by the Learning and Development team. Full system training will be provided to new recruits with top-up training provided where changes are introduced within the system. Work instructions and Guidance documentation is also produced as a reference tool.
3.6 - Appeals and review
If a customer feels a decision in relation to their student finance application has been made incorrectly, they have the right to appeal. An appeal is a formal request for the decision to be reviewed. Appeals can be submitted by post or email.
When submitting an appeal customers need to provide details of the award they’re appealing and why they consider SLC’s decision to be wrong. They should also enclose any evidence that supports their case.
The appeals process takes on average 40+ days and will be dealt with by the Appeals team with no automated processing. An appeals officer will review the original decision and details alongside any supporting evidence submitted as part of the appeal.
Submitted appeals will be rejected where it is not a genuine appeal, such as disagreeing with policy, or requiring more information regarding their application.
Tier 2 - Tool Specification
4.1.1 - System architecture
The cluster of services that encapsulate the business rules are implemented using a micro-services architecture. Each micro-service has been designed with a specific function which coordinates with other micro-services to fulfil the wider business process. Each micro-service is deployed independently using an automated deployment pipeline. These services are deployed across multiple development and test environments before reaching production; at each stage there is a suite of automated tests increasing in breadth and depth to maintain correctness. The services are deployed into production with multiple instances to provide high availability, robustness and scalability. Services can be updated regularly, with the processes allowing for multiple updates to the service daily, if necessary, with no visible impacts to users.
4.1.2 - Phase
Production
4.1.3 - Maintenance
The services that deliver the assessment process are supported, updated and maintained on a regular schedule:
- there are typically annual changes to government policies that necessitate an update of the business rules and processes implemented.
- periodically there are projects that take place that may result in changes to the integration of the services with other systems.
- there are irregular clarifications of government policy that may result in updates to the rules throughout the year.
- any defects that are reported/identified can be rectified on an on-going basis
- there are dashboards that monitor the system for any problems and members of staff that regularly inspect the system for problems. The deployments to the underlying services are in themselves automated and regularly maintained. The underlying infrastructure is scanned and patched on a monthly basis and the various technologies used are subject to a continuous upgrade process.
4.1.4 - Models
The core business rules contained within the services are implemented using a business rules engine named Clara, which in itself is written in the Clojure programming language. The data models consumed and produced by Postgraduate Rules are entirely in-house, relatively small, and completely independent per customer apart from shared reference input data such as course details.
Tier 2 - Model Specification
4.2.1 - Model name
Student finance policy for post graduate students in England and Wales
4.2.2 - Model version
Student finance policy for post graduate students in England and Wales for each academic year since 2016
Postgraduate Masters
- Student Finance England Annual Policy from 2016 onwards
- Student Finance Wales Annual Policy from 2017 onwards
Postgraduate Doctoral - Student Finance England Annual Policy from 2018 onwards - Student Finance Wales Annual Policy from 2018 onwards
4.2.3 - Model task
To determine the Post Graduate Student finance application eligibility and entitlement.
4.2.4 - Model input
PG Student finance application and course data
4.2.5 - Model output
PG Student finance application eligibility and entitlement based on the information and evidence supplied by the customer (and any sponsors, where required) on their application and the higher education provider about their course.
4.2.6 - Model architecture
Forward-inferencing rules engine using the RETE algorithm.
4.2.7 - Model performance
The postgraduate assessment rules were tested across the various pre-production environments in a variety of ways:
-
in all pre-production environments automated tests created targeted test data in order to ensure that the rules generate the expected outputs
-
in the test environments example test data is also created by the testing teams in order to ensure that the rules generate the expected outputs.
SLC has a rigorous testing approach which covers all areas of the test pyramid at various levels including unit, system, integration and manual testing. Automated tests run as part of every release, a full and rigorous test phase takes place every year as part of the annual service launch this includes end to end, performance and user acceptance testing. The system has also undergone penetration and code review plus regular security scanning.
Formal sign-off from a disparate set of senior stakeholders is required prior to each annual service launch.
4.2.8 - Datasets
PG Student finance applications and course data provided by students, sponsors and higher education providers.
In the pre-production environments test data is created by automated tests in order to ensure that the rules generate the expected outputs.
In the test environments test data is also created by the testing teams in order to ensure that the rules generate the expected outputs.
The SLC operational teams have created a specific dataset of information suitable for their needs in the training environment.
4.2.9 - Dataset purposes
Validation - In the pre-production environments test data is created by automated tests in order to ensure that the rules generate the expected outputs.
Testing - In the test environments test data is also created by the testing teams in order to ensure that the rules generate the expected outputs.
Training - In the training environment the SLC operational teams have created a specific dataset of information suitable for their needs.
Tier 2 - Data Specification
4.3.1 - Source data name
PG Student finance application and course data
4.3.2 - Data modality
Text
4.3.3 - Data description
PG Student finance application and course data
4.3.4 - Data quantities
The volumes of data used varies depending on the environment.
Since the launch for the postgraduate student finance service in 2016 for England and Wales there have been approximately 1 millions applications.
Data in test environments typically exceeds Production significantly due to automated, user and performance testing.
4.3.5 - Sensitive attributes
The dataset contains personal data including personal identifying information about SLC Customers (sponsors and students) including but not limited to: Name Date of Birth E-mail address Telephone Address National Insurance Number Passport Number or Identity Document information Bank Details
4.3.6 - Data completeness and representativeness
The production dataset uses information provided by customers as part of their PG Student finance application and course data provided by higher education providers so this data is complete and representative of the target population.
4.3.7 - Source data URL
N/A - Not a public dataset
4.3.8 - Data collection
The data was/is collected when customers apply for student finance, either online or on paper. This is the only purpose of the data.
Customer data (e.g. name, address) may already exist from previous student finance applications (e.g. for further or higher education) and can be updated/reused for subsequent postgraduate applications.
4.3.9 - Data cleaning
SLC uses data provided by customers and higher education providers without any pre-processing or cleaning.
Data in validated at the point of entry for online applications. Any data quality issues will be addressed via communication between student finance officers and customers.
4.3.10 - Data sharing agreements
SLC shares data with other government departments as part of the assessment process where required based on each individual application.
As part of the assessment process this can include: HMPO HMRC The Home Office
Customers provide their consent to share this data as part of the application process.
4.3.11 - Data access and storage
SLC is responsible for the storage of the data in the postgraduate assessment platform.
Data in the postgraduate assessment platform is stored in accordance with the principles of the UK General Data Protection Regulation (GDPR).
The data is encrypted at rest, when in transit and access is restricted via role-based access control.
Tier 2 - Risks, Mitigations and Impact Assessments
5.1 - Impact assessment
The postgraduate assessment platform is subject to annual updates based on changes to government policy. Each update to the rules engine is subject to DPIA [Data Protection Impact Assessment] review.
The latest DPIA assessment for the postgraduate assessment platform was approved by the IAO [Information Asset Owner] on the 7th June 2024. This covered annual rollover and policy changes requested by government. (e.g. changes to residence categories, termination provision, and removal of the Student Finance Wales Masters means-tested grant)
5.2 - Risks and mitigations
Information stored and processed by the SLC is valuable to bad actors. All services that store or process information in the post-graduate assessment platform is subject to regular review by the SLC security teams to look for data risks, and systems have robust security protections and security tools regularly scanning for vulnerabilities and intrusion.