Research and analysis

Changing the default: a field trial with Zurich Insurance to advertise all jobs as part-time

Published 17 November 2020

Research report

Rony Hacohen, Shoshana Davidson, Vivek Roy-Chowdhury, Daniel Bogiatzis-Gibbons, Hannah Burd and Tiina Likki

Acknowledgements

We wish to acknowledge the roles that the following individuals had in this project:

The following individuals at Zurich Insurance UK:

• Steve Collinson for sponsoring the project
• Emma Francis for leading the project, and supporting us to undertake quantitative and qualitative research
• Neil Capel for providing us with historical and trial data, and implementing the trial
• David Rickman and Leann O’Connor for answering questions on the historical and trial data, and for working closely with us during the earlier exploratory analysis phase of the project
• Richard Crown and the Talent Acquisition Team for implementing the intervention

The following individuals at the University of Exeter:

• Professor Michelle Ryan for advising on how to measure belonging and Associate Professor Oliver Hauser for ongoing academic advice

The following individual at Harvard Business School:

• Ariella Kristal for ongoing academic advice and support with qualitative interviews of Zurich staff

The following individuals at the Behavioural Insights Team (BIT):

• Natalia Shakhina for writing up the trial into this report
• Jessica Barnes, James Lawrence and Monica Wills Silva for their Quality Assurance input at various stages of the project

Executive summary

The Government Equalities Office (GEO) established the Gender and Behavioural Insights (GABI) Programme in partnership with The Behavioural Insights Team (BIT). The GABI programme aims to build evidence on what works to improve gender equality in the workplace. As part of this programme, BIT partnered with Zurich Insurance UK (Zurich) to analyse the drivers of its Gender Pay Gap (GPG), and develop, test and evaluate a behaviourally-informed solution. Both initial exploratory work and the trial were conducted before the introduction of lockdown measures in response to COVID-19 in the UK.

A comprehensive review of Zurich’s internal HR data on recruitment, progression and promotions revealed differences in outcomes between staff who worked full-time and those who worked part-time. The vast majority of part-time workers were women.

Findings showed that, relative to full-time employees, part-time employees were 35% less likely to apply for promotions, received raises that were 1.1% lower, and received lower scores on performance and potential to progress metrics.

Following these quantitative findings, we conducted interviews with 25 managers and employees at Zurich. These suggested that part-time employees at Zurich lacked clear progression paths – they tended to think they were unable to progress without increasing working hours, often because jobs are not explicitly advertised as being open to part-time work patterns. This, alongside a perceived lack of senior role models, seemed to harm career aspirations.

Our interviews also suggested that there was some stigma around working part-time, related to perceptions of commitment of part-time employees, and employees feared being judged for seeking to reduce their working hours. In addition, managers and employees saw part-time work as a privilege, rather than a right.

To address these concerns, we developed an intervention to normalise part-time work at Zurich, by opening all new positions to part-time work by default. The intervention was live for 12 months. Defaults refer to the ‘status quo’ or ‘business-as-usual’ option that is pre-selected, by design or by accident, by the architect of choice. Default options can have a profound impact on human decision making. This is because defaults can remove inertia and uncertainty in human decision making – and allow individuals to autopilot to the path of least resistance. Making an option the default increases the likelihood of adoption because it becomes very easy to follow, while effort is usually required to change to a different option.

Trial design and sample

The intervention aimed to normalise part-time work across all levels of the organisation. To achieve this, new positions at Zurich were advertised as open to part-time patterns or job-shares by default, unless the hiring manager provided a business case for why that was not possible. The intervention went live in March 2019 on Zurich’s hiring platform.

It was not possible to run the intervention as a randomised controlled trial (RCT).^{[footnote 1]} 

Therefore, we conducted a before-after analysis comparing the data from the pre-intervention period (before March 2019) with the intervention period (March 2019 – February 2020).

A total of 1,790 new vacancies were posted for the period January 2017 to February 2020 with 497 in the trial period and 1,293 in the pre-period. Excluding ineligible vacancies, there were 1,432 vacancies total with 391 in the trial period and 1,041 in the pre-period. It is worth noting that by ‘vacancy’, we mean a job posting (some of these vacancies will have been posted together if more than one position was open at once).

Results and implications

The main results of our trial were as follows:

• High compliance with the trial: after discounting exempt business areas, at least 78% of job adverts complied with the policy and advertised the positions as open to part-time patterns or job-shares

• We did not find an increase in the proportion of employees working at Zurich part-time. This could be due to a small increase in the number of retained employees working part-time being counteracted by turnover, with part-time employees disproportionately leaving and no statistically significant change in the proportion of new hires who worked part-time.

• A significant increase of 16.4% in the overall proportion of female applicants (+6 percentage points from the baseline of 36.4% to 42.4%), as well as the proportion of applicants who did not say they were male (+3 percentage points).

• A significant increase of 19.3% in the proportion of female applicants to senior roles (+6 percentage points from the baseline of 31.1% to 37.1%).

• A significant increase of 8% in the number of part-time employees reporting that they feel they ‘belong’ at Zurich (+0.38 percentage points).

To lend context to these findings we conducted additional analysis which found:^{[footnote 2]}

• There was a non-significant increase in the proportion of promotions that went to women - from 50.2% in the pre-period to 56.1% in the post-period (+5.9 percentage points).

• For the proportion of successful applicants within in-process hiring rounds (including both external hires and internal promotions), the results are mixed: there was a decrease (3.2 percentage points) for female applicants but a small increase in female and non-declaring applicants (1.3 percentage points).^{[footnote 3]} For senior roles, there was an increase in female applicants (4.3 percentage points). All these changes were not statistically significant.

These results indicate that the default option of advertising new positions as available to part-time and job-share options was broadly effective at encouraging managers to advertise their vacancies in this way. Furthermore, it may have encouraged more women to apply to vacancies at Zurich, including vacancies for senior roles. Finally, the availability and salience of part-time and job-share roles could have a positive impact on the sense of ‘belonging’ among Zurich’s part-time employees.

However, this was not an RCT and we cannot conclusively say that the above results were caused by the intervention as we are unable to infer causality. Importantly, there were structural and workforce changes over the trial period. This makes it difficult to separate the effect of these changes from the impact of the intervention. We recommend that further research should evaluate similar interventions using an RCT – we are planning to run one with another UK employer.

Introduction

From April 2018, British companies with more than 250 employees were legally required to publish gender pay gap (GPG) reports.^{[footnote 4]} 78% of employers who reported their data in 2019 had higher median hourly pay for men than for women.^{[footnote 5]} Furthermore, almost half of reporting organisations saw an increase in their GPG compared to the previous year.^{[footnote 6]} Part of the gap is associated with women having more caring responsibilities and being less likely to be able to work full-time, which strongly affects wage progression.^{[footnote 7]} The wage gap tends to increase after the birth of a woman’s first child. Approximately half of the widening is explained by the lower hourly rates of part-time work compared to full-time work.^{[footnote 8]} Women who need flexible working arrangements, such as part-time work, may struggle to progress into more senior roles within their organisation. They may also have limited opportunities for external promotions that meet their work-care requirements, as only 15% of advertised jobs explicitly offer flexible working.^{[footnote 9]}

As a result, some women decide to leave the labour market, while others get ‘stuck’ and find it difficult to move role, due to a lack of opportunities to apply for flexible roles.^{[footnote 10]} Some women also report experiencing discrimination when requesting flexibility during the recruitment process.^{[footnote 11]} Therefore, increasing the availability of flexible working arrangements could contribute to improving gender equality in the workplace and reducing the GPG, by making it easier for women to stay in the labour market and progress into jobs with higher salaries.

The Behavioural Insights Team (BIT) is partnering with the Government Equalities Office (GEO) to deliver a multi-year research programme - the Gender and Behavioural Insights (GABI) programme. The purpose of this programme is to create new evidence and drive behaviour change towards greater gender equality in the UK, with a focus on the labour market.

The GABI programme is focused on building the evidence base on what actions are effective at improving gender equality in the workplace, by using behavioural insights and empirical approaches. To accomplish this, GABI is working with a number of large employers to run trials. The methodology for these trials begins with a deep exploratory analysis of payroll, progression, performance management, recruitment and other HR data. This exploratory analysis is designed to identify the root drivers of an organisation’s GPG, and the most impactful areas for running an intervention to improve its GPG.

The implementation partner for this trial was Zurich Insurance UK (Zurich) which is part of Zurich Insurance Group, a global insurance company with about 55,000 employees worldwide, including 4,500 in the UK.^{[footnote 12]}  In 2018, Zurich reported median and mean GPG of around 23%.^{[footnote 13]} While these fell from over 27% in 2017, they were still higher than the UK median of 17.8% in 2018.^{[footnote 14]} Zurich’s GPG was affected by the lack of women in senior and technical roles, and by the vast majority of part-time employees being female (93%).^{[footnote 15]} Zurich aspires to significantly reduce their GPG and  to attract more women into senior roles.^{[footnote 16]} To facilitate these efforts and get an external perspective, Zurich partnered with BIT and the GEO to investigate the causes of its GPG, and to design, test and evaluate a behavioural solution.

The results of this work will be beneficial not just to Zurich, but also to other organisations that have a hiring-related gender problem or a part-time advancement penalty.

The rest of the report is structured as follows:

Section 2 sets out the research aims and methodology, including an overview of findings from initial exploratory work which fed into the development of the solution; the intervention design; the sample; and the components of the evaluation methodology, including outcome measures and a process evaluation.

Section 3 covers the results of the trial, including quantitative and qualitative measures.

Section 4 discusses the key findings from these results and outlines the conclusion.

Research aims and methodology

Exploratory data research and findings

We analysed Zurich’s internal HR data on recruitment, progression and promotions to examine the drivers of its GPG. Overall, our findings were in line with Zurich’s hypotheses about the factors contributing to GPG:

• Women were less likely to apply for roles at Zurich compared to men.

• When they applied, they were no less likely to be hired than men, although we were not able to control for factors such as education or experience level.

• Women working at Zurich did not progress at the same rates as men.

• Women who worked part-time were 4.8% less likely to apply for internal promotions than men that worked part-time. However, there was no statistically significant difference between application rates for full-time men and full-time women (there were very few part-time men).

• Men were 5% more likely to get promoted (simply comparing women and men who applied for promotions).^{[footnote 17]}

• There were no differences in performance scores.

• Women spent about 4 years longer in the same grade than men. Taking into account differences in part-time work, women still spent about 2.5 years longer in the same grade than men. This might be explained by women in lower grades staying there for longer periods of time.

• There were no significant differences in salaries or pay rises within similar roles. However, working part-time was associated with a 1.1% penalty in raises for women.

• Retention rates were similar on average for men and women, although there were indications that there may be lower retention of women among new joiners at more junior grades.

Overall, our analysis revealed that many of the gender differences described above seemed to be driven by worse outcomes for part-time employees (the vast majority of whom were women). Relative to full-time employees, part-time employees were 35% less likely to apply for promotions, received raises that were 1.1% lower, and received lower scores on performance and potential to progress metrics. We also examined the performance scores of employees shifting from full-time to part-time work, and found that they suffered penalties when they did so.

Qualitative research and findings

While our quantitative analysis revealed that there were penalties for part-time employees, we could not identify any established evidence on where these penalties stem from. To better understand why the disparities occur we conducted in-depth semi-structured interviews with 25 managers and employees at Zurich.

The interviews suggested that part-time employees at Zurich lacked clear progression paths. They tended to think that they were unable to progress without increasing their working hours, normally because jobs were not explicitly open to part-time employees. This, alongside a lack of senior role models, seemed to harm career aspirations. Our interviews also found that there was some stigma around working part-time. This was related to the perceived lack of commitment of part-time employees. Employees also feared being judged for seeking to reduce their working hours. In addition, managers and employees saw part-time work as a privilege, rather than a right.

The intervention and rationale

Following the analysis above and our discussions with Zurich, we decided to develop an intervention with the aim of mitigating part-time work penalties by normalising part-time work at all levels of seniority within Zurich. We did not identify any published research evaluating interventions to address this particular problem. However, previous GABI research suggested that one way to normalise part-time work could be through changing job ads.^{[footnote 18]} There is evidence that small changes in the wording of job ads can change applicant behaviour, potentially by changing women’s perceptions that they would belong in the organisation.^{[footnote 19]}

Our intervention made all new advertised roles available as part-time and/or as a job-share by default, in addition to being available full-time, unless the hiring manager provided a business case for why that was not possible. We applied the following behavioural principles in this intervention:

Defaults: our choices are strongly influenced by pre-set options or defaults.^{[footnote 20]} Making an option the default increases the likelihood of adoption because it becomes very easy to follow the default while effort is usually required to change to a different option.

Salience: we have a tendency to focus on information that grabs our attention, which in turn can influence our behaviour.^{[footnote 21]} This is because our attention is a limited resource and usually there are multiple stimuli competing for it. By making part-time working options salient at several points in the job ad, people seeking part-time work can quickly and easily see whether the role is open to them.

The part-time and job-share options were also made more salient by including them in the job title line. Figure 1 below illustrates how the job adverts looked during the trial period, while Figure 2 presents the initial theory of change behind the intervention.

Figure 1. Job advert example for the trial

Note: 1. PT options included in job title in heading. 2. PT options included at top of ad text. 3. Inclusive sentence included in job ad.

Figure 2. Theory of change for the intervention

Intervention:

• Making new positions available PT by default

Outputs and outcomes

• Introduces friction for managers to advertise as FT-only
• Signals a normative expectation to managers
• Even senior roles are advertised as potentially PT
• Encourages PT employees to apply for more senior positions
• Reduces stigma against PT employees
• Improves the sense of belonging of PT employees
• Attracts more women to Zurich

Impact

• Improves career aspirations and progression of PT employees
• Potentially reduces GPG (if women get into senior positions)

There were 2 conditions in the trial: 1) Pre-trial period or ‘Business as Usual’; and 2) Trial period or ‘PT default’. Further details on these conditions are available in Table 1 below.

Table 1. Trial details

Condition

Business as usual (pre-period)

Description

Job adverts at Zurich only mentioned the word “Flexwork” at the bottom of the job advert. In addition, some were advertised as available part-time, but this was extremely rare (less than 1% of positions).

Condition

PT default (trial period)

Description

The intervention included the following elements:

All new positions were open to part-time and/or job-shares in addition to full-time by default, unless the hiring manager provided a business case explaining why that was not possible. 3 areas of the business also asked to be exempt from the entire trial.
Part-time/job-share options were made salient by including them in the job title and at the top of the job advert. As well as this, the following ‘inclusive’ sentence was added to all job adverts by default:

‘This job is available part-time, as a job-share, or full-time. This is because we want the best people for our roles and we recognise that sometimes those people aren’t available full-time.’

These changes appeared both on Zurich’s website, and on other recruitment websites where Zurich shares job adverts (for example, LinkedIn and Glassdoor).

Managers were offered training about managing flexible teams. This element of the intervention was included to prevent a situation whereby more part-time employees were hired and subsequently mismanaged (for example, given workloads disproportionate to their available working hours). This was a concern raised by some part-time employees in the interviews we conducted.

Outcome measures

To assess the impact of the intervention we used one primary and 3 secondary outcome measures.

Primary outcome measure

Our main outcome measure was the proportion of employees working part-time at Zurich. Part-time employees were defined as employees (not including contractors) working at least 1 hour per week at Zurich Insurance UK whose contracted hours per week are 33 hours or less.

We decomposed this primary outcome into the following:

• the proportion of those who remained at Zurich after a company restructure which included redundancies, who changed to part-time working

• the proportion of those who worked part-time who were made redundant in the post-period

• the proportion of new hires who chose to work part-time^{[footnote 22]}

Secondary outcomes measures

We used the following secondary outcomes to investigate the impact of the intervention on the proportion of female applicants:^{[footnote 23]}

• the proportion of Zurich job applicants who say they are female

• the proportion of Zurich job applicants for senior roles (grades 4 and higher) who say they are female

• the proportion of Zurich job applicants who do not say they are male

We analysed this, because applicants could choose to disclose their gender as either ‘male’, ‘female’ or ‘prefer not to say’. We do not know whether people who did not disclose their gender were more likely to be male or female. As such, it is important to consider what happens to the proportion of non-declaring applicants. This is because, if the proportion of female applicants increased, but this was countered by a decrease in non-declaring applicants, we would not be able to confidently say whether the proportion of female applicants had increased relative to male applicants, or if the intervention had simply encouraged more applicants to disclose their gender.

Exploratory analysis (reported in Appendix 2)

We also wanted to explore the impact on hiring behaviour. Therefore, we introduced 3 exploratory outcomes:

• the proportion of new employees who say they are female

• the proportion of total applicants who are successful in a hiring round who say they are female (this includes new hires, promotions and lateral moves at the same grade)

• the proportion of promoted staff who say they are female

Changes to the outcome measures

There were a number of outcomes specified in the original trial protocol that were not measured. In particular, there were 2 outcomes which we could not measure, as we had hoped that they were collected in Zurich’s internal HR systems but they turned out not to be recorded at all:

• the proportion of applicants asking for part-time work, and their hiring outcomes

• the proportion of part-time workers who apply for promotions

Furthermore, we chose to omit measuring the proportion of employees working part-time, by full-time equivalent hours (FTE) and gender. This was planned as a descriptive outcome, but it was not useful because men almost never worked part-time. For example, in the quarter ending March 2018, 27% of women (644 of 2378) worked part-time while 1.5% of men worked part-time (43 of 2961) and men made up only 6.3% of part-time workers (43 of 687).

Design

The intervention was rolled out across the entire organisation at once. Therefore, we could only measure success by comparing the outcome measures before and after it was implemented.

For the primary outcome measure, the pre-intervention measurement was taken in February 2019 and the post-intervention measurement was taken in February 2020.

For the secondary outcome measures, the pre-intervention period was 1 January 2017 to 8 March 2019, and the post-intervention period was 11 March 2019 to 28 February 2020.

This method was chosen because more robust evaluation options were not feasible. It was deemed that an RCT would be too disruptive for the business because jobs would be open to part-time work in a random manner. Gradual implementation of the policy (a stepped wedge RCT) would be difficult to use as a method of evaluation because the business areas at Zurich are too different to each other. For example, HR applicants are majority female whereas almost all engineer surveyor applicants are male. This means that making comparisons between them is not useful. Consequently, a stepped wedge trial would be nearly equivalent in inferential terms to a set of unrelated pre-post analyses, as well as being practically more difficult to implement. Furthermore, the power for a stepped wedge RCT was much lower.

The weakness of this design is that we cannot distinguish between changes that were due to the treatment and those that were correlated with time. There were structural and workforce changes over the period when the intervention was implemented. This makes it difficult to separate the effect of these changes from the impact of the intervention. We therefore accompanied our evaluation with a thorough process evaluation.

Sample

Primary outcome (current employees)

The primary outcome data came from Zurich’s core administrative dataset on its current employees. It records a variety of characteristics for each employee at the point of measurement. We received end of month data pulled at the following months:

• 2017: March (start of pre-period), June, September and December
• 2018: March and December
• 2019: Feb (end of pre-period), May, August and November
• 2020: February (end of post-period)

Secondary outcome (applications)

For the secondary outcome, we used Zurich’s recruitment dataset comprising 32,694 applications to 1,790 vacancies for the period January 2017 - February 2020 with 497 in the trial period and 1,293 in the pre-period. Of these, there were 1,432 vacancies that were not exempt from the new policy (were not EIC, Evergreen, Engineer Surveyor or consultant roles) with 391 in the trial period and 1,041 in the pre-period^{[footnote 24]}. These vacancies received a total of 19,228 applications.

The dataset contained 35 variables and table 2 below lists the ones used in the analysis.

Table 2. Coverage of important variables

Variable name in dataset	Coverage
Job Family	100% of job adverts
Job Grade	~99% of job adverts
Country	100% of job adverts
Requisition Creation Date	100% of job adverts
Req ID	100% of job adverts
Gender	92.7% of applicants (7.3% “not known”)

Exploratory analysis

New external hires, promotions and applicants successful within in-process hiring rounds

We conducted analysis on hiring behaviour in 2 ways which had different samples:

1. We examined all applicants who were successful in in-process hiring rounds. There were 1,432 vacancies in this sample with 391 in the trial period and 1,041 appointments in the pre-period.
2. We examined employees at Zurich who were promoted (either by applying to in-process job postings included in the secondary analysis or through out-of-process hiring, for example direct appointments). This represents a different criterion for inclusion to the secondary analysis because the effects on promotions could have come from 2 channels: both directly from the adverts influencing staff to apply for roles and indirectly from changing the perception of Zurich’s culture around part-time and flexible working options. There were 712 promotions in this sample with 212 in the trial period and 512 in the pre-period.

Process evaluation

The aim of a process evaluation is to understand how an intervention was implemented and the ways in which it may have deviated from what was planned. It allows us to unpack why the intervention did or did not work. We conducted a light-touch process evaluation to measure changes in perceptions of part-time employees, understand policy adoption levels, and explore the opinions and buy-in of stakeholders. The process evaluation consisted of 3 components:

1. A survey

2. Policy compliance metrics

3. Interviews

Below we discuss each of these in more detail.

Survey

We ran a survey with part-time employees to capture their sense of belonging, organisational fit, and perceptions about any stigma or discrimination they faced as part-time employees. In particular, the survey measured the following elements:

• organisational identification (such as, how satisfied they are as an employee of Zurich)

• presence of role models in the organisation

• fitting in with successful people in the same business areas of the organisation

• culture of long working hours at the organisation

• perceptions of disadvantage faced by part-time employees at the organisation

These indices were based on the validated measures recommended by Prof. Michelle Ryan (Exeter University), and their inclusion in this trial was based on discussion with Prof. Ryan.

The survey was sent to all part-time employees (493 for wave 1 and 448 for wave 2) by email in 2 waves. Wave 1 was sent in January 2019, before the new policy was introduced in March 2019, and Wave 2 was sent in November 2019, at which point the intervention had been live for 9 months. 291 part-time staff responded to the wave 1 survey (a response rate of 59%), and 313 (63%) responded to the wave 2 survey (282 individuals responded to both waves of the survey).

Policy compliance

To understand whether a policy was successful, it is necessary to understand uptake on the ground. We therefore collected data on the proportion of adverts that were advertised according to the new policy: as part-time or job-shares, in addition to being available full-time. Even though this was the new default, managers could still actively choose not to do this. While Zurich was not able to capture this variable on their recruitment system (Taleo), roles advertised according to the new policy should have included the new offer in the job title. Therefore, we conducted analysis based on the recorded job titles of advertised positions which were extracted from Taleo and then BIT analysts used string-searching algorithms to check if they contained any of “P/T, job-share or F/T” which were the terms intended to be included in the job titles.

Interviews

We conducted interviews with hiring managers and job candidates to:

1. Assess the awareness and understanding of the new Zurich policy to advertise positions as part-time or job-shares by default

2. Capture some of the range of experiences and views regarding the new policy

We only had resources to conduct a small number of interviews, meaning the range and diversity of the sample were limited. We spoke to 5 hiring managers from different business areas who had posted a live job advert since the intervention went live. We chose not to contact hiring managers who opted out of the policy because we understood that the different reasons for opting out were being collected by Zurich’s Talent Acquisition team. In general, the interviewees understood the new policy well and were supportive of it.

We also interviewed 2 external new part-time hires and 2 internal full-time hires (who also worked full-time previously). These interviewees were aware of, and had positive sentiments towards, the new part-time policy, even when it was not directly relevant to them as full-time employees.

Results

Policy compliance

Three business areas asked to be exempt from the part-time default trial from the outset: Engineer Surveyors, Evergreen and EIC. This was because they needed more time to redesign their operating model first. Most vacancies were not from these areas – for example in the post-period just over 78% of vacancies were in non-exempt areas.

For non-exempt business areas, the majority of job adverts were advertised according to the new policy, meaning that it was rare for a hiring manager to opt out. During the intervention period (March 2019 - February 2020), at least 78% of job titles included at least one of “P/T, job-share or F/T”.

This figure might have been higher in actual fact. We report the figure from a report Zurich ran to collect job titles which included mention of part-time and/or job-share working (including variants, for example ‘PT’, ‘Part time’ etc), but some job titles appeared to have been cut-off (and therefore we may not see the “P/T, job-share or F/T” string which typically appears at the end) by the process that extracts them from Taleo. Unfortunately, there is no alternative way to confirm the compliance figure. 78% is therefore the minimum possible value for compliance, among the participating business areas.

Primary outcome

No evidence of a change in the overall share of Zurich employees working part-time

We do not find a significant difference in the proportion of employees working part-time before and after the intervention was implemented (see Figure 3) – this analysis compared the proportion of staff working part-time in February 2019 with the proportion in February 2020. The proportion of part-time workers at Zurich was driven by 3 factors:

1. Changes to the proportion of new hires at Zurich who work part-time.

2. Changes to the proportion of staff leaving Zurich who work part-time.

3. Existing Zurich staff being influenced by the intervention to think it is more acceptable to work part-time and therefore working part-time more often.

Note that we do not adjust for job grade for the primary outcome (Figure 3), whereas the secondary outcomes and exploratory analysis (Figures 4-7) are all adjusted for job grade.

Figure 3. Share of Zurich employees working part-time

We explored the data for these 3 groups separately (see Table 3) and found that disproportionate rates of part-time staff leaving Zurich may have cancelled out any positive effects of the intervention itself. Further analysis details are provided in Appendix 1.

Table 3. Findings for key groups of part-time employees

Group	What we found
Existing Zurich staff who continued to work at Zurich	This group of staff worked part-time more during the post-intervention period.
New hires	There was a small increase in the proportion of new hires working part-time.
Leavers	Those who left Zurich (including due to redundancies) were more likely to have been working part-time than those who remained. This appears to have been more pronounced in the post-intervention period, perhaps due to redundancies.

Secondary outcomes

The share of female applicants increased

In general, the share of female applicants at Zurich changes a lot over time and is sensitive to:

• the type of jobs being advertised (job family and grade) which we control for

• factors we cannot easily control for, for example chance events, other language used in job adverts and labour market conditions

We note that we have excluded jobs that were advertised ‘out of process’. We define these as jobs which were not advertised on Zurich’s website using a job advert and available to any potential applicants, but rather were direct appointments of individuals or advertised solely by recruitment agencies to a limited number of individuals. 

We found a significant increase in the share of female applicants overall when we controlled for job family, grade and the variation in the sample over time. There were 391 vacancies advertised during the trial period. The share of female applicants increased by 6 percentage points (from 36.4% to 42.4%).

Figure 4. Share of applicants stating they are female

We also separately looked at the share of female applicants for senior roles (grade 4 and above) - for which there were 654 vacancies advertised during the trial period. We found a 6 percentage point increase in the proportion of applicants stating that they were female (from 31.1% to 37.1%).

Figure 5. Share of applicants stating they are female (senior roles)

The increase is robust to possible changes in applicants declaring their gender 

17% of applicants did not declare their gender which could influence the headline results. Therefore, we analysed the proportion of non-male applicants which included both female and non-declaring applicants. We did this because if the proportion of female applicants increased but the proportion of non-declaring applicants decreased, the entire change could have been due to changes in rates for declaring gender rather than due to more women applying.

We found that the share of those stating that they were either female or not declaring a gender increased by 3.2 percentage points. This is less than the increase in the share of female applicants (which was 6 percentage points). Therefore, given that the share of non-declarers decreased following the intervention, the total number of women applying could be smaller overall and the treatment effect could be closer to 3 percentage points. However, we can see that it would still be positive and there is very likely to have been an increase in the number of female applicants.

Figure 6. Share of applicants not stating they are male

Limitations

The key limitation of this analysis stems from its design - it was not an RCT and we therefore cannot draw causal links. This is because other factors, outside of our control, may have influenced the number of women applying to roles at Zurich, and the number of Zurich employees working part-time, during the intervention period. If we had been able to implement an RCT we would have been able to infer causality as we would have been able to compare a control group to a treatment group. This is particularly important as there were structural and workforce changes over the period when the intervention was implemented, including redundancies at Zurich. This makes it difficult to separate the effect of these changes from the impact of the intervention.

For the primary outcome, we did not have data on the job family available for this analysis. For the secondary outcomes, we were able to take a more strenuous approach in trying to control for confounding variables in the following ways.

1. We included controls for the type of jobs posted before and after the intervention. This is important because different job types are likely to attract very different proportions of women, and there is some evidence that the vacancy mix was not the same.

2. We also tried to control for variation (both trend and seasonal) in the share of women applying to roles across Zurich. This is to ensure that our estimated intervention effect does not simply reflect “normal” trend or seasonal fluctuations in the outcome variable. A simple example would be if the share of women had been rising prior to the intervention and simply continued on that trend; without any time series controls, that would have been identified as a positive impact of the intervention.

Another important issue is that about 30% of vacancies in our data only had one applicant. This may have been due to out-of-process hiring, or data errors. We ran the analysis with and without those vacancies. Our results are almost identical when we exclude those vacancies.

Finally, for the exploratory analysis reported in Appendix 2, it is worth noting that we have termed a person a ‘new hire’ when they appear in the list of employees at Zurich UK in one quarter, having not been in the list in the previous quarter. This may include a very small number of international transfers. It also does not capture lateral moves, such as role changes at the same grade but into a new job family.

Survey results

A significant increase in ‘belonging’ at Zurich

All part-time employees received the survey twice by email. Wave 1 was sent before the policy was introduced in March 2019 and had a response rate of 59% (291 part-time employees). Wave 2 was sent in November 2019 and had a response rate of 63% (313 part-time employees). 282 individuals responded to both surveys. We conducted pre-post analysis using OLS regressions to estimate the average effect of the policy on part-time employees. We ran the analysis for both for the whole sample and for the subsample of individuals who answered the relevant questions in both survey waves. Survey scores were measured on a scale of 1 to 7 and a higher score is associated with a better outcome. Our full-sample analysis provided the following results.

Part-time employees felt better about being part of Zurich in the post-intervention period. There was an 8% improvement in organisational identification, which was roughly equivalent to 6.5 times the effect of a promotion.^{[footnote 25]}

Figure 7. Organisational belonging index

The perception of a long working hours culture slightly reduced post-intervention. Furthermore, part-time staff seemed less likely to say they faced disadvantage due to working part-time.^{[footnote 26]}

Figure 8. Working hours culture index

Figure 9. Being disadvantaged due to working part-time index

There were no significant changes in the other 2 indices capturing the presence of role models and ‘fitting in’. These results do not indicate causal effects of the policy because there could have been other developments between the 2 survey waves that affected responses. For example, there could have been an improvement in wider societal attitudes towards part-time employees which could have led to higher scores.

Findings and conclusion

The aim of the intervention was to mitigate part-time work penalties by normalising part-time work at all levels of seniority within Zurich. Results showed that the proportion of part-time workers, which was our primary outcome, remained stable after the intervention. We believe that there could be 3 possible explanations:

1) The intervention was ineffective at encouraging part-time work.

2) The intervention was effective, but its impact was offset by other factors influencing resignation rates, joining rates or working patterns since March 2019. These could be influenced by the restructuring undertaken by Zurich whereby some workers were made redundant, and the structure of the business was transformed.

3) The intervention may take longer to impact on the number of people working part-time at Zurich. It may still act as an initial signal that part-time work is acceptable and encouraged at Zurich - though there may be a time lag in translating this into staff working part-time. For instance, people who are considering working part-time in the near future may have applied to Zurich after seeing that adverts included part-time options, as they may have thought flexible and part time working patterns may be relevant to them later in their career.

Considering our secondary outcomes, proportionally more women appear to have applied per vacancy in the intervention period. Again, this could be due to the intervention or due to other changes, including changes to the labour market, Zurich’s job adverts or differences in the jobs advertised that we cannot record.

It is encouraging that both the number of female applicants and the number of those who did not declare that they were male (female plus non-declarers) rose. This means that the effect is unlikely to just have been due to non-declarers switching to stating their gender openly. The reduction in the number of non-declarers could be due to applicants feeling more comfortable about disclosing their gender.

Finally, our intervention was associated with an increased sense of ‘belonging’ among Zurich’s part-time employees – which may suggest some movement towards the normalisation of part-time work.

Overall, while the results of this trial are promising, we cannot make causal claims about the impact of our intervention. To generate further evidence, we are currently running a similar intervention as an RCT with a large retail company in the UK. We expect results to be available in mid-2021.

Appendix 1: Additional analysis of changes in part-time rates for different employee groups

Table A1 shows the changes in the part-time rate for:

• those retained by Zurich
• new hires
• those leaving Zurich

Note that these are not equal size groups. The next table (Table A1) shows how these produced the net zero effect on the part-time proportion. We note that the leavers working part-time in Zurich between March and December 2019 was only about 14%.

Table A1. Changes in part-time rates

	Number of employees in this period	% of retained employees who were working part-time	% of new hires who were working part-time	% of Zurich leaving the company who were working part-time
Fourth ‘post’ period (December 19 to February 20)	4646	11.9	6.8	15.1
Third ‘post’ period (September to November 19)	4671	11.6	6.8	9.8
Second ‘post’ period (June to August 2019)	4689	11.4	8.6	12.7
First ‘post’ period (March to May 19)	4814	11.3	6.5	21.4
Baseline period (December 18 to February 19)	4922	11.1	5.7	25.2

Table A2 decomposes changes to the part-time work rate into the cumulative net effect of leaving/hiring (negative) and the change to retained employees (positive). For example, the effect on retained employees is calculated by multiplying the change in the proportion of part-time employees by the proportion of retained employees.

Table A2. Cumulative net effects

	Number of employees in this period	Cumulative change in % part-time working since baseline period	Cumulative effect of changes to retained employees on part-time %	Cumulative effect of new hires and leavers on the part-time %
Fourth ‘post’ period (February 2020)	4646	0.12	0.74	-0.62
Third ‘post’ period (November 2019)	4671	-0.11	0.46	-0.57
Second ‘post’ period (August 2019)	4689	-0.26	0.27	-0.53
First ‘post’ period (May 2019)	4814	-0.40	0.18	-0.58

Appendix 2: Exploratory analysis

Exploratory results on interviewed applicants, new hires and promotions

Interviewed applicants

The share of interviewed applicants who stated their gender as female increased by 2 percentage points, but this was not statistically significant. This translates to an additional 125 applicants who stated they are female receiving an interview over and above what previous trends predicted. This was analysed using the same specification used for the secondary analysis of share of applicants (including lags of proportions to account for time trends in labour markets).

Figure A1. Share of interviewed applicants stating they are female

As a reminder, we conducted analysis on hiring behaviour in 2 different ways which each had different samples (for all of them the outcome is related to the proportion of women):

1. We examined all applicants who were successful in in-process hiring rounds.

2. We examined employees at Zurich who were promoted, which included all Zurich staff who were promoted (either by applying to vacancies included in the secondary analysis or through a direct process). This is a different criterion for inclusion to the secondary analysis because the effects on promotions could come from 2 channels: both directly from the ads influencing staff to apply for roles and indirectly from changing the perception of Zurich’s culture around part-time and flexible working options.

There was not a large sample size for any of these analyses, for example in the promotion analysis, there were only 712 eligible promotions. These analyses do not exactly overlap, so should be treated as separate analyses. For example, if there were any internal applicants to out-of-process hiring rounds that are lateral moves, they are excluded.^{[footnote 27]}

Promotions

We also investigated the effect of the intervention on promotions of existing Zurich staff, which we have restricted to increases in grade from one data pull to the next and have included both within and outside of hiring rounds.

Of these, there was an increase in the proportion of promotions that went to women - from 50.2% in the pre-period to 56.1% in the post-period, so an increase of 5.9 percentage points (4.9 percentage points when adjusted for grade). This was not statistically significant. It is worth noting that like hires, the likely reason is that there may have been effects of women working in different job families (though we did not have the data to measure this).

Candidates that succeed in hiring rounds

Finally, as hires typically occur in hiring rounds, we also analysed all successful candidates within in-process hiring rounds, which includes:

• external hires through externally advertised hiring rounds
• internal promotions if they were through a competitive process
• internal applicants who were successful at applying for a job at the same grade in the company, usually in a different team (such as lateral moves)

We found no statistically significant difference between the proportion of successful candidates that were female in the pre-intervention and post-intervention periods. We see a small decrease of 3.2 percentage points when we adjust for grade, job family and month of the year the decrease is. If we include female and non-declaring applicants, there is an increase of 1.2 percentage points that was also not statistically significant.^{[footnote 28]}

We also investigated the change for successful candidates in hiring rounds in higher grades – where we also found no statistically significant effects on the number of female successful applicants. For grades 5 and above, we find a 4.3 percentage point increase adjusted for grade, job family and month of the year – however, these are still statistically insignificant.

Appendix 3: Analytical strategy

The analytical strategy described below was specified prior to randomisation and was used to generate the findings described in the Results section.

Primary outcome

The pre-specified Analysis of the primary outcome encompasses a simple test of the difference in the proportions of Zurich employees working part-time before and after the intervention. For this final analysis, we measured this outcome in February 2019 (before) and February 2020 (after).

It is worth noting that the analysis in this section did not exclude the exempt categories, but it was checked to check that it made no material difference to the analysis.

This analysis does not give an accurate representation of the changes by itself, because:

• there have been changes to the organisational composition of Zurich, in particular to the job grades due to the restructure

• the redundancies may have directly affected the outcome as they were disproportionately of part-time employees

• this test does not account for the gender or job grade of the applicant, and therefore does not maximise our ability to detect an effect of the intervention 

Therefore, we also run the following OLS regression: y_{i, t} = α + β₁ * lastperiod + β₂ ᐧ Δ_t + β₃ * ​​Λ_{i, t} + ϵ_{i, t} (1)

where:

• y_i is whether employee i is working part-time at time t

• lastperiod_t is a dummy variable for whether the observation was in the February 2020 dataset

• ​​Δ_t is a vector of dummy variables for the other post periods

• ​​Λ_{i, t} is a vector for the covariates for individual i at time t which are job grade and gender. We could not adjust for job family or function because the available data was not complete enough.

The baseline was data from February 2019, so we had one pre-period and 4 post-periods – which were 1) March to May 2019, 2) June to August 2019, 3) September to November 2019 and 4) December 2019 to February 2020 – in the regression as this best matched the original idea of doing a proportions test.

We were primarily interested in β₁, the effect of being in the last period. We discussed whether it would be appropriate to use fixed effects here, to attempt to control for other time-invariant characteristics of individual employees. However, this would only measure the effect on staff who did not join in the post-period as for any new joiners their fixed effects would be perfectly collinear with the post-period dummy variables. Therefore, we chose not to include employee-level fixed effects.

Secondary outcomes

As robustness checks, we examined how the results differed under the following tweaks to the specification: dropping the second lag term, and, dropping both lag terms.

1. A randomised controlled trial (RCT) is a method for evaluating the impact of an intervention, whereby trial participants are randomly assigned to one of 2 (or more) groups to test an intervention. One group (the ‘treatment’ group) will experience the intervention being tested, while one group (the ‘control’ group) will not experience the intervention (or may receive a placebo). The number and characteristics of individuals in both groups are balanced. This allows researchers to measure outcomes for both groups to understand whether and how effective the intervention was, and specifically, to infer causality. ↩

2. It should be noted that the analysis that produced these results were not pre-specified before the trial, and therefore are only exploratory in nature. ↩

3. We define in-process hiring as jobs which were advertised on Zurich’s website using a job ad and available to any potential applicants, whereas out-of-process hiring refers to direct appointments of individuals or jobs advertised solely by recruitment agencies to a limited number of individuals. ↩

4. The regulation came into effect in April 2017. https://www.legislation.gov.uk/ukdsi/2017/9780111152010 ↩

5. House of Commons. (2020). The Gender Pay Gap. Briefing paper Number 7068, 6 March 2020 ↩

6. https://www.bbc.co.uk/news/business-47822291 ↩

7. Costa Dias, M., Joyce, R., & Parodi, F. (2018). Wage progression and the gender wage gap: the causal impact of hours of work. The Institute of Fiscal Studies. ↩

8. Costa Dias, M., Joyce, R., & Parodi, F. (2018). Wage progression and the gender wage gap: the causal impact of hours of work. The Institute of Fiscal Studies. ↩

9. Timewise. (2019). The Timewise Flexible Jobs Index 2019. https://timewise.co.uk/wp-content/uploads/2019/09/TW_Flexible_Jobs_Index_2019.pdf ↩

10. Grant, Linda & Yeandle, Sue & Buckner, Lisa. (2006). Working below Potential: Women and Part-Time Work. ↩

11. EHRC [Equality and Human Rights Commission] (2016b), Pregnancy and maternity-related discrimination and disadvantage: experiences of mothers. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/50950 1/BIS-16-146-pregnancy-and-maternity-related-discrimination-and-disadvantageexperiences-of-mothers.pdf ↩

12. https://www.zurich.co.uk/en/about-us ↩

13. https://www.zurich.co.uk/en/about-us/media-centre/company-news/2019/zurich-publishes-its-2018-uk-gender-pay-gap ↩

14. https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/earningsandworkinghours/bulletins/genderpaygapintheuk/2019#the-gender-pay-gap ↩

15. https://www.zurich.co.uk/-/media/project/zwp/united-kingdom/docs/about-us/cr/uk_gender_pay_gap_report_2018.pdf ↩

16. https://www.zurich.co.uk/-/media/project/zwp/united-kingdom/docs/about-us/cr/zurich-uk-gender-pay-gap-2019.pdf ↩

17. This analysis compared men and women in similar roles/grades, and with similar performance scores in prior years, tenure, service years and whether they are part or full-time. ↩

18. Nicks, L., Burd, H., & Barnes, J. (2019). Flexible working qualitative analysis. The Behavioural Insights team. Government Equalities Office. Retrieved from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/790354/Flexible-working-qualitative-analysis2.pdf ↩

19. Gaucher, D., Friesen, J., & Kay, A. C. (2011). Evidence that gendered wording in job advertisements exists and sustains gender inequality. Journal of personality and social psychology, 101(1), 109. Gee, L. K. (2019). The more you know: information effects on job application rates in a large field experiment. Management Science, 65(5), 2077-2094. ↩

20. Jachimowicz, J. M., Duncan, S., Weber, E. U., & Johnson, E. J. (2019). When and why defaults influence decisions: A meta-analysis of default effects. Behavioural Public Policy, 3(2), 159-186. ↩

21. Kahneman, D., & Thaler, R. H. (2006). Anomalies: Utility maximization and experienced utility. Journal of Economic Perspectives, 20(1), 221-234. ↩

22. We defined new hires to be anyone working at the company in period t who was not working there in period t-1. This may include some transfers from other parts of Zurich’s International offices, but we cannot adjust for this. This analysis excluded Engineer Surveyors. ↩

23. This analysis excluded Engineer Surveyors. ↩

24. Three areas of the business asked to be exempt from the part-time default trial from the outset: Engineer Surveyors, Evergreen and EIC. This was because they needed more time to redesign their operating model first. ↩

25. Statistically significant at 1% level. For the subsample, there was a statistically significant increase at 5% level. ↩

26. The improvement in the index of a culture of long working hours is significant at 10% level both in the full sample and in the subsample. The improvement in the index of perceptions of part-time disadvantage is significant at 10% level in the full sample and not significant in the subsample. ↩

27. We define lateral moves as internal role changes at the same grade but into a new job family. ↩

28. This analysis could not adjust for the lagged proportions of women in prior months in the same way as we did for interviewed applicants and applicants overall, as we have a very small sample size of hiring rounds. ↩