Research and analysis

Fake online reviews research: executive summary

Published 25 April 2023

Read the full report on Fake online reviews research: estimating the prevalence and impact of fake online reviews.

Context and objectives

Online consumer reviews play an important role in the purchasing decisions other consumers make online. These reviews serve as an important source of information to mitigate uncertainty around product quality, particularly when consumers have not seen the products themselves beforehand (Manes and Tchetchik 2018). Consumers generally perceive the information posted in online reviews as unbiased, and reviews can often “make or break” the success of a product or service (de Langhe, Fernbach, and Lichtenstein 2016). This provides an incentive for product sellers to manipulate their online reviews by purchasing or anonymously posting fake reviews intended to deceive consumers and increase sales.

Fake reviews can be favourable towards the seller’s product or unfavourable towards the products sold by competing businesses. Both strategies are intended to make consumers purchase products that they might not have in the absence of fake reviews. For the purposes of this research, we define a fake review as a review of a product or service which does not reflect a genuine experience of that product or service and has been designed to mislead consumers.

There is no consensus on how consumer choice, trust and future behaviour are impacted by fake reviews. Consumers might be aware that manipulation is taking place through fake reviews and adjust their interpretations of online opinions accordingly (Zhuang, Cui and Peng 2018). Alternatively, consumers might not be able to correct for the bias in evaluating product quality introduced by fake reviews if they cannot distinguish between fake and genuine reviews (Hu, Liu and Sambamurthy 2011). In addition, few papers have distinguished between different types of fake reviews (such as fake reviews that are more or less obviously written), and most previous research focuses on consumers in the US rather than the UK.

Alma Economics (the authors of this study) was commissioned by the Department for Business and Trade (DBT) to answer the following research questions:

1. What is the prevalence of online fake reviews on popular third-party UK e-commerce websites?^{[footnote 1]}
2. How do online fake reviews influence consumer choice when making online purchases?
3. What is the harm to consumers caused by fake reviews?
4. How effective are potential non-regulatory interventions in avoiding consumers being misled by fake reviews?

Methodology

As part of this study, we combined 2 separate approaches to understand the impact of online fake reviews on UK consumers.

First, we built a machine learning model to predict whether reviews were genuine or fake.^{[footnote 2]} Predictions were based on characteristics of reviews used in previous detection models (such as similarity with other reviews, review posting history and average review length), but also included network features that took into account whether products shared reviewers with other products. As fake reviews have become increasingly similar to genuine reviews over time as people who write fake reviews try to avoid detection, the content of the review itself has become less helpful in distinguishing genuine and fake reviews. As a result, characteristics of reviews not related to content, such as network features, are key in providing additional predictive power to the model.

Once the model was built, we then trained it on a dataset of known fake reviews collected from private Facebook groups where sellers buy reviews (He et al. 2022b). Previous models were trained on datasets of AI-generated fake reviews (using language models such as GPT-2) or platform-filtered reviews, which are not necessarily similar to the fake reviews seen by UK consumers when they shop online. Because we know for certain which reviews are fake and which are genuine, our model can provide more accurate predictions when deployed on the reviews of e-commerce platforms. This trained model was then applied to a dataset of 2.1 million product reviews across 9 popular UK e-commerce platforms (this larger dataset was unlabelled, which means we do not know for certain which reviews are genuine or fake). The outputs from this model allowed us to estimate the percentage of product reviews on these 9 platforms predicted to be fake.

Second, we carried out an online experiment with 4,900 participants in the UK who had previously shopped online. In this experiment, participants were asked to complete an online shopping task and purchase one of 3 similar products.^{[footnote 3]} The online shopping task was fully interactive and was designed to be as realistic as possible to the practice of shopping and reading reviews on a popular e-commerce site.

As part of the online shopping task, some participants only viewed genuine reviews when they clicked on a specific product page, while other participants viewed a mix of genuine and fake reviews^{[footnote 4]}. In addition, some participants saw a text box (displayed above all product reviews) stating that steps had been taken to moderate misleading content, including misleading reviews, on the platform. Following the online shopping task, participants completed a follow-up questionnaire covering their choices in the online shopping task, general shopping behaviour/preferences and demographics questions.

Based on the experiment described above, we compared the purchasing decisions made by participants who only viewed genuine product reviews with those who viewed both genuine and fake product reviews and those who saw the warning text box. We then assessed whether fake reviews (or warning consumers about fake reviews) changed the probability that a product with fake reviews was purchased. We also explored whether the impact of fake reviews differed across product type, product price and participant demographic characteristics, and how exposure to fake reviews changed consumer trust in platforms and future shopping behaviour.

To supplement these findings, we built a simple indicative model quantifying the annual harm to UK consumers caused by fake reviews on third-party platforms. This model was based on the idea that consumers misled by fake reviews make suboptimal choices, purchasing products that are lower in quality or do not align with their individual preferences (compared to the product they would have purchased in the absence of fake reviews).

Key findings

Our results present a nuanced picture of how consumers are impacted by fake online product reviews:

1. The prevalence of fake reviews differs across product categories and platforms. For e-commerce platforms widely used by UK consumers, we estimate that 11% to 15% of all reviews for 3 common product categories (consumer electronics, home and kitchen, sports and outdoors) are fake.
2. Network features (whether a product had a reviewer in common with another product) are stronger predictors of fake reviews than the content of reviews. Products with fake reviews have more reviewers in common than products that only have genuine reviews. This aligns with empirical evidence that most fake reviews are written by a small pool of individuals who participate in incentivised review service marketplaces (compared to the millions of users that buy products online and write genuine reviews).
3. Consumers are 5.3% less likely to purchase a product with poorly written (“strong”) fake reviews and 3.1% more likely to purchase a product with well-written (“subtle”) fake reviews. However, the size of this impact depends on the price and category of the product. Fake reviews had a greater impact on consumer behaviour for consumer electronics and higher-priced products, and in particular consumers were 9.2% more likely to purchase a product with subtle fake reviews if the product price was greater than £80.
4. Informing consumers that steps have been taken to moderate misleading content on the platform does not impact consumer purchasing behaviour. There was not a statistically significant difference in the likelihood of choosing a product with fake reviews when participants saw a banner with this additional information. However, other non-regulatory interventions may be effective in counteracting the influence of fake reviews on consumers and should be tested in future research.
5. Exposure to fake reviews generally does not impact consumer trust and future behaviour. Despite being exposed to fake reviews on the online platform, we did not observe consumers adapting their purchasing behaviour, leaving them potentially susceptible to being affected by further misinformation in the future.
6. The impact of fake reviews on consumers does not vary depending on their demographic characteristics. We did not find any differences in the effect of fake reviews on characteristics such as age, sex and ethnicity. This suggests that fake reviews have a similar impact on different groups of UK consumers.
7. Fake review text on products alone causes an estimated £50 million to £312 million in total annual harm to UK consumers. However, this estimate does not include the impact of fake reviews on consumers who purchase services, on future consumer behaviour or the separate impact of inflated star ratings (which often accompany fake reviews). As a result, this is a conservative estimate and the true consumer detriment arising from fake reviews is likely to be higher.

Limitations

There are some important limitations to the approaches used for this study.

1. While products for the experiment shopping platform were chosen to be similar in star rating, price and other characteristics, there were still differences in visual appearance and key characteristics that may have influenced consumers’ decision (in addition to variation in the content of reviews).
2. Participants were only asked to make a purchasing decision as part of the online shopping task that aligned with how they would act in the real world (instead of actually spending their own money, which means they may have been less motivated to find the “best” or “highest quality” product).
3. We only examined the prevalence and impact of fake review text on consumer products. As such, we cannot determine whether these findings also extend to services purchased online or other misleading review practices. Additionally, the estimated total harm caused to consumers does not include the impact of inflated star ratings.

However, the size of the experiment (based on the total number of participants), our integration of a bespoke online shopping platform that closely resembled an actual shopping experience and our use of a broad range of fake review types means we can be confident that our study robustly estimates the impact of fake reviews on UK consumers.

Policy implications

Our key findings, in particular (i) at least 10% of all product reviews on third-party e-commerce platforms are likely to be fake, and (ii) the presence of well-written “subtle” fake reviews leads to a statistically significant increase in the proportion of consumers buying the product with these fake reviews, highlight the importance of taking steps to reduce the prevalence of online fake product reviews. Our findings suggest that consumers are more susceptible to being misled by well-written fake reviews when purchasing products where reviews play a more prominent role in consumer decisions (such as consumer electronics or higher-priced products). If consumers are generally not able to distinguish genuine and well-written “subtle” fake reviews, and fake reviews are becoming more sophisticated and difficult to detect over time, the negative impacts of fake reviews on consumers are likely to increase over time as well. This suggests 3 main areas for future policy to consider:

Automated means of review moderation should focus on the characteristics of reviewers at least as much as the characteristics of reviews, given that network features (based on which products had reviewers in common with other products) are stronger predictors of whether reviews are fake than the content of reviews themselves.

The high levels of data and computational power required to generate product-reviewer networks for popular e-commerce platforms suggest that e-commerce platforms are better positioned to spot fake reviews compared to consumers. This aligns with previous research finding that even trained researchers cannot consistently and accurately distinguish between genuine and fake reviews (Plotkina, Munzel and Pallud 2020).

Our research found that informing consumers that steps have been taken to moderate misleading content (such as misleading customer reviews) does not seem to counteract the influence of fake reviews. This provides evidence that consumer trust in product reviews tend to be grounded in prior online shopping experiences and cannot easily be altered. It is possible that interventions that use stronger language or are more salient to consumers could increase awareness of fake reviews and encourage consumers to be more cautious. Future research should test whether these types of non-regulatory interventions can be effective despite the evidence indicating that consumers struggle to spot well written fake reviews, as these can be straightforward and cost-effective for platforms to implement.

1. Third-party e-commerce websites are online marketplaces that manage and host sales for other businesses. ↩

2. Machine learning models are based on the concept of learning algorithms: by ‘training’ a model on a dataset of product reviews which have been labelled as fake or genuine, it then becomes possible for the model to classify unlabelled reviews (that is reviews for which we do not know whether they are fake or genuine) without being explicitly programmed with the steps required to complete the task. ↩

3. There were 11 product types considered in this study: Bluetooth headphones, irons, kettles, desk chairs, smart speakers, keyboards, power banks, re-usable water bottles, yoga mats, sunscreen and vacuums. ↩

4. The fake reviews were written by the research team and we received feedback on the fake reviews from a representative at Which?, the UK consumer advocacy group. ↩