Using natural language processing to structure market research
Learn how a market research start-up used classification and clustering to gain insights to their free-text survey responses.
This guidance is part of a wider collection about using artificial intelligence (AI) in the public sector.
AI techniques and application used:
- classification
- clustering
- natural language processing
Objective
A market research start-up needed to access insights from their free-text survey responses.
Situation
A market research start-up was collecting free-text survey responses rather than structured answers. This presented a challenge, as the startup needed to analyse insights in real-time. The number of responses the start-up was gathering meant this was not something a human could do.
Action
The start-up applied a classification algorithm to the text data to match survey responses to pre-identified trends. The start-up then used natural language processing to analyse over 15,000 free text survey responses to identify consumer trends relevant to a particular client.
In parallel, the start-up built a clustering algorithm to draw out themes in the data and identify unexpected topics and responses.
Impact
The startup deployed the model as a dashboard which allowed employees to gain real-time insights and rapid analysis of free-text survey responses. This is a notoriously difficult action to do given the highly qualitative nature of free-text surveys. By clustering topics and responses, the start-up could efficiently unlock valuable insights and themes to inform their ongoing business strategy.
Related guides
- Understanding artificial intelligence
- Assessing if artificial intelligence is the right solution
- Planning and preparing for artificial intelligence Implementation
- Managing your AI project
- Understanding artificial intelligence ethics and safety
- Examples of real-world artificial intelligence use
- National Cyber Security Centre guidance for assessing intelligent tools for cyber security
- The Data Ethics Framework
- The Technology Code of Practice