Creativity is a major aspect of our work at BLUSPHINX®. Combining disciplines is a great way to come up with new ideas and innovations, sometimes before others. We like to be ahead of the game, especially in hot topics like AI!
We started to use Artificial Intelligence (AI) techniques to enhance data analytics in 2015, back when AI was a barely known topic in companies and data analytics usually meant Excel or statistical tools like SPSS or R. In the context of a consumer insights project for a luxury company, we developed a custom AI algorithm to solve a common problem related to the analysis of consumer surveys and save time for data analysts.
Though we have developed more complex AI innovations since then, this simple example remains a good illustration of how efficient AI can be to automate and speed up the execution of specific tasks, notably related to data analytics and data cleaning.
Even at successful global companies with dedicated consumer analytics teams, the analysis of unstructured data can be a challenge. One example is free-form text answers commonly used in online surveys. This is the data of survey questions where respondents type their own answers instead of selecting from a pre-defined set of options, for instance “What is your favorite brand?”. Free-form text answers provide better insights into consumer preferences and attitudes, but can contain spelling mistakes, typos, missing space between words – all of which make the data hard to analyze. French luxury brands are known around the world, but the correct spelling of their names much less so!
Below is an example of all the different spellings of the brand Louis Vuitton typed by survey respondents.
Traditionally, this type of data is analyzed manually (i.e. a human analyst reads and counts all responses). This is obviously very time-consuming and impractical with large surveys with thousands of respondents. Our solution was to develop an algorithm to automatically group all (mis)spellings of a brand in order to calculate the total number of occurrences. Specifically, we leveraged Machine Learning techniques related to Natural-Language Processing (NLP) to predict the actual brand that a (mis)spelling most likely referred to. As a result, instead of having a human analyst count manually the number of occurrences of each brand (including misspellings), the algorithm produced the results automatically and in seconds.
The output of the algorithm was the list of brands mentioned in the responses along with their number of occurrences. This in turn was incorporated into the overall data analysis of the survey (performed by humans) to find interesting insights. The algorithm ultimately saved time for the human analyst who could focus more time on generating insights rather than doing a tedious task.
AI & automation can save a lot of time and bring significant productivity gains by automating only parts of a process, instead of a whole process. Here, we did not automate the whole analysis of a survey, only the parsing of unstructured free-form text data. The key to make AI & automation work is to choose wisely which parts of a process to automate and to define clearly upfront what the outputs should be.
Aurelie Beaumel is an expert in digital strategy, data insights, resilience, and fine experiences. She has done multiple projects in the luxury sector.
