Analyzing comments and free text responses

This is available as standard in Insights, available straight after your survey has closed. The analysis provides you both themes and sentiment. You can also easily – from within the Application – download the text data for review elsewhere, for example in a spreadsheet.

Text Analysis

Text analysis (sometimes referred to as Natural Language Processing, or NLP) is the system for determining the theme and sentiment of a comment. For any open text questions (where the answer provided by the respondent is a written comment), we apply machine learning algorithms to provide the best estimate of what the theme of the comment is, and whether the sentiment is overly positive or negative. This is a very valuable part of using Qlearsite:

• it helps you summarise a large volume of comments in a quantitative way;
• it helps you find patterns in the text data that can be compared against the way respondents scored their other answers;
• it helps you identify potential root causes for particular issues.

Natural language processing is a subset of the broad field of artificial intelligence. It is applied to problems involving analytics, semantic search, text generation, optical character recognition, etc that allow computers to manipulate text. We specifically use it for topic modelling to find the topics (what we call ‘themes’) and meaning in the comments respondents write in their survey replies.

However we don’t share the details of our models as they are proprietary. In summary, our models are trained to find the most appropriate themes in a collection of sentences using the words, sentence structure and many other characteristics. Our models go beyond simple text classification, so called ‘bag of words’ models, often used for word frequencies and sentiment analysis.

Topic modelling assumes that each complex answer to an open question is a mixture of potential themes. We train the model to spot these hidden, or ‘latent’, themes to find the dominant topics in the text. This is important so that the model doesn’t rely on the actual key word to be present in order to spot the theme. The more data we have about a domain (such as employee engagement) the more it allows us to refine the model to be highly sensitive to recurring themes (such as pay, or diversity, etc).

The competition

We use sophisticated deep learning techniques to surface themes and sentiment, employing models that have been specifically trained on our ‘employee feedback’ domain.  Many tools count words, our language analysis technology reads and understands them. We analyse the whole comment, to go deep into it’s meaning. Why? Because context matters and changes meaning.

For example, in this hypothetical sentence “This is not a black and white issue“

…basic, less accurate tools might think it is about ‘Diversity & Inclusion’. That’s because they scan for keywords – ‘black’ and ‘white’. Our technology understands the whole sentence and confidently classifies it as ‘Decision Making’.

On such deeply sensitive topics, being more accurate is essential. That’s why we’re trusted by the leading employers to analyse their valuable, sensitive employee feedback.

Handling Bias

It is inevitable to have bias in NLP text analysis because models are trained on a human language, that will potentially include gender bias and stereotypes. Machine Learning models will therefore learn these implicit biases and patterns. He have counteracted for bias by:

• Taking care in how we select our training data sets;
• Involving a broad range of staff from different backgrounds in annotating the training data set.

We are also looking at building a shorter feedback loop within the actual product, whereby our users can provide direct feedback to the model on specific issues and biases they see in their results, helping us speed up the rate of learning. Providing feedback is valuable to us and ask that you continue to send us questions and comments whenever they arise to support@qlearsite.com.

Accuracy

Very accurate, our models are trained to find the most appropriate themes in a collection of sentences using the words, sentence structure and many other characteristics. Our models go beyond simple text classification, so called ‘bag of words’ models, often used for word frequencies and sentiment analysis.

Unfortunately, no NLP model based on machine learning is ever 100% accurate as they are based on probabilities. Should the tool not be able to recognise the sentiment of a comment than it will fall back on connecting specific words to themes. Over the longer term, we look to balance the model’s aggressiveness in finding themes, versus the level of accuracy we demand when it thinks it has found the most suitable theme. Too aggressive and the accuracy suffers, too accurate and the model’s usefulness suffers. Finding this balance improves with time, more data and training from us.

We’ve spent considerable time over the last few years, training the models on a wide variety of texts. But when the model is faced with a context or phrase it has not seen before, there will always be a chance that it gets the theme wrong. The chance of this happening continues to diminish over time with more and more data.

Employee Privacy and Personal Details

Survey administrators can hide comments so that they are not visible when sharing the survey results with managers or teams. Admin users will continue to see all comments, but are able to identify those that have been hidden.

To hide a comment, simply hover over a comment and click the flag symbol. This will open a dialogue box that lets you identify the reason why the comment is being flagged so that we can see if we need to make any improvements to our text and sentiment analysis.

Support for multiple languages

Yes, the Platform handles multiple languages. Our technique and model are proprietary but we would be happy to discuss this in more detail with you in person. Please contact us to discuss further.

Custom Themes

We have trained our models across a wide variety companies and industries and believe you will not need to customise the themes for your business. If you are concerned about a specific issue or bad experience from the past, please contact us so we can put your mind at ease.