User permissions required: ‘View Sources’ AND ‘Review and label’
TABLE OF CONTENTS
- What is label sentiment analysis?
- When should you use label sentiment analysis?
- Enabling label sentiment analysis
- Assigning labels with sentiment enabled
- Choosing which sentiment to apply
- Applying multiple sentiments
What is label sentiment analysis?
Label sentiment analysis is a feature that allows labels to be assigned with either positive or negative sentiment, depending on how that label concept is expressed within the verbatim.
Every assigned label needs to be given a positive or negative sentiment (there is no neutral sentiment), whilst multiple labels assigned on the same verbatim can have different sentiments depending on how they are expressed.
The benefits of this feature is being able to report on the sentiment within a dataset for specific topics, and a number of charts are available in Reports that relate to sentiment.
When should you use label sentiment analysis?
Label sentiment analysis is typically only appropriate for customer feedback related datasets. This is because they contain many more identifiable expressions of sentiment than other datasets, which tend to be much more neutral by nature.
It's important to be sure that label sentiment analysis is appropriate for your use case, as once it's enabled during dataset creation, it cannot be disabled on that dataset.
The platform does have a pre-trained tone analysis model available, which predicts the overall tone, i.e. sentiment, of a message. This is typically appropriate and sufficient for all other use cases, e.g. email inbox analysis and automation.
Enabling label sentiment analysis
Label sentiment analysis is enabled at dataset creation and cannot be changed later. As you go through the dataset setup flow, you have the option to enable label sentiment analysis.
Tone analysis, which provides an overall sentiment score from -10 to 10 for a message, can be enabled at dataset creation, or later via the dataset settings. This can be turned off and on, so it's not important if you don't enable it during setup.
Assigning labels with sentiment enabled
Assigning labels with sentiment is very similar to assigning labels without sentiment – see Steps 1, 2 and 3 below, which demonstrate labelling a verbatim from a dataset of customer hotel reviews.
The main difference is in Step 2, where after typing the label name, you must always select either positive or negative sentiment, denoted by the green or red face icons (this step has been repeated for both the ‘Price’ and the ‘Room > Size’ labels).
One important thing to remember when applying labels with sentiment, is that it's important to create a taxonomy with neutral label names (where possible). For example, 'Price' has been used above rather than 'Expensive', as 'Price' is neutral whereas 'Expensive' in inherently negative.
The selection of negative sentiment for a label with a neutral name would capture the instances where the verbatim is expressing a negative perception of the label.
Choosing which sentiment to apply
Much of the time it will be obvious which sentiment you should choose when you apply a label, based on the inherent positivity or negativity of the language (e.g. the 'Price' and 'Room > Size' examples above).
For certain labels, the concept may not lend itself to a neutral name and will be inherently negative or positive, and thus always always be applied with only one sentiment. For example, 'Error' related labels will typically all be applied with negative sentiment. This is fine, but should be applied consistently.
Sometimes, however, it can be quite unclear. If the language in a verbatim is very neutral in tone, we need to think more carefully about which sentiment to apply. Here there are two main things to consider:
Verbatim metadata
The first is to look at the verbatim's metadata. For customer feedback related verbatims (the most common type of data in a sentiment-enabled dataset), there will often be some kind of score or rating associated with a verbatim (e.g. NPS score). You can often use these scores to gauge whether a comment that appears neutral in tone, is more positive or negative in sentiment - i.e. a customer rarely leaves an NPS score of 10 if they're unhappy.
If you consistently apply label sentiment for verbatims that are neutral in tone, based on a 'score' metadata field, the model can learn to pick up on this and predict the sentiment accordingly.
Consistency of application
The second is to be consistent in how you apply the sentiment for a label when it's quite neutral in tone, and there's no other differentiator (e.g. a 'score' related metadata field).
If it's more common for feedback to be positive for a certain label, assume it's positive unless the verbatim is explicitly negative, and vice versa. If you're not consistent, however, the model will struggle to predict the sentiment.
Applying multiple sentiments
Another important thing to consider when using sentiment analysis is that the model applies each label (root and leaf) independently, so you can have two leaf labels from the same parent label that have different sentiments.
In these instances you must then judge what the overall sentiment for the parent label is. In this example below, the parent label ‘Room’ is positive overall.
If both leaf labels have the same sentiment, then the model will infer that the parent label also has a negative sentiment and only the leaf labels will be shown as pinned labels (though that implies the parent label is also applied).
Example verbatim with both positive and negative sentiment labels pinned
Previous: Model training and labelling best practice | Next: Training chat and calls data