An algorithm to discern sentiment in a text

Finding patterns in a text could be very useful. It could reveal if the text has been written by a certain person or not, for example. Going a step further, finding sentiments in a text or a speech can also be very handy. These skills usually appear in procedural and spy movies and TV series. In these works of fiction, it is usually an intelligent human character the one who finds the pattern; in others, it is a computer program the discoverer. But what you seldom find in these movies and series, though, is a computer program that deciphers feelings or opinions based on emotions, in a word, sentiments. This is an exclusive human area of expertise, right? Because, could a mathematical algorithm even aspire to decipher human emotions hidden in a text or a speech? And if so, how could they do it?

Actually, sentiment analysis is an established subset of natural language processing problems. Sentiment analysis objective is to discern and evaluate, in an automated manner, opinions from written or spoken language. A common choice to achieve it is to compute a vector-based representation of the words, or embeddings, for all words in a sentence. Embeddings are then used by supervised learning algorithms to predict the presence or absence of an emotion in the text.

Gaussian Process models (GPM) are one example of kernel-based methods which have been successfully applied to emotion or sentiment classification. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets. For many algorithms that solve these tasks, the data in raw representation have to be explicitly transformed into feature vector representations via a user-specified feature map: in contrast, some methods require only a similarity function over pairs of data points in raw representation, what is called a kernel.

GPM rely on strong Bayesian inference foundations, allowing them to update when new evidence is revealed. In comparison to other regression methods, they provide not only a prediction of a given function, but also an estimate of the uncertainty of this prediction.

In all previous applications of GPM to natural language processing problems, the choice of the most appropriate kernel was made a priori, among some well-known kernel functions. While there is a repertoire of kernels available in the literature, the selection of the most suitable one for a given problem is not straightforward. Moreover, kernels usually have some parameters – the so-called hyperparameters – that need to be adjusted, which makes even more difficult the kernel selection problem.

Now, a team of researchers proposes ¹ a method that does not rely on an a priori specified kernel. Finding the kernel expression is part of the model selection process. This is the first time that evolved GPM have been used for a multi-objective problem.

The learning algorithm that is used, based on genetic programming, is able to learn the kernel expression, together with a set of the corresponding hyperparameters. These evolved kernels keep the convenient property of being able to estimate the uncertainty of the predictions. By using a multi-objective approach the genetic programming algorithm evaluates kernels using three different criteria. Two of them measure kernel accuracy, and the third criterion minimizes the computational time needed, thus indirectly penalizing complexity.

Beyond sentiment analysis, other semantic analysis tasks, such as the post-editing effort prediction, could be addressed using this approach.

Author: César Tomé López is a science writer and the editor of Mapping Ignorance

Disclaimer: Parts of this article may be copied verbatim or almost verbatim from the referenced research paper.