Algorithms Tell World What We’re Thinking, Discern our Moods

A machine called the Hedonometer—yes, you read that right—is now discerning how happy we are by analyzing the words and phrases we use online via algorithms that target our moods.

Distilling monumental amounts of data, the system can decipher how world events impact society in myriad ways. However, it also shows us just how much our privacy has deteriorated over the years, as our words are put through a kind of public mill to be spit out on the other end.

Chris Danforth, more than any other person, is responsible for the perfection of this mega machine that crunches our online thoughts and musings and comes out with reports on just how we are feeling on any given day. A graduate of the University of Vermont, the applied mathematics professor devotes much of his time to interpreting the results from the Hedonometer, which itself is the product of several decades of research at the University.

Operating on a 24/7 basis, the Hedonometer analyzes some 50 million tweets every day Twitter and then spits out an overall picture of the mood of the public.

Using computers to assess the emotional tone of words has been the goal of researchers for as many decades as there have been computers. However, in his construction of the Hedonometer, Danforth had to learn how to teach a machine to comprehend the emotions behind Tweets.

Sentiment analysis was spot-on for March of 2020

The process by which the machine understands emotions is called “sentiment analysis.”

Not surprisingly, the year 2020 was the saddest year for all of us if the Hedonometer is any measure of this, which it is.

The machine, which has been in use since 2008, got its biggest workout in March of 2020 as the pandemic hit the world in full force, engendering fear, unease, and even panic.

As reported in Knowable Magazine, Wednesday, March 11, 2020 was the apex of this arc of bad news, and it was reflected in the public discourse. That was when the NBA suspended its season; President Trump issued an order suspending some travel from Europe to the US; and the popular actor Tom Hanks announced that he and his wife, Rita Wilson, had come down with the virus.

The very next day, Danforth and his team noticed that the sentiment analysis showed that there had been a precipitous drop in the national mood. It also lasted for weeks, as Americans and others began to realize that the pandemic would upend their social lives, as well as the way they worked, studied, and traveled.

“In the entire history of our instrument, over a decade, we’ve never seen an event that effectively persists in our collective mood for more than a day or two,” Danforth said in an interview. “Since March 12, the mood (was) dramatically depressed on Twitter.”

Using machines to assess mood is a tool that lends itself naturally to many areas of endeavor, including marketing, medical, and other types of research—and the news media.

In addition to these more mundane areas in which gauging public sentiment is very advantageous, the Hedonometer can also be used for more ethereal purposes  including illustrating just how much sadder a minor chord in music is compared to a major chord.

Naturally, marketing insights are sought after by businesses which can use things like reviews on Yelp to determine exactly who to target in their campaigns.

Employers use algorithms to judge employees’ internal writings using algorithms

A bit more ominously, employers can utilize it to gauge the moods of their employees moods on their own internal social networks and correspondence. More helpfully, perhaps, the technology can be used to identify depressed individuals who may need immediate medical intervention.

Previously, gauging public sentiment was a much trickier endeavor. As Danforth says, “In social science we tend to measure things that are easy, like gross domestic product. Happiness is an important thing that is hard to measure.”

Computers are notoriously bad at deconstructing human language to mine it for emotional clues. However, there are many hints to the emotions behind all written text, which computers can take note of—even when they do not understand the words’ meaning.

An obvious way that the system can calculate our moods is to count certain words and determine how they stack up against each other, with the positive words weighed against the negative terms. Weighting each word for the amount of feeling behind it is even more effective.

Naturally the word “Excellent” is more powerful than “good.” This is where humans come in; Danforth’s team assigns these “weights” to words, which go on to become a part of word-to-emotion dictionaries, called lexicons.

These lexicons have now become the basis of nearly all sentiment analyses.

Of course, human language is infinitely complex, and things like humor—especially sarcasm—do not compute.

Other issues, such as word order, are also important in gauging our emotions. Simply counting the numbers of positive versus negative words does not work, as we can see in the following sentence: “I’m so glad the weather isn’t terrible and sweltering, like it was last week, which was unbearable.”

Obviously the person is happy at the present time, but the number of negative words, such as terrible, sweltering, and unbearable, outnumber the positive, so context matters.

To avoid this issue, there are now machine learning algorithms that teach computers to recognize patterns, including essential relationships between words. For example, the computer can be told that pairs of words, such as “bank” and “river” often are seen together. However, if “bank” and “money” appear in the same sentence, the writer is most likely referring to another kind of bank.

Neural networks are advanced types of algorithms

Word embeddings, developed by Tomas Mikolov of Google Brain in 2013, were another major breakthrough in the tools used to discern moods and meanings.  

In this method, each word is converted into a list of 50 to 300 numbers, called a vector. These numbers are like indelible fingerprints that not only describe each term but also the other words it is usually associated with.

This tool is part of what researchers call a neural network method. The more layers are added to these networks, the more information can be gleaned from our posts and writings of all kinds.

The earliest sentient analysis achieved approximately 74 percent accuracy. Today’s ultra-sophisticated neural nets perform with upwards of 94 percent accuracy—approaching that of a human being in their ability to judge emotion, according to Knowable.

The University of Vermont’s Hedonometer still uses a lexicon, and Danforth has no plans to change this method, which may have more to do with the vast amount of time it takes to “train” a computer to use the neural nets.

Mental health and our word choices

Early assessments of writings, begun by psychologists back in the 1960’s, revealed that patients diagnosed with depression used the pronouns “I” and “me” more often in their text. They also used more words with negative affects, and more death-related words, as well. Unsurprisingly, all this early research has been borne out in the most recent algorithm analysis of mood by the Hedonometer and other computer systems.

The University of Vermont’s Danforth, partnering with Harvard psychologist Andrew Reece, recently analyzed the Twitter posts of trials participants who had had formal diagnoses of depression or post-traumatic stress disorder. Using posts that were made prior to their diagnoses, signs of depression began to appear as many as nine months earlier, according to the researchers.

Facebook, often the bane of those who are worried about technological overreach and the obliteration of our privacy, has an algorithm to detect posters who appear to be at risk of suicide. In this case, Facebook provides human experts who review the cases and send the users prompts or helpline numbers if they deem it is warranted.

Stevie Chancellor, an expert in human-centered computing at Chicago’s Northwestern University, believes that sentiment analysis, despite its many drawbacks, could be useful for clinics, for example, when triaging a new patient.

Gauging our moods part of consumer culture today

Obviously, the business world has lost no time in taking up the tool for its own exploitation. Sentiment analysis is now widely used by companies, but since many don’t want to admit they employ it, there is no way to know exactly just how many are using it today.

Bing Liu, of the University of Illinois, who was one of the pioneers of sentiment analysis, says “Everyone is doing it: Microsoft, Google, Amazon, everyone,” adding that “Some of them have multiple research groups.”

Now, great numbers of both commercial and academic sentiment analysis software programs are widely available. Naturally, one of the data mines used by companies is social media in an effort for them to determine what their customers are saying about them and  their products.

Naturally, any way to keep an eye on employees is not missed by large employers, such as IBM, which developed a program called “Social Pulse” to monitor the company’s intranet to identify employee complaints.

Because of privacy concerns, the program only looked at posts that were shared with the entire company.

Despite this stipulation, such practices concern Danforth, who says “My concern would be the privacy of the employees not being commensurate with the bottom line of the company. It’s an ethically sketchy thing to be doing.”

However, no one assumes that sentiment analysis using algorithms to gauge our moods is going away any time soon.

All companies, including mental health professionals and any others that employ it, however, must keep in mind that the human mind is limitless in its complexity.

Understanding humans is a never-ending quest. As Liu says, “We don’t even understand what understanding is.”