Affect Recognition and its Use Today in Machine Learning

Table of Contents

  1. Introduction
  2. Background of affect recognition
  3. Uses of affect recognition
    1. Amazon Rekognition
    2. VCV.ai and AllyO
    3. HireVue
  4. Banning the technology
  5. Concluding Remarks


  1. Introduction

Computer vision has been increasingly applied to different areas of our lives. For self-driving, the technology is nearly ready. Companies are taking this technology beyond human performance. The technology can pick up on subtle nuances that we display in our facial muscles and how these relate to the way that we are feeling. The name for this sub-field of psychology is called affect recognition.

  1. Background of affect recognition

The field of affect recognition can be traced back quite far. The first comprehensive analysis of determining emotions based on facial features was published in 1978 by psychologists Paul Ekman and Wallace Friesen. The system is called the Facial Action Coding System (FACS) and consists of different action descriptors. An article from the BBC states that this theory is actually rather outdated and there is much higher variablility in emotional states. However, this is still used as a core concept in emotional recognition technology.

There is an injection of funding into the affect recognition machine learning space, with AI Now stating that the field may already be worth $20B. Affect recognition typically uses convolutional neural networks in order to recognize differently trained expressions that humans exhibit. Such distinctions are highly valuable since the potential is there to detect how humans are feeling. Asking someone to review a movie will often put their true words through a filter – reading expressions, if it is done with accuracy, can offer an unfiltered look into true feelings.

  1. Uses of affect recognition

This technology is not sci-fi, it is already being applied. Companies such as Microsoft and Amazon sell their own emotion recognition algorithms. There are several notable examples of the technology that are already commercially available.

  • Amazon Rekognition 

Perhaps the most notable is Amazon’s Rekognition product which claims to be able to detect the presence of fear among other emotions. Rekognition can detect activities as well as perform accurate facial analysis of the subject that is being analyzed. Some of the most notable clients that AWS Rekognition touts are the National Football League, CBS, and National Geographic. The versatility of Rekognition is truly impressive, where they claim you can perform facial analysis with ease as well as identification of completely different items such as outdoor scenery.


AWS discusses some of the success stories of the technology being applied. The NFL discusses the difficulty they have had in looking through all the media footage that they acquire to find the asset they’re searching for. Rekognition allows them to create metadata tags and allows much quicker searches for content than was previously possible. CBS has a similar success story where they detail using Rekognition for real-time screening of content. The use the platform for editing hundreds of hours of footage each month. Amazon has some of the most notable clients on their platform perhaps due to the plethora of other machine learning offerings that they have.

  • ai and AllyO

Besides Amazon there are several that are commercializing this technology as well. VCV.ai has raised substantial seed funding to build out its automated recruiting platform that utilizes facial recognition technology. On the platform, job applicants record video of themselves and the company then utilizes their recognition technology to detect negative behaviors such as nervousness as well as how well the applicant fits in with the target organization. AllyO plays in this same space with their end-to-end recruitment AI platform. AllyO offers automated recruitment bots that can text candidates and schedule interviews as well as perform initial screening. The product is used in 15% of the Fortunate 50.

  • HireVue

The most notable HR recruitment company is HireVue. HireVue has received the most notoriety, however most of it from a negative light. Though it is the most heavily funded out of the start-ups with an impressive $93 million in seed funding. HireVue has many of the same features as AllyO such as scheduling as well as interview pre-screening capabilities. However, since they are leading the space, they are caught in a shroud of lawsuits – one most notably from a civil rights group from Oregon.

  1. Banning the technology

The technology walks a fine line. At a certain point, using machines to understand humans becomes an ethical question where we must ask ourselves if we will trust a machine to make decisions that have traditionally been human ones to make.

One of the biggest opponents is the AI Now Institute from New York University which claims that the determination of mood based on facial expressions lacks any scientific evidence. They published a recent report that discusses from of the gravity of decisions that the technology could be in charge of making. Affect recognition does seem like it can be correlated with mood in many instances – of course not all cases. Those cases where it is wrong is primarily what is concerning people. Several in-depth reviews of the technology says that it should be used with caution. These types of algorithms can oversee making some of the huge decisions in our life and of course it is acceptable to expect a broad base of scientific proof that the concept really works.

  1. Concluding Remarks

As artificial intelligence continues to become more ever present in our daily lives, the concept of affect recognition needs to be considered and regulated. The reliability of this needs to always be questioned especially when it is used to make important decisions such as those in hiring processes.


Show More

Nick Allyn

Hello, my name is Nick Allyn. I am extremely passionate about the field of artificial intelligence. I believe that artificial intelligence will save millions of lives in the coming years due in higher cancer survival rates, cleaner air, as well as autonomous cars.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button