Latest News

Real-time facial expression estimation technology has robotic applications

Omron has developed technology capable of estimating seven different facial expressions — happiness, surprise, fear, disgust, anger, sadness, and neutral — all in real time.

By indicating the degree to which each expression is evident, even ambiguous expressions can be estimated.

The technology offers interesting possibilities in a variety of applications. If incorporated into a communication robot, for instance, the robot could react to a user's facial expressions.

The technology is also suitable for image search engines, or interactive games that respond to the facial expressions of players.

Omron's Facial Expression Estimation technology combines the company's proprietary 3D model-fitting technology and statistical classification method, which are based on a database of facial images that Omron has accumulated over many years.

The technology has up till now been used primarily to recognise individual faces, but by enhancing these technologies it has become possible to also estimate facial expressions.

Rather than relying on the detection of the shapes of individual facial parts such as the eyes and mouth, Omron's technology estimates facial expressions based on the relative position of all facial parts, thus enabling high-speed estimation while requiring only a small amount of system memory.

Since the process only requires a single facial image, real-time estimation of facial expressions is possible using not only video imagery but also still images.

In line with Omron's ambition of "bringing machines closer to people," the company has been working on development of its "OKAO Vision" face detection technology since 1995.

This facial image sensing technology is capable of detecting and extracting a wide range of information from facial images, and is now one of Omron's core technologies.

The developments to date have enabled such functions as Face Detection, Face Recognition, Smile Degree Estimation, Gaze and Blink Estimation, Eye Open-close Estimation, and Age and Gender Estimation.

"OKAO Vision" technology is also used in the auto-focus function of digital cameras and smartphones, as well as the skin beautification function of printers, image management for personal computers, and many, many other applications.

Omron's estimation of seven facial expressions.The addition of the new facial expression estimation capabilities to "OKAO Vision" enables smoother communication and interaction between people and machines and will help to further expand the range of applications for the technology.

Because expressions can be estimated from a single facial image, this technology can be used with both still and video imagery and is therefore suited for use with a wide range of devices.

The high-speed data processing requires only a small amount of system memory thus enabling real-time estimation.

Specifications and performance

  • Recommended face image size: 64 pixels or higher
  • Detectable expressions: Happiness, surprise, fear, disgust, anger, sadness, and neutral
  • Processing time: Approximately 10ms (Clock speed: Snapdragon at 1.0GHz)
  • Compatible operating systems: Windows XP/7, Android OS, iOS

Possible applications

  • Robotics: Robots can react to the facial expressions of users.
  • Games: Game avatars can mimic the facial expression of players in real time.
  • Photo decoration: Decorations that match the subject's facial expression can be added to photos automatically.
  • Image search: Photos can be searched according to the facial expressions and retrieved instantly.
Send this to a friend