• Home
  • Blog
  • News
  • What Is Face Tracking and How It Boosts Entertainment Apps

What Is Face Tracking and How It Boosts Entertainment Apps

This post is the first installment of our ongoing series to explore the numerous applications and solutions of face tracking. Here, we define what face tracking is, explain how it functions, and show its various applications in gaming, arts, and entertainment. If you want to learn more about the sought-after technology and how successful companies are implementing it, read on.

It’s no secret that augmented reality is currently one of the hottest topics of the tech world. Ever since the meteoric rise of Snapchat and Pokémon Go, companies, investors, and developers alike are delving into the auspicious domain by exploring the technology and its numerous applications. As AR continues to gain traction, the search for the most relevant and effective algorithmic solutions is as crucial as ever.

One of the most popular technologies that’s making its way into the AR world is markerless face tracking. Regardless of whether or not you’ve heard of it, chances are, you’ve experienced it in one way or another without even knowing it. If you’ve ever taken a photo or video using a cute face effect or camera sticker, then you already have an idea of some of the ways the transformative feature works.

Markerless face tracking, often referred to as facial motion capture (mo-cap), is a computer vision technology that obtains data from still images and video sequences by automatically tracking facial landmarks in real-time. The technology then analyzes the input to perceive head poses and facial expressions, and finally renders the information to an application. Unlike its marker-based predecessor (think Beowulf or The Polar Express), markerless facial tracking captures and converts a subject’s expressions and movements by pinpointing specific facial landmarks sans markers, wires, and high-def cameras. The more facial points it tracks, the more accurate the depiction of facial features will be. The top SDKs will even track the jawline–an integral and challenging facial area that some trackers forgo–and identify gaze direction.

With the growing number of entertainment, gaming, and photography apps in the marketplace, developers and companies need innovative, fun ways to engage their users, lest they risk becoming obsolete. A quick glimpse at the messaging app landscape, and it becomes clear that customizable selfies via masks, stickers, and filters are not just trending, but here to stay.

For some, the very idea of a technology scanning a person’s face conjures up dystopian ideas of Big Brother. To clarify, though facial recognition employs face tracking technology (we'll outline and explore the differences later), face tracking itself doesn’t analyze and archive your face and identity. On the contrary, it simply does what it suggests–detects and tracks facial movements. All you need is a camera.

If you’re a selfie-holic, then you understand that we’re amid a digital sticker movement, and plain ol’ selfies with banal filters are old news. Popular photography app Line Camera by LINE Corporation (the communication app boats over 220 million monthly active users) utilizes facial tracking through its motion stickers, as does Camera360  (over 700 million users worldwide). Both apps offer playful, transformative effects like face swapping that enliven smartphone photography with a whimsical, crafty feel.

Facerig’s newly launched mobile app also uses markerless face tracking to power its 3D animated avatars. Once the app detects the user’s face and captures a neutral expression, users can play the puppeteer with over 40 unique avatars with higher fidelity of facial expressions. It’s become such a hit that even YouTube sensations PewDiePie and jacksepticeye have featured the program.

These are just a few examples of how face tracking can combat app lethargy and boost enthusiasm. Fun and games aside, face tracking as a whole possesses deeper and wider applications beyond custom selfies and virtual cosplay. It’s one of the most important disruptive technologies in the IoT era that–along with machine learning–brings about numerous inventions in human-computer interaction applications. As it bridges the gap between human and computers, it’s revolutionizing business models and changing the technological landscapes in various industries from gaming, entertainment, retail, and robotics to biometrics, health, and distracted driving–which we will explore in the next post. 

Written by Angela H.