Red9 Shows off their system lipsync service that takes audio and maps it to facial motion.

Mark Jackson, CEO & Technical Director for Red9 Consultancy, posts a look into how Talkback works. Talkback is their audio-driven lipsyncing service offered by Red9 Consultancy to clients. The system maps facial animation to characters using audio data.

About Talkback and How it Works.

Red9 TalkBack is an audio-driven lipsyncing service offered by Red9 Consultancy to clients. Developed over the last 5 years, this uses state of the art algorithms and articulation processing to realistically drive any facial rig, purely from an audio file.

“This is an off-line process, allowing us to achieve far better results than standard engine driven systems but at the expense of being a non-real-time solution.” Perfect for mid-range cutscenes or projects that can’t run to the cost of full performance capture. Initially developed to help the Red9 team test Facial Rigs, TalkBack has matured into an extensive system that they can now offer to now offering clients.

Tested on Projects.

“We’ve already run through several high-end game projects (all awaiting release) as well as several VFX projects,” Jackson says. “The system is also flexible enough to even produce accurate results on cartoon data, and the more we process, the better the learning algorithms get.”

This YouTube demo is running on one of Red9’s FacialRigs in Maya, and the results are raw data, straight out of the solver. The only input was the audio file itself. “TalkBack now runs on any rig, with training, and is a service we offer to all clients.” Perfect if your budget won’t stretch to full performance capture!

For more information, contact Red9 to see how they can help streamline your production! info@red9consultancy.com