This event is part of a webinar series of Expert Talks organized by Virtuos, aimed at industry professionals and enthusiasts alike to discuss topics revolving around the future of game development.
Since the turn of the millenium, facial animation and motion capture has evolved by leaps and bounds, and technology has been a big part of creating iconic films and games, including Halo Infinite.
In October, we ran the fourth session of our webinar series on Twitch, themed “The Evolution of Facial Animation and Facial Motion Capture”. In this session, Alex Bittner, Senior Artist/Animator at NetherRealm Studios and Kyle Renteria, Creative Director at CounterPunch – a Virtuos Studio discussed how advancements in facial animation and motion capture have transformed the way video games are made today.
Moderated by Abigail Canavese, Co-development Success Manager at Virtuos, our experts went in-depth into the technological advancements of facial animation, how motion capture has transformed today’s games, best practices in motion capture, and shared advice for people aspiring to work in animation and animators looking to switch to the games industry.
Game changers in facial animation: Facial scanning and Facial Action Coding System
Both Alex and Kyle agreed that one of the greatest game changers in recent years is facial scanning, the use of facial scan data to create blendshapes driven by facial action units and being able to utilize them in-engine. “For anyone who’s not familiar with the process, we (usually) bring a model or an actor in, and they sit in this chair where they’re just in this dome, surrounded by tons and tons of cameras. The cameras will take three-dimensional scans of their face and head and generate that into a 3D model, which we can use for games and animation,” Alex explained. He added that having a model act out the facial expressions helps them create accurate face shapes and muscle movements.
Kyle also elaborated on another significant technological advancement in the field: the Facial Action Coding System (FACS). FACS is a comprehensive, anatomically-based system for describing all visually discernible facial movements. It breaks down facial expressions into individual components of muscle movement called Action Units (AUs). “Before (FACS), when it was just joints…you could only get so far with the expressions. But building it off how the muscles work, it really improved how much more emotion, expression, and different feelings you could get, and it feels more real to the audience,” Kyle shared.
How has motion capture transformed games today?
“Motion capture is this amazing tool that has given us the ability to greatly increase not only the speed and quantity of facial animation that we do but the quality and detail as well,” expressed Alex.
Adding on to Alex’s point, Kyle highlighted motion capture enables animators to have the ability to “trace” the video data of an actor’s performance and transfer the point data into the digital character’s poses, dramatically speeding up production. “The (motion tracking) dots also help us with consistency, improve accuracy, and help us to get closer to the end result that we’re trying to obtain, especially in large quantities of work,” Kyle added.
Best practices in motion capture
During the session, Alex and Kyle shared some tips and best practices for keeping video footage consistent for motion capture:
- Ensure that your head-mounted camera is framed consistently – if the camera is skewed, too high, or too low, the change in the angle of the actor’s face on the camera footage could affect the data getting translated to your 3D animations and motion tracking.
- Keep your lighting even – harsh overhead lighting may cause lens flares, which could distort or damage the motion-tracking, making it a lot more difficult to get a good capture out of the footage.
- Keep key facial features clear and unobstructed – in motion capture acting, there should be little to no objects and facial hair blocking the actor’s facial features, as the software relies on those features for tracking.
- Be gestural – even for facial motion capture, it adds a lot of life and subtle nuances to the facial performance that occurs on a subconscious level.
Advice for aspiring animators and those looking to join the games industry
“The advice I would give is to have passion for the craft. I wake up every day and the fact that I can act and help others act while having fun with it is a blessing,” Kyle said. “Facial animation is something I’m constantly observing and loving in my everyday life, so doing that for work is great for me.”
Another piece of advice he shared was to have patience.
“In live action, actors move, make their expressions, and it’s all great. But as animators, we have to make it understood and believable that these motions are real, that the characters are not just moving or rotating; they’re actually living. The nuance has to be there, and getting all of it right requires patience.”
He further encouraged aspiring animators to learn constantly, nurture their love for stories, and see themselves as digital actors.
Alex encouraged animators looking to join the games industry to be open to expanding their experience. “In game development, if you dip into any small part of it, you are a game developer. You will find yourself expanding into and getting your hands in all these different parts of it, so you just got to be willing and open to learning and being involved. Don’t ever close yourself off and think to yourself, “Well that’s not my job; that’s not my territory.”
Virtuos would like to thank all speakers and attendees for participating in our webinar. If you want to learn more about our animation capabilities, schedule a chat with us to find out more about our latest works here!
