Learning-Based Techniques for Facial Animation

Abstract

For decades, animation has been a popular storytelling technique. Traditional tools for creating animations are labor-intensive, requiring animators to painstakingly draw frames and motion curves by hand. An alternative workflow is to equip animators with direct real-time control over digital characters via performance, which offers a more immediate and efficient way to create animation. Even when using these existing expression transfer and lip sync methods, producing convincing facial animation in real-time is a challenging task. In this work, we present several deep learning techniques to model and automate the process of perceptually valid expression retargeting from humans to characters, real-time lip sync for animation, and building an emotionally aware embodied conversational agent. We also present the findings from user studies and some promising future directions in this domain.