Facial Type, Expression, and Viseme Generation

Abstract

Generating expressive, high-resolution 3D facial models and speech poses is essential for both feature animation and real-time interactive characters. Purely procedural models offer flexible sliders but often lack organic realism, whereas purely data-driven scan libraries achieve realism only at the cost of capturing exhaustive new 3D scans for every desired actor and expression.

Published at ACM SIGGRAPH 2007, our hybrid approach combines a sparse set of real-world 3D facial scans with a statistical learning pipeline that automatically extracts underlying geometric variation across three orthogonal dimensions:

1. Facial Identity / Archetype
2. Emotional Expression
3. Speech Visemes

Using lightweight user-assisted categorization of the sparse training scans, our system correlates subjective perceptual qualities with concrete geometric basis vectors—allowing artists to synthesize and deform entirely new 3D faces exhibiting combinations of identity, expression, and viseme never present in the original scan corpus.

Videos

Play