1. Introduction
Initial efforts to animate three-dimensional faces using computers go back more than 20 years. In recent years there has been a considerable progress in this area due to the growing performance of middle range personal computers.
There are several approaches to facial animation. In all approaches the surface of the face is manipulated over time by changing the position of the surface polygons (MESH) or the surface control points (NURBS).
Today the different techniques can be grouped in two fundamental animation types:
- Skin and muscle control
- Morphing technologies
2. Skin and muscle based facial animation
The anatomy of the head and face is a complex assembly of skin, muscles, nerves and other elements. To date no model has been developped with a high level of anatomical detail, but several tools exist which are based on a reduced complexity of the anatomy and mechanical properties of the face. Two categories of muscles affect the shape of the skin by bulging, stretching and pulling it in a number of directions.
The upper face muscles control the eyes and brows. There are four major muscles in the upper face:
- Frontalis: this muscle raises and lowers the eyebrows and moves during speech
- Curragator: this muscle assist in brow motion during emotions (anger, fear, sadness, ...)
- Levator Palpebrae: this muscle raises and lowers the eyelids
- Orbicularis Oculi: this muscle is used for eye movements
Seven muscles work together to move the lower face with the mouth and jaw:
- Orbicularis Oris: this muscle tightens the lips into a pursed position
- Zygomatic Major: this muscle pulls the face into a wide grin
- Levator Labii Superiors: this muscle raises the upper lip and the nose
- Triangularis: this muscle drives the corners of the lips downward
- Risorius: this muscle lowers the bottom lip
- Depressor Labii Inferioris: this muscle lowers the bottom lip
- Levator Menti: this muscle raises the bottom lip
The Compaq Research Laboratory in Cambridge Massachusetts has developped a "muscle-controlled" program that constructs a face model from four input data files and displays the result:
- faceline.dat
- index.dat
- muscle.dat
- expression-macros.dat
|
|
|
|
|
|
|
|
|
|
|
|
 |
The program is written in C and uses OpenGL display commands. An embeddable face OCX (Ole Custom Control) for the web (SimpleFace) using SAPI (Speech Application Programmers Interface) compliant text to speech engines to synchronize the mouth opening via text is also available in a beta version (V3.0b). A similar technique is used by FaceWorks Studio, a program also developped by Compaq. It allows you to transform any 2D image (JPEG) and map it to a 3D face model and to apply a sound file to generate automatic lip synchronous facial animations. FaceWork Studio was created for mainstream multimedia developers. |
|
Another example is the software package "Animation Master" from Hash Inc. which uses simple muscle motion for lip synchronization.
3. Morphing Technologies
Replacement animation is a precursor of the morphing technologies. It's a technique that uses several different objects, such as different mouth objects, to appear and disappear over time. Morphing is an enhanced technology which changes the shape of a base object into the shape of other target objects. The most common morphing animation is the interpolation of key frames. The desired shape articulation is specified at two time points (frames). An interpolation algorithm (linear, constant, spline, break, ...) is then used to generate the necessary poses for the in between frames.
Morphing is one basic technique of Poser animation. Poser is the premier 3D character animation and figure design tool. The standard Poser head morphs for face animations are:
- openLips
- smile
- frown
- mouth O
- mouth F
- mouth M
- tongue T
- tongue L
- brow down right/left
- brow up right/left
- worry right/left
- blink right/left
With these morphs, it's possible to create phonemes in Poser to create realistic facial animations. A phoneme is a linguistic term for the position of the lips, teeth and tongue as they make sounds.
There are several preset phonemes included in Poser. Some examples are shown in the following figures:
    
There are other animation programs using morphing technologies, some are listed below:
4. Lip Synchronization
Lip synchronization based on skin and muscle animation has been presented in chapter 2. This chapter focus on morphing technologies for producing lip synchronous face animations.
In Poser it's possible to produce lip synchronous animations by using the phoneme morphs
Paul Hafeli presented in 1998 in the Poser forum a way to accomplish the task of lip-syncing using Poser 3. He proposed to record a movie with a person saying the words you want to use in an animation in order to use it as a background in Poser. The frames per second and the aspect ratio must match the animation. If you left enough space on one side of the movie, you can place your 3D figure on the other side of the document window and set keyframes for the head in conformance with the lip movements of the person in the movie. The figure below illustrates the procedure.
Poser displays a graph palette which allows precise tunings on keyframes by syncing sound and motion.
But even using this technique, lip syncing remains a tedious task. LIPSinc developped a program to automate the the lip-syncing process based on the phoneme morphs. This enhancement product is described in chapter 5.
A list of other programs and plugin's for lip synchronization is shown below:
A web based poser lip synch utility is offered by Bill Underwood (Underdog) at his Fantasy3D site.
5. Mimic for Poser
Mimic1 from LIPSinc is a standalone program that works as an enhancement to Poser 3&4 providing speech animation by synchronizing the standard head morph parameters of Poser models with speech audio files.
I have participated in the beta test of the Mimic release 0.5 at the end of 1999. Mimic officially supports only standard U.S. english, but it works fine with german and french language and even with luxembourgish by using german phonemes.
I tried audio files with natural speech (french and german are common languages in Luxembourg used by everyone) and synthetic speech (Text to Speech engine from Digalo). A short animation with a french speaking girl (Yvette) and a german speaking boy (Karl) is shown below. |