Example PhD

Audio and Video Behaviour Transfer Between Computer Animated Faces

Supervisor: Professor A.D. Marshall

Keywords: Image Processing, Computer Vision.

This project aims to develop methods for automatic facial animation by transferring dynamics captured from one person and applying them to the graphical model of another person. In particular, we will use audio characteristics captured from the first person to better control the appearance of the synthesised model, thereby improving the level of realism. For example, if a significant level of anger is present in the audio signal, then this will modify the graphical model so that the face appears angry.

Many researchers in computer graphics consider the development of a video-realistic computer generated human as the holy grail of computer graphics. However, there are many technical difficulties to be overcome, in particular for animation of the face since humans are highly sensitive to facial behaviour and can easily identify flaws in synthetic facial animation. Also, current animation techniques require highly skilled animators and are time consuming and costly to produce. If the process could be in part automated then the benefits to the computer graphics, animation and multimedia industries are enormous.

Performance driven facial animation is the driving (i.e. animating) of a computer graphics face by means of data captured from another person's face. The benefits of this method is that it can produce more realistic dynamics, and requires less manual interaction by skilled animators. The proposed research project will combine audio information into this process.

The methodology is based on the previous work carried out at Cardiff University on developing a 2D and 3D image based system in which the texture and geometry of a face is represented by means of a model built using statistical analysis of input training examples of audio-visual head data. This parameterised model enables us to both analyse images (and fit the model) as well as synthesise new facial images. We have previously applied our work to facial video behaviour transfer. The proposed PhD will extend this work to incorporate the audio information.

Key Skills/Background: Strong computing and mathematical skills required

Contact: Professor A.D. Marshall to discuss this research topic.