Animating Virtual Speakers or Singers from Audio - Lip-Synching Facial Animation

Description

EURASIP Journal on Audio, Speech, and Music Processing, 2009, -66 pp.The aim of this special issue is to provide a detailed description of state-of-the-art systems for animating faces during speech, and identify new techniques that have recently emerged from both the audiovisual speech and computer graphics research communities. This special issue is a followup to the first LIPS Visual Speech Synthesis Challenge held as a special session at INTERSPEECH 2008 in Brisbane, Australia. As a motivation for the present special issue, we will report on the LIPS Challenge with respect to the synthesis techniques, andmore importantly the methods and results of the subjective evaluation. Facial speech animation can be found in a wide range of applications, among them the production of films and computer games, communication aids and tools for speech therapy, educational software, and various other kinds of information systems. The demands on facial animation differ largely with the application. Two main dimensions of the quality of speech animation can be identified: aesthetical and functional aspects. Naturalness and appeal are more connected to aesthetics; whilst intelligibility and listening effort define the function. The dimensions are orthogonal: there are animation systems of high naturalness whose output cannot be distinguished fromnatural video whilst the intelligibility can be very low. Conversely there are systems of clearly artificial appearance that provide intelligibility comparable to that of a natural speaker.Animating Virtual Speakers or Singers from Audio: Lip-Synching Facial Animation Optimization of an Image-Based Talking Head System Model-Based Synthesis of Visual Speech Movements from 3D Vide Lip-Synching Using Speaker-Specific Articulation, Shape and Appearance Models SynFace—Speech-Driven Facial Animation for Virtual Speech-Reading Support On the Importance of Audiovisual Coherence for the Perceived Quality of Synthesized Visual Speech