Embodied Conversational Agents with Realistic

Speech and Language: Science and Applications

Dom Massaro - University of California, Santa Cruz

Speech and language science evolved under the assumption that speech was a 
solely auditory event. However, a burgeoning record of research findings 
reveals that our perception and understanding are influenced by a speaker's 
face and accompanying gestures, as well as the actual sound of the speech. 
Perceivers expertly use these multiple sources of information to identify 
and interpret the language input. Given the value of face-to-face interaction 
and theoretical framework, our persistent goal has been to develop, evaluate, 
and apply animated agents to produce realistic and accurate speech. Baldi is 
an accurate three-dimensional animated talking face appropriately aligned 
with either synthesized or natural speech. Based on this research and 
technology, we have implemented animated agents as tutors for children with 
language challenges and persons learning a second language. Our 
language-training programs utilize these agents to guide students through a 
variety of exercises designed to teach vocabulary and grammar, to improve 
speech articulation, and to develop linguistic and phonological awareness.