Automatic Methods for Second-Language Fluency Assessment

Funded by the University of Illinois Critical Initiative, 2005-2008.

Richard Sproat, Chilin Shih, Mark Hasegawa-Johnson, Dan Roth, Kay Bock and Brian Ross

Project Summary

This project developed diagnostic tools and methods to aid in the development of spoken second language fluency. Since 9/11, there has been substantial interest in the Defense community and elsewhere in second language learning due to the realization that the United States is lackign in language skills. The building of a center for the development of technological aids to second language learning is therefore highly timely. We believe we can not only improve the teaching of foreign languages, but also develop technology that could have commercial opportunities. The focus on spoken language distinguishes this proposal from existing programs such as the Computer-Aided Language Learning (CALL) program already in existence at UIUC. Because of our emphasis on fluency as the ability to formulate and execute successful speech plans, in either the first or second language, this project is also distinct from existing programs that use automatic speech processing methods to help with accent correction.

What is second language fluency? For that matter, what does it mean to be a fluent speaker in one’s first language? Are these two uses of the word “fluency” related? Anecdotal evidence suggests a connection: those students who perform well at giving impromptu speeches in their own language seem to also perform well in a second language, subject to the limitations of their vocabulary and grammatical knowledge. Likewise, the common wisdom among second language instructors (and students) is that one must be naturally loquacious in order to be hired as a simultaneous translator.

In the early part of the twentieth century, second-language (L2) pedagogical theory emphasized the importance of listening, reading, and imitation. The importance of creative output for second language education has only been widely recognized since the mid-1980s (e.g., Gass and Selinker, 2001). In advanced Chinese language classes at the University of Illinois, co-PI Shih has developed a creative output paradigm based on the extemporaneous public speaking exercises of the Toastmasters. The curriculum centers around impromptu speaking by the students, in two formats, one called the “Variety Show” and the second called the “Debate”, both of which are designed to fit in a 50-minute class. The “Variety Show” incorporates a number of targeted scenarios and is run during most weeks in a semester. In a nutshell, students are asked to perform a speech on a predetermined topic, with minimal chance for preparation. The “Debate” format is run typically twice a semester, once near the middle of the term, and once near the end of the term. The “Debate” format trains students to reflect deeply about a topic and to carry out coherent arguments. So far the curriculum has mostly been evaluated qualitatively, but all measures suggest that the approach is successful.length increases over time, progressing from single words and short phrases in the beginning to complete sentences by the end of month. Their performances near the end of the first year (30 contact hours) show coherent organization, complex sentences, and the ability to use humor. One quantitative measure of success can be reported at this time: the implementation of this program in the 2003-2004 academic year resulted in a doubling of enrollment in third year Chinese for the 2004-2005 academic year.

The goal of this research is to develop and test scientific and technological models of second language fluency. In the pursuit of such models, students of Chinese will participate in both controlled psycholinguistic experiments and spontaneous extemporaneous speaking exercises in both their first and second languages. Extemporaneous speech, in both the first and second language, will be rated for subjective perceived fluency by a panel of judges. Statistical analysis will test the degree to which subjective fluency measures can be predicted by (1) psycholinguistic measures of first and second language speech planning fluency, (2) standardized language competence test scores, (3) objective measurements of the phonetic, prosodic, syntactic, and discourse features of speech.


The following datasets were created as part of this research.