Projects


Venu Govindaraju, Jinjun Xiong, Srirangaraj Setlur, Pamela Hadley, Julie Kientz, David Feil-Seifer, Brian Graham, Megan-Brette Hamilton, Anil K Jain, Ye Jia, Jennifer Taps Richard, Tracy A Sawicki, Alison Hendricks, Ifeoma Nwogu, Letitia Thomas, Christine Wang, Wenyao Xu, Maneesh Agrawala, Diego Aguirre, …, Shaofeng Zou. National AI Institute for Exceptional Education. NSF, 2023-2028


Mark Hasegawa-Johnson, Zsuzsanna Fagyal, Najim Dehak, Piotr Zelasko, & Laureano Moro-Velazquez. FAI: A New Paradigm for the Evaluation and Training of Inclusive Automatic Speech Recognition. no. 2147350, NSF, 2022-2025

Changdong Yoo, & Mark Hasegawa-Johnson. Deep F-measure Maximization for Fairness in Speech Understanding. Institute for Information & Communications Technology Promotion (IITP; Subaward by KAIST), Korea, 2020-2024


Mark Hasegawa-Johnson, Patricia Jones, Heejin Kim, Clarion Mendes, Meg Dickinson, & Erik Hege. Speech Accessibility Project. Nova Habilitas Consortium, 2022-2024


Nancy McElwain, Mark Hasegawa-Johnson, Dominika M Pindus, Bashima Islam and Charles Davies, & Maria Davila. Continuous Capture of Infant Feeding, Motor Activity, and Sleep Biorhythms via an Infant Wearable Platform and Machine Learning Approaches. University of Illinois Personalized Nutrition Initiative, 2022-2023

Amit Juneja, Mark Hasegawa-Johnson, & James Mireles. Virtual Reality Visualization of Complex and Unstructured Data and Relationships. no. FA864922P1171, USAF Research Lab (Subaward by Agile Data Decisions Inc.), 2022-2023

Najim Dehak, & Mark Hasegawa-Johnson. RI: Small: Collaborative Research: Automatic Creation of New Phone Inventories. no. 19-10319, NSF IIS, 2019-2022

Narendra Ahuja, Mark Hasegawa-Johnson, David Beiser, & David Chestek. Adding Audio-Visual Cues to Signs and Symptoms for Triaging Suspected or Diagnosed COVID-19 Patients. C3.ai Digital Transformation Institute, 2020-2022

Amit Juneja, Mark Hasegawa-Johnson, & James Mireles. Virtual Reality Visualization of Complex and Unstructured Data and Relationships. no. FA864922P0047, USAF Research LAb (Subaward by Agile Data Decisions Inc.), 2021-2022

Nancy McElwain. HEAL Consortium: Establishing Innovative Approaches for the HEALthy Brain and Child Development Study. no. 3R34 DA050256, NIH NIDA, 2019-2021

Nancy McElwain, Susan Caldecott-Johnson, Mark Hasegawa-Johnson, & Siraj Siddiqi and Romit Roy Choudhury. Early Detection Of Developmental Disorders Via A Remote Sensing Platform. Jump ARCHES, 2020-2021

Karrie Karahalios, Siraj Siddiqi, David Forsyth, & Mark Hasegawa-Johnson and Hedda Meadan. Visualizations Of Social Communication Behavior Of Children With Autism. Jump ARCHES, 2020-2021

Mark Hasegawa-Johnson, Suma Bhat, Dan Morrow, & James Graumlich. Using Conversational Agents To Support Older Adult Learning For Health. UIUC College of Education Technology Innovation in Educational Research and Design, 2020-2021


Katrin Kirchhoff, Mark Hasegawa-Johnson, & Gina Anne Levow. LanguageNet: Transfer Learning Across a Language Similarity Network. no. HR0011-15-2-0043, DARPA LO-REsource Languages for Emergent Incidents (LORELEI), 2015-2019


Mark Hasegawa-Johnson, Hanady Mansour Ahmed, Eiman Mustafawi, & Haitham El-Bashir. The Family as the Unit of Intervention for Speech-Generating Augmentative/Assistive Communication. no. NPRP 7-766-1-140, QNRF, 2014-2018


Mark Hasegawa-Johnson, Lav Varshney, & Preethi Jyothi. EAGER: Matching Non-Native Transcribers to the Distinctive Features of the Language Transcribed. NSF IIS, 2015-2018

Dan Morrow, Suma Pallathadka Bhat, Mark Hasegawa-Johnson, Thomas Huang and James Graumlich, Ann Willemsen-Dunlap, & Don Halpin. Interactive Technology Support for Patient Medication Self-Management. Jump ARCHES, 2016-2018

Yang Zhang, Gautham Mysore, & Mark Hasegawa-Johnson. Speech Resynthesis for Vocal Style Modification. Adobe Research, 2015-2017

Nancy Chen, & Mark Hasegawa-Johnson. Mismatched Crowdsourcing for 80-Language Speech Recognition. Institute for Infocomm Research (I$^2$R), Agency for Science, Technology, Advancement and Research (ASTAR), Singapore, 2015-2017


Mark Hasegawa-Johnson, Nancy Chen, & Boon Pang Lim. Noisy Channel Models for Massively Multilingual Automatic Speech Recognition. Advanced Digital Sciences Center (ADSC), Singapore, 2015-2017

Emmanuel Dupoux, Odette Scharenborg, Graham Neubig, Laurent Besacier, Mark Hasegawa-Johnson, Alan Black, Florian Metze, & Sebastian Stüker. The Speaking Rosetta Stone – Discovering Grounded Linguistic Units for Languages without Orthography. no. WS17, Frederick Jelinek Memorial Summer Workshop, 2017


Daniel G Morrow, Mark Hasegawa-Johnson, Thomas Huang, & William Schuh. Collaborative Patient Portals: Computer-based Agents and Patients’ Understanding of Numeric Health Information. no. R21-HS022948, AHRQ, 2014-2016

Xuesong Yang, Gregg Wilensky, & Mark Hasegawa-Johnson. Speech2Vec Speech-Based Semantic Vectors. Adobe Research, 2015-2016


Jia Chen Ren, Lawrence Angrave, & Mark Hasegawa-Johnson. Capturing; Transcribing; Searching; Analyzing; Adaptive: Learning in a curated classroom. Illinois Learning Sciences Design Initiative (ILSDI), University of Illinois, 2015-2016


Marshall Poole, David Forsyth, Feniosky Pena-Mora, Mark Hasegawa-Johnson and Kenton McHenry, & Peter Bajcsy. CDI-Type II: Collaborative Research: Groupscope: Instrumenting Research on Interaction Networks in Complex Social Contexts. no. 0941268, NSF BCS, 2010-2015

Mark Hasegawa-Johnson, Laura DeThorne, Tracy Gunderson, Julie Hengst, & Thomas Huang and Pat Malik. Pseudo-intelligent mediators (“Robo-Buddies”) to improve communication between students with and students without physical disabilities. Illinois Innovation Initiative (In3), 2011-2015

Mark Hasegawa-Johnson, Preethi Jyothi, Edmund Lalor, Adrian KC Lee, & Majid Mirbagheri. Probabilistic Transcription of Languages with No Native-Language Transcribers. no. WS15, Frederick Jelinek Memorial Summer Workshop, 2015


Richard Beraniuk, Volkan Cevher, Lydia Kavraki, Wotao Yin, John Benedetto, Rama Chellappa, Larry Davis, Tamer Basar, Mark Hasegawa-Johnson, Thomas Huang, Ronald Coifman, Lawrence Carin, & Stanley Osher. Opportunistic Sensing for Object and Activity Recognition from Multi-Modal, Multi-Platform Data. no. W911NF-09-1-0383, ARO MURI, 2009-2014


Mark Hasegawa-Johnson, Camille Goudeseune, Hank Kaczmarski, & Thomas Huang. FODAVA-Partner: Visualizing Audio for Anomaly Detection. no. 08-07329, NSF CCF, 2008-2013


Mark Hasegawa-Johnson, & Eiman Mustafawi. Multi-dialect phrase-based speech recognition and machine translation for Qatari broadcast TV. no. NPRP 09-410-1-069, Qatar National Research Fund, 2010-2013

Julie Hengst, Laura S. DeThorne, Ai Leen Choo, Mariana Aparacio Betancourt and Mark Hasegawa-Johnson, Karrie Karahalios, Paul Prior, Hedda Meadan-Kaplansky and David Gooler, Tracy Gunderson, & Andrew Moss. Conversation Strategies for Students With and Students Without Physical Disabilities. no. Social Interaction and Communicative Competence, University of Illinois Graduate College Focal Point Program, 2012-2013

Carol Espy-Wilson, Louis Goldstein, Elliot Saltzman, & Jennifer Cole and Mark Hasegawa-Johnson. RI-Collaborative Research: Landmark-based robust speech recognition using prosody-guided models of speech variability. no. 07-03624, NSF IIS, 2007-2012


Torrey Loucks, Chilin Shih, Ryan Shosted, & Mark Hasegawa-Johnson. Speech Production Research Initiative. University of Illinois Graduate College Focal Point Program, 2010-2011

Mark Hasegawa-Johnson, Adrienne Perlman, Thomas Huan, & Jon Gunderson. Audiovisual Distinctive-Feature-Based Recognition of Dysarthric Speech. no. 05-34106, NSF IIS, 2005-2010

Richard Sproat, J. Kathryn Bock, Brian Ross, Mark Hasegawa-Johnson, & Chi-Lin Shih. DHB: Fluency and the Dynamics of Second Language Acquisition. no. 06-23805, NSF IIS, 2006-2010

Mark Hasegawa-Johnson, Thomas Huang, & Dirk Bernhardt-Walther. RI Medium: Audio Diarization – Towards Comprehensive Description of Audio Events. no. 08-03219, NSF IIS, 2008-2010


Mark Hasegawa-Johnson, Adrienne Perlman, Thomas S Huang, Jon Gunderson and Ken Watkin, & Heejin Kim. Audiovisual Description and Recognition of Audible and Visible Dysarthric Phonology. no. 5R21 DC008090, NIH NIDCD, 2006-2009

Mark Hasegawa-Johnson. CAREER: Landmark-Based Speech Recognition in Music and Speech Backgrounds. no. 01-32900, NSF IIS, 2002-2007

Mark Hasegawa-Johnson, Jennifer Cole, & Chi-Lin Shih. Prosodic, Intonational, and Voice Quality Correlates of Disfluency. no. 04-14117, NSF IIS, 2004-2007

Richard Sproat, Chilin Shih, Mark Hasegawa-Johnson, Dan Roth, & Kay Bock and Brian Ross. Automated Methods for Second-Language Fluency Assessment. University of Illinois Critical Research Initiative, 2005-2007

Thomas S Huang, & Mark Hasegawa-Johnson. Audiovisual Emotional Speech AVATAR. no. RPS 31, Motorola Communications Center, 2005-2007

Nash Borges, Ozgur Cetin, Mark Hasegawa-Johnson, Simon King, & Karen Livescu. Articulatory Feature-based Speech Recognition. no. WS06, Johns Hopkins Center for Language and Speech Processing, 2006

Mark Hasegawa-Johnson. Rhythmic Organization of Durations for Automatic Speech Recognition. UIUC Research Board, 2005-2006


Mark Hasegawa-Johnson, & Thomas S Huang. Audiovisual Speech Recognition: Data Collection and Feature Extraction in Automotive Environment. no. RPS 19, Motorola Communications Center, 2002-2005


Mark Hasegawa-Johnson, & Jennifer Cole. Prosody-Dependent Speech Recognition. University of Illinois Critical Research Initiative, 2002-2004

Weimo Zhu. Development and Validation of An E-diary System for Assessing Physical Activity and Travel Behaviors. no. 47334, Robert Wood Johnson Foundation, 2002-2004

Jim Baker, Steve Greenberg, Mark Hasegawa-Johnson, Katrin Kirchhoff and Jennifer Muller, & Kemal Sonmez. Landmark Based Speech Recognition. no. WS04, Johns Hopkins Center for Language and Speech Processing, 2004


Mark Hasegawa-Johnson. Immersive Headphone-free Virtual Reality Audio. University of Illinois Research Board, 2001-2002


Mark Hasegawa-Johnson. Acoustic Features for Phoneme Recognition. Phonetact Incorporated, 2002


Mark Hasegawa-Johnson. Factor Analysis of the Tongue Shapes of Speech. University of Illinois Research Board, 1999-2000

Mark Hasegawa-Johnson. Factor Analysis of MRI-Derived Articulator Shapes. Individual National Research Service Award, no. 1 F32 DC000323, NIH NIDCD, 1997-1999