Julia Hirschberg
Session Speaker
Julia Hirschberg has been a professor in the Department of Computer Science at Columbia Universitysince 2002, serving as Chair from July 2012 through June 2018. She was appointed Percy K. and VidaL. W. Hudson Professor of Computer Science in 2013. She received her PhD in Computer Science fromthe University of Pennsylvania. She worked at Bell Laboratories and AT&T Laboratories – Researchfrom 1985-2003 as a Member of Technical Staff, Department Head, and Technology Leader, creatingthe Human-Computer Interface Research Department at Bell Labs. She has done research on prosody,discourse structure, conversational implicature, text-to-speech synthesis, speech summarization, spokendialogue systems, emotional speech, deceptive speech, charismatic speech, entrainment, empatheticspeech and code-switching. She served as editor-in-chief of Computational Linguistics from 1993-2003and was an editor-in-chief of Speech Communication from 2003-2006 and is now on the EditorialBoard. She served on the Executive Board of the Association for Computational Linguistics (ACL)from 1993-2003, the IEEE Speech and Language Processing Technical Committee (SLTC) from 2011–,the executive board of the North American chapter of the Association for Computational Linguistics(NAACL), 2012-2015, the AAAI Council, 2012-2015, the Board of Directors of the Computing ResearchAssociation (CRA) 2013-15; 2018–, the ACM/AAAI Alan Newell Award Committee, 2018–; as ViceChair of the IEEE SPS Fellow Evaluation Committee, 2019–21; the American Philosophical SocietyClass 1 Memberships Committee, 2019–; the American Academy of Arts and Sciences Class I Section6 Membership Panel, 2020– (currently as Chair); NAE Section 5 as Vice Chair, 2019-20, and Chair,2020-21; the board of the International Speech Communication Association (ISCA) from 1999-2007 (asPresident 2005-2007); Advisory Council 2007–, chair of Distinguished Lecturers Selection Committee,2010–2012), and on the ISCA Diversity Committee (2018–). She is former co-chair of the CRAW (now CRA-WP), 2016–2019, and was a board member 2009-22. She has been a fellow of theAssociation for the Advancement of Artificial Intelligence since 1994, an ISCA Fellow since 2008,a founding Fellow of the ACL since 2011, an ACM fellow since 2015, an IEEE Fellow since 2017,a Fellow of the International Academy of Artificial Intelligence Sciences (AAIS) in 2024. She waselected to the American Philosophical Society in 2014, the National Academy of Engineering in 2017,the American Academy of Arts and Sciences in 2018, the National Academy of Artificial Intelligence(NAAI) Academy of Sciences in 2025, the Austrian Academy of Engineering (AAE) and the EuropeanAcademy of Engineering (EAE) in 2026. She has received honorary doctorates from the Royal Instituteof Technology (KTH) Stockholm and from Tilburg University in the Netherlands. She is the 2011recipient of the James L. Flanagan Speech and Audio Processing Award from IEEE and of the ISCAMedal for Scientific Achievement and received the ISCA Special Service medal in 2020. She receivedthe National Academy of Artificial Intelligence (NAAI) Artificial Intelligence Exploration Award andthe ACL Dragomir Radev Distinguished Service Medal in 2025. She was elected to the American Philosophical Society in 2014, was elected an honorary member of the Association for LaboratoryPhonology in 2014, and was selected as the Fillmore Professor at the 2017 LSA summer Institute. Shereceived a Columbia Engineering School Alumni Association (CESAA) Distinguished Faculty TeachingAward in 2009 and received the Columbia Janette and Armen Avanessians Diversity Award in 2018.She won best paper awards from EMNLP 2021 (Best Short Paper), Eurasip (2005) for Best Paperpublished in Speech Communication for 2003-2004, Interspeech 2006 (Best Student Paper), the AVIOSaward for Best Paper appearing in the Journal of AVIOS in 1994, Best Paper at CoginfoComm 2012,Best Student Paper at Speech Prosody 2014, Best Paper Award for Cognitive 2015, and 2016 BestPaper Award for an article appearing in Computer Speech and Language. She has received IBMFaculty Awards in 2005 and 2006 and has also received funding from the National Science Foundation(NSF), the National Institute of Health (NIH), the Department of Homeland Security, DARPA, Google,IARPA, KDD, Lockheed Martin, the Air Force Office of Scientific Research, the Simons Foundationand Amazon. She was an Amazon Scholar (2020-2025). She holds 6 patents for text-to-speech synthesisand audio browsing/retrieval. Her h-index is 82 and citations are 34649.
Title: Code-Switching in Dialogue Speech and Text: How is this produced and understood? Abstract: For people who speak more than one language, code-switching (CSW) is a common phenomenon. However, spoken language recognition systems, including voice assistants, find it difficult to understand and appropriately reacting to this multilingual speech. We are studying how spoken and written CSW interacts with other aspects of communication, including the production of named entities and dialogue acts and the influence of entrainment, empathy, prosody, formality, and information load. Our goals are to improve prediction of when, why, and to what effect CSW occurs as well as how to produce appropriate code-switched responses to inform further development of voice assistants and their ability to successfully interact with multilingual users. We have studied many aspects of CSW to date: Does the degree of formality of a conversation influence the degree of CSW in it? 2) What is the role of information load in predicting and explaining CSW? 3) Do speakers entrain on strategies of CSW in speech?: 4) Is there a quantifiable relationship between CSW and empathy in speech? We are current examining: 5) Which dialogue acts tend to be produced most often in CSW? 6) Does the presence of named entities prime CSW? 7) How do speakers produce intonational contours (ToBI) when they perform CSW -- Do these match either of their languages or is it different from both? We are testing all these topics on speech and lexical features of: Standard American English with Spanish, Mandarin Chinese, and Hindi.