Dialogue Systems Go Multimodal: The SmartKom Experience.- Facts and Figures About the SmartKom Project.- An Exemplary Interaction with SmartKom.- Multimodal Input Analysis.- The SmartKom Architecture: A Framework for Multimodal Dialogue Systems.- Modeling Domain Knowledge: Know-How and Know-What.- Speech Recognition.- Class-Based Language Model Adaptation.- The Dynamic Lexicon.- The Prosody Module.- The Sense of Vision: Gestures and Real Objects.- The Facial Expression Module.- Multiple Biometrics.- Natural Language Understanding.- The Gesture Interpretation Module.- Multimodal Dialogue Processing.- Modality Fusion.- Discourse Modeling.- Overlay: The Basic Operation for Discourse Processing.- In Context: Integrating Domain- and Situation-Specific Knowledge.- Intention Recognition.- Plan-Based Dialogue Management for Multiple Cooperating Applications.- Emotion Analysis and Emotion-Handling Subdialogues.- Problematic, Indirect, Affective, and Other Nonstandard Input Processing.- Multimodal Output Generation.- Realizing Complex User Wishes with a Function Planning Module.- Intelligent Integration of External Data and Services into SmartKom.- Multimodal Fission and Media Design.- Natural Language Generation with Fully Specified Templates.- Multimodal Speech Synthesis.- Scenarios and Applications.- Building Multimodal Dialogue Applications: System Integration in SmartKom.- SmartKom-English: From Robust Recognition to Felicitous Interaction.- SmartKom-Public.- SmartKom-Home: The Interface to Home Entertainment.- SmartKom-Mobile: Intelligent Interaction with a Mobile System.- SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment.- Data Collection and Evaluation.- Wizard-of-Oz Recordings.- Annotation of Multimodal Data.- Multimodal Emogram, Data Collection and Presentation.- Empirical Studies for Intuitive Interaction.- Evaluation of Multimodal Dialogue Systems.
About the Author: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster is the Director and CEO of the German Research Center for Artificial Intelligence (DFKI GmbH) and a Professor of Computer Science at the Universität des Saarlandes, Saarbrücken. In 2000, he was coopted as a Professor of Computational Linguistics at the same university. In addition, he is the Head of the Intelligent User Interfaces Lab at DFKI.
He was the Scientific Director of the Verbmobil consortium on spontaneous speech translation (1993-2000), the SmartKom consortium on multimodal dialog systems (1999-2003), and the SmartWeb consortium on mobile multimodal access to semantic web services (2004-2008).
He has authored more than 150 technical papers and 6 books on language technology and intelligent user interfaces, and edited many books, among them the Springer titles "Verbmobil: Foundations of Speech-to-Speech Translation", "SmartKom: Foundations of Multimodal Dialogue Systems", and "SemProM: Foundations of Semantic Product Memories for the Internet of Things". His research includes multimodal and perceptive user interfaces, user modeling, embodied conversational agents, smart navigation systems, semantic web services, and resource-adaptive cognitive technologies.
He was awarded the Deutscher Zukunftspreis (German Future Award) in 2001 and he received the Federal Cross of Merit, First Class, the Officer's Cross of the Order of Merit of the Federal Republic of Germany in 2006.
He is the Chief Academic Advisor for Information and Communication Technologies (ICT) in the Research Union of the German Government, he is a member or chair of many key international scientific advisory and governance boards, including the National Institute of Informatics (NII) of Japan, the jury for the European ICT Prize of the European Commission, the Singapore Agency for Science, Technology and Research, the SAP Next Business & Technology Advisory Board, the Center for Advanced Security Research in Darmstadt, the Intel Visual Computing Institute, the Fondazione Bruno Kessler, and the Future Internet Public-Private Partnership of the European Commission.
He is a member of many academies, including the German Academy of Sciences and Literature, Mainz, the Royal Swedish Academy of Sciences, Stockholm, the German Academy of Natural Scientists Leopoldina, Acatech, the Council for Engineering Sciences of the Union of German Academies of Science and Humanities, and the Berlin-Brandenburg Academy of Sciences and Humanities.
He has chaired and given invited talks at the key international conferences in computational linguistics and artificial intelligence, he is a board member of the key AI journals, and he is a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), the European Coordinating Committee for Artificial Intelligence (ECCAI), and the Gesellschaft für Informatik (GI).
