Jeff Dalton

From Conversational Search to Real-World Tasks and Beyond: The Future of Multi-Modal Assistance

Conversational assistants are rapidly evolving with significant potential, but with fundamental challenges that need to be overcome to realize their potential. This talk presents current challenges and future directions for knowledge grounded conversational agents. It discusses current progress from experience developing systems and benchmarks in conversational search, with leading media organizations, and the development of the winning agent in the Alexa TaskBot challenge. It presents fundamental challenges in conversational knowledge-grounded language understanding and discusses issues on controllability of neural generative models. It highlights future directions including multi-modal understanding, new forms of mixed-initiative and rich interactions, and increasingly complex tasks.

Dr. Jeff Dalton is an Associate Professor (Senior Lecturer) in the School of Computing Science at the University of Glasgow where he leads the Glasgow Representation and Information Learning Lab (GRILL) ( His research focuses on text understanding and conversational information seeking. He completed his Ph.D. at the University of Massachusetts Amherst in the Center for Intelligent Information Retrieval. Later in Google Research, he worked on Information Extraction as part of the Knowledge Discovery Team (Knowledge Vault) and on language understanding in the Assistant Response Ranking team. He is the lead organizer for the TREC Conversational Assistance Track (CAsT) ( benchmark, the longest running conversational search benchmark. He is the recipient of a prestigious UKRI Turing AI Acceleration Fellowship on Neural Conversational Assistants and received research awards from Google, Amazon, and Bloomberg. He is the faculty advisor for the UoG GRILLBot team that won the 2021/2022 Alexa Prize Taskbot Competition, the first grant challenge to include multi-modal aspects. He holds multiple patents in retrieval, information extraction, and question answering.

Buttontwitter Buttonlinkedin
This website uses cookies to ensure you get the best experience. Learn more