Amazon today announced that a team from the University of California, Santa Barbara (UCSB) has won the overall first-place prize for the Alexa Prize SocialBot Grand Challenge 5 (SGC5). In addition, a team from Stanford University was awarded the first-place prize for scientific innovation — a category that was newly introduced for SGC5 with the aim of allowing teams to focus on advancing the field of conversational AI through a deeper study of the fundamentals of open dialogue conversations.
SGC5 teams competed to build socialbots with the aim of providing a compelling interactive multimodal and conversational user experience, integrating speech with visuals. Teams pursued a broad range of approaches including emotive avatars, synchronized graphics and multimedia, image generation, and multimodal dialogue using hints and touch input.
The GauchoChat team of four UCSB students, advised by Xifeng Yan, earned $250,000 for its overall first-place performance. The Stanford Chirpy Cardinal team, consisting of nine students advised by Christopher Manning, was also awarded $250,000. The work of both teams, along with that of the other SGC5 participants, is now captured in a series of research papers.
“This is high recognition to our efforts during the past eight months for the deployment, testing, and optimization on our socialbot,” said Hong Wang, the UCSB GauchoChat team leader. “Additionally, it is also strong proof of the effectiveness and user-preference towards our bot.”
“The innovations we’ve brought to the table, thanks to SGC5, have really transformed the way we think about open-domain conversation, and we’re confident that many others in the dialogue community will share the same view as well,” said Ryan Chi, the Stanford Chirpy Cardinal team lead.
This was the first iteration of the SocialBot Grand Challenge to incorporate multimodal customer experiences. In addition to verbal conversations, customers with Echo screen devices or a Fire TV were presented with images or text meant to enhance the conversational experience. Teams were offered the opportunity to improve their customer interactions by including text and images that provide more diverse and meaningful information.
“We are most proud that we constructed a personalized, proactive, and robust socialbot system which steadily achieved the first place on the user-rating leaderboard,” said Rajan Saini, a GauchoChat team member.
Team NAM from the Stevens Institute of Technology, advised by Jia Xu, and team Alquist from Czech Technical University, advised by Jan Sedivy, took second and third place, respectively, in the overall prize. Team Athena from the University of California, Santa Cruz, advised by Xin Eric Wang; and team HokieBot from Virginia Tech, advised by Lifu Huang, earned second and third in the scientific innovation category. The teams were each awarded $50,000 for second and $25,000 for third.
“This year, SGC teams have pushed the boundaries of conversational AI by harnessing the potential of large language models to craft robust open-domain socialbots and identified key opportunities to enhance LLMs for building the next-generation interactive and multi-modal conversational AI assistants,” said Reza Ghanadan, a senior principal scientist in Alexa AI and head of Alexa AI Prize.
Alexa customers interacted with the university socialbots by saying “Alexa, open Alexa Prize” on Amazon Echo or Fire TV devices. Customer ratings and feedback helped the student teams improve their bots. During the finals phase, university teams implemented their latest innovations and adjusted their approach based on customer feedback.
Each university selected for the challenge received a $250,000 research grant, Alexa-enabled devices, free Amazon Web Services (AWS) cloud computing services to support their research and development efforts, access to Amazon scientists, the CoBot (conversational bot) AI/ML toolkit and other tools such as automated speech recognition through Alexa, neural detection and response generation models, conversational datasets, user ratings and feedback, and design guidance and development support from the Alexa Prize team.
The ultimate goal is to meet the Grand Challenge: earn a composite score of 4.0 or higher (out of 5) from a panel of judges, and have those judges find that at least two-thirds of their conversations with the socialbot in the final round of judging remain coherent and engaging for at least 20 minutes. Although the first team to meet that benchmark would have been eligible to receive a $1 million research grant for their university, no teams met that benchmark in the SGC5 competition.
“I’m very proud to see the extraordinary commitment and tireless effort of students from diverse SGC teams worldwide, leveraging the CoBot AI platform and tools to conceptualize, experiment, and validate their scientific innovations using real-world online feedback they received from Alexa customers,” Ghanadan said. “Their journey of learning, collaboration, and growth throughout the nine-month program has been truly remarkable. As evidenced in this year’s socialbot proceedings, SGC5 teams have made significant scientific contributions in advancing the science of human-AI Interaction, demonstrating their socialbot innovations in the real-world environment.”
In previous challenges, participating teams have improved the state of the art for open domain dialogue systems by developing improved natural language understanding (NLU) systems, neural response generation models, common sense knowledge modeling, and dialogue policies leading to smoother, and more engaging conversations.
The “Alquist” team from Czech Technical University won the fourth challenge in SGC4, with teams from Stanford and the University of Buffalo earning second- and third-place prizes, respectively. The publications from that challenge can be found here.
Winning teams from previous years also include Emory University, the University of Washington, and the University of California, Davis.
