Amazon today announced it is launching a new Alexa Prize SimBot Challenge, a competition focused on helping advance development of next-generation virtual assistants that will assist humans in completing real-world tasks by harnessing generalizable AI methodologies such as continuous learning, teachable AI, multimodal understanding, and reasoning.
The challenge is an extension of the Alexa Prize program, which includes existing TaskBot and Socialbot challenges.
“Next-generation assistants won’t just be sitting in your kitchen or living room, and responding to your questions or requests,” said Dilek Hakkani-Tür, Alexa AI senior principal scientist. “They will be ambient and mobile, helping customers accomplish tasks in the real-world through better situated intelligence. The goal of the SimBot Challenge is to advance the science so that future generations of virtual assistants are capable of learning continuously, and are imbued with a more generalized intelligence.”
SimBot Challenge
Human-robot interaction has long been investigated within the field of artificial intelligence, including using dialogue as the interaction mechanism for completing tasks. The SimBot Challenge will focus on navigation, object manipulation, and machine perception and reasoning within a virtual world.
The SimBot Challenge will have two phases: A public benchmark phase, and a live interactions phase.
The goal of the SimBot Challenge is to advance the science so that future generations of virtual assistants are capable of learning continuously, and are imbued with a more generalized intelligence.
Dilek Hakkani-Tür, Alexa AI senior principal scientist
Participants in both phases will build machine-learning models for natural language understanding, human-robot interaction, and robotic task completion. Artificial intelligence challenges addressed in the competition relate to reasoning on language and scene understanding, learning from demonstration, self-learning, and task completion utilizing natural language.
Unlike previous Alexa Prize competitions, the public benchmark challenge phase will be open not only to teams of university students but also to individuals in academia and industry interested in advancing the science of AI and engaging top researchers from around the globe. The SimBot Challenge public benchmark phase is similar to existing competitions on language-guided visual navigation and task completion.
The live interactions challenge phase will comprise university teams who will compete to develop the bot that best responds to customer commands and multimodal sensor inputs from within a virtual world. Similar to previous Socialbot challenges, customers will participate in this phase; Alexa customers will play the game on their Amazon Echo Show devices, seeking to solve mysteries and progressively harder tasks within the virtual environment.
University teams selected for the SimBot Challenge following the public benchmark phase will receive a research grant, Alexa-enabled devices, free Amazon Web Services (AWS) cloud computing services to support their research and development efforts, as well as other resources, and Alexa team support. The winning team will receive a $500,000 prize, while the second- and third-place teams will receive prizes of $100,000 and $50,000, respectively.
“Our Alexa Prize Socialbot Challenge introduced a new means by which industry and academia could collaborate to advance the science behind conversational AI,” said Prem Natarajan, Alexa AI vice president of Natural Understanding. “Just as the Socialbot Challenge broke new ground in enabling university students to advance the state of conversational AI by leveraging real-time feedback from millions of Alexa customers, the SimBot Challenge will enable university students to advance multimodal human-robot interaction and teachable AI much faster than previously possible. We are excited to see how the participating teams will help evolve this area of study from science fiction to science fact.”
The application period for university teams looking to participate in the Alexa Prize SimBot Challenge begins today, and extends to October 31, 2021. More information about the Alexa Prize SimBot Challenge and the application process is available on www.alexaprize.com.
TEACh Dataset
In conjunction with the announcement of the SimBot Challenge, Amazon today is publicly releasing TEACh, a new dataset of more than 3,000 human-to-human dialogues between a simulated user and simulated robot communicating with each other to complete household tasks.
New dataset for training household robots
On the blog, Alexa AI applied scientist Aishwara Padmakumar provides greater detail and context about TEACh, a new dataset that contains more than 3,000 simulated dialogues, in which a human instructs a robot in the completion of household tasks, and associated visual data from a simulated environment.
In TEACh, the simulated user cannot interact with objects in the environment and the simulated robot does not know the task to be completed, requiring them to communicate and collaborate to successfully complete tasks. The public benchmark phase of the SimBot Challenge will be based on the TEACh dataset Execution from Dialog History (EDH) benchmark which evaluates a model’s ability to predict subsequent simulated robot actions, given the dialogue history between the user and the robot, and past robot actions and observations.
Due to the unconstrained dialog interface used in the collection the dataset, TEACh dialogue sessions demonstrate a wide range of interesting dialogue phenomena including variation in instruction granularity, completeness, relevance and repetition, coreference to previously mentioned entities, past actions and locations, language-guided backtracking and correction of mistakes – all of which will be important aspects in the SimBot Challenge.
A research paper about TEACh is available on arXiv.
Alexa Prize TaskBot and Socialbot challenges
The new challenge will be conducted in parallel with the Alexa Prize TaskBot Challenge, which was announced earlier this year. Ten university teams from three continents are currently participating in the TaskBot Challenge, the first conversational AI challenge to incorporate multimodal (voice and vision) customer experiences.
The SimBot Challenge will also run in parallel with future editions of the Alexa Prize Socialbot Grand Challenge.
In August, Amazon announced that a team from Czech Technical University in Prague won the fourth SocialBot challenge. Previous challenge winners include teams from the University of Washington, the University of California, Davis, and Emory University.
Each of the nine participating teams in Alexa Prize Grand Challenge 4 published a research paper outlining their technical approaches to this year’s competition. The papers are available on the Alexa Prize website.
Details on Alexa Prize Socialbot Grand Challenge 5 will be available in the coming months.
