Amazon and the Max Planck Society (also known as Max-Planck-Gesellschaft, or MPG) announced the formation of a Science Hub in May 2022. The collaboration marked the first Amazon Science Hub outside the United States. The hub’s goal is to promote research in AI, computer vision, and machine learning whose benefits are shared broadly across all sectors of society.
In line with that goal, the Science Hub is sponsoring the following research projects:
“Non-verbal interpersonal synchrony analysis tool (nVISA-online)”: Senya Polikovsky, research engineer, Max Planck Institute for Intelligent Systems
“Interpersonal synchrony (In-Sync), the spontaneous synchronization of behavioral, affective, and even physiological responses in contexts of social interaction, offers a useful insight into the quality and smoothness of interactions between social partners. Automatic assessment of In-Sync could provide objective tools for studying the interactive abilities of people with psychiatric conditions and the degree of therapeutic alliance in psychotherapy, as well as online human-to-human and avatar-to-human interactions. Body movement dynamics are central to the analysis of In-Sync. Besides that, facial expression, gaze interaction, audio signals, and physiological measures such as electrodermal activity, heart rate variability, or breathing rates can be used to detect In-Sync. The goal of our proposed research is to extend In-Sync analysis to online scenarios. We would like to identify the instruments and the procedures for the detection of In-Sync in online therapy sessions.”
“Accurate 3D body shape and clothing estimation from a single image”: Gerard Pons-Moll, senior researcher, Max Planck Institute for Informatics, and professor, University of Tübingen, endowed by the Carl Zeiss Foundation
“This research aims to reconstruct a 3-D digital human from a single RGB photograph, using a combination of 3-D neural implicit functions to represent worn garments and a parametric model to represent and control the pose and shape of the underlying body. A key novelty is to incorporate feedback in the deep body estimation network, ensuring that the 3-D body mesh is consistent with its projection onto the photo.”
“Learning clothing pressure fields via high-fidelity finite element analysis”: Gokhan Serhat, research scientist, Max Planck Institute for Intelligent Systems, and assistant professor, KU Leuven
“Accurate estimation of clothing pressure on the human body is crucial to understanding physical factors that influence clothing comfort. Such estimation requires the ability to express contact pressure fields in terms of different body sizes and types and garment sizes and material properties. However, such pressure fields are generally too complex to be modeled by analytical methods. Experimentation also has certain drawbacks, since it does not provide information about internal stresses, and precise measurement of the contact pressure between a pair of curved soft bodies is difficult. The intricate anatomy of the human body may require elaborate finite-element models with several thousand elements for capturing deformation mechanics accurately. Such large models inherently induce high computational costs and are likely to suffer from the aforementioned numerical issues. The proposed study aims to address this problem by developing a deformation model based on deep neural networks trained with the data from finite-element analysis.”
“Material estimation of clothing for controllable human avatars”: Justus Thies, research group leader, Max Planck Institute for Intelligent Systems, and professor, Technical University Darmstadt
“A key challenge for controllable human avatars is the reconstruction and animation of clothing. While data-driven, person-specific reconstruction methods show promising results, the extrapolation to novel poses with realistic deformations of clothing is limited. In contrast to data-driven methods, physics simulation can be used to model the dynamically changing clothing. In this project, we want to explore material estimation of human clothing for such simulations. Specifically, we are interested in predicting material properties such as stiffness or stretch that influence the motion-dependent deformation of the clothing. Besides these mechanical material properties, we are also interested in predicting material properties for the appearance of clothing that allow for re-renderings under novel poses.”
