Figure 1: Visualizations of Motion Sequences Showing Virtual Characters Performing Various Intended Actions with Different Objects, as Generated by Our Method. We synthesize the full-body pose sequences along with the 3D object positions from textual inputs. Our method can synthesize single-handed as well as two-handed interactions depending on the intent and the type of the object used.


Can we make virtual characters in a scene interact with their surrounding objects through simple instructions? Is it possible to synthesize such motion plausibly with a diverse set of objects and instructions? Inspired by these questions, we present the first framework to synthesize the full-body motion of virtual human characters performing specified actions with 3D objects placed within their reach. Our system takes as input textual instructions specifying the objects and the associated `intentions' of the virtual characters and outputs diverse sequences of full-body motions. This contrasts existing works, where full-body action synthesis methods generally do not consider object interactions and human-object interaction methods focus mainly on synthesizing hand or finger movements for grasping objects. We accomplish our objective by designing an intent-driven full-body motion generator, which uses a pair of decoupled conditional variational auto-regressors to learn the motion of the body parts in an autoregressive manner. We also optimize the 6DoF pose of the objects such that they plausibly fit within the hands of the synthesized characters. We compare our proposed method with the existing methods of motion synthesis and establish a new and stronger state-of-the-art for the task of intent-driven motion synthesis.


Qualitative Results for various Intents


  • arXiv

  • Source Code (github)


title={IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions},
author={Ghosh, Anindita and Dabral, Rishabh and Golyanik, Vladislav and Theobalt, Christian and Slusallek, Philipp},


For questions, clarifications, please get in touch with:
Anindita Ghosh

This page is Zotero translator friendly. Imprint. Data Protection.