Physically Plausible Monocular 3D Motion Capture
in Real Time

ACM Transactions on Graphics
(Proceedings of SIGGRAPH Asia 2020)
(with audio)


Marker-less 3D human motion capture from a single colour camera has seen significant progress. However, it is a very challenging and severely ill-posed problem. In consequence, even the most accurate state-of-the-art approaches have significant limitations. Purely kinematic formulations on the basis of individual joints or skeletons, and the frequent frame-wise reconstructionin state-of-the-art methods greatly limit 3D accuracy and temporal stability compared to multi-view or marker-based motion capture. Further, captured 3D poses are often physically incorrect and biomechanically implausible, or exhibit implausible environment interactions (floor penetration, foot skating, unnatural body leaning and strong shifting in depth), which is problematic for any use case in computer graphics. We, therefore, present PhysCap, the first algorithm for physically plausible, real-time and marker-less human 3D motion capture with a single colour camera at 25 fps. Our algorithm first captures 3D human poses purely kinematically. To this end, a CNN infers 2D and 3D joint positions, and subsequently, an inverse kinematics step finds space-time coherent jointangles and global 3D pose. Next, these kinematic reconstructions are used as constraints in a real-time physics-based pose optimiser that accounts forenvironment constraints (e.g.,collision handling and floor placement), gravity, and biophysical plausibility of human postures. Our approach employs a combination of ground reaction force and residual force for plausible root control, and uses a trained neural network to detect foot contact events in images. Our method captures physically plausible and temporally stable global3D human motion, without physically implausible postures, floor penetrations or foot skating, from video in real time and in general scenes.PhysCap achieves state-of-the-art accuracy on established pose benchmarks, and we propose new metrics to demonstrate the improved physical plausibility and temporal stability.



BibTeX, 1 KB

	author = {Shimada, Soshi and Golyanik, Vladislav and Xu, Weipeng and Theobalt, Christian},
	title = {PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time},
	journal = {ACM Transactions on Graphics}, 
	month = {dec},
	volume = {39},
	number = {6}, 
	articleno = {235},
	year = {2020}, 
	publisher = {ACM}, 
	keywords = {physics-based, 3D, motion capture, real time}


This work was supported by the ERC Consolidator Grant 4DReply (770784). We also thank Gereon Fox for his help with the narration on the supplementary video.


For questions, clarifications, please get in touch with:
Soshi Shimada
Vladislav Golyanik

This page is Zotero translator friendly. Page last updated Imprint. Data Protection.