Estimating Egocentric 3D Human Pose in Global Space

ICCV (Oral)
Code and Dataset will be released soon.
(with audio)

Abstract

Egocentric 3D human pose estimation using a single fisheye camera has become popular recently as it allows capturing a wide range of daily activities in unconstrained environments, which is difficult for traditional outside-in motion capture with external cameras. However, existing methods have several limitations. A prominent problem is that the estimated poses lie in the local coordinate system of the fisheye camera, rather than in the world coordinate system, which is restrictive for many applications. Furthermore, these methods suffer from limited accuracy and temporal instability due to ambiguities caused by the monocular setup and the severe occlusion in a strongly distorted egocentric perspective. To tackle these limitations, we present a new method for egocentric global 3D body pose estimation using a single head-mounted fisheye camera. To achieve accurate and temporally stable global poses, a spatio-temporal optimization is performed over a sequence of frames by minimizing heatmap reprojection errors and enforcing local and global body motion priors learned from a mocap dataset. Experimental results show that our approach outperforms state-of-the-art methods both quantitatively and qualitatively.

Downloads


Citation

BibTeX, 1 KB

@misc{wang2021estimating,
      title={Estimating Egocentric 3D Human Pose in Global Space}, 
      author={Jian Wang and Lingjie Liu and Weipeng Xu and Kripasindhu Sarkar and Christian Theobalt},
      year={2021},
      eprint={2104.13454},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
				

Acknowledgments

Jian Wang, Kripasindhu Sarkar and Christian Theobalt have been supported by the ERC Consolidator Grant 4DReply (770784) and Lingjie Liu has been supported by Lise Meitner Postdoctoral Fellowship. We also thank Gereon Fox for his help with the narration on the supplementary video.

Contact

For questions, clarifications, please get in touch with:
Jian Wang jianwang@mpi-inf.mpg.de
Lingjie Liu lliu@mpi-inf.mpg.de

This page is Zotero translator friendly. Page last updated Imprint. Data Protection.