NRST: Non-rigid Surface Tracking from Monocular Video

German Conference on Pattern Recognition(2018), Stuttgart, Germany

Download Video: HD (MP4, 41 MB)


We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-rigidly to each frame.We formulate the per-frame registration as an optimization problem that includes a novel texture term specifically tailored towards tracking objects with uniform texture but fine-scale structure, such as the regular micro-structural patterns of fabric. Our texture term exploits the orientation information in the micro-structures of the objects, e.g., the yarn patterns of fabrics. This enables us to accurately track uniformly colored materials that have these high frequency micro-structures, for which traditional photometric terms are usually less effective. The results demonstrate the effectiveness of our method on both general textured non-rigid objects and monochromatic fabrics.



BibTeX, 1 KB

 author = {Habermann, Marc and Xu, Weipeng and Rhodin, Helge and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian},
 title = {{NRST: Non-rigid Surface Tracking from Monocular Video}},
 journal = {German Conference on Pattern Recognition (GCPR)},
 issue_date = {February 2019},
 volume = {11269},
 month = October,
 year = {2018},
 pages = {335-348},
 numpages = {14},
 issn = {978-3-030-12939-2},
 url = {},
 doi = {10.1007/978-3-030-12939-2_23},
 publisher = {Springer},
 address = {Cham, Switzerland},
 keywords = {Non-rigid surface deformation},


This work is funded by the ERC Starting Grant project CapReal (335545).


For questions, clarifications, please get in touch with:
Marc Habermann

This page is Zotero translator friendly. Imprint. Data Protection.