User-Centric Computational Videography

Abstract

Digital video is ubiquitous: virtually every mobile device comes with at least one high-resolution video camera, and users often upload video to community websites – 300 hours of video are uploaded to YouTube alone every minute. Yet, many commercial solutions for capturing, editing and browsing videos are difficult to use for end users, and hence constrict user creativity.

Video capture requires framing techniques and shot planning to effectively convey the intended message and be comfortable to watch. Handheld capture with mobile devices in particular often results in shaky and wobbly footage. In addition, traditional video editing tools are not keeping pace with the proliferation of video content, and some provide little more than an image-editing interface with a timeline. Unfortunately, this rather trivial addition of the temporal axis to images does not enable users to perform complex editing tasks that change the content of videos, like adding or removing objects in a video. Furthermore, existing community video collections typically treat videos like photos by having users navigate static thumbnails, instead of visualising the spatial or temporal overlaps between videos.

User-Centric Computational Videography aims to improve the quality and flexibility of capturing, editing, and exploring consumer videos. In this course, we discuss recent techniques in computer vision and graphics, and analyze how they have advanced towards this goal. By finding and exploiting inter- and intra-video content connections, these techniques make videos easier for amateur users, for example by enabling dynamic object removal in videos, and provide new empowering video experiences like content-based video browsing. We will take stock of the progress made so far on this topic, discuss current trends in the software industry as well as in research, and propose directions for future research.

Our course is targeted at a broad audience: enthusiastic video users will discover cutting-edge video processing techniques that may soon find their way into consumer applications, video editors will learn about powerful approaches that break from the norm of timeline editing, and researchers will benefit from a high-level overview and analysis of user-centric video techniques. We consider the first and last quarters of our course to be most suitable for beginners, as they provide the background on existing video tools and timeline editing, and discuss user interfaces for video exploration. The middle half of our course covers more advanced video editing techniques, which are also of interest in video production.