Light-field video has recently been used in virtual and augmented reality applications to increase realism and immersion. However, existing light-field methods are generally limited to static scenes due to the requirement to acquire a dense scene representation. The large amount of data and the absence of methods to infer temporal coherence pose major challenges in storage, compression and editing compared to conventional video. In this paper, we propose the first method to extract a spatio-temporally coherent light-field video representation. A novel method to obtain Epipolar Plane Images (EPIs) from a spare lightfield camera array is proposed. EPIs are used to constrain scene flow estimation to obtain 4D temporally coherent representations of dynamic light-fields. Temporal coherence is achieved on a variety of light-field datasets. Evaluation of the proposed light-field scene flow against existing multiview dense correspondence approaches demonstrates a significant improvement in accuracy of temporal coherence.


4D Temporally Coherent Light-field Video
Armin Mustafa, Marco Volino, Jean-Yves Guillemaut and Adrian Hilton
3DV 2017


				title = {4D Temporally Coherent Light-field Video},
				author={Mustafa, A. and Volino, M. and Guillemaut, J.-Y. and Hilton, A.}



This research was supported by the InnovateUK Live Action Lightfields for Immersive Virtual Reality Experiences (ALIVE) project (grant 102686).