Tragic Talkers: A Shakespearean Sound- and Light-Field Dataset for Audio-Visual Machine Learning Research

Dataset Overview

We present "Tragic Talkers", an audio-visual dataset consisting of excerpts from the "Romeo and Juliet" drama captured with microphone arrays and multiple co-located cameras for light-field video. Tragic Talkers provides ideal content for object-based media (OBM) production. It is designed to cover various conventional talking scenarios, such as monologues, two-people conversations, and interactions with considerable movement and occlusion, yielding 30 sequences captured from a total of 22 different points of view and two 16-element microphone arrays. Additionally, we provide voice activity labels, 2D face bounding boxes for each camera view, 2D pose detection keypoints, 3D tracking data of the mouth of the actors, and dialogue transcriptions.

Dataset Links

• The dataset requires registration, please check the license information and request access.


• Short video previews of the 30 sequences of the Tragic Talkers dataset can be viewed from this link.


• Useful python scripts to get started with the Tragic Talkers dataset (e.g. PyTorch dataloaders and audio-visual feature extractors) are available from our GitHub repository.


• If you have received the username and password, you can download the TragicTalkers dataset.

Paper

This is the author’s version of the paper. It is posted here for your personal use. This paper is published under a Creative Commons Attribution (CC-BY) license. The definitive version was published in the ACM Digital Library, https://doi.org/10.1145/3565516.3565522

License

The datasets are free for research use only.

This agreement must be confirmed by a senior representative of your organisation. To access and use this data you agree to the following conditions:

The copyright of the TragicTalkers dataset is owned by The Centre for Vision Speech and Signal Processing, University of Surrey, UK. The data should not be redistributed. Permission is hereby granted to use the TragicTalkers dataset for academic purposes only, provided that it is referenced in publications related to its use as follows:

D. Berghi, M. Volino and P. J. B. Jackson, "Tragic Talkers: A Shakespearean Sound- and Light-Field Dataset for Audio-Visual Machine Learning Research," European Conference on Visual Media Production (CVMP), 2022, doi: 10.1145/3565516.3565522.

    @inproceedings{Berghi:2022:TragicTalkers,
	AUTHOR = "Berghi, Davide and Volino, Marco and Jackson, Philip J. B.",
	TITLE = "Tragic {T}alkers: A {S}hakespearean Sound- and Light-Field Dataset 
		for Audio-Visual Machine Learning Research",
	BOOKTITLE = "European Conference on Visual Media Production (CVMP)",
	PUBLISHER = "Association for Computing Machinery",
	YEAR = "2022",
	DOI  = "10.1145/3565516.3565522"
    }
	    

Acknowledgments