The MHI-Mimicry database was created in order to study the phenomena occurring in social interactions between humans in more detail, and more particularly to analyse mimicry in human-human interaction. Specifically, the goal of the database is to provide a collection of recordings in which the participating subjects are acting with a significant amount of resemblance and/or synchrony.
Recordings were made of two experiments: a discussion on a political topic, and a role-playing game. In total there are 54 recordings, of which 34 are of the discussions and 20 of the role-playing game. Participants were seated facing each other approximately 6 feet apart, and equipped with both a headset personal microphone and a head-mounted tiara for head pose estimation. Apart from the recordings, the database contains annotations for many different phenomena, including dialogue acts, turn-taking, affect, head gestures, hand gestures, body movements and facial expressions. Accurate head pose estimation is also available.
In total, 40 participants were recruited, of which 28 were male and 12 female, aged between 18 and 40 years old at the time of the recordings. All of the participants self-reported their felt experiences after conduction of the experiments.
The recordings were made under controlled laboratory conditions using 15 cameras (res:1024×1024, frame-rate:58fps) and 3 microphones (sr:48kHz). Camera views include frontal and overhead (for each participant), and profile. All sensory data was synchronised with extreme accuracy (less than 10ns) using hardware triggers.
The paper regarding multi-sensor synchronization is:
J. Lichtenauer, J. Shen, M. F. Valstar, and M. Pantic.
“Cost-effective solution to synchronised audio-visual data capture using multiple sensors”
Image and Vision Computing Journal, vol. 29, pp. 666-680, 2011.
The paper describing the database (please cite the paper whenever using the data from the MHI-Mimicry database):
X. Sun, J. Lichtenauer, M. F. Valstar, A. Nijholt, and M. Pantic.
“A multimodal database for mimicry analysis”
Proc. of the Int’l Conf. Affective Computing and Intelligent Interfaces, 2011.
- year: 2011
- url: http://mahnob-db.eu/mimicry/
- main_author: MAHNOB
- license: Free for academic purposes
- subjects: 40
- recordings: 54
- duration: 13
- naturality: mixed
- media: Audio-visual: 15 video streams; 3 audio streams
Categories: face-analysis; gesture-analysis; voice-analysis; multimodal-analysis