The Social Signal Processing Network
SSPNet activities revolve around two research foci selected for their primacy in our everyday life:
- Social Signal Processing in Human-Human Interaction
- Social Signal Processing in Human-Computer Interaction
Hence, the main focus of the SSPNet is on developing and validating the scientific foundations and engineering principles (including resources for experimentation) required to address the problems of social behaviour analysis, interpretation, and synthesis. The project focuses on multimodal approaches aimed at: (i) interpreting information for better understanding of human social signals and implicit intentions, and (ii) generating socially adept behaviour of embodied conversational agents. It will consider how we can model, represent, and employ human social signals and behaviours to design autonomous systems able to know, either through their design or via a process of learning, how to understand and respond to human communicative signals and behaviour.
What are Social Signals?
A social signal is a communicative or informative signal that, either directly or indirectly, provides information concerning social interactions, social emotions, social attitudes or social relations. Social signals are manifested through a multiplicity of non-verbal behavioural cues including facial expressions, body postures, gestures, vocal outbursts, etc.
- See Section 2 of the SSPNet Research Agenda
- Watch the presentation by Isabella Poggi at the Workshop on Foundations of Social Signals
- Watch the presentation by Marc Mehu at the Workshop Foundations of Social Signals
- Watch the presentation by Roddy Cowie at the First International Workshop on Social Signal Processing
What is Social Signal Processing?
Social Signal Processing lies at the intersection of three main domains:
- Conceptual Modelling. The interactions and social relations that people have with each other are generally governed by principles and laws. The SSPNet investigates these laws, makes them explicit, and studies how these laws are expressed and influenced by social signals
- Automatic Analysis. Social interactions can be sensed with a wide array of devices (cameras, microphones, proximity detectors, smartphones, etc.). The SSPNet investigates automatic approaches aimed at understanding social signals captured with sensors
- Synthesis. Artificial agents display a wide spectrum of artificially generated nonverbal behavioural cues. The SSPNet investigates how these synthetic cues can be made capable of conveying social signals eliciting desired social perceptions
As a result, Social Signal Processing addresses three main research questions:
- Is it possible to identify, describe and conceptualize social signalling patterns that are stable at least for a given context and culture?
- Is it possible to automatically detect and understand nonverbal behavioural cues conveying social signals captured with sensors like microphones and cameras?
- Is it possible to synthesize nonverbal behavioural cues conveying desired social signals for embodiment of social behaviours in artificial agents, robots or other manufacts?
For more details about question 1 see the following:
- Section 2 of the SSPNet Research Agenda
- P. M. Brunet, H. Donnan, G. McKeown, E. Douglas-Cowie, and R. Cowie, “Social signal processing: What are the relevant variables? And in what ways do they relate?”, in Proceedings of the IEEE International Workshop on Social Signal Processing, 2009, pp. 1-6.
- I. Poggi and F. D’Errico, “Social signals and the action – Cognition loop. The case of overhelp and evaluation,” in Proceedings of IEEE International Conference on Affective Computing and Intelligent Interaction, 2009, pp. 1-8.
For more details about question 2, see the following:
- Section 3 of the SSPNet Research Agenda
- Explore the Virtual Learning Centre of the SSPNet
- A.Vinciarelli, M.Pantic and H.Bourlard, “Social Signal Processing: Survey of an Emerging Domain”, Image and Vision Computing, 27(12):1743-1759 (2009).
For more details about question 3, see the following
- Section 4 of the SSPNet Research Agenda
- Explore the Virtual Learning Centre of the SSPNet
- K. J. Heylen, M. Theune, H. J. A. op den Akker, and A. Nijholt, “Social Agents: the first generations”, in Proceedings of the International Workshop on Social Signals Processing, 2009, pp. 1-7.
Partners and Key Personnel
- University of Glasgow, United Kingdom
- Alessandro Vinciarelli, Coordinator
- IDIAP Research Institute, Switzerland
- Fabio Valente and Hervé Bourlard
- Imperial College, United Kingdom
- Maja Pantic, Scientific Coordinator, Konstantinos Bousmalis, Michel Valstar
- University of Edinburgh, United Kingdom
- Steve Renals; Jean Carletta; Johanna Moore; Simon King; Hiroshi Shimodaira
- University of Twente, the Netherlands
- Rieks op den Akker; Dirk Heylen; Anton Nijholt; Mannes Poel
- Università di Roma Tre, Italy
- Isabella Poggi; Raffaele Pozzi
- Queen’s University Belfast, United Kingdom
- Roddy Cowie; Ellen Douglas-Cowie
- DFKI, Germany
- Marc Schröder; Oytun Türk; Marcela Charfuelan; Sathish Chandra Pammi
- CNRS/TELECOM ParisTech, France
- Catherine Pelachaud; Ken Prepin
- Universite de Genève, Switzerland
- Klaus R. Scherer; Marc Mehu
- Delft University of Technology
- Emile Hendriks; Robert P.W. Duin; Marco Loog; David Tax; Laurens van der Maaten