Comunicazione non verbale: computazione affettiva

Transcript

1 Comunicazione non verbale: computazione affettiva Corso di Interazione uomo-macchina II Prof. Giuseppe Boccignone Dipartimento di Scienze dell Informazione Università di Milano Ipotesi di lavoro: interazione fra organismi riflessione ambiente riflessione azione percezione emozioni motivazioni azione percezione emozioni motivazioni metabolismo metabolismo verbale non verbale

2 Riconoscere emozioni Riconoscere emozioni AUTONOMIA / LIVELLI DI CONTROLLO CONTROLLO RIFLESSIVO / COGNITIVO riflessione ambiente Goals esterni CONTROLLO EMOTIVO azione percezione PROCESSI INTERATTIVI emozioni motivazioni CONTROLLO VISCERALE REGOLAZIONE OMEOSTATICA metabolismo Goals interni confine del corpo situato nell ambiente PROCESSI COSTITUTIVI

3 Emotional expression A. Vinciarelli, M. Pantic, H. Bourlard, Social Signal Processing: Survey of an Emerging Domain, Image and Vision Computing (2008) Emotional expression A. Vinciarelli, M. Pantic, H. Bourlard, Social Signal Processing: Survey of an Emerging Domain, Image and Vision Computing (2008)

4 Emotional expression Riconoscere emozioni da espressioni facciali Approcci possibili: Discrete (Ekman) vs. Dimensional (Russell) AU-based (Ekman) vs. holistic Paradigma dominante

5 Riconoscere emozioni da espressioni facciali Approcci possibili: Discrete (Ekman) vs. Dimensional (Russell) AU-based (Ekman) vs. holistic Paradigma dominante Approcci robusti Riconoscere emozioni da espressioni facciali Approcci possibili: Discrete (Ekman) vs. Dimensional (Russell) AU-based (Ekman) vs. holistic Meno consueti Usati soprattutto nella simulazione virtuali Analisi di emozioni in musica o nel parlato

6 Riconoscere emozioni da espressioni facciali // Riconoscere emozioni da espressioni facciali

7 Expression vs identity //A.J. Calder et al. : Vision Research 41 (2001) Expression vs identity //A.J. Calder et al. : Vision Research 41 (2001)

8 Expression vs identity //A.J. Calder et al. : Vision Research 41 (2001) Expression vs identity //A.J. Calder et al. : Vision Research 41 (2001)

9 Expression vs identity //A.J. Calder et al. : Vision Research 41 (2001) Schema generale di un sistema per AU detection

10 Riconoscere emozioni da espressioni facciali //Bartlett et al. Riconoscere emozioni da espressioni facciali //Bartlett et al.

11 Riconoscere emozioni da espressioni facciali //Bartlett et al. Unified Probabilistic Framework for Spontaneous Facial Action Modeling Tong et al (2010)

12 Unified Probabilistic Framework for Spontaneous Facial Action Modeling Tong et al (2010) Mind-Reading Machines: Automated Inference of Complex Mental States by Rana Ayman el Kaliouby

13 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

14 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

15 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

16 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

17 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

18 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

19 Mind-Reading Machines: Automated Inference of Complex Mental States Mind-Reading Machines: Automated Inference of Complex Mental States

20 Mind-Reading Machines: Automated Inference of Complex Mental States Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji

21 Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji

22 Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji

23 Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji

24 Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji

25 Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji Active and Dynamic Information Fusion for Facial Expression Understanding Zhang & Ji

26 Dimensional approach via Gabor wavelets Lyons et al. Dimensional approach via Gabor wavelets Lyons et al. Gabor and human similarity data was analyzed usingnon-metric multidimensional scaling (nmds) using thealscal algorithm [13]. nmds embeds points in a euclidean space in such a way that the distances between points preserves the rank order of the dissimilarity values betweenthose points. it was found that two dimensions provide an adequate embedding of the similarity data

27 Dimensional approach via Gabor wavelets Lyons et al. Gabor and human similarity data was analyzed usingnon-metric multidimensional scaling (nmds) using thealscal algorithm [13]. nmds embeds points in a euclidean space in such a way that the distances between points preserves the rank order of the dissimilarity values betweenthose points. it was found that two dimensions provide an adequate embedding of the similarity data Dimensional approach via Gabor wavelets

28 Emotion by dynamical simulation Essa & Pentland (1997): We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with geometric, physical and motion-based dynamic models describing the facial structure. Our method produces a reliable parametric representation of the face s independent muscle action groups, as well as an accurate estimate of facial motion. Emotion by dynamical simulation Essa & Pentland (1997):

29 Emotion by dynamical simulation Essa & Pentland (1997): Emotion by dynamical simulation Essa & Pentland (1997):

30 Emotion by dynamical simulation Essa & Pentland (1997): Emotion by dynamical simulation Essa & Pentland (1997):

31 Reconstruction of Facial Expressions in Embodied Systems Karl Grammer & Elisabeth Oberzaucher Reconstruction of Facial Expressions in Embodied Systems Karl Grammer & Elisabeth Oberzaucher

32 Reconstruction of Facial Expressions in Embodied Systems Karl Grammer & Elisabeth Oberzaucher Reconstruction of Facial Expressions in Embodied Systems Karl Grammer & Elisabeth Oberzaucher The activation of single AUs in a pleasure and arousal space. Note that these distributions are different for different Action Units.

33 Reconstruction of Facial Expressions in Embodied Systems Karl Grammer & Elisabeth Oberzaucher Reconstruction of Facial Expressions in Embodied Systems Karl Grammer & Elisabeth Oberzaucher

34 Sociable robots Kismet Sociable robots Kismet

35 Sociable robots Kismet Sociable robots Kismet

36 Sociable robots Kismet Sociable robots Kismet

37 Sociable robots Kismet