Preface xix
Acknowledgments xxvii
Contributors xxix
1 Introduction to Emotion Recognition 1
Amit Konar, Anisha Halder, and Aruna Chakraborty
1.1 Basics of Pattern Recognition, 1
1.2 Emotion Detection as a Pattern Recognition Problem, 2
1.3 Feature Extraction, 3
1.4 Feature Reduction Techniques, 15
1.5 Emotion Classification, 17
1.6 Multimodal Emotion Recognition, 24
1.7 Stimulus Generation for Emotion Arousal, 24
1.8 Validation Techniques, 26
1.9 Summary, 27
References, 28
Author Biographies, 44
2 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial Action Recognition 47
Yan Tong and Qiang Ji
2.1 Introduction, 48
2.2 Related Work, 49
2.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN, 50
2.4 Experimental Results, 60
2.5 Conclusion, 64
References, 64
Author Biographies, 66
3 Facial Expressions: A Cross-Cultural Study 69
Chandrani Saha, Washef Ahmed, Soma Mitra, Debasis Mazumdar, and Sushmita Mitra
3.1 Introduction, 69
3.2 Extraction of Facial Regions and Ekman's Action Units, 71
3.3 Cultural Variation in Occurrence of Different AUs, 76
3.4 Classification Performance Considering Cultural Variability, 79
3.5 Conclusion, 84
References, 84
Author Biographies, 86
4 A Subject-Dependent Facial Expression Recognition System 89
Chuan-Yu Chang and Yan-Chiang Huang
4.1 Introduction, 89
4.2 Proposed Method, 91
4.3 Experiment Result, 103
4.4 Conclusion, 109
Acknowledgment, 110
References, 110
Author Biographies, 112
5 Facial Expression Recognition Using Independent Component Features and Hidden Markov Model 113
Md. Zia Uddin and Tae-Seong Kim
5.1 Introduction, 114
5.2 Methodology, 115
5.3 Experimental Results, 123
5.4 Conclusion, 125
Acknowledgments, 125
References, 126
Author Biographies, 127
6 Feature Selection for Facial Expression Based on Rough Set Theory 129
Yong Yang and Guoyin Wang
6.1 Introduction, 129
6.2 Feature Selection for Emotion Recognition Based on Rough Set Theory, 131
6.3 Experiment Results and Discussion, 137
6.4 Conclusion, 143
Acknowledgments, 143
References, 143
Author Biographies, 145
7 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets 147
Anisha Halder, Amit Konar, Aruna Chakraborty, and Atulya K. Nagar
7.1 Introduction, 148
7.2 Preliminaries on Type-2 Fuzzy Sets, 150
7.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition, 152
7.4 Fuzzy Type-2 Membership Evaluation, 157
7.5 Experimental Details, 161
7.6 Performance Analysis, 167
7.7 Conclusion, 175
References, 176
Author Biographies, 180
8 Emotion Recognition from Non-frontal Facial Images 183
Wenming Zheng, Hao Tang, and Thomas S. Huang
8.1 Introduction, 184
8.2 A Brief Review of Automatic Emotional Expression Recognition, 187
8.3 Databases for Non-frontal Facial Emotion Recognition, 191
8.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images, 196
8.5 Discussions and Conclusions, 205
Acknowledgments, 206
References, 206
Author Biographies, 211
9 Maximum a Posteriori Based Fusion Method for Speech Emotion Recognition 215
Ling Cen, Zhu Liang Yu, and Wee Ser
9.1 Introduction, 216
9.2 Acoustic Feature Extraction for Emotion Recognition, 219
9.3 Proposed Map-Based Fusion Method, 223
9.4 Experiment, 229
9.5 Conclusion, 232
References, 232
Author Biographies, 234
10 Emotion Recognition in Naturalistic Speech and Language-A Survey 237
Felix Weninger, Martin W¨ollmer, and Bjorn Schuller
10.1 Introduction, 238
10.2 Tasks and Applications, 239
10.3 Implementation and Evaluation, 244
10.4 Challenges, 253
10.5 Conclusion and Outlook, 257
Acknowledgment, 259
References, 259
Author Biographies, 267
11 EEG-Based Emotion Recognition Using Advanced Signal Processing Techniques 269
Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis
11.1 Introduction, 270
11.2 Brain Activity and Emotions, 271
11.3 EEG-ER Systems: An Overview, 272
11.4 Emotion Elicitation, 273
11.5 Advanced Signal Processing in EEG-ER, 275
11.6 Concluding Remarks and Future Directions, 287
References, 289
Author Biographies, 292
12 Frequency Band Localization on Multiple Physiological Signals for Human Emotion Classification Using DWT 295
M. Murugappan
12.1 Introduction, 296
12.2 Related Work, 297
12.3 Research Methodology, 299
12.4 Experimental Results and Discussions, 306
12.5 Conclusion, 310
12.6 Future Work, 310
Acknowledgments, 310
References, 310
Author Biography, 312
13 Toward Affective Brain-Computer Interface: Fundamentals and Analysis of EEG-Based Emotion Classification 315
Yuan-Pin Lin, Tzyy-Ping Jung, Yijun Wang, and Julie Onton
13.1 Introduction, 316
13.2 Materials and Methods, 323
13.3 Results and Discussion, 327
13.4 Conclusion, 332
13.5 Issues and Challenges Toward ABCIs, 332
Acknowledgments, 336
References, 336
Author Biographies, 340
14 Bodily Expression for Automatic Affect Recognition 343
Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian
14.1 Introduction, 344
14.2 Background and Related Work, 345
14.3 Creating a Database of Facial and Bodily Expressions: The FABO Database, 353
14.4 Automatic Recognition of Affect from Bodily Expressions, 356
14.5 Automatic Recognition of Bodily Expression Temporal Dynamics, 361
14.6 Discussion and Outlook, 367
14.7 Conclusions, 369
Acknowledgments, 370
References, 370
Author Biographies, 375
15 Building a Robust System for Multimodal Emotion Recognition 379
Johannes Wagner, Florian Lingenfelser, and Elisabeth Andre
15.1 Introduction, 380
15.2 Related Work, 381
15.3 The Callas Expressivity Corpus, 382
15.4 Methodology, 386
15.5 Multisensor Data Fusion, 390
15.6 Experiments, 395
15.7 Online Recognition System, 399
15.8 Conclusion, 403
Acknowledgment, 404
References, 404
Author Biographies, 410
16 Semantic Audiovisual Data Fusion for Automatic Emotion Recognition 411
Dragos Datcu and Leon J. M. Rothkrantz
16.1 Introduction, 412
16.2 Related Work, 413
16.3 Data Set Preparation, 416
16.4 Architecture, 418
16.5 Results, 431
16.6 Conclusion, 432
References, 432
Author Biographies, 434
17 A Multilevel Fusion Approach for Audiovisual Emotion Recognition 437
Girija Chetty, Michael Wagner, and Roland Goecke
17.1 Introduction, 437
17.2 Motivation and Background, 438
17.3 Facial Expression Quantification, 440
17.4 Experiment Design, 444
17.5 Experimental Results and Discussion, 450
17.6 Conclusion, 456
References, 456
Author Biographies, 459
18 From a Discrete Perspective of Emotions to Continuous, Dynamic, and Multimodal Affect Sensing 461
Isabelle Hupont, Sergio Ballano, Eva Cerezo, and Sandra Baldassarri
18.1 Introduction, 462
18.2 A Novel Method for Discrete Emotional Classification of Facial Images, 465
18.3 A 2D Emotional Space for Continuous and Dynamic Facial Affect Sensing, 469
18.4 Expansion to Multimodal Affect Sensing, 474
18.5 Building Tools That Care, 479
18.6 Concluding Remarks and Future Work, 486
Acknowledgments, 488
References, 488
Author Biographies, 491
19 Audiovisual Emotion Recognition Using Semi-Coupled Hidden Markov Model with State-Based Alignment Strategy 493
Chung-Hsien Wu, Jen-Chun Lin, and Wen-Li Wei
19.1 Introduction, 494
19.2 Feature Extraction, 495
19.3 Semi-Coupled Hidden Markov Model, 500
19.4 Experiments, 504
19.5 Conclusion, 508
References, 509
Author Biographies, 512
20 Emotion Recognition in Car Industry 515
Christos D. Katsis, George Rigas, Yorgos Goletsis, and Dimitrios I. Fotiadis
20.1 Introduction, 516
20.2 An Overview of Application for the Car Industry, 517
20.3 Modality-Based Categorization, 517
20.4 Emotion-Based Categorization, 520
20.5 Two Exemplar Cases, 523
20.6 Open Issues and Future Steps, 536
20.7 Conclusion, 537
References, 537
Author Biographies, 543
Index 545
