Bayerl P. S., Tammewar A., Riedhammer K. and Riccardi G. Detecting Emotion Carriers By Combining Acoustic and Lexical Representations (Conference) IEEE Automatic Speech Recognition and Understanding Conference, 2021. (BibTeX | Tags: Affective Computing, Machine Learning) Torres M. J., Ravanelli M., Medina-Devilliers S., Lerner D. M. and Riccardi G. Interpretable SincNet-based Deep Learning for Emotion Recognition in Individuals with Autism (Conference) IEEE Conf. Engineering in Medicine and Biology, Conference, 2021. (Links | BibTeX | Tags: Affective Computing, Autism, Machine Learning, Signal Annotation and Interpretation) Tammewar A., Cervone A. and Riccardi G. Emotion Carrier Recognition from Personal Narratives (Conference) INTERSPEECH, 2021. (Links | BibTeX | Tags: Affective Computing) Torres M. J., Clarkson T., Hauschild K., Luhmann C. C., Lerner D. M. and Riccardi G. Facial emotions are accurately encoded in the brains of those with autism: A deep learning approach (Article) Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 2021. (Links | BibTeX | Tags: Affective Computing, Autism, Machine Learning, Signal Annotation and Interpretation) Tammewar A., Cervone A. and Riccardi G. Emotion Carrier Recognition from Personal Narratives (Article) 2020. (Links | BibTeX | Tags: Affective Computing, Natural Language Processing) Tammewar A., Cervone A.,Eva-Maria Messner, Riccardi G. Annotation of Emotion Carriers in Personal Narratives (Proceeding) 2020. (Links | BibTeX | Tags: Affective Computing, Natural Language Processing, Signal Annotation and Interpretation) Tammewar A., Cervone A., Messner E., Riccardi G. Modeling user context for valence prediction from narratives (Conference) 2019. (Links | BibTeX | Tags: Affective Computing, Natural Language Processing) Mayor Torres, J.M., Clarkson, T., Luhmann, C. C., Riccardi, G., Lerner, M.D. 2019. (Links | BibTeX | Tags: Affective Computing, Autism, Machine Learning) Alam F., Danieli M. and Riccardi G. Annotating and Modeling Empathy in Spoken Conversations (Article) Computer Speech and Language, 50 pp. 40-61, 2018. (Links | BibTeX | Tags: Affective Computing, Discourse, Signal Annotation and Interpretation) Mayor Torres, J.M., Clarkson, T., Stepanov E. A. , Luhmann C. C., Lerner, M.D., Riccardi, G. Enhanced Error Decoding from Error-Related Potentials using Convolutional Neural Networks (Conference) 2018. (Links | BibTeX | Tags: Affective Computing, Autism, Machine Learning) Mayor Torres, J.M., Libsack, E.J., Clarkson, T., Keifer, C.M., Riccardi, G., Lerner, M.D. 2018. (Links | BibTeX | Tags: Affective Computing, Autism) Cervone A., Gambi E., Tortoreto G., Stepanov E. A., and Riccardi G. Automatically Predicting User Ratings for Conversational Systems (Conference) 2018. (Links | BibTeX | Tags: Affective Computing, Conversational and Interactive Systems , Speech Analytics) Stepanov E. A., Lathuiliere S., Chowdhury S. A., Ghosh A., Vieriu R., Sebe N.and Riccardi G. Depression Severity Estimation from Multiple Modalities (Conference) 2018. (Links | BibTeX | Tags: Affective Computing, Health Analytics) Mayor Torres Juan M., Stepanov A. E. WI '17 Proceedings of the International Conference on Web Intelligence Pages 939-946, Leipzig, Germany - August 23 - 26, 2017, 2017. (Abstract | Links | BibTeX | Tags: Affective Computing, Interactive Systems, Signal Annotation and Interpretation) Cervone A., Stepanov E. A., Celli F., and Riccardi G. Irony Detection: from the Twittersphere to the News Space (Conference) 2017. (Abstract | Links | BibTeX | Tags: Affective Computing, Natural Language Processing) Chowdhury A. S., Stepanov E. A., Danieli M.. and Riccardi G. Functions of Silences towards Information Flow in Spoken Conversation (Conference) EMNLP 2017 Workshop on Speech-Centric Natural Language Processing, Copenhagen, 2017, 2017. (Abstract | Links | BibTeX | Tags: Affective Computing, Speech Processing) Tortoreto G., Ghosh A., Stepanov E. A., Danieli M.. and Riccardi G. Affective Behaviour Analysis of User Interactions in Support Web Group (Presentation) 01.01.2017. (Links | BibTeX | Tags: Affective Computing) Chowdhury A. S. and Riccardi G. A Deep Learning Approach To Modeling Competitiveness In Spoken Conversations (Conference) Proc. ICASSP, New Orleans, 2017, 2017. (Abstract | Links | BibTeX | Tags: Affective Computing, Speech Processing) Alam F. , Chowdhury S. , Danieli M. and Riccardi G. How Interlocutors Coordinate with each other within Emotional Segments? (Proceeding) Proc. COLING, Osaka, 2016., 2016. (Abstract | Links | BibTeX | Tags: Affective Computing, Conversational and Interactive Systems , Discourse, Interactive Systems) Chowdhury S. , Stepanov A. E. and Riccardi G. Predicting User Satisfaction from Turn-Taking in Spoken Conversations (Proceeding) Proc. INTERSPEECH, San Francisco, 2016., 2016. (Abstract | Links | BibTeX | Tags: Affective Computing, Conversational and Interactive Systems , Discourse, Interactive Systems, Signal Annotation and Interpretation, Speech Processing) Alam F., Celli F., Stepanov A. E., Ghosh A. and Riccardi G. The Social Mood of News: Self-reported Annotations to Design Automatic Mood Detection Systems (Proceeding) Proc. PEOPLES Workshop at COLING, Osaka 2016, 2016. (Abstract | Links | BibTeX | Tags: Affective Computing, Conversational and Interactive Systems ) Celli F., Stepanov A. E., Poesio M. and Riccardi G. Predicting Brexit: Classifying Agreement is Better than Sentiment and Pollsters (Proceeding) Proc. PEOPLES Workshop at COLING, Osaka 2016., 2016. (Abstract | Links | BibTeX | Tags: Affective Computing, Machine Learning, Signal Annotation and Interpretation) Danieli M. , Riccardi G. and Alam F. Emotion Unfolding and Affective Scenes: A Case Study in Spoken Conversations (Conference) 2015. (Abstract | Links | BibTeX | Tags: Affective Computing, Speech Processing) Danieli M., Ghosh A, Berra E., Testa E., Rabbia F., Veglio F. and Riccardi G. Comprendere l’Ipertensione Arteriosa Essenziale A Partire da Costrutti Psicologici e Segnali Fisiologici (Conference) 2015. (Links | BibTeX | Tags: Affective Computing, Health Analytics) Celli F. and Riccardi G. and Ghosh A. CorEA: Italian News Corpus with Emotions and Agreement (Conference) 2014. (Abstract | Links | BibTeX | Tags: Affective Computing, Natural Language Processing, Signal Annotation and Interpretation) Danieli M. , Riccardi G. and Alam F. Annotation of Complex Emotions in Real-Life Dialogues: The Case of Empathy (Conference) 2014. (Abstract | Links | BibTeX | Tags: Affective Computing, Signal Annotation and Interpretation) Alam F. and Riccardi G. Fusion of Acoustic, Linguistic and Psycholinguistic Features for Speaker Personality Traits Recognition (Conference) 2014. (Abstract | Links | BibTeX | Tags: Affective Computing) Alam F. and Riccardi G. Predicting Personality Traits using Multimodal Information (Conference) 2014. (Abstract | Links | BibTeX | Tags: Affective Computing) Alam F. and Riccardi G. Comparative Study of Speaker Personality Traits Recognition in Conversational and Broadcast News Speech (Conference) 2013. (Abstract | Links | BibTeX | Tags: Affective Computing, Speech Processing) Riccardi G., Ghosh A., Chowdhury S. A. and Bayer A. O. Motivational Feedback in Crowdsourcing: a Case Study in Speech Transcriptions (Conference) 2013. (Abstract | Links | BibTeX | Tags: Affective Computing, Conversational and Interactive Systems , Machine Learning, Signal Annotation and Interpretation) Alam F. ,Stepanov E. and Riccardi G. Personality Traits Recognition on Social Network - Facebook (Conference) 2013. (Abstract | Links | BibTeX | Tags: Affective Computing, Natural Language Processing) Ivanov A. V. and Riccardi G. Kolmogorov-Smirnov Test for Feature Selection in Emotion Recognition From Speech (Conference) 2012. (Abstract | Links | BibTeX | Tags: Affective Computing) Ivanov A. V., Riccardi G., Sporka A. J. and Franc J. Recognition of Personality Traits from Human Spoken Conversations (Conference) 2011. (Abstract | Links | BibTeX | Tags: Affective Computing) Riccardi G. and Hakkani-Tur D. Grounding Emotions in Human-Machine Conversational Systems (Article) Lecture Notes in Computer Science, Springer-Verlag, , pp. 144 – 154, 2005, 2005. (Abstract | Links | BibTeX | Tags: Affective Computing, Conversational and Interactive Systems ) Liscombe J., Riccardi G. and Hakkani-Tur D. Using Context to Improve Emotion Detection in Spoken Dialog Systems (Conference) 2005. (Abstract | Links | BibTeX | Tags: Affective Computing, Speech Processing)2021
title = {Detecting Emotion Carriers By Combining Acoustic and Lexical Representations},
author = {Bayerl P. S., Tammewar A., Riedhammer K. and Riccardi G.},
year = {2021},
date = {2021-10-01},
publisher = {IEEE Automatic Speech Recognition and Understanding Conference},
keywords = {Affective Computing, Machine Learning}
}
title = {Interpretable SincNet-based Deep Learning for Emotion Recognition in Individuals with Autism},
author = {Torres M. J., Ravanelli M., Medina-Devilliers S., Lerner D. M. and Riccardi G.},
url = {https://arxiv.org/pdf/2107.10790.pdf},
year = {2021},
date = {2021-07-18},
publisher = {IEEE Conf. Engineering in Medicine and Biology, Conference},
keywords = {Affective Computing, Autism, Machine Learning, Signal Annotation and Interpretation}
}
title = {Emotion Carrier Recognition from Personal Narratives},
author = {Tammewar A., Cervone A. and Riccardi G.},
url = {https://arxiv.org/abs/2008.07481},
year = {2021},
date = {2021-06-24},
publisher = {INTERSPEECH},
keywords = {Affective Computing}
}
title = {Facial emotions are accurately encoded in the brains of those with autism: A deep learning approach},
author = {Torres M. J., Clarkson T., Hauschild K., Luhmann C. C., Lerner D. M. and Riccardi G.},
url = {https://www.sciencedirect.com/science/article/abs/pii/S2451902221001075?via%3Dihub},
year = {2021},
date = {2021-04-16},
journal = {Biological Psychiatry: Cognitive Neuroscience and Neuroimaging},
keywords = {Affective Computing, Autism, Machine Learning, Signal Annotation and Interpretation}
}
2020
title = {Emotion Carrier Recognition from Personal Narratives },
author = {Tammewar A., Cervone A. and Riccardi G.},
editor = {arXiv.org, 2020},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2020/12/2008.07481.pdf},
year = {2020},
date = {2020-08-17},
keywords = {Affective Computing, Natural Language Processing}
}
title = {Annotation of Emotion Carriers in Personal Narratives},
author = {Tammewar A., Cervone A.,Eva-Maria Messner, Riccardi G.},
editor = {Proc. Language Resources and Evaluation Conference , Marseille*, 2020},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2020/12/LREC20-EmotionCarriers.pdf},
year = {2020},
date = {2020-05-11},
keywords = {Affective Computing, Natural Language Processing, Signal Annotation and Interpretation}
}
2019
title = {Modeling user context for valence prediction from narratives},
author = {Tammewar A., Cervone A., Messner E., Riccardi G.},
editor = {INTERSPEECH, Graz},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2019/11/IS19-ValencePredictionNarratives.pdf},
year = {2019},
date = {2019-01-01},
keywords = {Affective Computing, Natural Language Processing}
}
title = {Distinct but Effective Neural Networks for Facial Emotion Recognition in Individuals with Autism: A Deep Learning Approach},
author = {Mayor Torres, J.M., Clarkson, T., Luhmann, C. C., Riccardi, G., Lerner, M.D.},
url = {https://disi.unitn.it/~riccardi/papers2/INSAR_JMM_2019_Deep_Learning.pdf},
year = {2019},
date = {2019-01-01},
keywords = {Affective Computing, Autism, Machine Learning}
}
2018
title = {Annotating and Modeling Empathy in Spoken Conversations},
author = {Alam F., Danieli M. and Riccardi G.},
url = {https://www.sciencedirect.com/science/article/pii/S088523081730133X},
year = {2018},
date = {2018-07-01},
journal = {Computer Speech and Language},
volume = {50},
pages = {40-61},
keywords = {Affective Computing, Discourse, Signal Annotation and Interpretation}
}
title = {Enhanced Error Decoding from Error-Related Potentials using Convolutional Neural Networks},
author = {Mayor Torres, J.M., Clarkson, T., Stepanov E. A. , Luhmann C. C., Lerner, M.D., Riccardi, G.},
editor = {IEEE Conf. Engineering in Biology and Medicine Society, Honolulu},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2019/11/EMBC18-Enhanced-Error-Decoding.pdf},
year = {2018},
date = {2018-01-01},
keywords = {Affective Computing, Autism, Machine Learning}
}
title = {EEG-based Single trial Classification Emotion Recognition: A Comparative Analysis in Individuals with and without Autism Spectrum Disorder},
author = {Mayor Torres, J.M., Libsack, E.J., Clarkson, T., Keifer, C.M., Riccardi, G., Lerner, M.D.},
editor = {Annual Meeting of the International Society for Autism Research, Rotterdam},
url = {https://insar.confex.com/imfar/2018/webprogram/Paper27651.html},
year = {2018},
date = {2018-01-01},
keywords = {Affective Computing, Autism}
}
title = {Automatically Predicting User Ratings for Conversational Systems},
author = {Cervone A., Gambi E., Tortoreto G., Stepanov E. A., and Riccardi G.},
editor = {Fifth Italian Conference on Computational Linguistics , Turin},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2019/11/Clicit18-PredictingUserRatings.pdf},
year = {2018},
date = {2018-01-01},
keywords = {Affective Computing, Conversational and Interactive Systems , Speech Analytics}
}
title = {Depression Severity Estimation from Multiple Modalities},
author = {Stepanov E. A., Lathuiliere S., Chowdhury S. A., Ghosh A., Vieriu R., Sebe N.and Riccardi G.},
editor = {AVEC Challenge},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2019/11/1711.060951.pdf},
year = {2018},
date = {2018-01-01},
keywords = {Affective Computing, Health Analytics}
}
2017
title = {Enhanced face/audio emotion recognition: video and instance level classification using ConvNets and restricted Boltzmann Machines},
author = {Mayor Torres Juan M., Stepanov A. E.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2017/09/ACMWI2017MayorStepanov.pdf
https://dl.acm.org/citation.cfm?id=3109423},
year = {2017},
date = {2017-08-23},
publisher = {WI '17 Proceedings of the International Conference on Web Intelligence Pages 939-946, Leipzig, Germany - August 23 - 26, 2017},
abstract = {Face-based and audio-based emotion recognition modalities have been studied profusely obtaining successful classification rates for arousal/valence levels and multiple emotion categories settings. However, recent studies only focus their attention on classifying discrete emotion categories with a single image representation and/or a single set of audio feature descriptors. Face-based emotion recognition systems use a single image channel representations such as principal-components-analysis whitening, isotropic smoothing, or ZCA whitening. Similarly, audio emotion recognition systems use a standardized set of audio descriptors, including only averaged Mel-Frequency Cepstral coefficients. Both approaches imply the inclusion of decision-fusion modalities to compensate the limited feature separability and achieve high classification rates. In this paper, we propose two new methodologies for enhancing face-based and audio-based emotion recognition based on a single classifier decision and using the EU Emotion Stimulus dataset: (1) A combination of a Convolutional Neural Networks for frame-level feature extraction with a k-Nearest Neighbors classifier for the subsequent frame-level aggregation and video-level classification, and (2) a shallow Restricted Boltzmann Machine network for arousal/valence classification.},
keywords = {Affective Computing, Interactive Systems, Signal Annotation and Interpretation}
}
title = {Irony Detection: from the Twittersphere to the News Space},
author = {Cervone A., Stepanov E. A., Celli F., and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2017/10/Clicit17-IronyDetection.pdf},
year = {2017},
date = {2017-01-01},
journal = {Proc. Fourth Conference on Computational Linguistics , Rome, 2017},
abstract = {Automatic detection of irony is one of the hot topics for sentiment analysis, as it changes the polarity of text. Most of the work has been focused on the detection of figurative language in Twitter data due to relative ease of obtaining annotated data, thanks to the use of hashtags to signal irony. However, irony is present generally in natural language conversations and in particular in online public fora. In this paper, we present a comparative evaluation of irony detection from Italian news fora and Twitter posts. Since irony is not a very frequent phenomenon, its automatic detection suffers from data imbalance and feature sparseness problems. We experiment with different representations of text – bag-of-words, writing style, and word embeddings to address the feature sparseness; and balancing techniques to address the data imbalance.},
keywords = {Affective Computing, Natural Language Processing}
}
title = {Functions of Silences towards Information Flow in Spoken Conversation},
author = {Chowdhury A. S., Stepanov E. A., Danieli M.. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2017/10/2017_SCNLP_Chowdhury_etal.pdf},
year = {2017},
date = {2017-01-01},
publisher = {EMNLP 2017 Workshop on Speech-Centric Natural Language Processing, Copenhagen, 2017},
abstract = {Silence is an integral part of the most frequent turn-taking phenomena in spoken conversations. Silence is sized and placed within the conversation flow and it is coordinated by the speakers along with the other speech acts. The objective of this analytical study is twofold: to explore the functions of silence with duration of one second and above, towards information flow in a dyadic conversation utilizing the sequences of dialog acts present in the turns surrounding the silence itself; and to design a feature space useful for clustering the silences using a hierarchical concept formation algorithm. The resulting clusters are manually grouped into functional categories based on their similarities. It is observed that the silence plays an important role in response preparation, also can indicate speakers’ hesitation or indecisiveness. It is also observed that sometimes long silences can be used deliberately to get a forced response from another speaker thus making silence a multi-functional and an important catalyst towards information flow.
},
keywords = {Affective Computing, Speech Processing}
}
title = {Affective Behaviour Analysis of User Interactions in Support Web Group},
author = {Tortoreto G., Ghosh A., Stepanov E. A., Danieli M.. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2017/10/2017_ECP_Tortoreto_etal_poster.pdf},
year = {2017},
date = {2017-01-01},
journal = {Proc. European Congress of Psychology, Amsterdam , 2017},
keywords = {Affective Computing},
tppubtype = {presentation}
}
title = {A Deep Learning Approach To Modeling Competitiveness In Spoken Conversations},
author = {Chowdhury A. S. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2017/10/2017_ICASSP_Chowdhury_Riccardi.pdf},
year = {2017},
date = {2017-01-01},
publisher = {Proc. ICASSP, New Orleans, 2017},
abstract = {The motivation behind the research on overlapping speech has always been dominated by the need to model humanmachine interaction for dialog systems and conversation analysis. To have more complex insights of the interlocutors’ intentions behind the interaction, we need to understand the type of overlaps. Overlapping speech signals the interlocutor’s intention to grab the floor. This act could be a competitive or non-competitive act, which either signals a problem or indicates assistance in communication. In this paper, we present a Deep Learning approach to modeling competitiveness in overlapping speech using acoustic and lexical features and their combination. We compare a fully-connected feed-forward neural network to the Support Vector Machine (SVM) models on real call center human-human conversations. We have observed that feature combination with DNN (significantly) outperforms SVM models, both the individual feature baselines and the feature combination model by 4% and 2% respectively.},
keywords = {Affective Computing, Speech Processing}
}
2016
title = {How Interlocutors Coordinate with each other within Emotional Segments?},
author = {Alam F. , Chowdhury S. , Danieli M. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2016/11/Coling16-CoordinationEmotionalSegments.pdf},
year = {2016},
date = {2016-11-01},
publisher = {Proc. COLING, Osaka, 2016.},
abstract = {In this paper, we aim to investigate the coordination of interlocutors behavior in different emotional segments. Conversational coordination between the interlocutors is the tendency of speakers to predict and adjust each other accordingly on an ongoing conversation. In order to find such a coordination, we investigated 1) lexical similarities between the speakers in each emotional segments,
2) correlation between the interlocutors using psycholinguistic features, such as linguistic styles, psychological process, personal concerns among others, and 3) relation of interlocutors turn-taking behaviors such as competitiveness. To study the degree of coordination in different emotional segments, we conducted our experiments using real dyadic conversations collected from call centers in which agent’s emotional state include empathy and customer’s emotional states include anger and frustration. Our findings suggest that the most coordination occurs between the interlocutors inside anger segments, where as, a little coordination was observed when the agent was empathic, even though an increase in the amount of non-competitive overlaps was observed. We found no significant difference between anger and frustration segment in terms of turn-taking behaviors. However, the length of pause significantly decreases in the preceding segment of anger where as it increases in the preceding segment of frustration.},
keywords = {Affective Computing, Conversational and Interactive Systems , Discourse, Interactive Systems}
}
2) correlation between the interlocutors using psycholinguistic features, such as linguistic styles, psychological process, personal concerns among others, and 3) relation of interlocutors turn-taking behaviors such as competitiveness. To study the degree of coordination in different emotional segments, we conducted our experiments using real dyadic conversations collected from call centers in which agent’s emotional state include empathy and customer’s emotional states include anger and frustration. Our findings suggest that the most coordination occurs between the interlocutors inside anger segments, where as, a little coordination was observed when the agent was empathic, even though an increase in the amount of non-competitive overlaps was observed. We found no significant difference between anger and frustration segment in terms of turn-taking behaviors. However, the length of pause significantly decreases in the preceding segment of anger where as it increases in the preceding segment of frustration.
title = {Predicting User Satisfaction from Turn-Taking in Spoken Conversations},
author = {Chowdhury S. , Stepanov A. E. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2016/11/IS16-PredictingUserSatisfactionTurnTaking.pdf},
year = {2016},
date = {2016-11-01},
publisher = {Proc. INTERSPEECH, San Francisco, 2016.},
abstract = {User satisfaction is an important aspect of the user experience while interacting with objects, systems or people. Traditionally user satisfaction is evaluated a-posteriori via spoken or written questionnaires or interviews. In automatic behavioral analysis we aim at measuring the user emotional states and its descriptions as they unfold during the interaction. In our approach, user satisfaction is modeled as the final state of a sequence of emotional states and given ternary values positive, negative, neutral. In this paper, we investigate the discriminating power of turn-taking in predicting user satisfaction in spoken conversations. Turn-taking is used for discourse organization of a conversation by means of explicit phrasing, intonation, and pausing. In this paper, we train different characterization of turn-taking, such as competitiveness of the speech overlaps. To extract turn-taking features we design a turn segmentation and labeling system that incorporates lexical and acoustic information. Given a human-human spoken dialog, our system automatically infers any of the three values of the state of the user satisfaction. We evaluate the classification system on real-life call-center human-human dialogs. The comparative performance analysis shows that the contribution of the turn-taking features outperforms both prosodic and lexical features.},
keywords = {Affective Computing, Conversational and Interactive Systems , Discourse, Interactive Systems, Signal Annotation and Interpretation, Speech Processing}
}
title = {The Social Mood of News: Self-reported Annotations to Design Automatic Mood Detection Systems},
author = {Alam F., Celli F., Stepanov A. E., Ghosh A. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2016/11/Coling16PEOPLE-MoodClassification.pdf},
year = {2016},
date = {2016-11-01},
publisher = {Proc. PEOPLES Workshop at COLING, Osaka 2016},
abstract = {In this paper, we address the issue of automatic prediction of readers’ mood from newspaper articles and comments. As online newspapers are becoming more and more similar to social media platforms, users can provide affective feedback, such as mood and emotion. We exploited the self-reported annotation of mood categories obtained from the metadata of the Italian online newspaper corriere.it to design and evaluate a system for predicting five different mood categories from news articles and comments: indignation, disappointment, worry, satisfaction and amusement. The outcome of our experiments shows that overall, bag-of-word-ngrams performs better compared to all other feature sets, however, stylometric features perform better for the mood score prediction of articles. Our study shows that such self-reported annotations can be used to design automatic systems.},
keywords = {Affective Computing, Conversational and Interactive Systems }
}
title = {Predicting Brexit: Classifying Agreement is Better than Sentiment and Pollsters},
author = {Celli F., Stepanov A. E., Poesio M. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2016/11/Coling16PEOPLE-BrexitPaper.pdf},
year = {2016},
date = {2016-11-01},
publisher = {Proc. PEOPLES Workshop at COLING, Osaka 2016.},
abstract = {On June 23rd 2016, UK held the referendum which ratified the exit from the EU. While most of the traditional pollsters failed to forecast the final vote, there were online systems that hit the result with high accuracy using opinion mining techniques and big data.
Starting one month before, we collected and monitored millions of posts about the referendum from social media conversations, and exploited Natural Language Processing techniques to predict the referendum outcome. In this paper we discuss the methods used by traditional pollsters and compare it to the predictions based on different opinion mining techniques. We find that opinion mining based on agreement/disagreement classification works better than opinion mining based on polarity classification in the forecast of the referendum outcome.},
keywords = {Affective Computing, Machine Learning, Signal Annotation and Interpretation}
}
Starting one month before, we collected and monitored millions of posts about the referendum from social media conversations, and exploited Natural Language Processing techniques to predict the referendum outcome. In this paper we discuss the methods used by traditional pollsters and compare it to the predictions based on different opinion mining techniques. We find that opinion mining based on agreement/disagreement classification works better than opinion mining based on polarity classification in the forecast of the referendum outcome.2015
title = {Emotion Unfolding and Affective Scenes: A Case Study in Spoken Conversations},
author = {Danieli M. , Riccardi G. and Alam F.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2015/11/ICMI15-ERM4CT.pdf},
year = {2015},
date = {2015-11-09},
journal = {Proc. ICMI Workshop on Representations and Modelling for Companion Systems, Seattle, 2015},
abstract = {The manifestation of human emotions evolves over time and space. Most of the work on affective computing research is limited to the association of context-free signal segments, such as utterances and images, to basic emotions. In this paper, we discuss the hypothesis that interpreting emotions
requires a conceptual description of their dynamics within the context of their manifestations. We describe the unfolding of emotions through the proposed affective scene framework.
Affective scenes are defined in terms of who first expresses the variation in their emotional state in a conversation, how this affects the other speaker’s emotional appraisal and response, and which modifications occur from the initial through the final state of the scene. This conceptual framework is applied and evaluated on real human-human conversations drawn from call centers. We show that the automatic classification of affective scenes achieves more than satisfactory results and it benefits from acoustic, lexical and psycholinguistic features of the speech and linguistics signals.},
keywords = {Affective Computing, Speech Processing}
}
requires a conceptual description of their dynamics within the context of their manifestations. We describe the unfolding of emotions through the proposed affective scene framework.
Affective scenes are defined in terms of who first expresses the variation in their emotional state in a conversation, how this affects the other speaker’s emotional appraisal and response, and which modifications occur from the initial through the final state of the scene. This conceptual framework is applied and evaluated on real human-human conversations drawn from call centers. We show that the automatic classification of affective scenes achieves more than satisfactory results and it benefits from acoustic, lexical and psycholinguistic features of the speech and linguistics signals.
title = {Comprendere l’Ipertensione Arteriosa Essenziale A Partire da Costrutti Psicologici e Segnali Fisiologici},
author = {Danieli M., Ghosh A, Berra E., Testa E., Rabbia F., Veglio F. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2015/11/SIIA-2015-ComprendereIpertensionePoster.pdf},
year = {2015},
date = {2015-09-24},
journal = {XXXI Congresso Societa’ Italiana Ipertensione Arteriosa, Bologna, 2015. (Poster)},
keywords = {Affective Computing, Health Analytics}
}
2014
title = {CorEA: Italian News Corpus with Emotions and Agreement},
author = {Celli F. and Riccardi G. and Ghosh A.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/AI-IA14-CoreaAgreementDisagreement-1.pdf},
year = {2014},
date = {2014-11-01},
journal = {Conferenza di Linguistica Computazionale, Pisa, 2014},
abstract = {In this paper, we describe an Italian corpus of news blogs, including bloggers’ emotion tags, and annotations of agreement relations amongst blogger- comment pairs. The main contributions of this work are: the formalization of the agreement relation, the design of guide- lines for its annotation, the quantitative analysis of the annotators’ agreement.},
keywords = {Affective Computing, Natural Language Processing, Signal Annotation and Interpretation}
}
title = {Annotation of Complex Emotions in Real-Life Dialogues: The Case of Empathy},
author = {Danieli M. , Riccardi G. and Alam F.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/AI-IA14-EmpathyCorpusAnnotation.pdf},
year = {2014},
date = {2014-01-01},
journal = {Conferenza di Linguistica Computazionale, Pisa, 2014},
abstract = {In this paper we discuss the problem of an-notating emotions in reallife spoken conversations by investigating the special case of empathy. We propose an annotation model based on the situated theories of emotions. The annotation scheme is directed to ob-serve the natural unfolding of empathy during the conversations. The key component of the protocol is the identification of the annotation unit based both on linguistic and paralinguistic cues. In the last part of the paper we evaluate the reliability of the annotation model.},
keywords = {Affective Computing, Signal Annotation and Interpretation}
}
title = {Fusion of Acoustic, Linguistic and Psycholinguistic Features for Speaker Personality Traits Recognition},
author = {Alam F. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/ICASSP14-FusionAcLingPsychPersonalityReco.pdf},
year = {2014},
date = {2014-01-01},
journal = {ICASSP, Florence, 2014},
abstract = {Behavioral analytics is an emerging research area that aims at automatic understanding of human behavior. For the advancement of this research area, we are interested in the problem of learning the personality traits from spoken data. In this study, we investigated the contribution of different types of speech features to the automatic recognition of Speaker Personality Trait (SPT) across diverse speech corpora (broadcast news and spoken conversation). We have extracted acoustic, linguistic, and psycholinguistic features and modeled their combination as input to the classification task. For the classification, we used Sequential Minimal Optimization for Support Vector Machine (SMO) together with Relief feature selection. The present study shows different levels of performance for automatically selected feature sets, and overall improved performance with their combination across diverse corpora.},
keywords = {Affective Computing}
}
title = {Predicting Personality Traits using Multimodal Information},
author = {Alam F. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/ACMM14-PersonalitytraitsFromMultimodal.pdf},
year = {2014},
date = {2014-01-01},
journal = {ACM International Conference on Multimedia, Workshop on Computational Personality Recognition, Orlando, 2014},
abstract = {Measuring personality traits has a long story in psychology where analysis has been done by asking sets of questions. These question sets (inventories) have been designed by investigating lexical terms that we use in our daily communications or by analyzing biological phenomena. Whether consciously or unconsciously we express our thoughts and behaviors when communicating with others, either verbally, non-verbally or using visual expressions. Recently, research in behavioral signal processing has focused on automatically measuring personality traits using different behavioral cues that appear in our daily communication. In this study, we present an approach to automatically recognize personality traits using a video-blog (vlog) corpus, consisting of transcription and extracted audio-visual features. We analyzed linguistic, psycholinguistic and emotional features in addition to the audio-visual features provided with the dataset. We also studied whether we can better predict a trait by identifying other traits. Using our best models we obtained very promising results compared to the official baseline.},
keywords = {Affective Computing}
}
2013
title = {Comparative Study of Speaker Personality Traits Recognition in Conversational and Broadcast News Speech},
author = {Alam F. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/IS13-Personality.pdf},
year = {2013},
date = {2013-01-01},
journal = {INTERSPEECH, Lyon, 2013},
abstract = {Natural human-computer interaction requires, in addition to understand what the speaker is saying, recognition of behavioral descriptors, such as speaker’s personality traits (SPTs). The complexity of this problem depends on the high variability and dimensionality of the acoustic, lexical and situational context manifestations of the SPTs. In this paper, we present a comparative study of automatic speaker personality trait recognition from speech corpora that differ in the source speaking style (broadcast news vs. conversational) and experimental context. We evaluated different feature selection algorithms such as information gain, relief and ensemble classification methods to address the high dimensionality issues. We trained and evaluated ensemble methods to leverage base learners, using three different algorithms such as SMO (Sequential Minimal Optimization for Support Vector Machine), RF (Random Forest) and Adaboost. After that, we combined them using majority voting and stacking methods. Our study shows that, performance of the system greatly benefits from feature selection and ensemble methods across corpora.},
keywords = {Affective Computing, Speech Processing}
}
title = {Motivational Feedback in Crowdsourcing: a Case Study in Speech Transcriptions},
author = {Riccardi G., Ghosh A., Chowdhury S. A. and Bayer A. O.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/IS13-Crowdsourcing.pdf},
year = {2013},
date = {2013-01-01},
journal = {INTERSPEECH, Lyon, 2013},
abstract = {A widely used strategy in human and machine performance enhancement is achieved through feedback. In this paper we investigate the effect of live motivational feedback on motivating crowds and improving the performance of the crowdsourcing computational model. The provided feedback allows workers to react in real-time and review past actions (e.g. word deletions); thus, to improve their performance on the current and future (sub) tasks. The feedback signal can be controlled via clean (e.g. expert) supervision or noisy supervision in order to trade-off between cost and target performance of the crowd-sourced task. The feedback signal is designed to enable crowd workers to improve their performance at the (sub) task level. The type and performance of feedback signal is evaluated in the context of a speech transcription task. Amazon Mechanical Turk (AMT) platform is used to transcribe speech utterances from different corpora. We show that in both clean (expert) and noisy (worker/turker) real-time feedback conditions the crowd workers are able to provide significantly more accurate transcriptions in a shorter time.},
keywords = {Affective Computing, Conversational and Interactive Systems , Machine Learning, Signal Annotation and Interpretation}
}
title = {Personality Traits Recognition on Social Network - Facebook},
author = {Alam F. ,Stepanov E. and Riccardi G.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/WCPR13-PersonalityTraitFacebook.pdf},
year = {2013},
date = {2013-01-01},
journal = {ICWSM, Workshop on Computational Personality Recognition, Boston, 2013},
abstract = {For the natural and social interaction it is necessary to understand human behavior. Personality is one of the fundamental aspects, by which we can understand behavioral dispositions. It is evident that there is a strong correlation between users’ personality and the way they behave on online social network (e.g., Facebook). This paper presents automatic recognition of Big-5 personality traits on social network (Facebook) using users’ status text. For the automatic recognition we studied different classification methods such as SMO (Sequential Minimal Optimization for Support Vector Machine), Bayesian Logistic Regression (BLR) and Multinomial Naïve Bayes (MNB) sparse modeling. Performance of the systems had been measured using macro-averaged precision, recall and F1; weighted average accuracy (WA) and un-weighted average accuracy (UA). Our comparative study shows that MNB performs better than BLR and SMO for personality traits recognition on the social network data.},
keywords = {Affective Computing, Natural Language Processing}
}
2012
title = {Kolmogorov-Smirnov Test for Feature Selection in Emotion Recognition From Speech},
author = {Ivanov A. V. and Riccardi G.},
url = {https://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=6289074&contentType=Conference+Publications},
year = {2012},
date = {2012-01-01},
journal = { ICASSP, Kyoto, 2012},
abstract = {Automatic emotion recognition from speech is limited by the ability to discover the relevant predicting features. The common approach is to extract a very large set of features over a generally long analysis time window. In this paper we investigate the applicability of two-sample Kolmogorov-Smirnov statistical test (KST) to the problem of segmental speech emotion recognition. We train emotion classifiers for each speech segment within an utterance. The segment labels are then combined to predict the dominant emotion label. Our findings show that KST can be successfully used to extract statistically relevant features. KST criterion is used to optimize the parameters of the statistical segmental analysis, namely the window segment size and shift. We carry out seven binary class emotion classification experiments on the Emo-DB and evaluate the impact of the segmental analysis and emotion-specific feature selection.},
keywords = {Affective Computing}
}
2011
title = {Recognition of Personality Traits from Human Spoken Conversations},
author = {Ivanov A. V., Riccardi G., Sporka A. J. and Franc J.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/IS11-PersonalityRecog.pdf},
year = {2011},
date = {2011-01-01},
journal = {INTERSPEECH, Florence, 2011},
abstract = {We are interested in understanding human personality and its manifestations in human interactions. The automatic analysis of such personality traits in natural conversation is quite complex due to the user-profiled corpora acquisition, annotation task and multidimensional modeling. While in the experimental psychology research this topic has been addressed extensively, speech and language scientists have recently engaged in limited experiments. In this paper we describe an automated system for speaker-independent personality prediction in the context of human-human spoken conversations. The evaluation of such system is carried out on the PersIA human-human spoken dialog corpus annotated with user self-assessments of the Big-Five personality traits. The personality predictor has been trained on paralinguistic features and its evaluation on five personality traits shows encouraging results for the conscientiousness and extroversion labels.},
keywords = {Affective Computing}
}
2005
title = {Grounding Emotions in Human-Machine Conversational Systems},
author = {Riccardi G. and Hakkani-Tur D.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/intetain05.pdf},
year = {2005},
date = {2005-01-01},
journal = {Lecture Notes in Computer Science, Springer-Verlag, , pp. 144 – 154, 2005},
abstract = {Abstract. In this paper we investigate the role of user emotions in human-machine goal-oriented conversations. There has been a growing interest in predicting emotions from acted and non-acted spontaneous speech. Much of the research work has gone in determining what are the correct labels and improving emotion prediction accuracy. In this paper we evaluate the value of user emotional state towards a computational model of emotion processing. We consider a binary representation of emotions (positive vs. negative) in the context of a goal-driven conversational system. For each human-machine interaction we acquire the temporal emotion sequence going from the initial to the final conversational state. These traces are used as features to characterize the user state dynamics. We ground the emotion traces by associating its patterns to dialog strategies and their effectiveness. In order to quantify the value of emotion indicators, we evaluate their predictions in terms of speech recognition and spoken language understanding errors as well as task success or failure. We report results on the 11.5K dialog corpus samples from the How may I Help You? corpus.},
keywords = {Affective Computing, Conversational and Interactive Systems }
}
title = {Using Context to Improve Emotion Detection in Spoken Dialog Systems},
author = {Liscombe J., Riccardi G. and Hakkani-Tur D.},
url = {https://sisl.disi.unitn.it/wp-content/uploads/2014/11/IS05-EmotRecog.pdf},
year = {2005},
date = {2005-01-01},
journal = {INTERSPEECH, Lisbon, Sept. 2005},
abstract = {Most research that explores the emotional state of users of spoken dialog systems does not fully utilize the contextual nature that the dialog structure provides. This paper reports results of machine learning experiments designed to automatically classify the emotional state of user turns using a corpus of 5,690 dialogs collected with the “How May I Help You SM ” spoken dialog system. We show that augmenting standard lexical and prosodic features with contextual features that exploit the structure of spoken dialog and track user state increases classification accuracy by 2.6%.},
keywords = {Affective Computing, Speech Processing}
}