Affective Analysis and Summarization of Conversations
ith the expansion of the call center industry, spoken conversation data is being generated in overwhelming amounts. In call centers, where the goals are to evaluate the expertise of operators, as well as to understand the content of the call in terms of topics, callers’ concerns and emotions, an automatic summary should contain a range of indicators that are useful for monitoring call quality addressing all these aspects.
The SENSEI automatic spoken conversation summary is organized in terms of several dimensions -– objective conversation descriptions –- such as factual metrics, emotional labels, discourse, and written synopses. Together these dimensions form an affective and content summary of a call and provide a wider perspective on the conversation.
Extractive and Abstractive Summarization
Call-center conversation synopses are short summaries of the events taking place during a conversation between a caller (or user) and one or more agents. Such a synopsis should contain a description of the user need or problem, and how the agent solves that problem. It might also describe the attitude of the caller and the agent.
While extractive synopses are generated by selected the most important conversation turns, the abstractive summarization is template based. The templates are learned by extracting frequent patterns from hand-written synopses, generalizing slot variables and filling the templates with entities extracted from a conversation transcript.
Overlap Discourse
Overlapping speech is a frequently occurring event in human-human conversations and it indicates the level of co-operation between the speakers. The description consists of statistics on competitive and non-competitive overlaps occurring in a conversation and the high amount of competitive overlaps signals problematic calls.
Emotion Recognition
Besides providing an affective description of a conversation, the identification of basic and complex emotions such as anger, frustration, empathy and satisfaction has a straightforward application to the evaluation of the call itself, as well as the operator expertise in handing situations. The emotion recognition models detect the presence of an empathy on the agents side, and the presence of anger, frustration and satisfaction on the client side.
Stepanov E. A., Favre B., Alam F., Chowdhury A. S., Singla K., Trione J., Bechet F. and Riccardi G., “Automatic Summarization of Call-Center Conversations”, IEEE ASRU, Scottsdale, 2015.
Alam F., Danieli M. and Riccardi G.,“Annotating and Modeling Empathy in Spoken Conversations”, Computer Speech and Language, July , v. 50, pp. 40-61, 2018.