Hierarchical Attention Network for Evaluating Therapist Empathy in Counseling Session

Counseling typically takes the form of spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered to be an essential quality factor of counseling outcome. This paper proposes a hierarchical recurrent network combined with two-level attention mechanisms to determine the therapist's empathy level solely from the acoustic features of conversational speech in a counseling session. The experimental results show that the proposed model can achieve an accuracy of 72.1% in classifying the therapist's empathy level as being "high" or "low". It is found that the speech from both the therapist and the client are contributing to predicting the empathy level that is subjectively rated by an expert observer. By analyzing speaker turns assigned with high attention weights, it is observed that 2 to 6 consecutive turns should be considered together to provide useful clues for detecting empathy, and the observer tends to take the whole session into consideration when rating the therapist empathy, instead of relying on a few specific speaker turns.Comment: Submitted to INTERSPEECH 202

A Study on Prosodic Entrainment in Relation to Therapist Empathy in Counseling Conversation

Counseling is carried out as spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered an important index of the quality of counseling and often assessed by an observer or the client. This research investigates the entrainment of speech prosody in relation to subjectively rated empathy. Experimental results show that the entrainment of intensity is more influential to empathy observation than that of pitch or speech rate in client-therapist interaction. The observer and the client have different perceptions of therapist empathy with the same entrained phenomena in pitch and intensity. The client's intention to make adjustment on pitch variation and intensity of speech is considered an indicator of the client's perception of counseling quality.Comment: Accepted by INTERSPEECH 202

Modeling Intrapersonal and Interpersonal Influences for Automatic Estimation of Therapist Empathy in Counseling Conversation

Counseling is usually conducted through spoken conversation between a therapist and a client. The empathy level of therapist is a key indicator of outcomes. Presuming that therapist's empathy expression is shaped by their past behavior and their perception of the client's behavior, we propose a model to estimate the therapist empathy by considering both intrapersonal and interpersonal influences. These dynamic influences are captured by applying an attention mechanism to the therapist turn and the historical turns of both therapist and client. Our findings suggest that the integration of dynamic influences enhances empathy level estimation. The influence-derived embedding should constitute a minor portion in the target turn representation for optimal empathy estimation. The client's turns (interpersonal influence) appear to slightly surpass the therapist's own turns (intrapersonal influence) in empathy estimation effectiveness. It is noted that concentrating exclusively on recent historical turns can significantly impact the estimation of therapist empathy

Anno-Mi : A dataset of expert- annotated counselling dialogues

This work has been funded by the EC in the H2020 Marie Skłodowska-Curie PhilHumans project, contract no. 812882. We would also like to thank Dr. Mark Aloia for his guidance and support

A speech-based empathy training system - initial design insights

Empathy is an essential component of human communication since it increases our understanding and perception of others. However, studies show that students\u27 empathy skills have declined rapidly in the last decades. Against this background, practitioner reports predict that the importance of empathy will increase as a skill for successful agile teamwork in the future. Therefore, researchers have designed information systems to train empathy abilities of learners in different domains. Nevertheless, research on automated speech-based training is rather scarce. Hence, we aim to investigate how to design a speech-based empathy training system that helps students react emotionally adequately in communication. This research in progress paper presents five initial requirements that guide future research and development of a speech-based empathy training system intended to support students\u27 self-regulated learning. With this, we hope to provide guidance for the design and embedding of speech-based empathy training systems at scale

Using fundamental frequency of cancer survivors’ speech to investigate emotional distress in out-patient visits

Objective Emotions, are in part conveyed by varying levels of fundamental frequency of voice pitch (f0). This study tests the hypothesis that patients display heightened levels of emotional arousal (f0) during Verona Coding Definitions of Emotional Sequences (VR-CoDES) cues and concerns versus during neutral statements. Methods The audio recordings of sixteen head and neck cancer survivors’ follow-up consultations were coded for patients’ emotional distress. Pitch (f0) of coded cues and concerns, including neutral statements was extracted. These were compared using a hierarchical linear model, nested for patient and pitch range, controlling for statement speech length. Utterance content was also explored. Results Clustering by patient explained 30% of the variance in utterances f0. Cues and concerns were on average 13.07 Hz higher than neutral statements (p = 0.02). Cues and concerns in these consultations contained content with a high proportion of recurrence fears. Conclusion The present study highlights the benefits and challenges of adding f0 and potential other prosodic features to the toolkit of coding emotional distress in the health communication setting. Practice implications The assessment of f0 during clinical conversations can provide additional information for research into emotional expression.PostprintPeer reviewe