Towards an emotionally communicative robot : feature analysis for multimodal support of affective touch recognition

Human affective state extracted from touch interaction takes advantage of natural communication of emotion through physical contact, enabling applications like robot therapy, intelligent tutoring systems, emotionally-reactive smart tech, and more. This work focused on the emotionally aware robot pet context and produced a custom, low-cost piezoresistive fabric touch sensor at 1-inch taxel resolution that accommodates the flex and stretch of the robot in motion. Using established machine learning techniques, we built classification models of social and emotional touch data. We present an iteration of the human-robot interaction loop for an emotionally aware robot through two distinct studies and demonstrate gesture recognition at roughly 85% accuracy (chance 14%). The first study collected social touch gesture data (N=26) to assess data quality of our custom sensor under noisy conditions: mounted on a robot skeleton simulating regular breathing, obscured under fur casings, placed over deformable surfaces. Our second study targeted affect with the same sensor, wherein participants (N=30) relived emotionally intense memories while interacting with a smaller stationary robot, generating touch data imbued with the following: Stressed, Excited, Relaxed, or Depressed. A feature space analysis triangulating touch, gaze, and physiological data highlighted the dimensions of touch that suggest affective state. To close the interactive loop, we had participants (N=20) evaluate researcherdesigned breathing behaviours on 1-DOF robots for emotional content. Results demonstrate that these behaviours can display human-recognizable emotion as perceptual affective qualities across the valence-arousal emotion model. Finally, we discuss the potential impact of a system capable of emotional “conversation” with human users, referencing specific applications.Science, Faculty ofComputer Science, Department ofGraduat

From devices to data and back again : a tale of computationally modelling affective touch

Emotionally responsive Human-Robot Interaction (HRI) has captured our curiosity and imagination in fantastical ways throughout much of modern media. With touch being a valuable yet sorely missed emotion communication channel when in-person interaction is unrealistic for practical reasons, we could look to machine-mediated ways to bridge that distance. In this thesis, we investigate how we might enable machines to recognize natural and spontaneous emotional touch expressions in two parts. First, we take a close look at ways machines engage with human emotion by examining examples of machines in three emotionally communicative roles: as a passive witness receiving and logging the emotional state of their (N=30) human counterparts, as an influential actor whose own breathing behaviour alters human fear response (N=103), and as a conduit for the transmission of emotion expression between human users (N=10 dyads and N=21 individuals). Next, we argue that in order for devices to be truly emotionally reactive, they should address the time-varying and dynamic nature of emotional lived experience. Any computational or emotion recognition engine intended for use under realistic conditions should acknowledge that emotions will evolve over time. Machine responses may change with changing ‘emotion direction’ – acting in an encouraging way when the user is `happy and getting happier' vs. presenting calming behaviours for `happy but getting anxious'. To that end, we develop a multi-stage emotion self-reporting procedure for collecting N=16 users’ dynamic emotion expression during videogame play. From their keypress force controlling their in-game character, we benchmark individualized recognition performance for emotion direction, even finding it to exceed that of brain activity (as measured by continuous Electroencephalography (EEG)). For a proof-of-concept of a training process that generates models of true and spontaneous emotion expression evolving with the user, we then revise our protocol to be more flexible to naturalistic emotion expression. We build a custom tool to help with data collection and labelling of personal storytelling sessions and evaluate user impressions (N=5 with up to 3 stories each for a total of 10 sessions). Finally, we conclude with actionable recommendations for advancing the training and machine recognition of naturalistic and dynamic emotion expression.Science, Faculty ofComputer Science, Department ofGraduat

Discerning Affect from Touch and Gaze During Interaction with a Robot Pet

Practical affect recognition needs to be efficient and unobtrusive in interactive contexts. One approach to a robust real-time system is to sense and automatically integrate multiple nonverbal sources. We investigated how users' touch, and secondarily gaze, perform as affect-encoding modalities during physical interaction with a robot pet, in comparison to more-studied biometric channels. To elicit authentically experienced emotions, participants recounted two intense memories of opposing polarity in Stressed-Relaxed or Depressed-Excited conditions. We collected data (N=30) from a touch sensor embedded under robot fur (force magnitude and location), a robot-adjacent gaze tracker (location), and biometric sensors (skin conductance, blood volume pulse, respiration rate). Cross-validation of Random Forest classifiers achieved best-case accuracy for combined touch-with-gaze approaching that of biometric results: where training and test sets include adjacent temporal windows, subject-dependent prediction was 94% accurate. In contrast, subject-independent Leave-One-participant-Out predictions resulted in 30% accuracy (chance 25%). Performance was best where participant information was available in both training and test sets. Addressing computational robustness for dynamic, adaptive real-time interactions, we analyzed subsets of our multimodal feature set, varying sample rates and window sizes. We summarize design directions based on these parameters for this touch-based, affective, and hard, real-time robot interaction application.acceptedVersionPeer reviewe

Touch Challenge ‘15: Recognizing social touch gestures

Advances in the field of touch recognition could open up applications for touch-based interaction in areas such as Human-Robot Interaction (HRI). We extended this challenge to the research community working on multimodal interaction with the goal of sparking interest in the touch modality and to promote exploration of the use of data processing techniques from other more mature modalities for touch recognition. Two data sets were made available containing labeled pressure sensor data of social touch gestures that were performed by touching a touch-sensitive surface with the hand. Each set was collected from similar sensor grids, but under conditions reflecting different application orientations: CoST: Corpus of Social Touch and HAART: The Human-Animal Affective Robot Touch gesture set. In this paper we describe the challenge protocol and summa- rize the results from the touch challenge hosted in conjunction with the 2015 ACM International Conference on Multi- modal Interaction (ICMI). The most important outcomes of the challenges were: (1) transferring techniques from other modalities, such as image processing, speech, and human action recognition provided valuable feature sets; (2) gesture classification confusions were similar despite the various data processing methods used.\ud Categorie

Sketching CuddleBits: Coupled Prototyping of Body and Behaviour for an Affective Robot Pet

Social robots that physically display emotion invite natural communication with their human interlocutors, enabling ap- plications like robot-assisted therapy where a complex robot’s breathing influences human emotional and physiological state. Using DIY fabrication and assembly, we explore how sim- ple 1-DOF robots can express affect with economy and user customizability, leveraging open-source designs. We developed low-cost techniques for coupled iteration of a simple robot’s body and behaviour, and evaluated its potential to display emotion. Through two user studies, we (1) vali- dated these CuddleBits’ ability to express emotions (N=20); (2) sourced a corpus of 72 robot emotion behaviours from participants (N=10); and (3) analyzed it to link underlying parameters to emotional perception (N=14). We found that CuddleBits can express arousal (activation), and to a lesser degree valence (pleasantness). We also show how a sketch-refine paradigm combined with DIY fabrication and novel input methods enable parametric design of physical emotion display, and discuss how mastering this parsimonious case can give insight into layering simple behaviours in more complex robots

CEPC Technical Design Report -- Accelerator

The Circular Electron Positron Collider (CEPC) is a large scientific project initiated and hosted by China, fostered through extensive collaboration with international partners. The complex comprises four accelerators: a 30 GeV Linac, a 1.1 GeV Damping Ring, a Booster capable of achieving energies up to 180 GeV, and a Collider operating at varying energy modes (Z, W, H, and ttbar). The Linac and Damping Ring are situated on the surface, while the Booster and Collider are housed in a 100 km circumference underground tunnel, strategically accommodating future expansion with provisions for a Super Proton Proton Collider (SPPC). The CEPC primarily serves as a Higgs factory. In its baseline design with synchrotron radiation (SR) power of 30 MW per beam, it can achieve a luminosity of 5e34 /cm^2/s^1, resulting in an integrated luminosity of 13 /ab for two interaction points over a decade, producing 2.6 million Higgs bosons. Increasing the SR power to 50 MW per beam expands the CEPC's capability to generate 4.3 million Higgs bosons, facilitating precise measurements of Higgs coupling at sub-percent levels, exceeding the precision expected from the HL-LHC by an order of magnitude. This Technical Design Report (TDR) follows the Preliminary Conceptual Design Report (Pre-CDR, 2015) and the Conceptual Design Report (CDR, 2018), comprehensively detailing the machine's layout and performance, physical design and analysis, technical systems design, R&D and prototyping efforts, and associated civil engineering aspects. Additionally, it includes a cost estimate and a preliminary construction timeline, establishing a framework for forthcoming engineering design phase and site selection procedures. Construction is anticipated to begin around 2027-2028, pending government approval, with an estimated duration of 8 years. The commencement of experiments could potentially initiate in the mid-2030s

CEPC Technical Design Report -- Accelerator

International audienceThe Circular Electron Positron Collider (CEPC) is a large scientific project initiated and hosted by China, fostered through extensive collaboration with international partners. The complex comprises four accelerators: a 30 GeV Linac, a 1.1 GeV Damping Ring, a Booster capable of achieving energies up to 180 GeV, and a Collider operating at varying energy modes (Z, W, H, and ttbar). The Linac and Damping Ring are situated on the surface, while the Booster and Collider are housed in a 100 km circumference underground tunnel, strategically accommodating future expansion with provisions for a Super Proton Proton Collider (SPPC). The CEPC primarily serves as a Higgs factory. In its baseline design with synchrotron radiation (SR) power of 30 MW per beam, it can achieve a luminosity of 5e34 /cm^2/s^1, resulting in an integrated luminosity of 13 /ab for two interaction points over a decade, producing 2.6 million Higgs bosons. Increasing the SR power to 50 MW per beam expands the CEPC's capability to generate 4.3 million Higgs bosons, facilitating precise measurements of Higgs coupling at sub-percent levels, exceeding the precision expected from the HL-LHC by an order of magnitude. This Technical Design Report (TDR) follows the Preliminary Conceptual Design Report (Pre-CDR, 2015) and the Conceptual Design Report (CDR, 2018), comprehensively detailing the machine's layout and performance, physical design and analysis, technical systems design, R&D and prototyping efforts, and associated civil engineering aspects. Additionally, it includes a cost estimate and a preliminary construction timeline, establishing a framework for forthcoming engineering design phase and site selection procedures. Construction is anticipated to begin around 2027-2028, pending government approval, with an estimated duration of 8 years. The commencement of experiments could potentially initiate in the mid-2030s

CEPC Technical Design Report -- Accelerator

International audienceThe Circular Electron Positron Collider (CEPC) is a large scientific project initiated and hosted by China, fostered through extensive collaboration with international partners. The complex comprises four accelerators: a 30 GeV Linac, a 1.1 GeV Damping Ring, a Booster capable of achieving energies up to 180 GeV, and a Collider operating at varying energy modes (Z, W, H, and ttbar). The Linac and Damping Ring are situated on the surface, while the Booster and Collider are housed in a 100 km circumference underground tunnel, strategically accommodating future expansion with provisions for a Super Proton Proton Collider (SPPC). The CEPC primarily serves as a Higgs factory. In its baseline design with synchrotron radiation (SR) power of 30 MW per beam, it can achieve a luminosity of 5e34 /cm^2/s^1, resulting in an integrated luminosity of 13 /ab for two interaction points over a decade, producing 2.6 million Higgs bosons. Increasing the SR power to 50 MW per beam expands the CEPC's capability to generate 4.3 million Higgs bosons, facilitating precise measurements of Higgs coupling at sub-percent levels, exceeding the precision expected from the HL-LHC by an order of magnitude. This Technical Design Report (TDR) follows the Preliminary Conceptual Design Report (Pre-CDR, 2015) and the Conceptual Design Report (CDR, 2018), comprehensively detailing the machine's layout and performance, physical design and analysis, technical systems design, R&D and prototyping efforts, and associated civil engineering aspects. Additionally, it includes a cost estimate and a preliminary construction timeline, establishing a framework for forthcoming engineering design phase and site selection procedures. Construction is anticipated to begin around 2027-2028, pending government approval, with an estimated duration of 8 years. The commencement of experiments could potentially initiate in the mid-2030s

CEPC Technical Design Report -- Accelerator

International audienceThe Circular Electron Positron Collider (CEPC) is a large scientific project initiated and hosted by China, fostered through extensive collaboration with international partners. The complex comprises four accelerators: a 30 GeV Linac, a 1.1 GeV Damping Ring, a Booster capable of achieving energies up to 180 GeV, and a Collider operating at varying energy modes (Z, W, H, and ttbar). The Linac and Damping Ring are situated on the surface, while the Booster and Collider are housed in a 100 km circumference underground tunnel, strategically accommodating future expansion with provisions for a Super Proton Proton Collider (SPPC). The CEPC primarily serves as a Higgs factory. In its baseline design with synchrotron radiation (SR) power of 30 MW per beam, it can achieve a luminosity of 5e34 /cm^2/s^1, resulting in an integrated luminosity of 13 /ab for two interaction points over a decade, producing 2.6 million Higgs bosons. Increasing the SR power to 50 MW per beam expands the CEPC's capability to generate 4.3 million Higgs bosons, facilitating precise measurements of Higgs coupling at sub-percent levels, exceeding the precision expected from the HL-LHC by an order of magnitude. This Technical Design Report (TDR) follows the Preliminary Conceptual Design Report (Pre-CDR, 2015) and the Conceptual Design Report (CDR, 2018), comprehensively detailing the machine's layout and performance, physical design and analysis, technical systems design, R&D and prototyping efforts, and associated civil engineering aspects. Additionally, it includes a cost estimate and a preliminary construction timeline, establishing a framework for forthcoming engineering design phase and site selection procedures. Construction is anticipated to begin around 2027-2028, pending government approval, with an estimated duration of 8 years. The commencement of experiments could potentially initiate in the mid-2030s

CEPC Technical Design Report -- Accelerator

International audienceThe Circular Electron Positron Collider (CEPC) is a large scientific project initiated and hosted by China, fostered through extensive collaboration with international partners. The complex comprises four accelerators: a 30 GeV Linac, a 1.1 GeV Damping Ring, a Booster capable of achieving energies up to 180 GeV, and a Collider operating at varying energy modes (Z, W, H, and ttbar). The Linac and Damping Ring are situated on the surface, while the Booster and Collider are housed in a 100 km circumference underground tunnel, strategically accommodating future expansion with provisions for a Super Proton Proton Collider (SPPC). The CEPC primarily serves as a Higgs factory. In its baseline design with synchrotron radiation (SR) power of 30 MW per beam, it can achieve a luminosity of 5e34 /cm^2/s^1, resulting in an integrated luminosity of 13 /ab for two interaction points over a decade, producing 2.6 million Higgs bosons. Increasing the SR power to 50 MW per beam expands the CEPC's capability to generate 4.3 million Higgs bosons, facilitating precise measurements of Higgs coupling at sub-percent levels, exceeding the precision expected from the HL-LHC by an order of magnitude. This Technical Design Report (TDR) follows the Preliminary Conceptual Design Report (Pre-CDR, 2015) and the Conceptual Design Report (CDR, 2018), comprehensively detailing the machine's layout and performance, physical design and analysis, technical systems design, R&D and prototyping efforts, and associated civil engineering aspects. Additionally, it includes a cost estimate and a preliminary construction timeline, establishing a framework for forthcoming engineering design phase and site selection procedures. Construction is anticipated to begin around 2027-2028, pending government approval, with an estimated duration of 8 years. The commencement of experiments could potentially initiate in the mid-2030s