Building Embodied Conversational Agents:Observations on human nonverbal behaviour as a resource for the development of artificial characters

"Wow this is so cool!" This is what I most probably yelled, back in the 90s, when my first computer program on our MSX computer turned out to do exactly what I wanted it to do. The program contained the following instruction: COLOR 10(1.1) After hitting enter, it would change the screen color from light blue to dark yellow. A few years after that experience, Microsoft Windows was introduced. Windows came with an intuitive graphical user interface that was designed to allow all people, so also those who would not consider themselves to be experienced computer addicts, to interact with the computer. This was a major step forward in human-computer interaction, as from that point forward no complex programming skills were required anymore to perform such actions as adapting the screen color. Changing the background was just a matter of pointing the mouse to the desired color on a color palette. "Wow this is so cool!". This is what I shouted, again, 20 years later. This time my new smartphone successfully skipped to the next song on Spotify because I literally told my smartphone, with my voice, to do so. Being able to operate your smartphone with natural language through voice-control can be extremely handy, for instance when listening to music while showering. Again, the option to handle a computer with voice instructions turned out to be a significant optimization in human-computer interaction. From now on, computers could be instructed without the use of a screen, mouse or keyboard, and instead could operate successfully simply by telling the machine what to do. In other words, I have personally witnessed how, within only a few decades, the way people interact with computers has changed drastically, starting as a rather technical and abstract enterprise to becoming something that was both natural and intuitive, and did not require any advanced computer background. Accordingly, while computers used to be machines that could only be operated by technically-oriented individuals, they had gradually changed into devices that are part of many people’s household, just as much as a television, a vacuum cleaner or a microwave oven. The introduction of voice control is a significant feature of the newer generation of interfaces in the sense that these have become more "antropomorphic" and try to mimic the way people interact in daily life, where indeed the voice is a universally used device that humans exploit in their exchanges with others. The question then arises whether it would be possible to go even one step further, where people, like in science-fiction movies, interact with avatars or humanoid robots, whereby users can have a proper conversation with a computer-simulated human that is indistinguishable from a real human. An interaction with a human-like representation of a computer that behaves, talks and reacts like a real person would imply that the computer is able to not only produce and understand messages transmitted auditorily through the voice, but also could rely on the perception and generation of different forms of body language, such as facial expressions, gestures or body posture. At the time of writing, developments of this next step in human-computer interaction are in full swing, but the type of such interactions is still rather constrained when compared to the way humans have their exchanges with other humans. It is interesting to reflect on how such future humanmachine interactions may look like. When we consider other products that have been created in history, it sometimes is striking to see that some of these have been inspired by things that can be observed in our environment, yet at the same do not have to be exact copies of those phenomena. For instance, an airplane has wings just as birds, yet the wings of an airplane do not make those typical movements a bird would produce to fly. Moreover, an airplane has wheels, whereas a bird has legs. At the same time, an airplane has made it possible for a humans to cover long distances in a fast and smooth manner in a way that was unthinkable before it was invented. The example of the airplane shows how new technologies can have "unnatural" properties, but can nonetheless be very beneficial and impactful for human beings. This dissertation centers on this practical question of how virtual humans can be programmed to act more human-like. The four studies presented in this dissertation all have the equivalent underlying question of how parts of human behavior can be captured, such that computers can use it to become more human-like. Each study differs in method, perspective and specific questions, but they are all aimed to gain insights and directions that would help further push the computer developments of human-like behavior and investigate (the simulation of) human conversational behavior. The rest of this introductory chapter gives a general overview of virtual humans (also known as embodied conversational agents), their potential uses and the engineering challenges, followed by an overview of the four studies

Building Embodied Conversational Agents:Observations on human nonverbal behaviour as a resource for the development of artificial characters

"Wow this is so cool!" This is what I most probably yelled, back in the 90s, when my first computer program on our MSX computer turned out to do exactly what I wanted it to do. The program contained the following instruction: COLOR 10(1.1) After hitting enter, it would change the screen color from light blue to dark yellow. A few years after that experience, Microsoft Windows was introduced. Windows came with an intuitive graphical user interface that was designed to allow all people, so also those who would not consider themselves to be experienced computer addicts, to interact with the computer. This was a major step forward in human-computer interaction, as from that point forward no complex programming skills were required anymore to perform such actions as adapting the screen color. Changing the background was just a matter of pointing the mouse to the desired color on a color palette. "Wow this is so cool!". This is what I shouted, again, 20 years later. This time my new smartphone successfully skipped to the next song on Spotify because I literally told my smartphone, with my voice, to do so. Being able to operate your smartphone with natural language through voice-control can be extremely handy, for instance when listening to music while showering. Again, the option to handle a computer with voice instructions turned out to be a significant optimization in human-computer interaction. From now on, computers could be instructed without the use of a screen, mouse or keyboard, and instead could operate successfully simply by telling the machine what to do. In other words, I have personally witnessed how, within only a few decades, the way people interact with computers has changed drastically, starting as a rather technical and abstract enterprise to becoming something that was both natural and intuitive, and did not require any advanced computer background. Accordingly, while computers used to be machines that could only be operated by technically-oriented individuals, they had gradually changed into devices that are part of many people’s household, just as much as a television, a vacuum cleaner or a microwave oven. The introduction of voice control is a significant feature of the newer generation of interfaces in the sense that these have become more "antropomorphic" and try to mimic the way people interact in daily life, where indeed the voice is a universally used device that humans exploit in their exchanges with others. The question then arises whether it would be possible to go even one step further, where people, like in science-fiction movies, interact with avatars or humanoid robots, whereby users can have a proper conversation with a computer-simulated human that is indistinguishable from a real human. An interaction with a human-like representation of a computer that behaves, talks and reacts like a real person would imply that the computer is able to not only produce and understand messages transmitted auditorily through the voice, but also could rely on the perception and generation of different forms of body language, such as facial expressions, gestures or body posture. At the time of writing, developments of this next step in human-computer interaction are in full swing, but the type of such interactions is still rather constrained when compared to the way humans have their exchanges with other humans. It is interesting to reflect on how such future humanmachine interactions may look like. When we consider other products that have been created in history, it sometimes is striking to see that some of these have been inspired by things that can be observed in our environment, yet at the same do not have to be exact copies of those phenomena. For instance, an airplane has wings just as birds, yet the wings of an airplane do not make those typical movements a bird would produce to fly. Moreover, an airplane has wheels, whereas a bird has legs. At the same time, an airplane has made it possible for a humans to cover long distances in a fast and smooth manner in a way that was unthinkable before it was invented. The example of the airplane shows how new technologies can have "unnatural" properties, but can nonetheless be very beneficial and impactful for human beings. This dissertation centers on this practical question of how virtual humans can be programmed to act more human-like. The four studies presented in this dissertation all have the equivalent underlying question of how parts of human behavior can be captured, such that computers can use it to become more human-like. Each study differs in method, perspective and specific questions, but they are all aimed to gain insights and directions that would help further push the computer developments of human-like behavior and investigate (the simulation of) human conversational behavior. The rest of this introductory chapter gives a general overview of virtual humans (also known as embodied conversational agents), their potential uses and the engineering challenges, followed by an overview of the four studies

Advances in air quality research – current and emerging challenges

This review provides a community\u27s perspective on air quality research focusing mainly on developments over the past decade. The article provides perspectives on current and future challenges as well as research needs for selected key topics. While this paper is not an exhaustive review of all research areas in the field of air quality, we have selected key topics that we feel are important from air quality research and policy perspectives. After providing a short historical overview, this review focuses on improvements in characterizing sources and emissions of air pollution, new air quality observations and instrumentation, advances in air quality prediction and forecasting, understanding interactions of air quality with meteorology and climate, exposure and health assessment, and air quality management and policy. In conducting the review, specific objectives were (i) to address current developments that push the boundaries of air quality research forward, (ii) to highlight the emerging prominent gaps of knowledge in air quality research, and (iii) to make recommendations to guide the direction for future research within the wider community. This review also identifies areas of particular importance for air quality policy. The original concept of this review was borne at the International Conference on Air Quality 2020 (held online due to the COVID 19 restrictions during 18–26 May 2020), but the article incorporates a wider landscape of research literature within the field of air quality science. On air pollution emissions the review highlights, in particular, the need to reduce uncertainties in emissions from diffuse sources, particulate matter chemical components, shipping emissions, and the importance of considering both indoor and outdoor sources. There is a growing need to have integrated air pollution and related observations from both ground-based and remote sensing instruments, including in particular those on satellites. The research should also capitalize on the growing area of low-cost sensors, while ensuring a quality of the measurements which are regulated by guidelines. Connecting various physical scales in air quality modelling is still a continual issue, with cities being affected by air pollution gradients at local scales and by long-range transport. At the same time, one should allow for the impacts from climate change on a longer timescale. Earth system modelling offers considerable potential by providing a consistent framework for treating scales and processes, especially where there are significant feedbacks, such as those related to aerosols, chemistry, and meteorology. Assessment of exposure to air pollution should consider the impacts of both indoor and outdoor emissions, as well as application of more sophisticated, dynamic modelling approaches to predict concentrations of air pollutants in both environments. With particulate matter being one of the most important pollutants for health, research is indicating the urgent need to understand, in particular, the role of particle number and chemical components in terms of health impact, which in turn requires improved emission inventories and models for predicting high-resolution distributions of these metrics over cities. The review also examines how air pollution management needs to adapt to the above-mentioned new challenges and briefly considers the implications from the COVID-19 pandemic for air quality. Finally, we provide recommendations for air quality research and support for policy

Advances in air quality research – current and emerging challenges

© Author(s) 2022. This work is distributed under the Creative Commons Attribution 4.0 License. https://creativecommons.org/licenses/by/4.0/This review provides a community’s perspective on air quality research focusing mainly on developmentsover the past decade. The article provides perspectives on current and future challenges as well asresearch needs for selected key topics. While this paper is not an exhaustive review of all research areas in thefield of air quality, we have selected key topics that we feel are important from air quality research and policy perspectives. After providing a short historical overview, this review focuses on improvements in characterizingsources and emissions of air pollution, new air quality observations and instrumentation, advances in air qualityprediction and forecasting, understanding interactions of air quality with meteorology and climate, exposure andhealth assessment, and air quality management and policy. In conducting the review, specific objectives were(i) to address current developments that push the boundaries of air quality research forward, (ii) to highlightthe emerging prominent gaps of knowledge in air quality research, and (iii) to make recommendations to guidethe direction for future research within the wider community. This review also identifies areas of particular importancefor air quality policy. The original concept of this review was borne at the International Conferenceon Air Quality 2020 (held online due to the COVID 19 restrictions during 18–26 May 2020), but the articleincorporates a wider landscape of research literature within the field of air quality science. On air pollutionemissions the review highlights, in particular, the need to reduce uncertainties in emissions from diffuse sources,particulate matter chemical components, shipping emissions, and the importance of considering both indoor andoutdoor sources. There is a growing need to have integrated air pollution and related observations from bothground-based and remote sensing instruments, including in particular those on satellites. The research shouldalso capitalize on the growing area of low-cost sensors, while ensuring a quality of the measurements which areregulated by guidelines. Connecting various physical scales in air quality modelling is still a continual issue,with cities being affected by air pollution gradients at local scales and by long-range transport. At the same time,one should allow for the impacts from climate change on a longer timescale. Earth system modelling offers considerablepotential by providing a consistent framework for treating scales and processes, especially where thereare significant feedbacks, such as those related to aerosols, chemistry, and meteorology. Assessment of exposureto air pollution should consider the impacts of both indoor and outdoor emissions, as well as application of moresophisticated, dynamic modelling approaches to predict concentrations of air pollutants in both environments.With particulate matter being one of the most important pollutants for health, research is indicating the urgentneed to understand, in particular, the role of particle number and chemical components in terms of health impact,which in turn requires improved emission inventories and models for predicting high-resolution distributions ofthese metrics over cities. The review also examines how air pollution management needs to adapt to the abovementionednew challenges and briefly considers the implications from the COVID-19 pandemic for air quality.Finally, we provide recommendations for air quality research and support for policy.Peer reviewe

Exploring the practical use of a collaborative robot for academic purposes

This article presents a set of experiences related to the setup and exploration of potential educational uses of a collaborative robot (cobot). The basic principles that have guided the work carried out have been three. First and foremost, study of all the functionalities offered by the robot and exploration of its potential academic uses both in subjects focused on industrial robotics and in subjects of related disciplines (automation, communications, computer vision). Second, achieve the total integration of the cobot at the laboratory, seeking not only independent uses of it but also seeking for applications (laboratory practices) in which the cobot interacts with some of the other devices already existing at the laboratory (other industrial robots and a flexible manufacturing system). Third, reuse of some available components and minimization of the number and associated cost of required new components. The experiences, carried out following a project-based learning methodology under the framework of bachelor and master subjects and thesis, have focused on the integration of mechanical, electronic and programming aspects in new design solutions (end effector, cooperative workspace, artificial vision system integration) and case studies (advanced task programming, cybersecure communication, remote access). These experiences have consolidated the students' acquisition of skills in the transition to professional life by having the close collaboration of the university faculty with the experts of the robotics company.Postprint (published version