Communication, Machines & Human Augmentics

This essay reformulates the question of human augmentation as a problem of advanced human-machine communication, theorizing that such communication implies robust artificial intelligence and necessitates understanding the relational role new technologies play in human-machine communication. We focus on the questions, “When do electronic tools cease to be ‘simply’ tools, and become meaningfully part of ourselves,” and, “When might we think of these tools as augmenting our selves, rather than simply amplifying our capabilities?” These questions, already important to the medical and rehabilitative fields, loom larger with increasing commodification of pervasive augmentation technologies, and indicate the verge on which human-machine communication now finds itself. Through analyses of human and machine agency, mediated through a theory of close human-machine communication, we argue that the critical element in discussions of human-machine communication is an increase in sense of agency, extending the traditional human-computer interface dictum to provide an internal locus of control

Eu Sonar : uso de computação vestível para o auxílio a deficientes visuais

Monografia (graduação)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2016.Neste trabalho, realizamos um estudo sobre o histórico da computação vestível e propomos um novo sistema baseado no uso de sensores e micromotores, para a detecção de obstáculos e correto direcionamento de pessoas com deficiência visual. Para isso, desenvolvemos um sistema que recebe dados através de comunicação sem fio via interface bluetooth de um dispositivo móvel equipado com GPS e bússola, usado para alertar e guiar o deficiente visual a seu destino por meio de vibrações. Testes de laboratório foram realizados a fim de comprovar a eficácia do sistema proposto, bem como identificar os principais problemas que podem advir da sua implementação prática. Um protótipo plenamente funcional foi construído e testado com usuários deficientes visuais, bem como usuários não-deficientes. Os resultados obtidos demonstram que o sistema proposto é promissor e pontos de melhoria são sugeridos.In this paper, we carried out a study on the history of wearable computing and propose a new system based on the use of sensors and micromotors, for the detection of obstacles and correct guiding of people with visual impairment. For this, we developed a system that receives data through wireless communication via Bluetooth interface for a mobile device equipped with GPS and compass, used to alert and guide the visually impaired to their destination through vibrations. Laboratory tests were conducted to prove the effectiveness of the proposed system and to identify the main problems that may arise from its practical implementation. A fully functional prototype was built and tested with visually impaired users as well as non-disabled users. The results show that the proposed system is promising and points of improvement are suggested

Gapeau: Enhancing the Sense of Distance to Others with a Head-Mounted Sensor

Human perception lacks the capabilities to accurately assess distance. The recent Covid-19 pandemic outbreak rendered this ability particularly important. Augmenting our sense of distance can help maintain safe separation from others when required. To explore how systems can help users maintain physical distance, we designed, implemented and evaluated Gapeau - a head-mounted system for augmenting the sense of distance. Our system uses proximity sensors and thermal sensing to detect and measure the distance to other people. We conducted a validation protocol, an experiment, in which we compared different feedback modalities, and an in-the-wild study to evaluate Gapeau\u27s performance and suitability for use in social contexts. We found that our system enabled users to more accurately determine whether they were maintaining a safe distance from others. Vibration and auditory feedback were found most effective and usable. Gapeau was perceived as socially acceptable. Our work contributes insights for augmented sensing systems with social relevance

Ubiquitous haptic feedback in human-computer interaction through electrical muscle stimulation

[no abstract

Social Emotions in Multiagent Systems

Tesis por compendioA lo largo de los últimos años, los sistemas multi-agente (SMA) han demostrado ser un paradigma potente y versátil, con un gran potencial a la hora de resolver problemas complejos en entornos dinámicos y distribuidos. Este potencial no se debe principalmente a sus características individuales (como son su autonomía, su capacidad de percepción, reacción y de razonamiento), sino que también a la capacidad de comunicación y cooperación a la hora de conseguir un objetivo. De hecho, su capacidad social es la que más llama la atención, es este comportamiento social el que dota de potencial a los sistemas multi-agente. Estas características han hecho de los SMA, la herramienta de inteligencia artificial (IA) más utilizada para el diseño de entornos virtuales inteligentes (IVE), los cuales son herramientas de simulación compleja basadas en agentes. Sin embargo, los IVE incorporan restricciones físicas (como gravedad, fuerzas, rozamientos, etc.), así como una representación 3D de lo que se quiere simular. Así mismo, estas herramientas no son sólo utilizadas para la realización de simulaciones. Con la aparición de nuevas aplicaciones como \emph{Internet of Things (IoT)}, \emph{Ambient Intelligence (AmI)}, robot asistentes, entre otras, las cuales están en contacto directo con el ser humano. Este contacto plantea nuevos retos a la hora de interactuar con estas aplicaciones. Una nueva forma de interacción que ha despertado un especial interés, es el que se relaciona con la detección y/o simulación de estados emocionales. Esto ha permitido que estas aplicaciones no sólo puedan detectar nuestros estados emocionales, sino que puedan simular y expresar sus propias emociones mejorando así la experiencia del usuario con dichas aplicaciones. Con el fin de mejorar la experiencia humano-máquina, esta tesis plantea como objetivo principal la creación de modelos emocionales sociales, los cuales podrán ser utilizados en aplicaciones MAS permitiendo a los agentes interpretar y/o emular diferentes estados emocionales y, además, emular fenómenos de contagio emocional. Estos modelos permitirán realizar simulaciones complejas basadas en emociones y aplicaciones más realistas en dominios como IoT, AIm, SH.Over the past few years, multi-agent systems (SMA) have proven to be a powerful and versatile paradigm, with great potential for solving complex problems in dynamic and distributed environments. This potential is not primarily due to their individual characteristics (such as their autonomy, their capacity for perception, reaction and reasoning), but also the ability to communicate and cooperate in achieving a goal. In fact, its social capacity is the one that draws the most attention, it is this social behavior that gives potential to multi-agent systems. These characteristics have made the SMA, the artificial intelligence (AI) tool most used for the design of intelligent virtual environments (IVE), which are complex agent-based simulation tools. However, IVE incorporates physical constraints (such as gravity, forces, friction, etc.), as well as a 3D representation of what you want to simulate. Also, these tools are not only used for simulations. With the emergence of new applications such as \emph {Internet of Things (IoT)}, \emph {Ambient Intelligence (AmI)}, robot assistants, among others, which are in direct contact with humans. This contact poses new challenges when it comes to interacting with these applications. A new form of interaction that has aroused a special interest is that which is related to the detection and / or simulation of emotional states. This has allowed these applications not only to detect our emotional states, but also to simulate and express their own emotions, thus improving the user experience with those applications. In order to improve the human-machine experience, this thesis aims to create social emotional models, which can be used in MAS applications, allowing agents to interpret and / or emulate different emotional states, and emulate phenomena of emotional contagion. These models will allow complex simulations based on emotions and more realistic applications in domains like IoT, AIm, SH.Al llarg dels últims anys, els sistemes multi-agent (SMA) han demostrat ser un paradigma potent i versàtil, amb un gran potencial a l'hora de resoldre problemes complexos en entorns dinàmics i distribuïts. Aquest potencial no es deu principalment a les seues característiques individuals (com són la seua autonomia, la seua capacitat de percepció, reacció i de raonament), sinó que també a la capacitat de comunicació i cooperació a l'hora d'aconseguir un objectiu. De fet, la seua capacitat social és la que més crida l'atenció, és aquest comportament social el que dota de potencial als sistemes multi-agent. Aquestes característiques han fet dels SMA, l'eina d'intel·ligència artificial (IA) més utilitzada per al disseny d'entorns virtuals intel·ligents (IVE), els quals són eines de simulació complexa basades en agents. No obstant això, els IVE incorporen restriccions físiques (com gravetat, forces, fregaments, etc.), així com una representació 3D del que es vol simular. Així mateix, aquestes eines no són només utilitzades per a la realització de simulacions. Amb l'aparició de noves aplicacions com \emph{Internet of Things (IOT)}, \emph{Ambient Intelligence (AmI)}, robot assistents, entre altres, les quals estan en contacte directe amb l'ésser humà. Aquest contacte planteja nous reptes a l'hora d'interactuar amb aquestes aplicacions. Una nova forma d'interacció que ha despertat un especial interès, és el que es relaciona amb la detecció i/o simulació d'estats emocionals. Això ha permès que aquestes aplicacions no només puguen detectar els nostres estats emocionals, sinó que puguen simular i expressar les seues pròpies emocions millorant així l'experiència de l'usuari amb aquestes aplicacions. Per tal de millorar l'experiència humà-màquina, aquesta tesi planteja com a objectiu principal la creació de models emocionals socials, els quals podran ser utilitzats en aplicacions MAS permetent als agents interpretar i/o emular diferents estats emocionals i, a més, emular fenòmens de contagi emocional. Aquests models permetran realitzar simulacions complexes basades en emocions i aplicacions més realistes en dominis com IoT, AIM, SH.Rincón Arango, JA. (2018). Social Emotions in Multiagent Systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/98090TESISCompendi

Enriching mobile interaction with garment-based wearable computing devices

Wearable computing is on the brink of moving from research to mainstream. The first simple products, such as fitness wristbands and smart watches, hit the mass market and achieved considerable market penetration. However, the number and versatility of research prototypes in the field of wearable computing is far beyond the available devices on the market. Particularly, smart garments as a specific type of wearable computer, have high potential to change the way we interact with computing systems. Due to the proximity to the user`s body, smart garments allow to unobtrusively sense implicit and explicit user input. Smart garments are capable of sensing physiological information, detecting touch input, and recognizing the movement of the user. In this thesis, we explore how smart garments can enrich mobile interaction. Employing a user-centered design process, we demonstrate how different input and output modalities can enrich interaction capabilities of mobile devices such as mobile phones or smart watches. To understand the context of use, we chart the design space for mobile interaction through wearable devices. We focus on the device placement on the body as well as interaction modality. We use a probe-based research approach to systematically investigate the possible inputs and outputs for garment based wearable computing devices. We develop six different research probes showing how mobile interaction benefits from wearable computing devices and what requirements these devices pose for mobile operating systems. On the input side, we look at explicit input using touch and mid-air gestures as well as implicit input using physiological signals. Although touch input is well known from mobile devices, the limited screen real estate as well as the occlusion of the display by the input finger are challenges that can be overcome with touch-enabled garments. Additionally, mid-air gestures provide a more sophisticated and abstract form of input. We present a gesture elicitation study to address the special requirements of mobile interaction and present the resulting gesture set. As garments are worn, they allow different physiological signals to be sensed. We explore how we can leverage these physiological signals for implicit input. We conduct a study assessing physiological information by focusing on the workload of drivers in an automotive setting. We show that we can infer the driver´s workload using these physiological signals. Beside the input capabilities of garments, we explore how garments can be used as output. We present research probes covering the most important output modalities, namely visual, auditory, and haptic. We explore how low resolution displays can serve as a context display and how and where content should be placed on such a display. For auditory output, we investigate a novel authentication mechanism utilizing the closeness of wearable devices to the body. We show that by probing audio cues through the head of the user and re-recording them, user authentication is feasible. Last, we investigate EMS as a haptic feedback method. We show that by actuating the user`s body, an embodied form of haptic feedback can be achieved. From the aforementioned research probes, we distilled a set of design recommendations. These recommendations are grouped into interaction-based and technology-based recommendations and serve as a basis for designing novel ways of mobile interaction. We implement a system based on these recommendations. The system supports developers in integrating wearable sensors and actuators by providing an easy to use API for accessing these devices. In conclusion, this thesis broadens the understanding of how garment-based wearable computing devices can enrich mobile interaction. It outlines challenges and opportunities on an interaction and technological level. The unique characteristics of smart garments make them a promising technology for making the next step in mobile interaction

Extending MAM5 Meta-Model and JaCalIVE Framework to Integrate Smart Devices from Real Environments

[EN] This paper presents the extension of a meta-model (MAM5) and a framework based on the model (JaCalIVE) for developing intelligent virtual environments. The goal of this extension is to develop augmented mirror worlds that represent a real and virtual world coupled, so that the virtual world not only reflects the real one, but also complements it. A new component called a smart resource artifact, that enables modelling and developing devices to access the real physical world, and a human in the loop agent to place a human in the system have been included in the meta-model and framework. The proposed extension of MAM5 has been tested by simulating a light control system where agents can access both virtual and real sensor/actuators through the smart resources developed. The results show that the use of real environment interactive elements (smart resource artifacts) in agent-based simulations allows to minimize the error between simulated and real system.This work is partially supported by the TIN2009-13839-C03-01, TIN2011-27652-C03-01, 547CSD2007-00022, COST Action IC0801, FP7-294931 and the FPI grant AP2013-01276 548 awarded to Jaime-Andres Rincon.Rincón Arango, JA.; Poza Luján, JL.; Julian Inglada, VJ.; Posadas Yagüe, JL.; Carrascosa Casamayor, C. (2016). Extending MAM5 Meta-Model and JaCalIVE Framework to Integrate Smart Devices from Real Environments. PLoS ONE. 11(2):1-27. https://doi.org/10.1371/journal.pone.0149665S127112Luck, M., & Aylett, R. (2000). Applying artificial intelligence to virtual reality: Intelligent virtual environments. Applied Artificial Intelligence, 14(1), 3-32. doi:10.1080/088395100117142Barella A, Ricci A, Boissier O, Carrascosa C. MAM5: Multi-Agent Model For Intelligent Virtual Environments. In: 10th European Workshop on Multi-Agent Systems (EUMAS 2012); 2012. p. 16–30.Omicini, A., Ricci, A., & Viroli, M. (2008). Artifacts in the A&A meta-model for multi-agent systems. Autonomous Agents and Multi-Agent Systems, 17(3), 432-456. doi:10.1007/s10458-008-9053-xYu Ch, Nagpal R. Distributed Consensus and Self-Adapting Modular Robots. In: IROS-2008 workshop on Self-Reconfigurable Robots and Applications; 2008. Available from: http://www.isi.edu/robots/iros08wksp/Papers/iros08-wksp-paper.pdfLidoris G, Buss M. A Multi-Agent System Architecture for Modular Robotic Mobility Aids. In: European Robotics Symposium 2006; 2006. p. 15–26. Available from: http://link.springer.com/chapter/10.1007/11681120_2Yu, C.-H., & Nagpal, R. (2010). A Self-adaptive Framework for Modular Robots in a Dynamic Environment: Theory and Applications. The International Journal of Robotics Research, 30(8), 1015-1036. doi:10.1177/0278364910384753Barbero A, González-Rodríguez MS, de Lara J, Alfonseca M. Multi-Agent Simulation of an Educational Collaborative Web System. In: European Simulation and Modelling Conference; 2007. Available from: http://sistemas-humano-computacionais.wikidot.com/local--files/capitulo:colaboracao-auxiliada-por-computador/%5BBarbero%202007%5D%20Multi-Agent%20Simulation%20of%20an%20Educational%20Collaborative%20Web%20System.pdfRanathunga S, Cranefield S, Purvis MK. Interfacing a cognitive agent platform with a virtual world: a case study using Second Life. In: AAMAS; 2011. p. 1181–1182. Available from: http://www.aamas-conference.org/Proceedings/aamas2011/papers/B20.pdfAndreoli R, De Chiara R, Erra U, Scarano V. Interactive 3d environments by using videogame engines. In: Information Visualisation, 2005. Proceedings. Ninth International Conference on. IEEE; 2005. p. 515–520. Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1509124Dignum, F. (2011). Agents for games and simulations. Autonomous Agents and Multi-Agent Systems, 24(2), 217-220. doi:10.1007/s10458-011-9169-2dos Santos C, Osorio F. AdapTIVE: an intelligent virtual environment and its application in e-commerce. In: Computer Software and Applications Conference, 2004. COMPSAC 2004. Proceedings of the 28th Annual International; 2004. p. 468–473 vol.1.Kazemi, A., Fazel Zarandi, M. H., & Moattar Husseini, S. M. (2008). A multi-agent system to solve the production–distribution planning problem for a supply chain: a genetic algorithm approach. The International Journal of Advanced Manufacturing Technology, 44(1-2), 180-193. doi:10.1007/s00170-008-1826-5Dimuro GP, Costa ACdR, Gonçalves LV, Hubner A. Interval-valued Hidden Markov Models for recognizing personality traits in social exchanges in open multiagent systems. Repositório Institucional da Universidade Federal do Rio Grande. 2008;.Woźniak, M., Graña, M., & Corchado, E. (2014). A survey of multiple classifier systems as hybrid systems. Information Fusion, 16, 3-17. doi:10.1016/j.inffus.2013.04.006Jia L, Zhenjiang M. Entertainment Oriented Intelligent Virtual Environment with Agent and Neural Networks. In: IEEE International Workshop on Haptic, Audio and Visual Environments and Games, 2007. HAVE 2007; 2007. p. 90–95.Corchado, E., Woźniak, M., Abraham, A., de Carvalho, A. C. P. L. F., & Snášel, V. (2014). Recent trends in intelligent data analysis. Neurocomputing, 126, 1-2. doi:10.1016/j.neucom.2013.07.001Ricci A, Viroli M, Omicini A. Give agents their artifacts: the A&A approach for engineering working environments in MAS. In: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems; 2007. p. 150. Available from: http://dl.acm.org/citation.cfm?id=1329308Barella, A., Valero, S., & Carrascosa, C. (2009). JGOMAS: New Approach to AI Teaching. IEEE Transactions on Education, 52(2), 228-235. doi:10.1109/te.2008.925764Behrens, T. M., Hindriks, K. V., & Dix, J. (2010). Towards an environment interface standard for agent platforms. Annals of Mathematics and Artificial Intelligence, 61(4), 261-295. doi:10.1007/s10472-010-9215-9Ricci A, Viroli M, Omicini A. A general purpose programming model & technology for developing working environments in MAS. In: 5th International Workshop Programming Multi-Agent Systems(PROMAS 2007); 2007. p. 54–69. Available from: http://lia.deis.unibo.it/~ao/pubs/pdf/2007/promas.pdfChee-Yee Chong, & Kumar, S. P. (2003). Sensor networks: Evolution, opportunities, and challenges. Proceedings of the IEEE, 91(8), 1247-1256. doi:10.1109/jproc.2003.814918Kushner D. The making of arduino. IEEE Spectrum. 2011;26.Schmidt, A., & van Laerhoven, K. (2001). How to build smart appliances? IEEE Personal Communications, 8(4), 66-71. doi:10.1109/98.944006Salzmann C, Gillet D. Smart device paradigm standardization for online labs. In: 4th IEEE Global Engineering Education Conference (EDUCON); 2013.Gonzalez-Jorge, H., Riveiro, B., Vazquez-Fernandez, E., Martínez-Sánchez, J., & Arias, P. (2013). Metrological evaluation of Microsoft Kinect and Asus Xtion sensors. Measurement, 46(6), 1800-1806. doi:10.1016/j.measurement.2013.01.011Cook, D. J., & Das, S. K. (2007). How smart are our environments? An updated look at the state of the art. Pervasive and Mobile Computing, 3(2), 53-73. doi:10.1016/j.pmcj.2006.12.001Compton, M., Barnaghi, P., Bermudez, L., García-Castro, R., Corcho, O., Cox, S., … Taylor, K. (2012). The SSN ontology of the W3C semantic sensor network incubator group. Journal of Web Semantics, 17, 25-32. doi:10.1016/j.websem.2012.05.003Munera, E., Poza-Lujan, J.-L., Posadas-Yagüe, J.-L., Simó-Ten, J.-E., & Noguera, J. (2015). Dynamic Reconfiguration of a RGBD Sensor Based on QoS and QoC Requirements in Distributed Systems. Sensors, 15(8), 18080-18101. doi:10.3390/s150818080Castrillón-Santan, M., Lorenzo-Navarro, J., & Hernández-Sosa, D. (2014). Conteo de personas con un sensor RGBD comercial. Revista Iberoamericana de Automática e Informática Industrial RIAI, 11(3), 348-357. doi:10.1016/j.riai.2014.05.006Rincon JA, Julian V, Carrascosa C. An Emotional-based Hybrid Application for Human-Agent Societies. In: 10th International Conference on Soft Computing Models in Industrial and Environmental Applications. vol. 368; 2015. p. 203–214.Rincon JA, Julian V, Carrascosa C. Applying a Social Emotional Model in Human-Agent Societies. In: Workshop WIHAS’15. Intelligent Human-Agent Societies‥ vol. 524 of CCIS; 2015. p. 377–388.Leccese, F., Cagnetti, M., & Trinca, D. (2014). A Smart City Application: A Fully Controlled Street Lighting Isle Based on Raspberry-Pi Card, a ZigBee Sensor Network and WiMAX. Sensors, 14(12), 24408-24424. doi:10.3390/s141224408Mateevitsi V, Haggadone B, Leigh J, Kunzer B, Kenyon RV. Sensing the environment through SpiderSense. In: Proceedings of the 4th Augmented Human International Conference. ACM; 2013. p. 51–57.Kavitha R, Thiyagarajan N. Distributed Intelligent Street Lamp Monitoring and Control System Based on Zigbee. International Journal of Science and Research (IJSR) PP; p. 2319–7064.Pan, M.-S., Yeh, L.-W., Chen, Y.-A., Lin, Y.-H., & Tseng, Y.-C. (2008). A WSN-Based Intelligent Light Control System Considering User Activities and Profiles. IEEE Sensors Journal, 8(10), 1710-1721. doi:10.1109/jsen.2008.2004294Villarrubia, G., De Paz, J., Bajo, J., & Corchado, J. (2014). Ambient Agents: Embedded Agents for Remote Control and Monitoring Using the PANGEA Platform. Sensors, 14(8), 13955-13979. doi:10.3390/s14081395