Human-document interaction systems: a new frontier for document image analysis

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.All indications show that paper documents will not cede in favour of their digital counterparts, but will instead be used increasingly in conjunction with digital information. An open challenge is how to seamlessly link the physical with the digital – how to continue taking advantage of the important affordances of paper, without missing out on digital functionality. This paper presents the authors’ experience with developing systems for Human-Document Interaction based on augmented document interfaces and examines new challenges and opportunities arising for the document image analysis field in this area. The system presented combines state of the art camera-based document image analysis techniques with a range of complementary technologies to offer fluid Human-Document Interaction. Both fixed and nomadic setups are discussed that have gone through user testing in real-life environments, and use cases are presented that span the spectrum from business to educational applications.Peer ReviewedPostprint (author's final draft

Robots and IoT devices for assistive automation

In recent years, we live every day in an ever more connected world. This phenomenon has developed the technologies of the Internet of Things (IoT). At the same time, advances in robotics now make it possible to have autonomous service robots in their mobility and in the accomplishment of their missions. So far the question is whether it might be useful to unite these two areas. Just as humans do, how do these service robots integrate the objects of IoT into their environment to facilitate the fulfillment of their mission? This report focuses on how it is possible to integrate IoT objects into the environment of a PAL Robotics robot, TIAGo. It is the result of twelve weeks of internship within the laboratory of Perception and Manipulation of the Institut de Rob`otica i Inform`atica Industrial of Barcelona.Postprint (updated version

Security Analysis of Home IoT Network

Táto práca sa zaoberá problematikou bezpečnosti domácej IoT siete. Zameriava sa na bezpečnostné problémy komerčných IoT zariadení a ich automatizačných systémov. V samotnej práci boli najskôr analýzou zistené konkrétne bezpečnostné hrozby v sieti. Pre riešenie týchto hrozieb, boli nahradené pôvodné automatizačné systémy a firmware na zariadeniach open-source implementáciami, ktoré boli nakonfigurované tak aby riešili konkrétne objavené hrozby. Aj keď sa v sieti s open-source implementáciou nachádza niekoľko bezpečnostných hrozieb, podarilo sa eliminovať veľkú časť objavených bezpečnostných hrozieb. Výsledky tejto práce umožňujú nakonfigurovať domácu IoT sieť použitím open-source automatizačného systému a firmware a dokazujú, že takáto sieť je bezpečnejšia oproti sieti kde sa používaju pôvodné, výrobcom dodané automatizačné systémy a firmware na zariadeniach.This work focuses on the issue of the  security in home IoT network. Specifically, it focuses on the security issues of commercial IoT devices and their automation systems. In the work itself, it was firstly analyzed and identified specific threats specific for our network. To address these threats, the original automation systems and firmware on the devices were replaced by open-source implementations that were configured to address discovered threats. Although, there are still several security issues in a network with an open-source implementation, a large part of the security issues which were discovered has been eliminated.  The results of this work allows to configure a home IoT network using an opens-source automation system and firmware and proves that such a network is more secure than the network where the original factory-supplied automation systems and firmware on devices are used.

A smart home environment to support safety and risk monitoring for the elderly living independently

The elderly prefer to live independently despite vulnerability to age-related challenges. Constant monitoring is required in cases where the elderly are living alone. The home environment can be a dangerous environment for the elderly living independently due to adverse events that can occur at any time. The potential risks for the elderly living independently can be categorised as injury in the home, home environmental risks and inactivity due to unconsciousness. The main research objective was to develop a Smart Home Environment (SHE) that can support risk and safety monitoring for the elderly living independently. An unobtrusive and low cost SHE solution that uses a Raspberry Pi 3 model B, a Microsoft Kinect Sensor and an Aeotec 4-in-1 Multisensor was implemented. The Aeotec Multisensor was used to measure temperature, motion, lighting, and humidity in the home. Data from the multisensor was collected using OpenHAB as the Smart Home Operating System. The information was processed using the Raspberry Pi 3 and push notifications were sent when risk situations were detected. An experimental evaluation was conducted to determine the accuracy with which the prototype SHE detected abnormal events. Evaluation scripts were each evaluated five times. The results show that the prototype has an average accuracy, sensitivity and specificity of 94%, 96.92% and 88.93% respectively. The sensitivity shows that the chance of the prototype missing a risk situation is 3.08%, and the specificity shows that the chance of incorrectly classifying a non-risk situation is 11.07%. The prototype does not require any interaction on the part of the elderly. Relatives and caregivers can remotely monitor the elderly person living independently via the mobile application or a web portal. The total cost of the equipment used was below R3000

A smart home environment to support safety and risk monitoring for the elderly living independently

The elderly prefer to live independently despite vulnerability to age-related challenges. Constant monitoring is required in cases where the elderly are living alone. The home environment can be a dangerous environment for the elderly living independently due to adverse events that can occur at any time. The potential risks for the elderly living independently can be categorised as injury in the home, home environmental risks and inactivity due to unconsciousness. The main research objective was to develop a Smart Home Environment (SHE) that can support risk and safety monitoring for the elderly living independently. An unobtrusive and low cost SHE solution that uses a Raspberry Pi 3 model B, a Microsoft Kinect Sensor and an Aeotec 4-in-1 Multisensor was implemented. The Aeotec Multisensor was used to measure temperature, motion, lighting, and humidity in the home. Data from the multisensor was collected using OpenHAB as the Smart Home Operating System. The information was processed using the Raspberry Pi 3 and push notifications were sent when risk situations were detected. An experimental evaluation was conducted to determine the accuracy with which the prototype SHE detected abnormal events. Evaluation scripts were each evaluated five times. The results show that the prototype has an average accuracy, sensitivity and specificity of 94%, 96.92% and 88.93% respectively. The sensitivity shows that the chance of the prototype missing a risk situation is 3.08%, and the specificity shows that the chance of incorrectly classifying a non-risk situation is 11.07%. The prototype does not require any interaction on the part of the elderly. Relatives and caregivers can remotely monitor the elderly person living independently via the mobile application or a web portal. The total cost of the equipment used was below R3000

Development and implementation of a domotic system based in open source software for a single family home

Grado en Ingeniería de Tecnologías de Telecomunicació

Design, analysis and control of solar heating system with seasonal thermal energy storage

The majority of the electricity consumption in Canadian single-family house is for space heating and water heating. Currently, around 99% house in Canada using the conventional grid electricity for those purposes. To utilize the sun’s free energy for space heating and domestic water heating, first a sessional solar thermal energy storage system (SSTES) has been designed and investigated to determine their thermal and electrical performance of a house consists of four persons in the Canadian climate environment. The detailed mathematical formulation and sizing of each SSTES system component have been developed. Similarly, the components mathematical modelling, sizing of solar collector-based TES system, a hybrid solar Photovoltaic thermal (PV/T) based TES system, a Photovoltaic based TES system has been designed. To validate the feasibility and numerical studies of the developed STES configurations, all configurations have been simulated in a professional thermal simulation software named as PolySun, and only solar collector-based TES system has been designed and simulated at MATLAB/Simulink environment packages. The proposed TES system performance have been compared with the existing conventional system. All configurations have been tested using the solar radiation and other weather data of St. John’s city/NL city in Canada. The main objective to design a suitable TES system for space heating and water heating so that the residence can save high monthly electricity bill. For experimental validation, an open source IoT platform named openHAB smart home automation is used as a home server, an ESP32 Thing microcontroller board has been used as Microcontroller unit where all sensors and output devices (relays, thermostat settings channels) are connected for data acquisition and control. The proposed setup is able to monitor the TES system parameters, and able to control locally/remotely and manually/automatically. The proposed system is the low cost, low power consumption prototype which will be a commercial solution of TES system monitoring and remote control

Mobile Pen and Paper Interaction

Although smartphones, tablets and other mobile devices become increasingly popular, pen and paper continue to play an important role in mobile settings, such as note taking or creative discussions. However, information on paper documents remains static and usage practices involving sharing, researching, linking or in any other way digitally processing information on paper are hindered by the gap between the digital and physical worlds. A considerable body of research has leveraged digital pen technology in order to overcome this problem with respect to static settings, however, systematically neglecting the mobile domain. Only recently, several approaches began exploring the mobile domain and developing initial insights into mobile pen-and-paper interaction (mPPI), e.g., to publish digital sketches, [Cowan et al., 2011], link paper and digital artifacts, [Pietrzak et al., 2012] or compose music, [Tsandilas, 2012]. However, applications designed to integrate the most common mobile tools pen, paper and mobile devices, thereby combining the benefits of both worlds in a hybrid mPPI ensemble, are hindered by the lack of supporting infrastructures and limited theoretical understanding of interaction design in the domain. This thesis advances the field by contributing a novel infrastructural approach toward supporting mPPI. It allows applications employing digital pen technology in controlling interactive functionality while preserving mobile characteristics of pen and paper. In addition, it contributes a conceptual framework of user interaction in the domain suiting to serve as basis for novel mPPI toolkits. Such toolkits ease development of mPPI solutions by focusing on expressing interaction rather than designing user interfaces by means of rigid widget sets. As such, they provide the link between infrastructure and interaction in the domain. Lastly, this thesis presents a novel, empirically substantiated theory of interaction in hybrid mPPI ensembles. This theory informs interaction design of mPPI, ultimately allowing to develop compelling and engaging interactive systems employing this modality