Integration of mobile devices in home automation with use of machine learning for object recognition

The concept of smart homes is increasingly expanding and the number of objects we have at home that are connected grows exponentially. The so-called internet of things is increasingly englobing more home devices and the need to control them is also growing. However, there are numerous platforms that integrate numerous protocols and devices in many ways, many of them being unintuitive. Something that we always carry with us is our mobile devices and with the evolution of technology, they have become increasingly powerful and equipped with lots of sensors. One of the bridges to the real world in these devices is the camera and its many potentials. The amount of information gathered can be used in a variety of ways and one topic that has also gathered tremendous relevance is Artificial Intelligence and Machine Learning algorithms. Thus, with the correct processing, data collected by the sensors could be used intuitively to interact with such devices present at home. This dissertation presents the prototype of a system that integrates mobile devices in home automation platforms by detecting objects in the information collected by their cameras, consequently allowing the user to interact with them in an intuitive way. The main contribution of the work developed is the non-explored until then integration, in the home automation context, of cutting-edge algorithms capable of easily outperforming humans into analyzing and processing data acquired by our mobile devices. Throughout the dissertation the referred concepts are explored as well as the potentiality of this integration and the results obtained.O conceito de casas inteligentes está cada vez mais em constante expansão e o número de objetos que temos em casa que estão conectados cresce exponencialmente. A tão chamada internet das coisas abrange cada vez mais dispositivos domésticos crescendo também a necessidade de os controlar. No entanto existem inúmeras plataformas que integram inúmeros protocolos e dispositivos, de inúmeras maneiras, muitas delas pouco intuitivas. Algo que transportamos sempre connosco são os nossos dispositivos móveis e com a evolução da tecnologia, estes vieram-se tornando cada vez mais potentes e munidos de variados sensores. Uma das portas para o mundo real nestes dispositivos é a câmara e as suas inúmeras potencialidades. Uma temática que tem vindo também a ganhar enorme relevância é a Inteligência Artificial e os algoritmos de Aprendizagem Máquina. Assim, com o processamento correto os dados recolhidos pelos sensores poderiam ser utilizados de maneira intuitiva para interagir com os tais dispositivos presentes em casa. Nesta dissertação é apresentado o protótipo de um sistema que integra os dispositivos móveis nas plataformas de automação de casas através da deteção de objetos na informação recolhida pela câmara dos mesmos, permitindo assim ao utilizador interagir com eles de forma intuitiva. A principal contribuição do trabalho desenvolvido é a integração não explorada até então, no contexto da automação de casas, de algoritmos de ponta capazes de superar facilmente os seres humanos na análise e processamento de dados adquiridos pelos nossos dispositivos móveis. Ao longo da dissertação são explorados os conceitos referidos, bem como a potencialidade dessa integração e os resultados obtidos

Automated license plate recognition for resource-constrained environments

The incorporation of deep-learning techniques in embedded systems has enhanced the capabilities of edge computing to a great extent. However, most of these solutions rely on high-end hardware and often require a high processing capacity, which cannot be achieved with resource-constrained edge computing. This study presents a novel approach and a proof of concept for a hardware-efficient automated license plate recognition system for a constrained environment with limited resources. The proposed solution is purely implemented for low-resource edge devices and performed well for extreme illumination changes such as day and nighttime. The generalisability of the proposed models has been achieved using a novel set of neural networks for different hardware configurations based on the computational capabilities and low cost. The accuracy, energy efficiency, communication, and computational latency of the proposed models are validated using different license plate datasets in the daytime and nighttime and in real time. Meanwhile, the results obtained from the proposed study have shown competitive performance to the state-of-the-art server-grade hardware solutions as well

SLAM research for port AGV based on 2D LIDAR

With the increase in international trade, the transshipment of goods at international container ports is very busy. The AGV (Automated Guided Vehicle) has been used as a new generation of automated container horizontal transport equipment. The AGV is an automated unmanned vehicle that can work 24 hours a day, increasing productivity and reducing labor costs compared to using container trucks. The ability to obtain information about the surrounding environment is a prerequisite for the AGV to automatically complete tasks in the port area. At present, the method of AGV based on RFID tag positioning and navigation has a problem of excessive cost. This dissertation has carried out a research on applying light detection and ranging (LIDAR) simultaneous localization and mapping (SLAM) technology to port AGV. In this master's thesis, a mobile test platform based on a laser range finder is developed to scan 360-degree environmental information (distance and angle) centered on the LIDAR and upload the information to a real-time database to generate surrounding environmental maps, and the obstacle avoidance strategy was developed based on the acquired information. The effectiveness of the platform was verified by the experiments from multiple scenarios. Then based on the first platform, another experimental platform with encoder and IMU sensor was developed. In this platform, the functionality of SLAM is enabled by the GMapping algorithm and the installation of the encoder and IMU sensor. Based on the established environment SLAM map, the path planning and obstacle avoidance functions of the platform were realized.Com o aumento do comércio internacional, o transbordo de mercadorias em portos internacionais de contentores é muito movimentado. O AGV (“Automated Guided Vehicle”) foi usado como uma nova geração de equipamentos para transporte horizontal de contentores de forma automatizada. O AGV é um veículo não tripulado automatizado que pode funcionar 24 horas por dia, aumentando a produtividade e reduzindo os custos de mão-de-obra em comparação com o uso de camiões porta-contentores. A capacidade de obter informações sobre o ambiente circundante é um pré-requisito para o AGV concluir automaticamente tarefas na área portuária. Atualmente, o método de AGV baseado no posicionamento e navegação de etiquetas RFID apresenta um problema de custo excessivo. Nesta dissertação foi realizada uma pesquisa sobre a aplicação da tecnologia LIDAR de localização e mapeamento simultâneo (SLAM) num AGV. Uma plataforma de teste móvel baseada num telémetro a laser é desenvolvida para examinar o ambiente em redor em 360 graus (distância e ângulo), centrado no LIDAR, e fazer upload da informação para uma base de dados em tempo real para gerar um mapa do ambiente em redor. Uma estratégia de prevenção de obstáculos foi também desenvolvida com base nas informações adquiridas. A eficácia da plataforma foi verificada através da realização de testes com vários cenários e obstáculos. Por fim, com base na primeira plataforma, uma outra plataforma experimental com codificador e sensor IMU foi também desenvolvida. Nesta plataforma, a funcionalidade do SLAM é ativada pelo algoritmo GMapping e pela instalação do codificador e do sensor IMU. Com base no estabelecimento do ambiente circundante SLAM, foram realizadas as funções de planeamento de trajetória e prevenção de obstáculos pela plataforma

UWB system and algorithms for indoor positioning

This research work presents of study of ultra-wide band (UWB) indoor positioning considering different type of obstacles that can affect the localization accuracy. In the actual warehouse, a variety of obstacles including metal, board, worker and other obstacles will have NLOS (non-line-of-sight) impact on the positioning of the logistics package, which influence the measurement of the distance between the logistics package and the anchor , thereby affecting positioning accuracy. A new developed method attempts to improve the accuracy of UWB indoor positioning, through and improved positioning algorithm and filtering algorithm. In this project, simulate the warehouse environment in the laboratory, several simulation proves that the used Kalman filter algorithm and Markov algorithm can effectively reduce the error of NLOS. Experimental validation is carried out considering a mobile tag mounted on a robot platform.Este trabalho de pesquisa apresenta um estudo de posicionamento de banda ultra-larga (UWB) em ambientes internos considerando diferentes tipos de obstáculos que podem afetar a precisão de localização. No armazém real, uma variedade de obstáculos incluindo metal, placa, trabalhador e outros obstáculos terão impacto NLOS (não linha de visão) no posicionamento do pacote logístico, o que influencia a medição da distância entre o pacote logístico e a âncora, afetando assim a precisão do posicionamento. Um novo método desenvolvido tenta melhorar a precisão do posicionamento interno UWB, através de um algoritmo de posicionamento e algoritmo de filtragem aprimorados. Neste projeto, para simular o ambiente de warehouse em laboratório, diversas simulações comprovam que o algoritmo de filtro de Kalman e o algoritmo de Markov usados podem efetivamente reduzir o erro de NLOS. A validação experimental é realizada considerando um tag móvel montado em uma plataforma de robô

Reading Robot

Presently, there is an insufficient availability of human experts to assist students in reading competency and comprehension. Our team’s goal was to create an improved socially assistive robot for use by therapists, teachers, and parents to help children and adults develop reading skills while they do not have access to specialists. HAPI is a socially assistive robot that we created with the goal of helping students practice their reading comprehension skills. HAPI enables a student to improve their reading skills without an educator present, while enabling educators to review the student\u27s performance remotely. Design constraints included: physical size, weight, duration of user engagement, chamber temperature, device memory size, and durability. From three different initial concepts, we chose HAPI the Librarian (Hand Articulating Phone Interface), named after the idea of securing the phone in the robot’s hands, much like a librarian reading during story-time. By creating an initial prototype, we were able to verify the functionality of several key design features, including the phone tilt mechanism and the antenna tilt mechanism. In our analysis, we used calculation to find the ideal DC motor speed and maximum current draw. After exploring possibilities of using a third-party vendor to print the robot, we concluded it was not the most economical option. We were able to print the robot in one of the team member’s homes using an Ultimaker 2+ printer. Our printed circuit board (PCB) design was manufactured via a third-party vendor, Sunstone circuits. Components were ordered online, and the robot was assembled by team members. In our final design, main components include the phone tilt mechanism, antenna wiggle mechanism, LED screen, and android phone. The phone user interface guides the user through reading a passage, then answering reading comprehension questions. The robot gives positive or constructive feedback based on the correctness of response through audio, antenna movement, LED screen changes, and changes in the phone screen. Major electrical components include the Raspberry Pi 4, which controls the motors and power; the Power Distribution Board (PDB), which provides power isolation; and the LED Array, which displays the robot facial expressions. The entire assembly was designed for ease of assembly and disassembly. To verify the functionality of the device, we tested the electrical wiring integrity, durability, and steady state chamber temperature. We also completed user testing on 5 participants, ages 6 through 9. Robot usability was accessed over video call during the session and through a participant survey. Based on the performance, we made changes to the final design. The team plans on passing off the robot to the project sponsors so development of the robot can continue. Next steps will likely include more user testing, as well as testing focused specifically on students with speech impairments. To improve the current system, we recommend improving the software for reading analysis and making changes to increase the compatibility and accessibility of the device

Vacancy state detector oriented to convolutional neural network, background subtraction and embedded systems

Dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáMuch has been discussed recently related to population ascension, the reasons for this event, and, in particular, the aspects of society affected. Over the years, the city governments realized a higher level of growth, mainly in terms of urban scale, technology, and individuals numbers. It comprises improvements and investments in their structure and policies, motivated by improving conditions in population live quality and reduce environmental, energy, fuel, time, and money resources, besides population living costs, including the increasing demand for parking structures accessible to the general or private-public, and a waste of substantial daily time and fuel, disturbing the population routinely. Therefore, one way to achieve that challenge is focused on reducing energy, money, and time costs to travel to work or travel to another substantial location. That work presents a robust, and low computational power Smart Parking system adaptive to several environments changes to detect and report vacancy states in a parking space oriented to Deep Learning, and Embedded Systems. This project consists of determining the parking vacancy status through statistical and image processing methods, creates a robust image data set, and the Convolutional Neural Network model focused on predict three final classes. In order to save computational power, this approach uses the Background Subtraction based on the Mixture of Gaussian method, only updating parking space status, in which large levels of motion are detected. The proposed model presents 94 percent of precision at the designed domain.Muito se discutiu recentemente sobre a ascensão populacional, as razões deste evento e, em particular, os aspectos da sociedade afetados. Ao longo dos anos, os governos perceberam um grande nível de crescimento, principalmente em termos de escala urbana, tecnologia e número de indivíduos. Este fato deve-se a melhorias e investimentos na estrutura urbana e políticas motivados por melhorar as condições de qualidade de vida da população e reduzir a utilização de recursos ambientais, energéticos, combustíveis, temporais e monetários, além dos custos de vida da população, incluindo a crescente demanda por estruturas de estacionamento acessíveis ao público em geral ou público-privado. Portanto, uma maneira de alcançar esse desafio é manter a atenção na redução de custos de energia, dinheiro e tempo para viajar para o trabalho ou para outro local substancial. Esse trabalho apresenta um sistema robusto de Smart Parking, com baixo consumo computacional, adaptável a diversas mudanças no ambiente observado para detectar e relatar os estados das vagas de estacionamento, orientado por Deep Learning e Embedded Systems. Este projeto consiste em determinar o status da vaga de estacionamento por meio de métodos estatísticos e de processamento de imagem, criando um conjunto robusto de dados e um modelo de Rede Neuronal Convolucional com foco na previsão de três classes finais. A fim de reduzir consumo computacional, essa abordagem usa o método de Background Subtraction, somente atualizando o status do espaço de estacionamento em que grandes níveis de movimento são detectados. O modelo proposto apresenta 94 porcento da precisão no domínio projetado

Proceedings of the 5th Baltic Mechatronics Symposium - Espoo April 17, 2020

The Baltic Mechatronics Symposium is annual symposium with the objective to provide a forum for young scientists from Baltic countries to exchange knowledge, experience, results and information in large variety of fields in mechatronics. The symposium was organized in co-operation with Taltech and Aalto University. Due to Coronavirus COVID-19 the symposium was organized as a virtual conference. The content of the proceedings1. Monitoring Cleanliness of Public Transportation with Computer Vision2. Device for Bending and Cutting Coaxial Wires for Cryostat in Quantum Computing3. Inertial Measurement Method and Application for Bowling Performance Metrics4. Mechatronics Escape Room5. Hardware-In-the-Loop Test Setup for Tuning Semi-Active Hydraulic Suspension Systems6. Newtonian Telescope Design for Stand-off Laser Induced Breakdown Spectroscopy7. Simulation and Testing of Temperature Behavior in Flat Type Linear Motor Carrier8. Powder Removal Device for Metal Additive Manufacturing9. Self-Leveling Spreader Beam for Adjusting the Orientation of an Overhead Crane Loa

Development of a smart weed detector and selective herbicide sprayer

Abstract: The fourth industrial revolution has brought about tremendous advancements in various sectors of the economy including the agricultural domain. Aimed at improving food production and alleviating poverty, these technological advancements through precision agriculture has ushered in optimized agricultural processes, real-time analysis and monitoring of agricultural data. The detrimental effects of applying agrochemicals in large or hard-to-reach farmlands and the need to treat a specific class of weed with a particular herbicide for effective weed elimination gave rise to the necessity of this research work...M.Ing. (Mechanical Engineering

Research on port AGV composite positioning based on UWB/RFID

In recent years, ports in various countries have successively carried out research and application of fully automated terminal. The terminal adopts the "Double car shore bridge + AGV + ARMG" automation process, which is the most widely used and relatively mature fully automated solution. At present, the AGV navigation of the terminal is based on RFID magnetic nail positioning and the accuracy is good. However, nowadays UWB technology has become the most popular technology in ranging and positioning. The research in this work is based on UWB/RFID composite positioning, which is mainly used for the specific localization tasks in the port and it can accurately locate the position of the AGV. This MSc work studies the UWB positioning system first and then researches the traditional 3D positioning algorithm. Importance contribution expressed by 3D TOA localization algorithm. For RFID system, this connection between the reader and the carrier is designed, and the reference tag is buried. At last, data-based on RFID localization algorithm in scene analysis method is adopted for positioning. Secondly, the basis of the composite positioning system is data fusion technology. The most widely used and mature fusion algorithm is the Kalman filter algorithm and Particle filter. Finally, the experimental analysis of UWB and RFID composite positioning system is implemented. The results indicate that UWB and RFID composite positioning system can reduce the cost of the positioning system. Higher positioning accuracy and robustness are characterizing the developed system.Nos últimos anos, portos de vários países realizaram sucessivamente pesquisas e aplicações de terminais totalmente automatizados. O terminal adota o processo de automação "Double car shore bridge + AGV + ARMG", que é a solução totalmente automatizada mais amplamente utilizada e relativamente madura. Atualmente, a navegação AGV do terminal é baseada no posicionamento da etiqueta RFID e a precisão é boa. No entanto, hoje em dia, a tecnologia UWB tornou-se na tecnologia mais popular relativamente ao alcance e posicionamento. A pesquisa neste trabalho é baseada no posicionamento composto por UWB / RFID, usado principalmente para tarefas de localização específicas nos portos, podendo desta forma localizar-se com precisão a posição do AGV. Este projeto de mestrado estuda em primeiro lugar o sistema de posicionamento UWB, e depois um algoritmo tradicional de posicionamento 3D. A contribuição da importância expressa pelo algoritmo de posicionamento “time of arrival” (TOA) 3D foi proposta. Para o sistema de posicionamento RFID, a conexão entre o leitor e a transportadora é projetada e a etiqueta de referência é ocultada. Por fim, o algoritmo de “k-nearest neighbor” baseado numa base de dados e no método de análise de cena é adotado para realizar o posicionamento. Em segundo lugar, a base do sistema de posicionamento composto é a tecnologia de fusão de dados. O algoritmo de fusão mais amplamente utilizado e maduro é o algoritmo de filtro Kalman e o filtro de partículas. Finalmente, é realizada a análise experimental do sistema de posicionamento composto UWB e RFID. Os resultados experimentais mostram que o sistema de posicionamento composto UWB e RFID pode reduzir o custo do sistema de posicionamento. O sistema desenvolvido é caracterizado por uma maior precisão de posicionamento e robustez