Computer Vision Algorithms for Mobile Camera Applications

Wearable and mobile sensors have found widespread use in recent years due to their ever-decreasing cost, ease of deployment and use, and ability to provide continuous monitoring as opposed to sensors installed at fixed locations. Since many smart phones are now equipped with a variety of sensors, including accelerometer, gyroscope, magnetometer, microphone and camera, it has become more feasible to develop algorithms for activity monitoring, guidance and navigation of unmanned vehicles, autonomous driving and driver assistance, by using data from one or more of these sensors. In this thesis, we focus on multiple mobile camera applications, and present lightweight algorithms suitable for embedded mobile platforms. The mobile camera scenarios presented in the thesis are: (i) activity detection and step counting from wearable cameras, (ii) door detection for indoor navigation of unmanned vehicles, and (iii) traffic sign detection from vehicle-mounted cameras. First, we present a fall detection and activity classification system developed for embedded smart camera platform CITRIC. In our system, the camera platform is worn by the subject, as opposed to static sensors installed at fixed locations in certain rooms, and, therefore, monitoring is not limited to confined areas, and extends to wherever the subject may travel including indoors and outdoors. Next, we present a real-time smart phone-based fall detection system, wherein we implement camera and accelerometer based fall-detection on Samsung Galaxy S™ 4. We fuse these two sensor modalities to have a more robust fall detection system. Then, we introduce a fall detection algorithm with autonomous thresholding using relative-entropy within the class of Ali-Silvey distance measures. As another wearable camera application, we present a footstep counting algorithm using a smart phone camera. This algorithm provides more accurate step-count compared to using only accelerometer data in smart phones and smart watches at various body locations. As a second mobile camera scenario, we study autonomous indoor navigation of unmanned vehicles. A novel approach is proposed to autonomously detect and verify doorway openings by using the Google Project Tango™ platform. The third mobile camera scenario involves vehicle-mounted cameras. More specifically, we focus on traffic sign detection from lower-resolution and noisy videos captured from vehicle-mounted cameras. We present a new method for accurate traffic sign detection, incorporating Aggregate Channel Features and Chain Code Histograms, with the goal of providing much faster training and testing, and comparable or better performance, with respect to deep neural network approaches, without requiring specialized processors. Proposed computer vision algorithms provide promising results for various useful applications despite the limited energy and processing capabilities of mobile devices

Iranian cashes recognition using mobile

In economical societies of today, using cash is an inseparable aspect of human life. People use cashes for marketing, services, entertainments, bank operations and so on. This huge amount of contact with cash and the necessity of knowing the monetary value of it caused one of the most challenging problems for visually impaired people. In this paper we propose a mobile phone based approach to identify monetary value of a picture taken from cashes using some image processing and machine vision techniques. While the developed approach is very fast, it can recognize the value of cash by average accuracy of about 95% and can overcome different challenges like rotation, scaling, collision, illumination changes, perspective, and some others.Comment: arXiv #13370

The Visual Social Distancing Problem

One of the main and most effective measures to contain the recent viral outbreak is the maintenance of the so-called Social Distancing (SD). To comply with this constraint, workplaces, public institutions, transports and schools will likely adopt restrictions over the minimum inter-personal distance between people. Given this actual scenario, it is crucial to massively measure the compliance to such physical constraint in our life, in order to figure out the reasons of the possible breaks of such distance limitations, and understand if this implies a possible threat given the scene context. All of this, complying with privacy policies and making the measurement acceptable. To this end, we introduce the Visual Social Distancing (VSD) problem, defined as the automatic estimation of the inter-personal distance from an image, and the characterization of the related people aggregations. VSD is pivotal for a non-invasive analysis to whether people comply with the SD restriction, and to provide statistics about the level of safety of specific areas whenever this constraint is violated. We then discuss how VSD relates with previous literature in Social Signal Processing and indicate which existing Computer Vision methods can be used to manage such problem. We conclude with future challenges related to the effectiveness of VSD systems, ethical implications and future application scenarios.Comment: 9 pages, 5 figures. All the authors equally contributed to this manuscript and they are listed by alphabetical order. Under submissio

RIDI: Robust IMU Double Integration

This paper proposes a novel data-driven approach for inertial navigation, which learns to estimate trajectories of natural human motions just from an inertial measurement unit (IMU) in every smartphone. The key observation is that human motions are repetitive and consist of a few major modes (e.g., standing, walking, or turning). Our algorithm regresses a velocity vector from the history of linear accelerations and angular velocities, then corrects low-frequency bias in the linear accelerations, which are integrated twice to estimate positions. We have acquired training data with ground-truth motions across multiple human subjects and multiple phone placements (e.g., in a bag or a hand). The qualitatively and quantitatively evaluations have demonstrated that our algorithm has surprisingly shown comparable results to full Visual Inertial navigation. To our knowledge, this paper is the first to integrate sophisticated machine learning techniques with inertial navigation, potentially opening up a new line of research in the domain of data-driven inertial navigation. We will publicly share our code and data to facilitate further research

PlaceRaider: Virtual Theft in Physical Spaces with Smartphones

As smartphones become more pervasive, they are increasingly targeted by malware. At the same time, each new generation of smartphone features increasingly powerful onboard sensor suites. A new strain of sensor malware has been developing that leverages these sensors to steal information from the physical environment (e.g., researchers have recently demonstrated how malware can listen for spoken credit card numbers through the microphone, or feel keystroke vibrations using the accelerometer). Yet the possibilities of what malware can see through a camera have been understudied. This paper introduces a novel visual malware called PlaceRaider, which allows remote attackers to engage in remote reconnaissance and what we call virtual theft. Through completely opportunistic use of the camera on the phone and other sensors, PlaceRaider constructs rich, three dimensional models of indoor environments. Remote burglars can thus download the physical space, study the environment carefully, and steal virtual objects from the environment (such as financial documents, information on computer monitors, and personally identifiable information). Through two human subject studies we demonstrate the effectiveness of using mobile devices as powerful surveillance and virtual theft platforms, and we suggest several possible defenses against visual malware

Advancing prevention of sexually transmitted infections through point-of-care testing : target product profiles and landscape analysis

Objectives: Advancing the field of point-of-care testing (POCT) for STIs can rapidly and substantially improve STI control and prevention by providing targeted, essential STI services (case detection and screening). POCT enables definitive diagnosis and appropriate treatment in a single visit and home and community-based testing. Methods: Since 2014, the WHO Department of Reproductive Health and Research, in collaboration with technical partners, has completed four landscape analyses of promising diagnostics for use at or near the point of patient care to detect syphilis, Neisseria gonorrhoeae, Chlamydia trachomatis, Trichomonas vaginalis and the human papillomavirus. The analyses comprised a literature review and interviews. Two International Technical Consultations on STI POCTs (2014 and 2015) resulted in the development of target product profiles (TPP). Experts in STI microbiology, laboratory diagnostics, clinical management, public health and epidemiology participated in the consultations with representation from all WHO regions. Results: The landscape analysis identified diagnostic tests that are either available on the market, to be released in the near future or in the pipeline. The TPPs specify 28 analytical and operational characteristics of POCTs for use in different populations for surveillance, screening and case management. None of the tests that were identified in the landscape analysis met all of the targets of the TPPs. Conclusion: More efforts of the global health community are needed to accelerate access to affordable quality-assured STI POCTs, particularly in low-and middle-income countries, by supporting the development of new diagnostic platforms as well as strengthening the validation and implementation of existing diagnostics according to internationally endorsed standards and the best available evidence

Smart bus stop: people counting in a multi-view camera environment

As paragens de autocarros nos dias de hoje tem de estar cada vez mais ao serviço dos utentes, esta dissertação explora as ideias fundamentais sobre o que deve ser uma paragem de autocarro inteligente, reunindo num texto os conceitos mais utilizados e as mais recentes tecnologias sobre este tópico. Os fundamentos do que é uma paragem de autocarro inteligente são explorados, bem como a arquitetura de todo o sistema, não só a paragem propriamente dita. Ao analisar a bibliografia já existentes compreende-se que a paragem de autocarro não é uma entidade totalmente independente, pois esta está dependente de informação vinda de variadíssimas fontes. Assim sendo, a paragem de autocarro inteligente será um subsistema de um sistema muito mais complexo, composto pela própria paragem, pelo autocarro e por uma central. Em que a comunicação flui entre estes de forma a manter toda a informação do sistema atualizada em tempo real. O autocarro recolherá informação, como quantos passageiros tem abordo e a sua localização geográfica por exemplo. A central receberá toda a informação de todos os autocarros existentes assim como de todas as paragens de autocarro existentes. Por sua vez a paragem de autocarro, recolherá dados também, tais como quantas pessoas estão na paragem, temperatura, humidade, emissões de dióxido de carbono, ruido, entre outros. A paragem de autocarro deverá contar com um conjunto de interfaces de comunicação, tais como Bluetooth e/ou NFC, hi-fi e RFID ou Beacons, para que possam ser feitas comunicações com os utilizadores, com os autocarros e com a central. Deverá ter também ecrãs interativos que poderão ser acedidos usando gestos e/ou toque e/ou voz para que se possam efetuar as ações pretendidas. A informação não será apenas transmitida nos ecrãs interativos, será transmitida também através de som. A informação contida na paragem pode ser de todo o tipo, desde as rotas, horários, posição atual do próximo autocarro, assim como o número do mesmo, publicidade animada, etc. A paragem conta também com outras funcionalidades como conectores onde se possam carregar dispositivos móveis, aquecimento, iluminação controlada face à afluência de utilizadores e horário, um sistema de armazenamento de energia pois deverá contar com fontes de energia renováveis para que possa ser o mais autossustentável possível, e obviamente câmeras de vigilância para segurança dos utilizadores. Sendo o principal objetivo deste trabalho, o desenvolvimento de um algoritmo capaz de contar quantas pessoas se encontram na paragem de autocarro, através do processamento das imagens vindas de várias câmaras, o foco principal é explorar as tecnologias de visão computacional e como estas podem ser utilizadas dentro do conceito da paragem de autocarro inteligente. Uma vez que o mundo da visão computacional evoluiu muito nos últimos anos e as suas aplicações são quase ilimitadas, dai a sua implementação nas mais diversas áreas, como reconstrução de cenários, deteção de eventos, monitorização de vídeo, reconhecimento de objetos, estimativa de movimento, restauração de imagem, etc. Ao combinar os diferentes algoritmos das diferentes aplicações, podem ser criadas ferramentas mais poderosas. Assim sendo o algoritmo desenvolvido utiliza redes neuronais convulsionais para detetar todas as pessoas de uma imagem, devolvendo uma região de interesse. Essa região de interesse é processada em busca de caras e caso estas existam essa informação é guardada no perfil da pessoa. Isto é possível através da utilização de reconhecimento facial, que utiliza um algoritmo de Deep Learning (DL). Essa região de interesse também é convertida para uma escala de cinzentos e posteriormente para uma matriz, essa matriz será também guardada no perfil do utilizador. Está informação é necessária para que se possa treinar um modelo que utiliza algoritmos de aprendizagem de máquina (Support Vector Machine - SVM). Os algoritmos de DL e SVM são necessários para que se possa fazer a identificação dos utilizadores a cada imagem e para que se possa cruzar os vários perfis vindos das várias origens, para que possa eliminar os perfis repetidos. Com isto a mesma pessoa é contada as vezes que apareça nas imagens, em função do número de câmeras existentes na paragem. Assim sendo é preciso eliminar essas repetições de forma a ter um número de pessoas correto. Num ambiente controlado o algoritmo proposto tem uma taxa de sucesso elevada, praticamente sem falhas, mas quando testado no ambiente para o qual foi desenhado já não é bem assim, ou porque numa paragem de autocarro as pessoas estão em contante movimento ou porque ficam na frente umas das outras e não é possível visualizá-las a todas. Mesmo com muitas câmeras colocadas no local, acabam sempre por haver pontos mortos, devido à estrutura da paragem ou até mesmo devido ao meio, por exemplo árvores ou um carro mal-estacionado, etc.Bus stops nowadays have to be increasingly at the user’s service, this thesis explores the fundamentals ideas of what a Smart Bus Stop should be and bring all together into one concept using today’s technologies. Although the fundamentals of a Smart Bus Stop (SBS) are explored, the primary focus here is to explore computer vision technology and how they can be used inside the Smart Bus Stop concept. The world of computer vision has evolved a lot in recent years and its applications are almost limitless, so they have been incorporated into many different areas like scene reconstruction, event detection, video tracking, object recognition, motion estimation, image restoration, etc. When combining the different algorithms of the different applications more powerful tools can be created. This work uses a Convolutional Neural Network (CNN) based algorithm to detect people in a multi video feeds. It also counts the number of persons in the SBS, using facial recognition, using with Deep Learning algorithm, and Support Vector Machine algorithm. It is important to stress, these last two are used to keep track of the user and also to remove the repeated profiles gathered in the different video sources, since the SBS is in a multi-camera environment. Combining these technologies was possible to count how many people were in the SBS. In laboratory the propose algorithm presents an extremely high success rate, when applied to real bus stops que success rate decreases due to blind spots for instance