Aerial LiDAR-based 3D Object Detection And Tracking For Traffic Monitoring

The proliferation of Light Detection and Ranging (LiDAR) technology in the automotive industry has quickly promoted its use in many emerging areas in smart cities and internet-of-things. Compared to other sensors, like cameras and radars, LiDAR provides up to 64 scanning channels, vertical and horizontal field of view, high precision, high detection range, and great performance under poor weather conditions. In this paper, we propose a novel aerial traffic monitoring solution based on Light Detection and Ranging (LiDAR) technology. By equipping unmanned aerial vehicles (UAVs) with a LiDAR sensor, we generate 3D point cloud data that can be used for object detection and tracking. Due to the unavailability of LiDAR data from the sky, we propose to use a 3D simulator. Then, we implement Point Voxel-RCNN (PV-RCNN) to perform road user detection (e.g., vehicles and pedestrians). Subsequently, we implement an Unscented Kalman filter, which takes a 3D detected object as input and uses its information to predict the state of the 3D box before the next LiDAR scan gets loaded. Finally, we update the measurement by using the new observation of the point cloud and correct the previous prediction\u27s belief. The simulation results illustrate the performance gain (around 8 %) achieved by our solution compared to other 3D point cloud solutions

Multisensor navigation systems: a remedy for GNSS vulnerabilities?

Space-based positioning, navigation, and timing (PNT) technologies, such as the global navigation satellite systems (GNSS) provide position, velocity, and timing information to an unlimited number of users around the world. In recent years, PNT information has become increasingly critical to the security, safety, and prosperity of the World's population, and is now widely recognized as an essential element of the global information infrastructure. Due to its vulnerabilities and line-of-sight requirements, GNSS alone is unable to provide PNT with the required levels of integrity, accuracy, continuity, and reliability. A multisensor navigation approach offers an effective augmentation in GNSS-challenged environments that holds a promise of delivering robust and resilient PNT. Traditionally, sensors such as inertial measurement units (IMUs), barometers, magnetometers, odometers, and digital compasses, have been used. However, recent trends have largely focused on image-based, terrain-based and collaborative navigation to recover the user location. This paper offers a review of the technological advances that have taken place in PNT over the last two decades, and discusses various hybridizations of multisensory systems, building upon the fundamental GNSS/IMU integration. The most important conclusion of this study is that in order to meet the challenging goals of delivering continuous, accurate and robust PNT to the ever-growing numbers of users, the hybridization of a suite of different PNT solutions is required

Edistysaskeleita liikkeentunnistuksessa mobiililaitteilla

Motion sensing is one of the most important sensing capabilities of mobile devices, enabling monitoring physical movement of the device and associating the observed motion with predefined activities and physical phenomena. The present thesis is divided into three parts covering different facets of motion sensing techniques. In the first part of this thesis, we present techniques to identify the gravity component within three-dimensional accelerometer measurements. Our technique is particularly effective in the presence of sustained linear acceleration events. Using the estimated gravity component, we also demonstrate how the sensor measurements can be transformed into descriptive motion representations, able to convey information about sustained linear accelerations. To quantify sustained linear acceleration, we propose a set of novel peak features, designed to characterize movement during mechanized transportation. Using the gravity estimation technique and peak features, we proceed to present an accelerometer-based transportation mode detection system able to distinguish between fine-grained automotive modalities. In the second part of the thesis, we present a novel sensor-assisted method, crowd replication, for quantifying usage of a public space. As a key technical contribution within crowd replication, we describe construction and use of pedestrian motion models to accurately track detailed motion information. Fusing the pedestrian models with a positioning system and annotations about visual observations, we generate enriched trajectories able to accurately quantify usage of public spaces. Finally in the third part of the thesis, we present two exemplary mobile applications leveraging motion information. As the first application, we present a persuasive mobile application that uses transportation mode detection to promote sustainable transportation habits. The second application is a collaborative speech monitoring system, where motion information is used to monitor changes in physical configuration of the participating devices.Liikkeen havainnointi ja analysointi ovat keskeisimpiä kontekstitietoisten mobiililaitteiden ominaisuuksia. Tässä väitöskirjassa tarkastellaan kolmea eri liiketunnistuksen osa-aluetta. Väitöskirjan ensimmäinen osa käsittelee liiketunnistuksen menetelmiä erityisesti liikenteen ja ajoneuvojen saralla. Väitöskirja esittelee uusia menetelmiä gravitaatiokomponentin arviointiin tilanteissa, joissa laitteeseen kohdistuu pitkäkestoista lineaarista kiihtyvyyttä. Gravitaatiokomponentin tarkka arvio mahdollistaa ajoneuvon liikkeen erottelun muista laitteeseen kohdistuvista voimista. Menetelmän potentiaalin havainnollistamiseksi työssä esitellään kiihtyvyysanturipohjainen kulkumuototunnistusjärjestelmä, joka perustuu eri kulkumuotojen erotteluun näiden kiihtyvyysprofiilien perusteella. Väitöskirjan toinen osa keskittyy tapoihin mitata ja analysoida julkisten tilojen käyttöä liikkeentunnistuksen avulla. Työssä esitellään menetelmä, jolla kohdealueen käyttöä voidaan arvioida yhdistelemällä suoraa havainnointia ja mobiililaitteilla suoritettua havainnointia. Tämän esitellyn ihmisjoukkojen toisintamiseen (crowd replication) perustuvan menetelmän keskeisin tekninen kontribuutio on liikeantureihin perustuva liikkeenmallinnusmenetelmä, joka mahdollistaa käyttäjän tarkan askelten ja kävelyrytmin tunnistamisen. Yhdistämällä liikemallin tuottama tieto paikannusmenetelmään ja tutkijan omiin havaintoihin väitöskirjassa osoitetaan, kuinka käyttäjän osalta saadaan tallennettua tarkat tiedot hänen aktiviteeteistään ja liikeradoistaan sekä tilan että ajan suhteen. Väitöskirjan kolmannessa ja viimeisessä osassa esitellään kaksi esimerkkisovellusta liikkeentunnistuksen käytöstä mobiililaitteissa. Ensimmäinen näistä sovelluksista pyrkii edistämään ja tukemaan käyttäjää kohti kestäviä liikkumistapoja. Sovelluksen keskeisenä komponenttina toimii automaattinen kulkumuototunnistus, joka seuraa käyttäjän liikkumistottumuksia ja näistä koituvaa hiilidioksidijalanjälkeä. Toinen esiteltävä sovellus on mobiililaitepohjainen, yhteisöllinen puheentunnistus, jossa liikkeentunnistusta käytetään seuraamaan mobiililaiteryhmän fyysisen kokoonpanon pysyvyyttä

Investigation of Computer Vision Concepts and Methods for Structural Health Monitoring and Identification Applications

This study presents a comprehensive investigation of methods and technologies for developing a computer vision-based framework for Structural Health Monitoring (SHM) and Structural Identification (St-Id) for civil infrastructure systems, with particular emphasis on various types of bridges. SHM is implemented on various structures over the last two decades, yet, there are some issues such as considerable cost, field implementation time and excessive labor needs for the instrumentation of sensors, cable wiring work and possible interruptions during implementation. These issues make it only viable when major investments for SHM are warranted for decision making. For other cases, there needs to be a practical and effective solution, which computer-vision based framework can be a viable alternative. Computer vision based SHM has been explored over the last decade. Unlike most of the vision-based structural identification studies and practices, which focus either on structural input (vehicle location) estimation or on structural output (structural displacement and strain responses) estimation, the proposed framework combines the vision-based structural input and the structural output from non-contact sensors to overcome the limitations given above. First, this study develops a series of computer vision-based displacement measurement methods for structural response (structural output) monitoring which can be applied to different infrastructures such as grandstands, stadiums, towers, footbridges, small/medium span concrete bridges, railway bridges, and long span bridges, and under different loading cases such as human crowd, pedestrians, wind, vehicle, etc. Structural behavior, modal properties, load carrying capacities, structural serviceability and performance are investigated using vision-based methods and validated by comparing with conventional SHM approaches. In this study, some of the most famous landmark structures such as long span bridges are utilized as case studies. This study also investigated the serviceability status of structures by using computer vision-based methods. Subsequently, issues and considerations for computer vision-based measurement in field application are discussed and recommendations are provided for better results. This study also proposes a robust vision-based method for displacement measurement using spatio-temporal context learning and Taylor approximation to overcome the difficulties of vision-based monitoring under adverse environmental factors such as fog and illumination change. In addition, it is shown that the external load distribution on structures (structural input) can be estimated by using visual tracking, and afterward load rating of a bridge can be determined by using the load distribution factors extracted from computer vision-based methods. By combining the structural input and output results, the unit influence line (UIL) of structures are extracted during daily traffic just using cameras from which the external loads can be estimated by using just cameras and extracted UIL. Finally, the condition assessment at global structural level can be achieved using the structural input and output, both obtained from computer vision approaches, would give a normalized response irrespective of the type and/or load configurations of the vehicles or human loads

Automotive Interior Sensing - Temporal Consistent Human Body Pose Estimation

Com o surgimento e desenvolvimento de veículos autónomos, surgiu igualmente uma necessidade de monitorizar e identificar objetos e ações que ocorrem no ambiente que rodeia o veículo. Este tipo de monitorização é particularmente importante no caso de veículos partilhados, dada a necessidade de identificar ações não só no exterior mas também no interior do veículo devido à ausência de um condutor humano que possa detetar, por exemplo, potenciais ações de violência entre passageiros e/ou situações onde estes necessitem de assistência. Englobado neste contexto, a Bosch desenvolveu uma solução de estimação de postura humana com o objetivo de extrapolar a pose de todos os ocupantes presentes numa dada imagem, inferir o comportamento de cada passageiro e, consequentemente, identificar ações potencialmente maliciosas. Porém, para que este algoritmo possa ser aplicado não apenas a imagens isoladas mas também a vídeos é necessário adicionar contexto temporal entre frames. Por outras palavras, é necessário associar a estimação de pose de uma dada pessoa para uma dada frame às estimações de pose para a mesma pessoa em frames subsequentes de modo a que a identificação dessa pessoa (ou qualquer outra presente numa dada frame) ao longo do vídeo seja correta e consistente. O tópico de associação temporal, também conhecido como "pose tracking", é abordado e desenvolvido ao longo do presente projeto, culminando na proposta e implementação de uma solução que melhora consideravelmente a consistência temporal do algoritmo de estimação de pose humana da Bosch. A solução desenvolvida utiliza uma mistura de abordagens clássicas e atuais de associação de informação, como por exemplo o "Hungarian algorithm" e "Intersection over Union", e abordagens de lógica de informação desenvolvidas especificamente para o caso em questão. A performance do algoritmo implementado no presente projeto é avaliada usando duas das mais recorrentes métricas de avaliação em casos de rastreamento de pose.With the emergence and development of autonomous vehicles, a necessity to constantly monitor and identify objects and action that occur in the surrounding environment of the vehicle itself was also created. This type of monitoring is particularly important in the case of shared vehicles, given the necessity to identify actions not only in the exterior but also in the interior of the vehicle due to the absence of a human driver that can detect, for instance, potential violent actions between passengers and/or cases where assistence is required. Encompassed in this context, Bosch has developed a human body pose estimation solution in order to extrapolate the pose of all vehicle occupants present in a given image, infere the behaviour of each passenger and, consequently, identify potentially malicious actions. However, in order to apply this algorithm not only to isolated images but also to videos it is necessary to add temporal context between frames. In other words, an association is required between the body pose estimation for a given person in a given frame and the body pose estimations for the same person in subsequent frames in order to ensure that the identification of that passenger (or any other passenger present in the same frame) is accurate and consistent throughout the entire video. The temporal association topic, also known as pose tracking, is addressed and developed during the present project, culminating in the proposal and implementation of a solution that considerably improves the temporal consistency of the human body pose estimation algorithm developed by Bosch. The implemented solution uses a mixture of currently relevant classical approaches for data association, such as the Hungarian algorithm e Intersection over Union techniques, and approaches based on data logic developed specifically for the present case. Regarding performance, the developed algorithm is evaluated using two of the most recurrent metrics for pose tracking methods