Event-based Simultaneous Localization and Mapping: A Comprehensive Survey

In recent decades, visual simultaneous localization and mapping (vSLAM) has gained significant interest in both academia and industry. It estimates camera motion and reconstructs the environment concurrently using visual sensors on a moving robot. However, conventional cameras are limited by hardware, including motion blur and low dynamic range, which can negatively impact performance in challenging scenarios like high-speed motion and high dynamic range illumination. Recent studies have demonstrated that event cameras, a new type of bio-inspired visual sensor, offer advantages such as high temporal resolution, dynamic range, low power consumption, and low latency. This paper presents a timely and comprehensive review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks. The review covers the working principle of event cameras and various event representations for preprocessing event data. It also categorizes event-based vSLAM methods into four main categories: feature-based, direct, motion-compensation, and deep learning methods, with detailed discussions and practical guidance for each approach. Furthermore, the paper evaluates the state-of-the-art methods on various benchmarks, highlighting current challenges and future opportunities in this emerging research area. A public repository will be maintained to keep track of the rapid developments in this field at {\url{https://github.com/kun150kun/ESLAM-survey}}

DOES: A Deep Learning-based approach to estimate roll and pitch at sea

The use of Attitude and Heading Reference Systems (AHRS) for orientation estimation is now common practice in a wide range of applications, e.g., robotics and human motion tracking, aerial vehicles and aerospace, gaming and virtual reality, indoor pedestrian navigation and maritime navigation. The integration of the high-rate measurements can provide very accurate estimates, but these can suffer from errors accumulation due to the sensors drift over longer time scales. To overcome this issue, inertial sensors are typically combined with additional sensors and techniques. As an example, camera-based solutions have drawn a large attention by the community, thanks to their low-costs and easy hardware setup; moreover, impressive results have been demonstrated in the context of Deep Learning. This work presents the preliminary results obtained by DOES, a supportive Deep Learning method specifically designed for maritime navigation, which aims at improving the roll and pitch estimations obtained by common AHRS. DOES recovers these estimations through the analysis of the frames acquired by a low-cost camera pointing the horizon at sea. The training has been performed on the novel ROPIS dataset, presented in the context of this work, acquired using the FrameWO application developed for the scope. Promising results encourage to test other network backbones and to further expand the dataset, improving the accuracy of the results and the range of applications of the method as a valid support to visual-based odometry techniques

NeBula: TEAM CoSTAR’s robotic autonomy solution that won phase II of DARPA subterranean challenge

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved second and first place, respectively. We also discuss CoSTAR’s demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including (i) geometric and semantic environment mapping, (ii) a multi-modal positioning system, (iii) traversability analysis and local planning, (iv) global motion planning and exploration behavior, (v) risk-aware mission planning, (vi) networking and decentralized reasoning, and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g., wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition.Peer ReviewedAgha, A., Otsu, K., Morrell, B., Fan, D. D., Thakker, R., Santamaria-Navarro, A., Kim, S.-K., Bouman, A., Lei, X., Edlund, J., Ginting, M. F., Ebadi, K., Anderson, M., Pailevanian, T., Terry, E., Wolf, M., Tagliabue, A., Vaquero, T. S., Palieri, M., Tepsuporn, S., Chang, Y., Kalantari, A., Chavez, F., Lopez, B., Funabiki, N., Miles, G., Touma, T., Buscicchio, A., Tordesillas, J., Alatur, N., Nash, J., Walsh, W., Jung, S., Lee, H., Kanellakis, C., Mayo, J., Harper, S., Kaufmann, M., Dixit, A., Correa, G. J., Lee, C., Gao, J., Merewether, G., Maldonado-Contreras, J., Salhotra, G., Da Silva, M. S., Ramtoula, B., Fakoorian, S., Hatteland, A., Kim, T., Bartlett, T., Stephens, A., Kim, L., Bergh, C., Heiden, E., Lew, T., Cauligi, A., Heywood, T., Kramer, A., Leopold, H. A., Melikyan, H., Choi, H. C., Daftry, S., Toupet, O., Wee, I., Thakur, A., Feras, M., Beltrame, G., Nikolakopoulos, G., Shim, D., Carlone, L., & Burdick, JPostprint (published version

A predictive model for precision tree measurements using applied machine learning

Thesis (MEng)--Stellenbosch University, 2022.ENGLISH ABSTRACT: Accurately determining biological asset values is of great importance for forestry enterprises — the process ought to be characterised by the proper collection of tree data by means of utilising appropriate enumeration practices conducted at managed forest compartments. Currently, only between 5–20% of forest areas are enumerated which serve as a representative sample for the entire enclosing compartment. For forestry companies, timber volume estimations and future growth projections are based on these statistics, which may be accompanied by numerous unintentional errors during the data collection process. Many alternative methods towards estimating and inferring tree data accurately are available in the literature — the most popular characteristic is the so-called diameter at breast height (DBH), which can also be measured by means of remote sensing techniques. The advancements in laser scanning measurement apparatuses are significant in recent decades, however, these approaches are notably expensive and require specialised and technical skills for their operation. One of the main drawbacks associated with the measurement of DBH by means of laser scanning is the lack of scalability — equipment setup and data capture are arduous processes that take a significant amount of time to complete. Algorithmic breakthroughs in the domain of data science, predominantly spanning machine learning (ML) and deep learning (DL) approaches, warrant the selection and practical application of computer vision (CV) procedures. More specifically, an algorithmic approach towards monocular depth estimation (MDE) techniques was employed for the extraction of tree data features from video recordings (captured using no more than an ordinary smartphone device) and are investigated in this thesis. Towards this end, a suitable forest study area was identified to conduct the experiment and the industry partner of the project, i.e. the South African Forestry Company SOC Limited (SAFCOL) granted the necessary plantation access. The research methodology adopted for this thesis includes fieldwork at the given site, which involved first performing data collection steps according to accepted and standardised operating procedures developed for tree enumerations. This data set is regarded as the “ground truth” and comprises the target feature (i.e. actual DBH measurements) later used for modelling purposes. The video files were processed in a structured manner in order to extract tree segment patterns from the correspond ing imagery. Various ML models are then trained and tested in respect of the basic input feature data file, which produced a relative root mean squared error (RMSE %) between 14.1 and 18.3% for the study. The relative bias yields a score between −0.08% and 1.13% indicating that the proposed workflow solution exhibits a consis tent prediction result, but at an undesirable error rate (i.e. RMSE) deviation from the target output. Additionally, the suggested CV/ML workflow model is capable of generating a dis cernibly similar spatial representation upon visual inspection (when compared with the ground truth data set — i.e. tree coordinates captured during fieldwork). In the pursuit of precision forestry, the proposed predictive model developed for accurate tree measurements produce DBH estimations that approximate real-world values with a fair degree of accuracy.AFRIKAANSE OPSOMMING: Die akkurate bepaling van biologiese batewaardes is baie belangrik vir groot bos bou ondernemings — die proses word gekenmerk deur die korrekte versameling van boomdata, deur gebruik te maak van gepaste opsommingspraktyke wat in bestuurde bosbou kompartemente uitgevoer word. Tans word slegs tussen 5 en 20% van die bosareas opgesom wat as ‘n verteenwoordigende steekproef van die hele omhulde kompartement dien. Vir bosbou ondernemings is die beraming van houtvolumes en toekomstige groeiprojeksies gebaseer op hierdie statistieke, wat moontlik gepaard gaan met talle onbedoelde foute tydens die data-insamelingsproses. Baie alternatiewe metodes om boomdata akkuraat te bereken is in die literatuur beskikbaar — die gewildste data punt (kenmerkend in bosbou) is die sogenaamde diameter op borshoogte (DBH), wat selfs ook gemeet kan word deur middel van af standswaarnemings tegnieke. Die vooruitgang in meetapparate vir laserskandering is die afgelope dekades aansienlik verbeter, maar hierdie benaderings is veral duur en vereis gespesialiseerde en tegniese vaardighede vir die werking daarvan. Een van die belangrikste nadele verbonde aan die meting van DBH deur middel van hierdie laserskandering is die gebrek aan skaalbaarheid — die opstel van toerusting en die opneem van data is moeisame prosesse wat aansienlik baie Algoritmiese deurbrake op die gebied van data wetenskap, wat oorwegend masjien leer (ML) en diep leer (DL) benaderings bevat, regverdig die keuse en praktiese toepassing van rekenaarvisie (CV) prosedures. Meer spesifiek is die algoritmiese benadering ten opsigte van monokulˆere diepte skatting (MDE) tegnieke vir die ont trekking van boomdatafunksies vanuit video opnames (met nie meer as ‘n gewone slimfoonapparaat nie) en word in hierdie tesis deeglik ondersoek. Hiervoor is ‘n geskikte bosstudiegebied ge¨ıdentifiseer om die eksperiment uit te voer en die bedryfs vennoot van die projek, South African Forestry Company SOC Limited (SAFCOL) het die nodige toegang tot die plantasie verleen. Die navorsingsmetodologie wat vir hierdie proefskrif aangeneem is, bevat veldwerk op die gegewe terrein en die eerste stap van die uitgevoerde data insameling was volgens aanvaarde en gestandaardiseerde werkingsprosedures wat vir boomtellings neem om te voltooi. ontwikkel is. Hierdie opgawe en datastel word beskou as die “grondwaarheid” en be vat die teikenfunksie (werklike DBH metings), wat later vir modelleringsdoeleindes gebruik is. Die videolˆeers is op ‘n gestruktureerde manier verwerk om boomseg ment patrone uit die ooreenstemmende beelde te onttrek. Verskeie ML modelle word dan opgelei en getoets ten opsigte van die basiese invoerfunksiedatalˆeer, wat ‘n relatiewe wortel gemiddelde kwadraatfout (RMSE %) tussen 14.1% en 18.3% vir die studie opgelewer het. Die relatiewe vooroordeel lewer ‘n telling tussen −0.08% en 1.13% wat aandui dat die voorgestelde werkstroom oplossing ‘n konstante voor spellings resultaat toon, maar met ‘n ongewenste foutkoers (RMSE) afwyking vanaf die teikenuitset wat verlang word. Verder kan die voorgestelde CV/ML werkstroom model ook ‘n waarneembare en soortgelyke ruimtelike voorstelling genereer onder meer visuele inspeksie (in verge lyking met die grondwaarheids data stel — m.a.w. boomko¨ordinate wat tydens veld werk vasgelˆe is). In die strewe na presiese bosbou lewer hierdie voorspellingsmodel wat ontwikkel is vir boommetings (i.t.m. DBH beramings), die werklike waardes verteenwoordigend tot ‘n redelike mate van akkuraatheid.Master

Visual and Camera Sensors

This book includes 13 papers published in Special Issue ("Visual and Camera Sensors") of the journal Sensors. The goal of this Special Issue was to invite high-quality, state-of-the-art research papers dealing with challenging issues in visual and camera sensors

Novel Aggregated Solutions for Robust Visual Tracking in Traffic Scenarios

This work proposes novel approaches for object tracking in challenging scenarios like severe occlusion, deteriorated vision and long range multi-object reidentification. All these solutions are only based on image sequence captured by a monocular camera and do not require additional sensors. Experiments on standard benchmarks demonstrate an improved state-of-the-art performance of these approaches. Since all the presented approaches are smartly designed, they can run at a real-time speed