Accurate Calibration Scheme for a Multi-Camera Mobile Mapping System

Mobile mapping systems (MMS) are increasingly used for many photogrammetric and computer vision applications, especially encouraged by the fast and accurate geospatial data generation. The accuracy of point position in an MMS is mainly dependent on the quality of calibration, accuracy of sensor synchronization, accuracy of georeferencing and stability of geometric configuration of space intersections. In this study, we focus on multi-camera calibration (interior and relative orientation parameter estimation) and MMS calibration (mounting parameter estimation). The objective of this study was to develop a practical scheme for rigorous and accurate system calibration of a photogrammetric mapping station equipped with a multi-projective camera (MPC) and a global navigation satellite system (GNSS) and inertial measurement unit (IMU) for direct georeferencing. The proposed technique is comprised of two steps. Firstly, interior orientation parameters of each individual camera in an MPC and the relative orientation parameters of each cameras of the MPC with respect to the first camera are estimated. In the second step the offset and misalignment between MPC and GNSS/IMU are estimated. The global accuracy of the proposed method was assessed using independent check points. A correspondence map for a panorama is introduced that provides metric information. Our results highlight that the proposed calibration scheme reaches centimeter-level global accuracy for 3D point positioning. This level of global accuracy demonstrates the feasibility of the proposed technique and has the potential to fit accurate mapping purposes

Development of a robotic mobile mapping system by vision-aided inertial navigation:a geomatics approach

Vision-based inertial-aided navigation is gaining ground due to its many potential applications. In previous decades, the integration of vision and inertial sensors was monopolised by the defence industry due to its complexity and unrealistic economic burden. After the technology advancement, high-quality hardware and computing power became reachable for the investigation and realisation of various applications. In this thesis, a mapping system by vision-aided inertial navigation was developed for areas where GNSS signals are unreachable, for example, indoors, tunnels, city canyons, forests, etc. In this framework, a methodology on the integration of vision and inertial sensors was presented, analysed and tested when the only available information at the beginning is a number of features with known location/coordinates (with no GNSS signals accessibility), thus employing the method of "SLAM: Simultaneous Localisation And Mapping". SLAM is a term used in the robotics community to describe the problem of mapping the environment and at the same time using this map to determine (or to help in determining) the location of the mapping device. In addition to this, a link between the robotics and geomatics community was established where briefly the similarities and differences were outlined in terms of handling the navigation and mapping problem. Albeit many differences, the goal is common: developing a "navigation and mapping system" that is not bounded to the limits imposed by the used sensors. Classically, terrestrial robotics SLAM is approached using LASER scanners to locate the robot relative to a structured environment and to map this environment at the same time. However, outdoors robotics SLAM is not feasible with LASER scanners alone due to the environment's roughness and absence of simple geometric features. Recently in the robotics community, the use of visual methods, integrated with inertial sensors, has gained an interest. These visual methods rely on one or more cameras (or video) and make use of a single Kalman Filter with a state vector containing the map and the robot coordinates. This concept introduces high non-linearity and complications to the filter, which then needs to run at high rates (more than 20 Hz) with simplified navigation and mapping models. In this study, SLAM is developed using the Geomatics Engineering approach. Two filters are used in parallel: the Least-Squares Adjustment (LSA) for feature coordinates determination and the Kalman Filter (KF) for navigation correction. For this, a mobile mapping system (independent of GPS) is introduced by employing two CCD cameras (one metre apart) and one IMU. Conceptually, the outputs of the LSA photogrammetric resection (position and orientation) are used as the external measurements for the inertial KF. The filtered position and orientation are subsequently employed in the Photogrammetric intersection to map the surrounding features that are used as control points for the resection in the next epoch. In this manner, the KF takes the form of a navigation only filter, with a state vector containing the corrections to the navigation parameters. This way, the mapping and localisation can be updated at low rates (1 to 2 Hz) and use more complete modelling. Results show that this method is feasible with limitation induced from the quality of the images and the number of used features. Although simulation showed that (depending on the image geometry) determining the features' coordinates with an accuracy of 5-10 cm for objects at distances of up to 10 metres is possible, in practice this is not achieved with the employed hardware and pixel measurement techniques. Navigational accuracies depend as well on the quality of the images and the number and accuracy of the points used in the resection. While more than 25 points are needed to achieve centimetre accuracy from resection, they have to be within a distance of 10 metres from the cameras; otherwise, the resulting resection output will be of insufficient accuracy and further integration quality deteriorates. The initial conditions highly affect SLAM performance; these are the method of IMU initialisation and the a-priori assumptions on error distribution. The geometry of the system will furthermore have a consequence on possible applications. To conclude, the development consisted in establishing a mathematical framework, as well as implementing methods and algorithms for a novel integration methodology between vision and inertial sensors. The implementation and validation of the software have presented the main challenges, and it can be considered the first of a kind where all components were developed from scratch, with no pre-existing modules. Finally, simulations and practical tests were carried out, from which initial conclusions and recommendations were drawn to build upon. It is the author's hope that this work will stimulate others to investigate further this interesting problem taking into account the conclusions and recommendations sketched herein

MEMS Accelerometers

Micro-electro-mechanical system (MEMS) devices are widely used for inertia, pressure, and ultrasound sensing applications. Research on integrated MEMS technology has undergone extensive development driven by the requirements of a compact footprint, low cost, and increased functionality. Accelerometers are among the most widely used sensors implemented in MEMS technology. MEMS accelerometers are showing a growing presence in almost all industries ranging from automotive to medical. A traditional MEMS accelerometer employs a proof mass suspended to springs, which displaces in response to an external acceleration. A single proof mass can be used for one- or multi-axis sensing. A variety of transduction mechanisms have been used to detect the displacement. They include capacitive, piezoelectric, thermal, tunneling, and optical mechanisms. Capacitive accelerometers are widely used due to their DC measurement interface, thermal stability, reliability, and low cost. However, they are sensitive to electromagnetic field interferences and have poor performance for high-end applications (e.g., precise attitude control for the satellite). Over the past three decades, steady progress has been made in the area of optical accelerometers for high-performance and high-sensitivity applications but several challenges are still to be tackled by researchers and engineers to fully realize opto-mechanical accelerometers, such as chip-scale integration, scaling, low bandwidth, etc

Sistemas de posicionamento baseados em comunicação por luz para ambientes interiores

The demand for highly precise indoor positioning systems (IPSs) is growing rapidly due to its potential in the increasingly popular techniques of the Internet of Things, smart mobile devices, and artificial intelligence. IPS becomes a promising research domain that is getting wide attention due to its benefits in several working scenarios, such as, industries, indoor public locations, and autonomous navigation. Moreover, IPS has a prominent contribution in day-to-day activities in organizations such as health care centers, airports, shopping malls, manufacturing, underground locations, etc., for safe operating environments. In indoor environments, both radio frequency (RF) and optical wireless communication (OWC) based technologies could be adopted for localization. Although the RF-based global positioning system, such as, Global positioning system offers higher penetration rates with reduced accuracy (i.e., in the range of a few meters), it does not work well in indoor environments (and not at all in certain cases such as tunnels, mines, etc.) due to the very weak signal and no direct access to the satellites. On the other hand, the light-based system known as a visible light positioning (VLP) system, as part of the OWC systems, uses the pre-existing light-emitting diodes (LEDs)-based lighting infrastructure, could be used at low cost and high accuracy compared with the RF-based systems. VLP is an emerging technology promising high accuracy, high security, low deployment cost, shorter time response, and low relative complexity when compared with RFbased positioning. However, in indoor VLP systems, there are some concerns such as, multipath reflection, transmitter tilting, transmitter’s position, and orientation uncertainty, human shadowing/blocking, and noise causing the increase in the positioning error, thereby reducing the positioning accuracy of the system. Therefore, it is imperative to capture the characteristics of different VLP channel and properly model them for the dual purpose of illumination and localization. In this thesis, firstly, the impact of transmitter tilting angles and multipath reflections are studied and for the first time, it is demonstrated that tilting the transmitter can be beneficial in VLP systems considering both line of sight (LOS) and non-line of sight transmission paths. With the transmitters oriented towards the center of the receiving plane, the received power level is maximized due to the LOS components. It is also shown that the proposed scheme offers a significant accuracy improvement of up to ~66% compared with a typical non-tilted transmitter VLP. The effect of tilting the transmitter on the lighting uniformity is also investigated and results proved that the uniformity achieved complies with the European Standard EN 12464-1. After that, the impact of transmitter position and orientation uncertainty on the accuracy of the VLP system based on the received signal strength (RSS) is investigated. Simulation results show that the transmitter uncertainties have a severe impact on the positioning error, which can be leveraged through the usage of more transmitters. Concerning a smaller transmitter’s position epochs, and the size of the training set. It is shown that, the ANN with Bayesian regularization outperforms the traditional RSS technique using the non-linear least square estimation for all values of signal to noise ratio. Furthermore, a novel indoor VLP system is proposed based on support vector machines and polynomial regression considering two different multipath environments of an empty room and a furnished room. The results show that, in an empty room, the positioning accuracy improvement for the positioning error of 2.5 cm are 36.1, 58.3, and 72.2 % for three different scenarios according to the regions’ distribution in the room. For the furnished room, a positioning relative accuracy improvement of 214, 170, and 100 % is observed for positioning error of 0.1, 0.2, and 0.3 m, respectively. Ultimately, an indoor VLP system based on convolutional neural networks (CNN) is proposed and demonstrated experimentally in which LEDs are used as transmitters and a rolling shutter camera is used as receiver. A detection algorithm named single shot detector (SSD) is used which relies on CNN (i.e., MobileNet or ResNet) for classification as well as position estimation of each LED in the image. The system is validated using a real-world size test setup containing eight LED luminaries. The obtained results show that the maximum average root mean square positioning error achieved is 4.67 and 5.27 cm with SSD MobileNet and SSD ResNet models, respectively. The validation results show that the system can process 67 images per second, allowing real-time positioning.A procura por sistemas de posicionamento interior (IPSs) de alta precisão tem crescido rapidamente devido ao seu interesse nas técnicas cada vez mais populares da Internet das Coisas, dispositivos móveis inteligentes e inteligência artificial. O IPS tornou-se um domínio de pesquisa promissor que tem atraído grande atenção devido aos seus benefícios em vários cenários de trabalho, como indústrias, locais públicos e navegação autónoma. Além disso, o IPS tem uma contribuição destacada no dia a dia de organizações, como, centros de saúde, aeroportos, supermercados, fábricas, locais subterrâneos, etc. As tecnologias baseadas em radiofrequência (RF) e comunicação óptica sem fio (OWC) podem ser adotadas para localização em ambientes interiores. Embora o sistema de posicionamento global (GPS) baseado em RF ofereça taxas de penetração mais altas com precisão reduzida (ou seja, na faixa de alguns metros), não funciona bem em ambientes interiores (e não funciona bem em certos casos como túneis, minas, etc.) devido ao sinal muito fraco e falta de acesso direto aos satélites. Por outro lado, o sistema baseado em luz conhecido como sistema de posicionamento de luz visível (VLP), como parte dos sistemas OWC, usa a infraestrutura de iluminação baseada em díodos emissores de luz (LEDs) pré-existentes, é um sistemas de baixo custo e alta precisão quando comprado com os sistemas baseados em RF. O VLP é uma tecnologia emergente que promete alta precisão, alta segurança, baixo custo de implantação, menor tempo de resposta e baixa complexidade relativa quando comparado ao posicionamento baseado em RF. No entanto, os sistemas VLP interiores, exibem algumas limitações, como, a reflexão multicaminho, inclinação do transmissor, posição do transmissor e incerteza de orientação, sombra/bloqueio humano e ruído, que têm como consequência o aumento do erro de posicionamento, e consequente redução da precisão do sistema. Portanto, é imperativo estudar as características dos diferentes canais VLP e modelá-los adequadamente para o duplo propósito de iluminação e localização. Esta tesa aborda, primeiramente, o impacto dos ângulos de inclinação do transmissor e reflexões multipercurso no desempenho do sistema de posicionamento. Demonstra-se que a inclinação do transmissor pode ser benéfica em sistemas VLP considerando tanto a linha de vista (LOS) como as reflexões. Com os transmissores orientados para o centro do plano recetor, o nível de potência recebido é maximizado devido aos componentes LOS. Também é mostrado que o esquema proposto oferece uma melhoria significativa de precisão de até ~66% em comparação com um sistema VLP de transmissor não inclinado típico. O efeito da inclinação do transmissor na uniformidade da iluminação também é investigado e os resultados comprovam que a uniformidade alcançada está de acordo com a Norma Europeia EN 12464-1. O impacto da posição do transmissor e incerteza de orientação na precisão do sistema VLP com base na intensidade do sinal recebido (RSS) foi também investigado. Os resultados da simulação mostram que as incertezas do transmissor têm um impacto severo no erro de posicionamento, que pode ser atenuado com o uso de mais transmissores. Para incertezas de posicionamento dos transmissores menores que 5 cm, os erros médios de posicionamento são 23.3, 15.1 e 13.2 cm para conjuntos de 4, 9 e 16 transmissores, respetivamente. Enquanto que, para a incerteza de orientação de um transmissor menor de 5°, os erros médios de posicionamento são 31.9, 20.6 e 17 cm para conjuntos de 4, 9 e 16 transmissores, respetivamente. O trabalho da tese abordou a investigação dos aspetos de projeto de um sistema VLP indoor no qual uma rede neuronal artificial (ANN) é utilizada para estimativa de posicionamento considerando um canal multipercurso. O estudo considerou a influência do ruído como indicador de desempenho para a comparação entre diferentes abordagens de projeto. Três algoritmos de treino de ANNs diferentes foram considerados, a saber, Levenberg-Marquardt, regularização Bayesiana e algoritmos de gradiente conjugado escalonado, para minimizar o erro de posicionamento no sistema VLP. O projeto da ANN foi otimizado com base no número de neurónios nas camadas ocultas, no número de épocas de treino e no tamanho do conjunto de treino. Mostrou-se que, a ANN com regularização Bayesiana superou a técnica RSS tradicional usando a estimação não linear dos mínimos quadrados para todos os valores da relação sinal-ruído. Foi proposto um novo sistema VLP indoor baseado em máquinas de vetores de suporte (SVM) e regressão polinomial considerando dois ambientes interiores diferentes: uma sala vazia e uma sala mobiliada. Os resultados mostraram que, numa sala vazia, a melhoria da precisão de posicionamento para o erro de posicionamento de 2.5 cm são 36.1, 58.3 e 72.2% para três cenários diferentes de acordo com a distribuição das regiões na sala. Para a sala mobiliada, uma melhoria de precisão relativa de posicionamento de 214, 170 e 100% é observada para erro de posicionamento de 0.1, 0.2 e 0.3 m, respetivamente. Finalmente, foi proposto um sistema VLP indoor baseado em redes neurais convolucionais (CNN). O sistema foi demonstrado experimentalmente usando luminárias LED como transmissores e uma camara com obturador rotativo como recetor. O algoritmo de detecção usou um detector de disparo único (SSD) baseado numa CNN pré configurada (ou seja, MobileNet ou ResNet) para classificação. O sistema foi validado usando uma configuração de teste de tamanho real contendo oito luminárias LED. Os resultados obtidos mostraram que o erro de posicionamento quadrático médio alcançado é de 4.67 e 5.27 cm com os modelos SSD MobileNet e SSD ResNet, respetivamente. Os resultados da validação mostram que o sistema pode processar 67 imagens por segundo, permitindo o posicionamento em tempo real.Programa Doutoral em Engenharia Eletrotécnic

Accurate Calibration Scheme for a Multi-Camera Mobile Mapping System

Mobile mapping systems (MMS) are increasingly used for many photogrammetric and computer vision applications, especially encouraged by the fast and accurate geospatial data generation. The accuracy of point position in an MMS is mainly dependent on the quality of calibration, accuracy of sensor synchronization, accuracy of georeferencing and stability of geometric configuration of space intersections. In this study, we focus on multi-camera calibration (interior and relative orientation parameter estimation) and MMS calibration (mounting parameter estimation). The objective of this study was to develop a practical scheme for rigorous and accurate system calibration of a photogrammetric mapping station equipped with a multi-projective camera (MPC) and a global navigation satellite system (GNSS) and inertial measurement unit (IMU) for direct georeferencing. The proposed technique is comprised of two steps. Firstly, interior orientation parameters of each individual camera in an MPC and the relative orientation parameters of each cameras of the MPC with respect to the first camera are estimated. In the second step the offset and misalignment between MPC and GNSS/IMU are estimated. The global accuracy of the proposed method was assessed using independent check points. A correspondence map for a panorama is introduced that provides metric information. Our results highlight that the proposed calibration scheme reaches centimeter-level global accuracy for 3D point positioning. This level of global accuracy demonstrates the feasibility of the proposed technique and has the potential to fit accurate mapping purposes

SSTAC/ARTS review of the draft Integrated Technology Plan (ITP). Volume 8: Aerothermodynamics Automation and Robotics (A/R) systems sensors, high-temperature superconductivity

Viewgraphs of briefings presented at the SSTAC/ARTS review of the draft Integrated Technology Plan (ITP) on aerothermodynamics, automation and robotics systems, sensors, and high-temperature superconductivity are included. Topics covered include: aerothermodynamics; aerobraking; aeroassist flight experiment; entry technology for probes and penetrators; automation and robotics; artificial intelligence; NASA telerobotics program; planetary rover program; science sensor technology; direct detector; submillimeter sensors; laser sensors; passive microwave sensing; active microwave sensing; sensor electronics; sensor optics; coolers and cryogenics; and high temperature superconductivity

Modular Optical Wireless Elements

Optical wireless has gained attention in recent years as an e cient and secure way to provide broadband connectivity to mobile platforms, isolated communities, and crowded public events. Companies like NASA, Google, Facebook, and others have demonstrated its potential. However, current optical wireless technology remains mostly heavy, bulky, and expensive, making it impractical for many scenarios and inaccessible to most students/researchers. This work presents the concept of Modular Optical Wireless Elements (MOWE), a novel system composed of multiple electrically interconnected optical modules (i.e., elements) forming a at or curved terminal that is inexpensive, lightweight, and easy-to-assemble. The technology enables cost-eff ective access to wide eld-of-view optical communication for last-mile broadband connectivity. Smart modules provide recon gurability, as well as local and central processing capabilities. The modules enable innovative short- and medium-range applications for free-space optics (FSO) in indoor communication and navigation, MIMO, and optical sensing, among others. This dissertation introduces the MOWE concept and provides in-depth information about modeling, analysis, hardware, and rmware, along with proof-of-concept examples and demonstrations. The notions of software-de ned optics and cognitive optics are introduced and analyzed in a MOWE context. Several experiments and case studies covering a wide spectrum of applications-from intelligent power control to passive beam steering-are presented in detail. This dissertation also discusses the future of MOWE technology and suggests possible improvements for high performance systems

Fusion of low-cost and light-weight sensor system for mobile flexible manipulator

There is a need for non-industrial robots such as in homecare and eldercare. Light-weight mobile robots preferred as compared to conventional fixed based robots as the former is safe, portable, convenient and economical to implement. Sensor system for light-weight mobile flexible manipulator is studied in this research. A mobile flexible link manipulator (MFLM) contributes to high amount of vibrations at the tip, giving rise to inaccurate position estimations. In a control system, there inevitably exists a lag between the sensor feedback and the controller. Consequently, it contributed to instable control of the MFLM. Hence, there it is a need to predict the tip trajectory of the MFLM. Fusion of low cost sensors is studied to enhance prediction accuracy at the MFLM’s tip. A digital camera and an accelerometer are used predict tip of the MFLM. The main disadvantage of camera is the delayed feedback due to the slow data rate and long processing time, while accelerometer composes cumulative errors. Wheel encoder and webcam are used for position estimation of the mobile platform. The strengths and limitations of each sensor were compared. To solve the above problem, model based predictive sensor systems have been investigated for used on the mobile flexible link manipulator using the selected sensors. Mathematical models were being developed for modeling the reaction of the mobile platform and flexible manipulator when subjected to a series of input voltages and loads. The model-based Kalman filter fusion prediction algorithm was developed, which gave reasonability good predictions of the vibrations of the tip of flexible manipulator on the mobile platform. To facilitate evaluation of the novel predictive system, a mobile platform was fabricated, where the flexible manipulator and the sensors are mounted onto the platform. Straight path motions were performed for the experimental tests. The results showed that predictive algorithm with modelled input to the Extended Kalman filter have best prediction to the tip vibration of the MFLM

Modeling Navigation System Performance of a Satellite-Observing Star Tracker Tightly Integrated with an Inertial Measurement Unit

This dissertation evaluates a navigation system using satellite observations from a star tracker tightly-integrated with an inertial measurement unit (IMU) and a barometric altimeter using an extended Kalman filter. The star tracker measurement accuracy of a satellite is derived. Several system configurations are simulated comparing the performance of the estimate with respect to system parameters of the IMU, and star tracker, as well as comparing performance when providing a remote sensor satellite ephemeris error correction. Experimental observations are used to evaluate the model performance. Additionally, power requirements were calculated for a satellite signal operating in imaging bands, such that a Low Earth Orbiting satellite constellation could be detected during the day. This type of signal would make it possible to operate the star tracker integrated navigation system in GPS-degraded environments with similar duration and comparable accuracy of GPS

Small business innovation research. Abstracts of completed 1987 phase 1 projects

Non-proprietary summaries of Phase 1 Small Business Innovation Research (SBIR) projects supported by NASA in the 1987 program year are given. Work in the areas of aeronautical propulsion, aerodynamics, acoustics, aircraft systems, materials and structures, teleoperators and robotics, computer sciences, information systems, spacecraft systems, spacecraft power supplies, spacecraft propulsion, bioastronautics, satellite communication, and space processing are covered