BatMobility: Towards Flying Without Seeing for Autonomous Drones

Unmanned aerial vehicles (UAVs) rely on optical sensors such as cameras and lidar for autonomous operation. However, such optical sensors are error-prone in bad lighting, inclement weather conditions including fog and smoke, and around textureless or transparent surfaces. In this paper, we ask: is it possible to fly UAVs without relying on optical sensors, i.e., can UAVs fly without seeing? We present BatMobility, a lightweight mmWave radar-only perception system for UAVs that eliminates the need for optical sensors. BatMobility enables two core functionalities for UAVs -- radio flow estimation (a novel FMCW radar-based alternative for optical flow based on surface-parallel doppler shift) and radar-based collision avoidance. We build BatMobility using commodity sensors and deploy it as a real-time system on a small off-the-shelf quadcopter running an unmodified flight controller. Our evaluation shows that BatMobility achieves comparable or better performance than commercial-grade optical sensors across a wide range of scenarios

Visual Odometry and Control for an Omnidirectional Mobile Robot with a Downward-Facing Camera

©2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.DOI: 10.1109/IROS.2010.5649749Presented at the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 18-22 Oct. 2010, Taipei.An omnidirectional Mecanum base allows for more flexible mobile manipulation. However, slipping of the Mecanum wheels results in poor dead-reckoning estimates from wheel encoders, limiting the accuracy and overall utility of this type of base. We present a system with a downwardfacing camera and light ring to provide robust visual odometry estimates. We mounted the system under the robot which allows it to operate in conditions such as large crowds or low ambient lighting. We demonstrate that the visual odometry estimates are sufficient to generate closed-loop PID (Proportional Integral Derivative) and LQR (Linear Quadratic Regulator) controllers for motion control in three different scenarios: waypoint tracking, small disturbance rejection, and sideways motion. We report quantitative measurements that demonstrate superior control performance when using visual odometry compared to wheel encoders. Finally, we show that this system provides highfidelity odometry estimates and is able to compensate for wheel slip on a four-wheeled omnidirectional mobile robot base

Positioning device for outdoor mobile robots using optical sensors and lasers

We propose a novel method for positioning a mobile robot in an outdoor environment using lasers and optical sensors. Position estimation via a noncontact optical method is useful because the information from the wheel odometer and the global positioning system in a mobile robot is unreliable in some situations. Contact optical sensors such as computer mouse are designed to be in contact with a surface and do not function well in strong ambient light conditions. To mitigate the challenges of an outdoor environment, we developed an optical device with a bandpass filter and a pipe to restrict solar light and to detect translation. The use of two devices enables sensing of the mobile robot’s position, including posture. Furthermore, employing a collimated laser beam allows measurements against a surface to be invariable with the distance to the surface. In this paper, we describe motion estimation, device configurations, and several tests for performance evaluation. We also present the experimental positioning results from a vehicle equipped with our optical device on an outdoor path. Finally, we discuss an improvement in postural accuracy by combining an optical device with precise gyroscopes

A Survey on Odometry for Autonomous Navigation Systems

The development of a navigation system is one of the major challenges in building a fully autonomous platform. Full autonomy requires a dependable navigation capability not only in a perfect situation with clear GPS signals but also in situations, where the GPS is unreliable. Therefore, self-contained odometry systems have attracted much attention recently. This paper provides a general and comprehensive overview of the state of the art in the field of self-contained, i.e., GPS denied odometry systems, and identifies the out-coming challenges that demand further research in future. Self-contained odometry methods are categorized into five main types, i.e., wheel, inertial, laser, radar, and visual, where such categorization is based on the type of the sensor data being used for the odometry. Most of the research in the field is focused on analyzing the sensor data exhaustively or partially to extract the vehicle pose. Different combinations and fusions of sensor data in a tightly/loosely coupled manner and with filtering or optimizing fusion method have been investigated. We analyze the advantages and weaknesses of each approach in terms of different evaluation metrics, such as performance, response time, energy efficiency, and accuracy, which can be a useful guideline for researchers and engineers in the field. In the end, some future research challenges in the field are discussed

Unmanned Aerial Vehicle Navigation Using Wide-Field Optical Flow and Intertial Sensors

This paper offers a set of novel navigation techniques that rely on the use of inertial sensors and wide-field optical flow information. The aircraft ground velocity and attitude states are estimated with an Unscented Information Filter (UIF) and are evaluated with respect to two sets of experimental flight data collected from an Unmanned Aerial Vehicle (UAV). Two different formulations are proposed, a full state formulation including velocity and attitude and a simplified formulation which assumes that the lateral and vertical velocity of the aircraft are negligible. An additional state is also considered within each formulation to recover the image distance which can be measured using a laser rangefinder. The results demonstrate that the full state formulation is able to estimate the aircraft ground velocity to within 1.3 m/s of a GPS receiver solution used as reference "truth" and regulate attitude angles within 1.4 degrees standard deviation of error for both sets of flight data

Novel Camera Architectures for Localization and Mapping on Intelligent Mobile Platforms

Self-localization and environment mapping play a very important role in many robotics application such as autonomous driving and mixed reality consumer products. Although the most powerful solutions rely on a multitude of sensors including lidars and camera, the community maintains a high interest in developing cost-effective, purely vision-based localization and mapping approaches. The core problem of standard vision-only solutions is accuracy and robustness, especially in challenging visual conditions. The thesis aims to introduce new solutions to localization and mapping problems on intelligent mobile devices by taking advantages of novel camera architectures. The thesis investigates on using surround-view multi-camera systems, which combine the benefits of omni-directional measurements with a sufficient baseline for producing measurements in metric scale, and event cameras, that perform well under challenging illumination conditions and have high temporal resolutions. The thesis starts by looking into the motion estimation framework with multi-perspective camera systems. The framework could be divided into two sub-parts, a front-end module that initializes motion and estimates absolute pose after bootstrapping, and a back-end module that refines the estimate over a larger-scale sequence. First, the thesis proposes a complete real-time pipeline for visual odometry with non-overlapping, multi-perspective camera systems, and in particular presents a solution to the scale initialization problem, in order to solve the unobservability of metric scale under degenerate cases with such systems. Second, the thesis focuses on the further improvement of front-end relative pose estimation for vehicle-mounted surround-view multi-camera systems. It presents a new, reliable solution able to handle all kinds of relative displacements in the plane despite the possibly non-holonomic characteristics, and furthermore introduces a novel two-view optimization scheme which minimizes a geometrically relevant error without relying on 3D points related optimization variables. Third, the thesis explores the continues-time parametrization for exact modelling of non-holonomic ground vehicle trajectories in the back-end optimization of visual SLAM pipeline. It demonstrates the use of B-splines for an exact imposition of smooth, non-holonomic trajectories inside the 6 DoF bundle adjustment, and show that a significant improvement in robustness and accuracy in degrading visual conditions can be achieved. In order to deal with challenges in scenarios with high dynamics, low texture distinctiveness, or challenging illumination conditions, the thesis focuses on the solution to localization and mapping problem on Autonomous Ground Vehicle(AGV) using event cameras. Inspired by the time-continuous parametrizations of image warping functions introduced by previous works, the thesis proposes two new algorithms to tackle several motion estimation problems by performing contrast maximization approach. It firstly looks at the fronto-parallel motion estimation of an event camera, in stark contrast to the prior art, a globally optimal solution to this motion estimation problem is derived by using a branch-and-bound optimization scheme. Then, the thesis introduces a new solution to handle the localization and mapping problem of single event camera by continuous ray warping and volumetric contrast maximization, which can perform joint optimization over motion and structure for cameras exerting both translational and rotational displacements in an arbitrarily structured environment. The present thesis thus makes important contributions on both front-end and back-end of SLAM pipelines based on novel, promising camera architectures

Localization in Low Luminance, Slippery Indoor Environment Using Afocal Optical Flow Sensor and Image Processing

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·정보공학부, 2017. 8. 조동일.실내 서비스로봇의 위치 추정은 자율 주행을 위한 필수 요건이다. 특히 카메라로 위치를 추정하기 어려운 실내 저조도 환경에서 미끄러짐이 발생할 경우에는 위치 추정의 정확도가 낮아진다. 미끄러짐은 주로 카펫이나 문턱 등을 주행할 때 발생하며, 휠 엔코더 기반의 주행기록으로는 주행 거리의 정확한 인식에 한계가 있다. 본 논문에서는 카메라 기반 동시적 위치추정 및 지도작성 기술(simultaneous localization and mappingSLAM)이 동작하기 어려운 저조도, 미끄러운 환경에서 저가의 모션센서와 무한초점 광학흐름센서(afocal optical flow sensorAOFS) 및 VGA급 전방 단안카메라를 융합하여 강인하게 위치를 추정하는 방법을 제안했다. 로봇의 위치 추정은 주행거리 순간 변화량과 방위각 순간 변화량을 누적 융합하여 산출했으며, 미끄러운 환경에서도 좀 더 정확한 주행거리 추정을 위해 휠 엔코더와 AOFS로부터 획득한 이동 변위 정보를 융합했고, 방위각 추정을 위해 각속도 센서와 전방 영상으로부터 파악된 실내 공간정보를 활용했다. 광학흐름센서는 바퀴 미끄러짐에 강인하게 이동 변위를 추정 하지만, 카펫처럼 평평하지 않은 표면을 주행하는 이동 로봇에 광학흐름센서를 장착할 경우, 주행 중 발생하는 광학흐름센서와 바닥 간의 높이 변화가 광학흐름센서를 이용한 이동거리 추정 오차의 주요인으로 작용한다. 본 논문에서는 광학흐름센서에 무한초점계 원리를 적용하여 이 오차 요인을 완화하는 방안을 제시하였다. 로봇 문형 시스템(robotic gantry system)을 이용하여 카펫 및 세가지 종류의 바닥재질에서 광학흐름센서의 높이를 30 mm 에서 50 mm 로 변화시키며 80 cm 거리를 이동하는 실험을 10번씩 반복한 결과, 본 논문에서 제안하는 AOFS 모듈은 1 mm 높이 변화 당 0.1% 의 계통오차(systematic error)를 발생시켰으나, 기존의 고정초점방식의 광학흐름센서는 14.7% 의 계통오차를 나타냈다. 실내 이동용 서비스 로봇에 AOFS를 장착하여 카펫 위에서 1 m 를 주행한 결과 평균 거리 추정 오차는 0.02% 이고, 분산은 17.6% 인 반면, 고정초점 광학흐름센서를 로봇에 장착하여 같은 실험을 했을 때에는 4.09% 의 평균 오차 및 25.7% 의 분산을 나타냈다. 주위가 너무 어두워서 영상을 위치 보정에 사용하기 어려운 경우, 즉, 저조도 영상을 밝게 개선했으나 SLAM에 활용할 강인한 특징점 혹은 특징선을 추출하기 어려운 경우에도 로봇 주행 각도 보정에 저조도 이미지를 활용하는 방안을 제시했다. 저조도 영상에 히스토그램 평활화(histogram equalization) 알고리즘을 적용하면 영상이 밝게 보정 되면서 동시에 잡음도 증가하게 되는데, 영상 잡음을 없애는 동시에 이미지 경계를 뚜렷하게 하는 롤링 가이던스 필터(rolling guidance filterRGF)를 적용하여 이미지를 개선하고, 이 이미지에서 실내 공간을 구성하는 직교 직선 성분을 추출 후 소실점(vanishing pointVP)을 추정하고 소실점을 기준으로 한 로봇 상대 방위각을 획득하여 각도 보정에 활용했다. 제안하는 방법을 로봇에 적용하여 0.06 ~ 0.21 lx 의 저조도 실내 공간(77 sqm)에 카펫을 설치하고 주행했을 경우, 로봇의 복귀 위치 오차가 기존 401 cm 에서 21 cm로 줄어듦을 확인할 수 있었다.제 1 장 서 론 1 1.1 연구의 배경 1 1.2 선행 연구 조사 6 1.2.1 실내 이동형 서비스 로봇의 미끄러짐 감지 기술 6 1.2.2 저조도 영상 개선 기술 8 1.3 기여도 12 1.4 논문의 구성 14 제 2 장 무한초점 광학흐름센서(AOFS) 모듈 16 2.1 무한초점 시스템(afocal system) 16 2.2 바늘구멍 효과 18 2.3 무한초점 광학흐름센서(AOFS) 모듈 프로토타입 20 2.4 무한초점 광학흐름센서(AOFS) 모듈 실험 계획 24 2.5 무한초점 광학흐름센서(AOFS) 모듈 실험 결과 29 제 3 장 저조도영상의 방위각보정 활용방법 36 3.1 저조도 영상 개선 방법 36 3.2 한 장의 영상으로 실내 공간 파악 방법 38 3.3 소실점 랜드마크를 이용한 로봇 각도 추정 41 3.4 최종 주행기록 알고리즘 46 3.5 저조도영상의 방위각 보정 실험 계획 48 3.6 저조도영상의 방위각 보정 실험 결과 50 제 4 장 저조도 환경 위치인식 실험 결과 54 4.1 실험 환경 54 4.2 시뮬레이션 실험 결과 59 4.3 임베디드 실험 결과 61 제 5 장 결론 62Docto

Visual control of multi-rotor UAVs

Recent miniaturization of computer hardware, MEMs sensors, and high energy density batteries have enabled highly capable mobile robots to become available at low cost. This has driven the rapid expansion of interest in multi-rotor unmanned aerial vehicles. Another area which has expanded simultaneously is small powerful computers, in the form of smartphones, which nearly always have a camera attached, many of which now contain a OpenCL compatible graphics processing units. By combining the results of those two developments a low-cost multi-rotor UAV can be produced with a low-power onboard computer capable of real-time computer vision. The system should also use general purpose computer vision software to facilitate a variety of experiments. To demonstrate this I have built a quadrotor UAV based on control hardware from the Pixhawk project, and paired it with an ARM based single board computer, similar those in high-end smartphones. The quadrotor weights 980 g and has a flight time of 10 minutes. The onboard computer capable of running a pose estimation algorithm above the 10 Hz requirement for stable visual control of a quadrotor. A feature tracking algorithm was developed for efficient pose estimation, which relaxed the requirement for outlier rejection during matching. Compared with a RANSAC- only algorithm the pose estimates were less variable with a Z-axis standard deviation 0.2 cm compared with 2.4 cm for RANSAC. Processing time per frame was also faster with tracking, with 95 % confidence that tracking would process the frame within 50 ms, while for RANSAC the 95 % confidence time was 73 ms. The onboard computer ran the algorithm with a total system load of less than 25 %. All computer vision software uses the OpenCV library for common computer vision algorithms, fulfilling the requirement for running general purpose software. The tracking algorithm was used to demonstrate the capability of the system by per- forming visual servoing of the quadrotor (after manual takeoff). Response to external perturbations was poor however, requiring manual intervention to avoid crashing. This was due to poor visual controller tuning, and to variations in image acquisition and attitude estimate timing due to using free running image acquisition. The system, and the tracking algorithm, serve as proof of concept that visual control of a quadrotor is possible using small low-power computers and general purpose computer vision software

Planetary micro-rover operations on Mars using a Bayesian framework for inference and control

With the recent progress toward the application of commercially-available hardware to small-scale space missions, it is now becoming feasible for groups of small, efficient robots based on low-power embedded hardware to perform simple tasks on other planets in the place of large-scale, heavy and expensive robots. In this paper, we describe design and programming of the Beaver micro-rover developed for Northern Light, a Canadian initiative to send a small lander and rover to Mars to study the Martian surface and subsurface. For a small, hardware-limited rover to handle an uncertain and mostly unknown environment without constant management by human operators, we use a Bayesian network of discrete random variables as an abstraction of expert knowledge about the rover and its environment, and inference operations for control. A framework for efficient construction and inference into a Bayesian network using only the C language and fixed-point mathematics on embedded hardware has been developed for the Beaver to make intelligent decisions with minimal sensor data. We study the performance of the Beaver as it probabilistically maps a simple outdoor environment with sensor models that include uncertainty. Results indicate that the Beaver and other small and simple robotic platforms can make use of a Bayesian network to make intelligent decisions in uncertain planetary environments