Insect inspired visual motion sensing and flying robots

International audienceFlying insects excellently master visual motion sensing techniques. They use dedicated motion processing circuits at a low energy and computational costs. Thanks to observations obtained on insect visual guidance, we developed visual motion sensors and bio-inspired autopilots dedicated to flying robots. Optic flow-based visuomotor control systems have been implemented on an increasingly large number of sighted autonomous robots. In this chapter, we present how we designed and constructed local motion sensors and how we implemented bio-inspired visual guidance scheme on-board several micro-aerial vehicles. An hyperacurate sensor in which retinal micro-scanning movements are performed via a small piezo-bender actuator was mounted onto a miniature aerial robot. The OSCAR II robot is able to track a moving target accurately by exploiting the microscan-ning movement imposed to its eye's retina. We also present two interdependent control schemes driving the eye in robot angular position and the robot's body angular position with respect to a visual target but without any knowledge of the robot's orientation in the global frame. This "steering-by-gazing" control strategy, which is implemented on this lightweight (100 g) miniature sighted aerial robot, demonstrates the effectiveness of this biomimetic visual/inertial heading control strategy

Localization in Low Luminance, Slippery Indoor Environment Using Afocal Optical Flow Sensor and Image Processing

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·정보공학부, 2017. 8. 조동일.실내 서비스로봇의 위치 추정은 자율 주행을 위한 필수 요건이다. 특히 카메라로 위치를 추정하기 어려운 실내 저조도 환경에서 미끄러짐이 발생할 경우에는 위치 추정의 정확도가 낮아진다. 미끄러짐은 주로 카펫이나 문턱 등을 주행할 때 발생하며, 휠 엔코더 기반의 주행기록으로는 주행 거리의 정확한 인식에 한계가 있다. 본 논문에서는 카메라 기반 동시적 위치추정 및 지도작성 기술(simultaneous localization and mappingSLAM)이 동작하기 어려운 저조도, 미끄러운 환경에서 저가의 모션센서와 무한초점 광학흐름센서(afocal optical flow sensorAOFS) 및 VGA급 전방 단안카메라를 융합하여 강인하게 위치를 추정하는 방법을 제안했다. 로봇의 위치 추정은 주행거리 순간 변화량과 방위각 순간 변화량을 누적 융합하여 산출했으며, 미끄러운 환경에서도 좀 더 정확한 주행거리 추정을 위해 휠 엔코더와 AOFS로부터 획득한 이동 변위 정보를 융합했고, 방위각 추정을 위해 각속도 센서와 전방 영상으로부터 파악된 실내 공간정보를 활용했다. 광학흐름센서는 바퀴 미끄러짐에 강인하게 이동 변위를 추정 하지만, 카펫처럼 평평하지 않은 표면을 주행하는 이동 로봇에 광학흐름센서를 장착할 경우, 주행 중 발생하는 광학흐름센서와 바닥 간의 높이 변화가 광학흐름센서를 이용한 이동거리 추정 오차의 주요인으로 작용한다. 본 논문에서는 광학흐름센서에 무한초점계 원리를 적용하여 이 오차 요인을 완화하는 방안을 제시하였다. 로봇 문형 시스템(robotic gantry system)을 이용하여 카펫 및 세가지 종류의 바닥재질에서 광학흐름센서의 높이를 30 mm 에서 50 mm 로 변화시키며 80 cm 거리를 이동하는 실험을 10번씩 반복한 결과, 본 논문에서 제안하는 AOFS 모듈은 1 mm 높이 변화 당 0.1% 의 계통오차(systematic error)를 발생시켰으나, 기존의 고정초점방식의 광학흐름센서는 14.7% 의 계통오차를 나타냈다. 실내 이동용 서비스 로봇에 AOFS를 장착하여 카펫 위에서 1 m 를 주행한 결과 평균 거리 추정 오차는 0.02% 이고, 분산은 17.6% 인 반면, 고정초점 광학흐름센서를 로봇에 장착하여 같은 실험을 했을 때에는 4.09% 의 평균 오차 및 25.7% 의 분산을 나타냈다. 주위가 너무 어두워서 영상을 위치 보정에 사용하기 어려운 경우, 즉, 저조도 영상을 밝게 개선했으나 SLAM에 활용할 강인한 특징점 혹은 특징선을 추출하기 어려운 경우에도 로봇 주행 각도 보정에 저조도 이미지를 활용하는 방안을 제시했다. 저조도 영상에 히스토그램 평활화(histogram equalization) 알고리즘을 적용하면 영상이 밝게 보정 되면서 동시에 잡음도 증가하게 되는데, 영상 잡음을 없애는 동시에 이미지 경계를 뚜렷하게 하는 롤링 가이던스 필터(rolling guidance filterRGF)를 적용하여 이미지를 개선하고, 이 이미지에서 실내 공간을 구성하는 직교 직선 성분을 추출 후 소실점(vanishing pointVP)을 추정하고 소실점을 기준으로 한 로봇 상대 방위각을 획득하여 각도 보정에 활용했다. 제안하는 방법을 로봇에 적용하여 0.06 ~ 0.21 lx 의 저조도 실내 공간(77 sqm)에 카펫을 설치하고 주행했을 경우, 로봇의 복귀 위치 오차가 기존 401 cm 에서 21 cm로 줄어듦을 확인할 수 있었다.제 1 장 서 론 1 1.1 연구의 배경 1 1.2 선행 연구 조사 6 1.2.1 실내 이동형 서비스 로봇의 미끄러짐 감지 기술 6 1.2.2 저조도 영상 개선 기술 8 1.3 기여도 12 1.4 논문의 구성 14 제 2 장 무한초점 광학흐름센서(AOFS) 모듈 16 2.1 무한초점 시스템(afocal system) 16 2.2 바늘구멍 효과 18 2.3 무한초점 광학흐름센서(AOFS) 모듈 프로토타입 20 2.4 무한초점 광학흐름센서(AOFS) 모듈 실험 계획 24 2.5 무한초점 광학흐름센서(AOFS) 모듈 실험 결과 29 제 3 장 저조도영상의 방위각보정 활용방법 36 3.1 저조도 영상 개선 방법 36 3.2 한 장의 영상으로 실내 공간 파악 방법 38 3.3 소실점 랜드마크를 이용한 로봇 각도 추정 41 3.4 최종 주행기록 알고리즘 46 3.5 저조도영상의 방위각 보정 실험 계획 48 3.6 저조도영상의 방위각 보정 실험 결과 50 제 4 장 저조도 환경 위치인식 실험 결과 54 4.1 실험 환경 54 4.2 시뮬레이션 실험 결과 59 4.3 임베디드 실험 결과 61 제 5 장 결론 62Docto

MEMS Technology for Biomedical Imaging Applications

Biomedical imaging is the key technique and process to create informative images of the human body or other organic structures for clinical purposes or medical science. Micro-electro-mechanical systems (MEMS) technology has demonstrated enormous potential in biomedical imaging applications due to its outstanding advantages of, for instance, miniaturization, high speed, higher resolution, and convenience of batch fabrication. There are many advancements and breakthroughs developing in the academic community, and there are a few challenges raised accordingly upon the designs, structures, fabrication, integration, and applications of MEMS for all kinds of biomedical imaging. This Special Issue aims to collate and showcase research papers, short commutations, perspectives, and insightful review articles from esteemed colleagues that demonstrate: (1) original works on the topic of MEMS components or devices based on various kinds of mechanisms for biomedical imaging; and (2) new developments and potentials of applying MEMS technology of any kind in biomedical imaging. The objective of this special session is to provide insightful information regarding the technological advancements for the researchers in the community

Vision-based control of near-obstacle flight

Lightweight micro unmanned aerial vehicles (micro-UAVs) capable of autonomous flight in natural and urban environments have a large potential for civil and commercial applications, including environmental monitoring, forest fire monitoring, homeland security, traffic monitoring, aerial imagery, mapping and search and rescue. Smaller micro-UAVs capable of flying inside houses or small indoor environments have further applications in the domain of surveillance, search and rescue and entertainment. These applications require the capability to fly near to the ground and amongst obstacles. Existing UAVs rely on GPS and AHRS (attitude heading reference system) to control their flight and are unable to detect and avoid obstacles. Active distance sensors such as radars or laser range finders could be used to measure distances to obstacles, but are typically too heavy and power-consuming to be embedded on lightweight systems. In this thesis, we draw inspiration from biology and explore alternative approaches to flight control that allow aircraft to fly near obstacles. We show that optic flow can be used on flying platforms to estimate the proximity of obstacles and propose a novel control strategy, called optiPilot, for vision-based near-obstacle flight. Thanks to optiPilot, we demonstrate for the first time autonomous near-obstacle flight of micro-UAVs, both indoor and outdoor, without relying on an AHRS nor external beacons such as GPS. The control strategy only requires a small series of optic flow sensors, two rate gyroscopes and an airspeed sensor. It can run on a tiny embedded microcontroller in realtime. Despite its simplicity, optiPilot is able to fully control the aircraft, including altitude regulation, attitude stabilisation, obstacle avoidance, landing and take-off. This parsimony, inherited from the biology of flying insects, contrasts with the complexity of the systems used so far for flight control while offering more capabilities. The results presented in this thesis contribute to a better understanding of the minimal requirements, in terms of sensing and control architecture, that enable animals and artificial systems to fly and bring closer to reality the perspective of using lightweight and inexpensive micro-UAV for civilian purposes

Optic Flow for Obstacle Avoidance and Navigation: A Practical Approach

This thesis offers contributions and innovations to the development of vision-based autonomous flight control systems for small unmanned aerial vehicles operating in cluttered urban environments. Although many optic flow algorithms have been reported, almost none have addressed the critical issue of accuracy and reliability over a wide dynamic range of optic flow. My aim is to rigorously develop improved optic flow sensing to meet realistic mission requirements for autonomous navigation and collision avoidance. A review of related work enabled development of a new hybrid optic flow algorithm concept combining the best properties of image correlation and interpolation with additional innovations to enhance accuracy, computational speed and reliability. Key analytical work yielded a methodology for determining optic flow dynamic range requirements from system and sensor design parameters and a technique enabling a video sensor to operate as a passive ranging system for closed loop flight control. Detailed testing led to development of the hybrid image interpolation algorithm (HI2A) using improved correlation search strategies, sparse images to reduce processing loads, a solution tracking loop to bypass the more intensive initial estimation process, a frame look-back method to improve accuracy at low optic flow, a modified interpolation technique to improve robustness and an extensive error checking system for validating outputs. A realistic simulation system was developed incorporating independent, precision ground truthing to assess algorithm accuracy. Comparison testing of the HI2A against the commonly-used Lucas Kanade algorithm demonstrates major improvement in accuracy over greatly expanded dynamic range. A reactive flight controller using ranging data from a monocular, forward looking video sensor and rules-based logic was developed and tested in Monte Carlo simulations of a hundred flights. At higher flight speeds than reported in similar tests, collision-free results were obtained in a realistic urban canyon environment. The HI2A algorithm and flight controller software performance on a common PC was up to eight times faster than real-time for outputs of 250 measurements at 50 Hz. The feasibility of terrain mapping in real-time was demonstrated using 3D ranging data from optic flow in an overflight of the urban simulation environment indicating the potential for its use in path planning approaches to navigation and collision avoidance

Accelerated neuromorphic cybernetics

Accelerated mixed-signal neuromorphic hardware refers to electronic systems that emulate electrophysiological aspects of biological nervous systems in analog voltages and currents in an accelerated manner. While the functional spectrum of these systems already includes many observed neuronal capabilities, such as learning or classification, some areas remain largely unexplored. In particular, this concerns cybernetic scenarios in which nervous systems engage in closed interaction with their bodies and environments. Since the control of behavior and movement in animals is both the purpose and the cause of the development of nervous systems, such processes are, however, of essential importance in nature. Besides the design of neuromorphic circuit- and system components, the main focus of this work is therefore the construction and analysis of accelerated neuromorphic agents that are integrated into cybernetic chains of action. These agents are, on the one hand, an accelerated mechanical robot, on the other hand, an accelerated virtual insect. In both cases, the sensory organs and actuators of their artificial bodies are derived from the neurophysiology of the biological prototypes and are reproduced as faithfully as possible. In addition, each of the two biomimetic organisms is subjected to evolutionary optimization, which illustrates the advantages of accelerated neuromorphic nervous systems through significant time savings

Using reconstructed visual reality in ant navigation research

Insects have low resolution eyes and a tiny brain, yet they continuously solve very complex navigational problems; an ability that underpins fundamental biological processes such as pollination and parental care. Understanding the methods they employ would have profound impact on the fields of machine vision and robotics. As our knowledge on insect navigation grows, our physical, physiological and neural models get more complex and detailed. To test these models we need to perform increasingly sophisticated experiments. Evolution has optimised the animals to operate in their natural environment. To probe the fine details of the methods they utilise we need to use natural visual scenery which, for experimental purposes, we must be able to manipulate arbitrarily. Performing physiological experiments on insects outside the laboratory is not practical and our ability to modify the natural scenery for outdoor behavioural experiments is very limited. The solution is reconstructed visual reality, a projector that can present the visual aspect of the natural environment to the animal with high fidelity, taking the peculiarities of insect vision into account. While projectors have been used in insect research before, during my candidature I designed and built a projector specifically tuned to insect vision. To allow the ant to experience a full panoramic view, the projector completely surrounds her. The device (Antarium) is a polyhedral approximation of a sphere. It contains 20 thousand pixels made out of light emitting diodes (LEDs) that match the spectral sensitivity of Myrmecia. Insects have a much higher fusion frequency limit than humans, therefore the device has a very high flicker frequency (9kHz) and also a high frame rate (190fps). In the Antarium the animal is placed in the centre of the projector on a trackball. To test the trackball and to collect reference data, outdoor experiments were performed where ants were captured, tethered and placed on the trackball. The apparatus with the ant on it was then placed at certain locations relative to the nest and the foraging tree and the movements of the animal on the ball were recorded and analysed. The outdoor experiments proved that the trackball was well suited for our ants, and also provided the baseline behaviour reference for the subsequent Antarium experiments. To assess the Antarium, the natural habitat of the experimental animals was recreated as a 3-dimensional model. That model was then projected for the ants and their movements on the trackball was recorded, just like in the outdoor experiments Initial feasibility tests were performed by projecting a static image, which matches what the animals experienced during the outdoor experiments. To assess whether the ant was orienting herself relative to the scene we rotated the projected scene around her and her response monitored. Statistical methods were used to compare the outdoor and in-Antarium behaviour. The results proved that the concept was solid, but they also uncovered several shortcomings of the Antarium. Nevertheless, even with its limitations the Antarium was used to perform experiments that would be very hard to do in a real environment. In one experiment the foraging tree was repositioned in or deleted from the scene to see whether the animals go to where the tree is or where by their knowledge it should be. The results suggest the latter but the absence or altered location of the foraging tree certainly had a significant effect on the animals. In another experiment the scene, including the sky, were re-coloured to see whether colour plays a significant role in navigation. Results indicate that even very small amount of UV information statistically significantly improves the navigation of the animals. To rectify the device limitations discovered during the experiments a new, improved projector was designed and is currently being built