Ego-motion estimation using rectified stereo and bilateral transfer function

We describe an ego-motion algorithm based on dense spatio-temporal correspondences, using semi-global stereo matching (SGM) and bilateral image warping in time. The main contribution is an improvement in accuracy and robustness of such techniques, by taking care of speed and numerical stability, while employing twice the structure and data for the motion estimation task, in a symmetric way. In our approach we keep the tasks of structure and motion estimation separated, respectively solved by the SGM and by our pose estimation algorithm. Concerning the latter, we show the benefits introduced by our rectified, bilateral formulation, that provides at the same time more robustness to noise and disparity errors, at the price of a moderate increase in computational complexity, further reduced by an improved Gauss-Newton descent

Development of wireless control system for a spherical robot

The purpose of this thesis was to develop a control method which can reduce oscillation of lateral motion for a pendulum driven spherical robot operating on flat surface. The spherical robot provides a unique mobility and has several applications in surveillance and entertainment. Controlling a spherical robot is a challenging problem till today due to its nature of kinematics and dynamics. Firstly, its nonholonomic nature prohibits the use of conventional state feedback control laws. Secondly, kinematics of a spherical robot cannot be expressed as a chained-form system to utilize nonholonomic control algorithms. However, various types of nonlinear control algorithms were proposed to settle the problem though none of them provided satisfactory result. The kinematics and dynamics of the pendulum driven spherical robot was investigated followed by linearization for longitudinal and lateral motions through frequency and state space transformation. Moreover, the controllability of the states of the system was maintained during linearization. A robust self-tuning sliding mode con troller which suspends oscillation, maintains desired speed and compensates for unmodeled parameters was developed. The implemented control system consists of control station, prototype robot equipped with on-board microcontroller and sensors, and wireless communication link. Simulation and experimentation were conducted to test peformance of the control laws in suppressing oscillation and maintaining desired speed of the robot. The robot traveled to the commanded trajectory containing straight line and curve with relatively minimum oscillation at desired speed. Thus, the sliding mode control is an effective controller

Visual Tracking and Motion Estimation for an On-orbit Servicing of a Satellite

This thesis addresses visual tracking of a non-cooperative as well as a partially cooperative satellite, to enable close-range rendezvous between a servicer and a target satellite. Visual tracking and estimation of relative motion between a servicer and a target satellite are critical abilities for rendezvous and proximity operation such as repairing and deorbiting. For this purpose, Lidar has been widely employed in cooperative rendezvous and docking missions. Despite its robustness to harsh space illumination, Lidar has high weight and rotating parts and consumes more power, thus undermines the stringent requirements of a satellite design. On the other hand, inexpensive on-board cameras can provide an effective solution, working at a wide range of distances. However, conditions of space lighting are particularly challenging for image based tracking algorithms, because of the direct sunlight exposure, and due to the glossy surface of the satellite that creates strong reflection and image saturation, which leads to difficulties in tracking procedures. In order to address these difficulties, the relevant literature is examined in the fields of computer vision, and satellite rendezvous and docking. Two classes of problems are identified and relevant solutions, implemented on a standard computer are provided. Firstly, in the absence of a geometric model of the satellite, the thesis presents a robust feature-based method with prediction capability in case of insufficient features, relying on a point-wise motion model. Secondly, we employ a robust model-based hierarchical position localization method to handle change of image features along a range of distances, and localize an attitude-controlled (partially cooperative) satellite. Moreover, the thesis presents a pose tracking method addressing ambiguities in edge-matching, and a pose detection algorithm based on appearance model learning. For the validation of the methods, real camera images and ground truth data, generated with a laboratory tet bed similar to space conditions are used. The experimental results indicate that camera based methods provide robust and accurate tracking for the approach of malfunctioning satellites in spite of the difficulties associated with specularities and direct sunlight. Also exceptional lighting conditions associated to the sun angle are discussed, aimed at achieving fully reliable localization system in a certain mission

Data fusion framework for planetary and orbital robotics applications

In space robotics, a wide range of sensor data fusion methods are required to accomplish challenging objectives for exploration, science and commercial purposes. This includes navigation for planetary and guidance for orbital robotics, scientific prospecting, and on-orbit servicing. In Fuse provides a comprehensive data fusion framework or toolset to fuse and interpret sensor data from multiple sensors. This project represents an optimal approach to develop software for robotics: a standardized and comprehensive development environment for industrial applications, with particular focus on space applications where components can be connected, tested offline, evaluated and deployed in any preferred robotic framework, including those devised for space or terrestrial applications. This paper discusses the results of verification and validation of data fusion methods for robots deployed in orbital and planetary scenarios using data sets collected in simulation and outdoor analogue campaigns

Design and Operational Elements of the Robotic Subsystem for the e.deorbit Debris Removal Mission

This paper presents a robotic capture concept that was developed as part of the e.deorbit study by ESA. The defective and tumbling satellite ENVISAT was chosen as a potential target to be captured, stabilized, and subsequently de-orbited in a controlled manner. A robotic capture concept was developed that is based on a chaser satellite equipped with a seven degrees-of-freedom dexterous robotic manipulator, holding a dedicated linear two-bracket gripper. The satellite is also equipped with a clamping mechanism for achieving a stiff fixation with the grasped target, following their combined satellite-stack de-tumbling and prior to the execution of the de-orbit maneuver. Driving elements of the robotic design, operations and control are described and analyzed. These include pre and post-capture operations, the task-specific kinematics of the manipulator, the intrinsic mechanical arm flexibility and its effect on the arm's positioning accuracy, visual tracking, as well as the interaction between the manipulator controller and that of the chaser satellite. The kinematics analysis yielded robust reachability of the grasp point. The effects of intrinsic arm flexibility turned out to be noticeable but also effectively scalable through robot joint speed adaption throughout the maneuvers. During most of the critical robot arm operations, the internal robot joint torques are shown to be within the design limits. These limits are only reached for a limiting scenario of tumbling motion of ENVISAT, consisting of an initial pure spin of 5 deg/s about its unstable intermediate axis of inertia. The computer vision performance was found to be satisfactory with respect to positioning accuracy requirements. Further developments are necessary and are being pursued to meet the stringent mission-related robustness requirements. Overall, the analyses conducted in this study showed that the capture and de-orbiting of ENVISAT using the proposed robotic concept is feasible with respect to relevant mission requirements and for most of the operational scenarios considered. Future work aims at developing a combined chaser-robot system controller. This will include a visual servo to minimize the positioning errors during the contact phases of the mission (grasping and clamping). Further validation of the visual tracking in orbital lighting conditions will be pursued

Monocular 3D Pose Tracking of a Specular Object

A space object such as a satellite consists of highly specular surface, and when exposed to directional source of light, it is very difficult for visual tracking. However, camera-based tracking provides an inexpensive solution to the problem of on-orbit servicing of a satellite, such as orbital-life extension by repairing and refuelling, and debris removal. In this paper we present a real time pose tracking method applied to a such object under direct Sunlight, by adapting keypoint and edge-based approach, with known simple geometry. The implemented algorithm is relatively accurate and robust to specular reflection. We show the results which are based on real images from a simulation system of on-orbit servicing, consisting of two six degree of freedom robots, the Sun simulator and a full scale satellite mock-up

Tracking and Pose Estimation of Non-cooperative Satellite for On-orbit Servicing

We present a camera-based 3D feature tracking method, integrated into a bank of iterated, extended Kalman filters (IEKF), associated to each visual feature. The approach exploits motion field to estimate velocities of the rigid body. The depths are required only for initialization, and can be obtained either from joint estimation of depth and motion or from stereo correspondence. The motion of each 3D point cloud is predicted under a common rigid velocity constraint. A robust pose estimator, based on dual-quaternions and median statistics, is further applied to the estimated points. In case of temporarily missing measurements, the last estimated body velocity is used to predict the next poses. Results are shown on images of a satellite-mockup, to demonstrate performances for on-orbit servicing in space environment

Investigations about evaluation of nutritional and physiological effects of differently treated soybeans in chicken diets

Ziel der vorliegenden Arbeit war es, den Einfluss von unterschiedlich behandelten Sojabohnen in einer Broilerration (Mais und behandeltes Soja) auf die ernährungsphysiologischen Parameter beim wachsenden Küken zu prüfen. Unter Berücksichtigung der Einflussgrößen (z. B. Zeit, mechanischer Energieeinsatz und Futterstruktur), die die Effektivität der Futterbearbeitungsverfahren beeinflussen, wurde zunächst die Behandlung einer einheitlichen Charge von Vollfettsojabohnen unter folgenden technische Bedingungen vorgenommen: A: Walzenstuhl (WS) + Konditionierung (40 min./100 °C, Standardverfahren); B: WS + Konditionierung (40 min./100 °C) + Flockierung; C: WS + Konditionierung (10 min./100 °C); D: WS + Konditionierung (10 min. /100 °C) + Expandieren (20 kWh/t); E: Hammermühle (HM) + Konditionierung (10 min./100 °C) + Extrusion (15 kWh/t); F: HM + Konditionierung (10 min./100 °C), G: HM + Konditionierung (10 min./100 °C) + Expandieren (20 kWh/t); H: HM + Expandieren (20 kWh/t); I: HM + Expandieren (20 kWh/t) mit Dampfzufuhr; K: HM + Expandieren (40 kWh/t). Mit 6 ´ 10 männlichen Broilern der Herkunft Cobb wurde im Zeitraum vom 7. - 28. Lebenstag (LT) je Behandlung ein Wachstumsversuch durchgeführt. Während des Wachstumsversuchs wurden Daten zum Futterverzehr, Lebendmassezuwachs und Futteraufwand sowie Energie- und Proteinansatz über Körperanalyse erfaßt. Die ileale Aminosäureverdaulichkeit der einzelnen Mischungen wurde in jeweils 4 gepoolten Chymusproben (9 Tiere / gepoolte Probe) mit Hilfe eines Indikators (HCl-unlösliche Rohasche, Zusatz von 1 % Celite) nach Verfütterung vom 21. - 28. LT bestimmt. Parallel zum Wachstumsversuch wurde ein Stoffwechselversuch (6 männliche Küken der Herkunft Cobb je Behandlungsstufe) im Zeitraum vom 15. - 21. LT bei Fütterung der Versuchsmischung durchgeführt. Dabei erfolgte 3-mal täglich eine quantitative Exkrementsammlung. Neben der N- Bestimmung in den Exkrementen der Basis für die N-Bilanzmessung wurde der Bruttoenergiegehalt erfaßt und auf dieser Grundlage der Gehalt an N-korrigierter umsetzbarer Energie (MEn) in den Futtermischungen bestimmt. Weiterhin wurde aus den Daten der Bilanzmessungen der physiologische Nutzwert (PNu) als Kriterium der Proteinqualitätsbeurteilung sowie die Lysinwirksamkeit abgeleitet. Am Ende des Stoffwechselversuchs wurden die Tiere geschlachtet und die Trypsinaktivität im Chymus des Jejunums bestimmt. Die Ergebnisse lassen sich folgendermaßen zusammenfassen: 1. In Abhängigkeit von den gegebenen Bedingungen der Futterbehandlungsverfahren konnten unterschiedliche Trypsininhibitoraktivitäten (TIA) in den behandelten Sojabohnen erzielt werden. Eine starke Reduzierung des TIA war insbesondere bei den behandelten Sojabohnen A bis G erkennbar. Die relative Restaktivität lag bei diesen Gruppen (im Vergleich zur rohen Sojabohne) bei 16-27 %. Im Gegensatz dazu wurde eine höhere relative Restaktivität in den behandelten Sojabohnen H, I und K in Höhe von 75 %, 32 % und 48 % festgestellt . 2. Die Futteraufnahme, Lebendmassezunahme und Futterverwertung waren bei der Gruppe H infolge der höchsten Restaktivität des Trypsininhibitors signifikant niedriger als bei der Kontrolle A und anderen Gruppen (B, C, D, E, F, G, I und K). Analog dazu führte die höchste TIA in den Sojabohnen H ebenfalls zu einem signifikant verminderten Nährstoffansatz im Ganzkörper. Der Einfluss der höchsten Restaktivität des Trypsininhibitors auf den PNu war bei Gruppe H wenig ausgeprägt. 3. Die N-korrigierte umsetzbare Energie in den Futtermischungen B (14,87 MJ /kg T) und F (14,70 MJ /kg T) war signifikant höher als bei der Kontrolle A (13,96 MJ /kg). Der Grund für die Steigerung der N-korrigierten umsetzbaren Energie bei B und F kann dabei in der möglichen Verbesserung der Energieverfügbarkeit durch die Behandlungsverfahren Flockierung und Expander liegen. 4. Aufgrund der höchsten Restaktivität des Trypsininhibitors wurde bei der Gruppe H eine verminderte ileale Aminosäure- und Stickstoffverdaulichkeit im Vergleich zu Kontrolle A und den anderen Gruppen (B, C, D un E) festgestellt. Durch zusätzliche Dampfzufuhr und Erhöhung des Energieeintrages in den Gruppen I bzw. K konnten die antinutritiven Faktoren in den Sojabohnen reduziert werden und demzufolge die ileale Aminosäure- und Stickstoffverdaulichkeit bei den Gruppen I und K gegenüber der Gruppe H signifikant verbessert werden. Obwohl bei der Gruppe H die ileale Lysinverdaulichkeit gegenüber der Kontrolle A signifikant niedriger lag, wurde in Bezug auf die Lysinwirksamkeit kein signifikanter Unterschied zwischen den Gruppen H und der Kontrolle A festgestellt. 5. Die Trypsinaktivität im Chymus wurde auch in Abhängigkeit von der Restaktivität des Trypsininhibitors in den Sojabohnen unterschiedlich beeinflusst. Eine signifikant verminderte Trypsinaktivität im Chymus war insbesondere bei den Gruppen H sowie K feststellbar

Visual Tracking and Motion Estimation for an On-orbit Servicing of a Satellite

This thesis addresses visual tracking of a non-cooperative as well as a partially cooperative satellite, to enable close-range rendezvous between a servicer and a target satellite. Visual tracking and estimation of relative motion between a servicer and a target satellite are critical abilities for rendezvous and proximity operation such as repairing and deorbiting. For this purpose, Lidar has been widely employed in cooperative rendezvous and docking missions. Despite its robustness to harsh space illumination, Lidar has high weight and rotating parts and consumes more power, thus undermines the stringent requirements of a satellite design. On the other hand, inexpensive on-board cameras can provide an effective solution, working at a wide range of distances. However, conditions of space lighting are particularly challenging for image based tracking algorithms, because of the direct sunlight exposure, and due to the glossy surface of the satellite that creates strong reflection and image saturation, which leads to difficulties in tracking procedures. In order to address these difficulties, the relevant literature is examined in the fields of computer vision, and satellite rendezvous and docking. Two classes of problems are identified and relevant solutions, implemented on a standard computer are provided. Firstly, in the absence of a geometric model of the satellite, the thesis presents a robust feature-based method with prediction capability in case of insufficient features, relying on a point-wise motion model. Secondly, we employ a robust model-based hierarchical position localization method to handle change of image features along a range of distances, and localize an attitude-controlled (partially cooperative) satellite. Moreover, the thesis presents a pose tracking method addressing ambiguities in edge-matching, and a pose detection algorithm based on appearance model learning. For the validation of the methods, real camera images and ground truth data, generated with a laboratory tet bed similar to space conditions are used. The experimental results indicate that camera based methods provide robust and accurate tracking for the approach of malfunctioning satellites in spite of the difficulties associated with specularities and direct sunlight. Also exceptional lighting conditions associated to the sun angle are discussed, aimed at achieving fully reliable localization system in a certain mission

Appearance learning for 3D pose detection of a satellite at close-range

In this paper we present a learning-based 3D detection of a highly challenging specular object exposed to a direct sunlight at very close-range. An object detection is one of the most important areas of image processing, and can also be used for initialization of local visual tracking methods. While the object detection in 3D space is generally a difficult problem, it poses more difficulties when the object is specular and exposed to the direct sunlight as in a space environment. Our solution to a such problem relies on an appearance learning of a real satellite mock-up based on a vector quantization and the vocabulary tree. Our method, implemented on a standard computer (CPU), exploits a full perspective projection model and provides near real-time 3D pose detection of a satellite for close-range approach and manipulation. The time consuming part of the training (feature description, building the vocabulary tree and indexing, depth buffering and back-projection) are performed offline, while a fast image retrieval and 3D-2D registration are performed on-line. In contrast, the state of the art image-based 3D pose detection methods are slower on \{CPU\} or assume a weak perspective camera projection model. In our case the dimension of the satellite is larger than the distance to the camera, hence the assumption of the weak perspective model does not hold. To evaluate the proposed method, the appearance of a full scale mock-up of the rear part of the TerraSAR-X satellite is trained under various illumination and camera views. The training images are captured with a camera mounted on six degrees of freedom robot, which enables to position the camera in a desired view, sampled over a sphere. The views that are not within the workspace of the robot are interpolated using image-based rendering. Moreover, we generate ground truth poses to verify the accuracy of the detection algorithm. The achieved results are robust and accurate even under noise due to specular reflection, and able to initialize a local tracking method