Low latency vision-based control for robotics : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Mechatronics at Massey University, Manawatu, New Zealand

In this work, the problem of controlling a high-speed dynamic tracking and interception system using computer vision as the measurement unit was explored. High-speed control systems alone present many challenges, and these challenges are compounded when combined with the high volume of data processing required by computer vision systems. A semi-automated foosball table was chosen as the test-bed system because it combines all the challenges associated with a vision-based control system into a single platform. While computer vision is extremely useful and can solve many problems, it can also introduce many problems such as latency, the need for lens and spatial calibration, potentially high power consumption, and high cost. The objective of this work is to explore how to implement computer vision as the measurement unit in a high-speed controller, while minimising latencies caused by the vision itself, communication interfaces, data processing/strategy, instruction execution, and actuator control. Another objective was to implement the solution in one low-latency, low power, low cost embedded system. A field programmable gate array (FPGA) system on chip (SoC), which combines programmable digital logic with a dual core ARM processor (HPS) on the same chip, was hypothesised to be capable of running the described vision-based control system. The FPGA was used to perform streamed image pre-processing, concurrent stepper motor control and provide communication channels for user input, while the HPS performed the lens distortion mapping, intercept calculation and “strategy” control tasks, as well as controlling overall function of the system. Individual vision systems were compared for latency performance. Interception performance of the semi-automated foosball table was then tested for straight, moderate-speed shots with limited view time, and latency was artificially added to the system and the interception results for the same, centre-field shot tested with a variety of different added latencies. The FPGA based system performed the best in both steady-state latency, and novel event detection latency tests. The developed stepper motor control modules performed well in terms of speed, smoothness, resource consumption, and versatility. They are capable of constant velocity, constant acceleration and variable acceleration profiles, as well as being completely parameterisable. The interception modules on the foosball table achieved a 100% interception rate, with a confidence interval of 95%, and reliability of 98.4%. As artificial latency was added to the system, the performance dropped in terms of overall number of successful intercepts. The decrease in performance was roughly linear with a 60% in reduction in performance caused by 100 ms of added latency. Performance dropped to 0% successful intercepts when 166 ms of latency was added. The implications of this work are that FPGA SoC technology may, in future, enable computer vision to be used as a general purpose, high-speed measurement system for a wide variety of control problems

PANIC: the new panoramic NIR camera for Calar Alto

PANIC is a wide-field NIR camera, which is currently under development for the Calar Alto observatory (CAHA) in Spain. It uses a mosaic of four Hawaii-2RG detectors and covers the spectral range from 0.8-2.5 micron(z to K-band). The field-of-view is 30x30 arcmin. This instrument can be used at the 2.2m telescope (0.45arcsec/pixel, 0.5x0.5 degree FOV) and at the 3.5m telescope (0.23arcsec/pixel, 0.25x0.25 degree FOV). The operating temperature is about 77K, achieved by liquid Nitrogen cooling. The cryogenic optics has three flat folding mirrors with diameters up to 282 mm and nine lenses with diameters between 130 mm and 255 mm. A compact filter unit can carry up to 19 filters distributed over four filter wheels. Narrow band (1%) filters can be used. The instrument has a diameter of 1.1 m and it is about 1 m long. The weight limit of 400 kg at the 2.2m telescope requires a light-weight cryostat design. The aluminium vacuum vessel and radiation shield have wall thicknesses of only 6 mm and 3 mm respectively.Comment: This paper has been presented in the SPIE of Astronomical Telescopes and Instrumentation 2008 in Marseille (France

A Systems Approach to the Design of a Two Dimensional Cell Printer

Tissue engineering research has the potential to improve the current opportunities for replacement organs and tissues. Precise placement of cells has proven difficult in the early stages of this research. Bioprinting is an important research area that has the potential to accurately position cells and biological materials for tissue engineering. This paper describes the attempt to obtain micrometer accuracy in the placement of cells using a systems approach and technology derived from the Hewlett Packard 500 series of printers and their cartridges the HP26 series. The paper discusses the research and design of a custom printing system that allows control over printing parameters used to fire a droplet from the inkjet cartridge that include resolution, firing duration, and print speed. The proposed system also allows a larger workspace to print and the ability to print multiple cell types. These abilities are necessary in order to further research into this method of tissue engineering

Development of an intelligent safety gear system for high-rise elevators

Elevators have been a key element of buildings, especially tall buildings, since their widespread use began in the 19th century. As a matter of fact, high-rise buildings would not have existed without elevators. Elevators have a myriad of safety features and devices to ensure a safe journey for the passengers. One of these devices is the safety gear. Safety gears are emergency brakes that stop speeding elevators by gripping the guide rails. They are adjusted for a safe deceleration range by the technician during installation and exert a constant force. Due to their purely mechanical nature, once triggered, the safety gear is currently unable to actively adjust the braking force to counteract vibrations, to decelerate at different rates, or to stop the elevator at the closest landing. Therefore, the emergency braking event can be harsh and noticeable, leaving the passengers stuck in the elevator shaft after the braking event. This thesis aims to develop an intelligent safety gear system that is able to bring the elevator to a stop with a safe and adjustable deceleration rate. This was achieved by first, modeling a computer simulation of a small-scale elevator to be able to quickly simulate different braking event scenarios. Second, a small-scale elevator test rig was constructed to test the computer simulation with physical components. The test rig was validated by comparing its results with KONE’s high-rise safety gear test. The control system developed was able to safely stop the moving mass with the desired deceleration and a great deal of control over other parameters. Further development of the system could lead to a safer, more comfortable, and energy efficient elevator ride

The on-board calibration system of the X-ray Imaging Polarimetry Explorer (XIPE)

The calibration system for XIPE is aimed at providing a way to check and correct possible variations of performance of the Gas Pixel Detector during the three years of operation in orbit (plus two years of possible extended operation), while facilitating the observation of the celestial sources. This will be performed by using a filter wheel with a large heritage having a set of positions for the calibration and the observation systems. In particular, it will allow for correcting possible gain variation, for measuring the modulation factor using a polarized source, for removing non interesting bright sources in the field of view and for observing very bright celestial sources. The on-board calibration system is composed of three filter wheels, one for each detector and it is expected to operate for a small number of times during the year. Moreover, since it operates once at a time, within the observation mode, it allows for simultaneous calibration and acquisition from celestial sources on different detectors. In this paper we present the scope and the requirements of the on-board calibration system, its design, and a description of its possible use in space

The Solar Orbiter SPICE instrument: An extreme UV imaging spectrometer

Aims. The Spectral Imaging of the Coronal Environment (SPICE) instrument is a high-resolution imaging spectrometer operating at extreme ultraviolet (EUV) wavelengths. In this paper, we present the concept, design, and pre-launch performance of this facility instrument on the ESA/NASA Solar Orbiter mission. / Methods. The goal of this paper is to give prospective users a better understanding of the possible types of observations, the data acquisition, and the sources that contribute to the instrument’s signal. / Results. The paper discusses the science objectives, with a focus on the SPICE-specific aspects, before presenting the instrument’s design, including optical, mechanical, thermal, and electronics aspects. This is followed by a characterisation and calibration of the instrument’s performance. The paper concludes with descriptions of the operations concept and data processing. / Conclusions. The performance measurements of the various instrument parameters meet the requirements derived from the mission’s science objectives. The SPICE instrument is ready to perform measurements that will provide vital contributions to the scientific success of the Solar Orbiter mission

Modeling and Design of Digital Electronic Systems

The paper is concerned with the modern methodologies for holistic modeling of electronic systems enabling system-on-chip design. The method deals with the functional modeling of complete electronic systems using the behavioral features of Hardware Description Languages or high level languages then targeting programmable devices - mainly Field Programmable Gate Arrays (FPGAs) - for the rapid prototyping of digital electronic controllers. This approach offers major advantages such as: a unique modeling and evaluation environment for complete power systems, the same environment is used for the rapid prototyping of the digital controller, fast design development, short time to market, a CAD platform independent model, reusability of the model/design, generation of valuable IP, high level hardware/software partitioning of the design is enabled, Concurrent Engineering basic rules (unique EDA environment and common design database) are fulfilled. The recent evolution of such design methodologies is marked through references to case studies of electronic system modeling,simulation, controller design and implementation. Pointers for future trends / evolution of electronic design strategies and tools are given

Developing a Benchmark Suite for the Evaluation of Orientation Sensors

This paper examines the problem with the lack of standardization through which MEMS orientation sensors are evaluated. These sensors are sold with data sheets that outline their performance, but lack the conditions under which the testing takes place. In this research, a testing apparatus was developed, and testing routines were designed to evaluate the different characteristics of orientation sensors under different motion conditions. Three orientation sensors, each in a different price range, were evaluated with the benchmark suite. The testing apparatus is a turntable that can precisely spin an orientation sensor via a stepper motor, and can record its exact orientation along with the heading read from the orientation sensor. Sets of movements we call benchmark routines were implemented to test different properties of the sensors. The results show that the turntable performs correctly, and as expected, sensors with similar data sheets perform differently

ALL DIGITAL DESIGN AND IMPLEMENTAION OF PROPORTIONAL-INTEGRAL-DERIVATIVE (PID) CONTROLLER

Due to the prevalence of pulse encoders for system state information, an all-digital proportional-integral-derivative (ADPID) is proposed as an alternative to traditional analog and digital PID controllers. The basic concept of an ADPID stems from the use of pulse-width-modulation (PWM) control signals for continuous-time dynamical systems, in that the controllers proportional, integral and derivative actions are converted into pulses by means of standard up-down digital counters and other digital logic devices. An ADPID eliminates the need for analog-digital and digital-analog conversion, which can be costly and may introduce error and delay into the system. In the proposed ADPID, the unaltered output from a pulse encoder attached to the systems output can be interpreted directly. After defining a pulse train to represent the desired output of the encoder, an error signal is formed then processed by the ADPID. The resulting ADPID output or control signal is in PWM format, and can be fed directly into the target system without digital-to-analog conversion. In addition to proposing an architecture for the ADPID, rules are presented to enable control engineers to design ADPIDs for a variety of applications

Design of an Active Stereo Vision 3D Scene Reconstruction System Based on the Linear Position Sensor Module

Active vision systems and passive vision systems currently exist for three-dimensional (3D) scene reconstruction. Active systems use a laser that interacts with the scene. Passive systems implement stereo vision, using two cameras and geometry to reconstruct the scene. Each type of system has advantages and disadvantages in resolution, speed, and scene depth. It may be possible to combine the advantages of both systems as well as new hardware technologies such as position sensitive devices (PSDs) and field programmable gate arrays (FPGAs) to create a real-time, mid-range 3D scene reconstruction system. Active systems usually reconstruct long-range scenes so that a measurable amount of time can pass for the laser to travel to the scene and back. Passive systems usually reconstruct close-range scenes but must overcome the correspondence problem. If PSDs are placed in a stereo vision configuration and a laser is directed at the scene, the correspondence problem can be eliminated. The laser can scan the entire scene as the PSDs continually pick up points, and the scene can be reconstructed. By eliminating the correspondence problem, much of the computation time of stereo vision is removed, allowing larger scenes, possibly at mid-range, to be modeled. To give good resolution at a real-time frame rate, points would have to be recorded very quickly. PSDs are analog devices that give the position of a light spot and have very fast response times. The cameras in the system can be replaced by PSDs to help achieve real- time refresh rates and better resolution. A contribution of this thesis is to design a 3D scene reconstruction system by placing two PSDs in a stereo vision configuration and to use FPGAs to perform calculations to achieve real-time frame rates of mid-range scenes. The linear position sensor module (LPSM) made by Noah Corp is based on a PSD and outputs a position in terms of voltage. The LPSM is characterized for this application by testing it with different power lasers while also varying environment variables such as background light, scene type, and scene distance. It is determined that the LPSM is sensitive to red wavelength lasers. When the laser is reflected off of diffuse surfaces, the laser must output at least 500 mW to be picked up by the LPSM and the scene must be within 15 inches, or the power intensity will not meet the intensity requirements of the LPSM. The establishment of these performance boundaries is a contribution of the thesis along with characterizing and testing the LPSM as a vision sensor in the proposed scene reconstruction system. Once performance boundaries are set, the LPSM is used to model calibrated objects. LPSM sensitivity to power intensity changes seems to cause considerable error. The change in power appears to be a function of depth due to the dispersion of the laser beam. The model is improved by using a correction factor to find the position of the light spot. Using a better-focused laser may improve the results. Another option is to place two PSDs in the same configuration and test to see whether the intensity problem is intrinsic to all PSDs or if the problem is unique to the LPSM