Plenoptic Signal Processing for Robust Vision in Field Robotics

This thesis proposes the use of plenoptic cameras for improving the robustness and simplicity of machine vision in field robotics applications. Dust, rain, fog, snow, murky water and insufficient light can cause even the most sophisticated vision systems to fail. Plenoptic cameras offer an appealing alternative to conventional imagery by gathering significantly more light over a wider depth of field, and capturing a rich 4D light field structure that encodes textural and geometric information. The key contributions of this work lie in exploring the properties of plenoptic signals and developing algorithms for exploiting them. It lays the groundwork for the deployment of plenoptic cameras in field robotics by establishing a decoding, calibration and rectification scheme appropriate to compact, lenslet-based devices. Next, the frequency-domain shape of plenoptic signals is elaborated and exploited by constructing a filter which focuses over a wide depth of field rather than at a single depth. This filter is shown to reject noise, improving contrast in low light and through attenuating media, while mitigating occluders such as snow, rain and underwater particulate matter. Next, a closed-form generalization of optical flow is presented which directly estimates camera motion from first-order derivatives. An elegant adaptation of this "plenoptic flow" to lenslet-based imagery is demonstrated, as well as a simple, additive method for rendering novel views. Finally, the isolation of dynamic elements from a static background is considered, a task complicated by the non-uniform apparent motion caused by a mobile camera. Two elegant closed-form solutions are presented dealing with monocular time-series and light field image pairs. This work emphasizes non-iterative, noise-tolerant, closed-form, linear methods with predictable and constant runtimes, making them suitable for real-time embedded implementation in field robotics applications

Plenoptic Signal Processing for Robust Vision in Field Robotics

This thesis proposes the use of plenoptic cameras for improving the robustness and simplicity of machine vision in field robotics applications. Dust, rain, fog, snow, murky water and insufficient light can cause even the most sophisticated vision systems to fail. Plenoptic cameras offer an appealing alternative to conventional imagery by gathering significantly more light over a wider depth of field, and capturing a rich 4D light field structure that encodes textural and geometric information. The key contributions of this work lie in exploring the properties of plenoptic signals and developing algorithms for exploiting them. It lays the groundwork for the deployment of plenoptic cameras in field robotics by establishing a decoding, calibration and rectification scheme appropriate to compact, lenslet-based devices. Next, the frequency-domain shape of plenoptic signals is elaborated and exploited by constructing a filter which focuses over a wide depth of field rather than at a single depth. This filter is shown to reject noise, improving contrast in low light and through attenuating media, while mitigating occluders such as snow, rain and underwater particulate matter. Next, a closed-form generalization of optical flow is presented which directly estimates camera motion from first-order derivatives. An elegant adaptation of this "plenoptic flow" to lenslet-based imagery is demonstrated, as well as a simple, additive method for rendering novel views. Finally, the isolation of dynamic elements from a static background is considered, a task complicated by the non-uniform apparent motion caused by a mobile camera. Two elegant closed-form solutions are presented dealing with monocular time-series and light field image pairs. This work emphasizes non-iterative, noise-tolerant, closed-form, linear methods with predictable and constant runtimes, making them suitable for real-time embedded implementation in field robotics applications

Automated interpretation of benthic stereo imagery

Autonomous benthic imaging, reduces human risk and increases the amount of collected data. However, manually interpreting these high volumes of data is onerous, time consuming and in many cases, infeasible. The objective of this thesis is to improve the scientific utility of the large image datasets. Fine-scale terrain complexity is typically quantified by rugosity and measured by divers using chains and tape measures. This thesis proposes a new technique for measuring terrain complexity from 3D stereo image reconstructions, which is non-contact and can be calculated at multiple scales over large spatial extents. Using robots, terrain complexity can be measured without endangering humans, beyond scuba depths. Results show that this approach is more robust, flexible and easily repeatable than traditional methods. These proposed terrain complexity features are combined with visual colour and texture descriptors and applied to classifying imagery. New multi-dataset feature selection methods are proposed for performing feature selection across multiple datasets, and are shown to improve the overall classification performance. The results show that the most informative predictors of benthic habitat types are the new terrain complexity measurements. This thesis presents a method that aims to reduce human labelling effort, while maximising classification performance by combining pre-clustering with active learning. The results support that utilising the structure of the unlabelled data in conjunction with uncertainty sampling can significantly reduce the number of labels required for a given level of accuracy. Typically 0.00001–0.00007% of image data is annotated and processed for science purposes (20–50 points in 1–2% of the images). This thesis proposes a framework that uses existing human-annotated point labels to train a superpixel-based automated classification system, which can extrapolate the classified results to every pixel across all the images of an entire survey

Automated interpretation of benthic stereo imagery

Autonomous benthic imaging, reduces human risk and increases the amount of collected data. However, manually interpreting these high volumes of data is onerous, time consuming and in many cases, infeasible. The objective of this thesis is to improve the scientific utility of the large image datasets. Fine-scale terrain complexity is typically quantified by rugosity and measured by divers using chains and tape measures. This thesis proposes a new technique for measuring terrain complexity from 3D stereo image reconstructions, which is non-contact and can be calculated at multiple scales over large spatial extents. Using robots, terrain complexity can be measured without endangering humans, beyond scuba depths. Results show that this approach is more robust, flexible and easily repeatable than traditional methods. These proposed terrain complexity features are combined with visual colour and texture descriptors and applied to classifying imagery. New multi-dataset feature selection methods are proposed for performing feature selection across multiple datasets, and are shown to improve the overall classification performance. The results show that the most informative predictors of benthic habitat types are the new terrain complexity measurements. This thesis presents a method that aims to reduce human labelling effort, while maximising classification performance by combining pre-clustering with active learning. The results support that utilising the structure of the unlabelled data in conjunction with uncertainty sampling can significantly reduce the number of labels required for a given level of accuracy. Typically 0.00001–0.00007% of image data is annotated and processed for science purposes (20–50 points in 1–2% of the images). This thesis proposes a framework that uses existing human-annotated point labels to train a superpixel-based automated classification system, which can extrapolate the classified results to every pixel across all the images of an entire survey

Novel interactions in tropicalised ecosystems

Climate change has led to the range-expansion of novel consumers into kelp-dominated temperate reefs, which can result in a rapid deforestation of these ecosystems and the formation of turf habitats dominated by small, less complex algae. This thesis presents four data chapters exploring the mechanisms by which novel consumers (tropical range-expanding fishes) respond to and facilitate the maintenance of kelp-free reef states on the east coast of Australia. Firstly, an analysis of 17-years of survey data showed that local kelp loss has led to a shift in the trophic composition of tropicalised reefs, where a marked increase in abundances of herbivorous fishes and a decline in planktivorous fishes was observed (Chapter 2). Remarkably, despite sustained kelp loss, the overall diversity and abundance of both tropical and temperate fishes increased through time (Chapter 2). This finding is discussed within the context of global patterns of biodiversity homogenisation in response to environmental change. Secondly, the role of herbivory in directly influencing kelp loss was empirically tested via a manipulative field experiment (Chapters 3 and 4) where large herbivores were excluded from replicate reef patches and where spore supply was manipulated. Here I found that kelp juveniles only developed in plots where herbivores were excluded, and where propagules were supplied via the addition of reproductive kelp (Chapter 3). In contrast, where spore supply was not manipulated, there was very little change in benthic cover, where herbivory was allowed and also where it was excluded (Chapter 3). The influence of herbivory on benthic microbial assemblages was also tested as a potential interaction influencing kelp recruitment success (Chapter 4). Here, herbivory had a positive influence on the initial recruitment of kelp by influencing benthic microbial communities, but this did not lead to the development of kelp beyond the recruit stage (Chapter 4). A disturbance effect was observed on benthic microbial communities where herbivory maintained community stability and, independently, I observed that some benthic bacterial taxa may inhibit kelp recruitment where spores are available but herbivory is excluded (Chapter 4). Finally, the capacity for tropical fishes to adapt to novel diets was explored through an aquaria experiment where both tropical and temperate turf algal diets were made available to range-shifting juvenile fish (Chapter 5). Tropical fish exhibited higher bite rates on tropical algae but in total more temperate algae was lost (Chapter 5). This finding is attributed to the fact that tropical diets showed a higher nitrogen content, likely due to small particulate matter not captured by changes in algal cover (Chapter 5). Nevertheless, consumption of temperate turf suggests novel diets are not a barrier to range expansion for tropical fish (Chapter 5). Overall, it is likely that for diminished kelp populations in northern New South Wales, the interaction between increasing populations of novel herbivores and the proliferation of algal turfs maintain the system in a kelp-free stare which is unlikely to be reversed, despite extant kelp populations in close proximity to the focal reefs studied here. This thesis demonstrates the role of shifting species interactions in marine deforestation in temperate Australia and contributes to understanding the mechanisms that facilitate patterns of global tropicalisation

屋外調査用自律移動型ロボットの不整地移動性能

早大学位記番号:新7829早稲田大