Computer Vision for Marine Environmental Monitoring

Osterloff J. Computer Vision for Marine Environmental Monitoring. Bielefeld: Universität Bielefeld; 2018.Ocean exploration using imaging techniques has recently become very popular as camera systems became affordable and technique developed further. Marine imaging provides a unique opportunity to monitor the marine environment. The visual exploration using images allows to explore the variety of fauna, flora and geological structures of the marine environment. This monitoring creates a bottleneck as a manual evaluation of the large amounts of underwater image data is very time consuming. Information encapsulated in the images need to be extracted so that they can be included in statistical analyzes. Objects of interest (OOI) have to be localized and identified in the recorded images. In order to overcome the bottleneck, computer vision (CV) is applied in this thesis to extract the image information (semi-) automatically. A pre-evaluation of the images by marking OOIs manually, i.e. the manual annotation process, is necessary to provide examples for the applied CV methods. Five major challenges are identified in this thesis to apply of CV for marine environmental monitoring. The challenges can be grouped into challenges caused by underwater image acquisition and by the use of manual annotations for machine learning (ML). The image acquisition challenges are the optical properties challenge, e.g. a wavelength dependent attenuation underwater, and the dynamics of these properties, as different amount of matter in the water column affect colors and illumination in the images. The manual annotation challenges for applying ML for underwater images are, the low number of available manual annotations, the quality of the annotations in terms of correctness and reproducibility and the spatial uncertainty of them. The latter is caused by allowing a spatial uncertainty to speed up the manual annotation process e.g. using point annotations instead of fully outlining OOIs on a pixel level. The challenges are resolved individually in four different new CV approaches. The individual CV approaches allow to extract new biologically relevant information from time-series images recorded underwater. Manual annotations provide the ground truth for the CV systems and therefore for the included ML. Placing annotations manually in underwater images is a challenging task. In order to assess the quality in terms of correctness and reproducibility a detailed quality assessment for manual annotations is presented. This includes the computation of a gold standard to increase the quality of the ground truth for the ML. In the individually tailored CV systems, different ML algorithms are applied and adapted for marine environmental monitoring purposes. Applied ML algorithms cover a broad variety from unsupervised to supervised methods, including deep learning algorithms. Depending on the biologically motivated research question, systems are evaluated individually. The first two CV systems are developed for the _in-situ_ monitoring of the sessile species _Lophelia pertusa_. Visual information of the cold-water coral is extracted automatically from time-series images recorded by a fixed underwater observatory (FUO) located at 260 m depth and 22 km off the Norwegian coast. Color change of a cold water coral reef over time is quantified and the polyp activity of the imaged coral is estimated (semi-) automatically. The systems allow for the first time to document an _in-situ_ change of color of a _Lophelia pertusa_ coral reef and to estimate the polyp activity for half a year with a temporal resolution of one hour. The third CV system presented in this thesis allows to monitor the mobile species shrimp _in-situ_. Shrimp are semitransparent creating additional challenges for localization and identification in images using CV. Shrimp are localized and identified in time-series images recorded by the same FUO. Spatial distribution and temporal occurrence changes are observed by comparing two different time periods. The last CV system presented in this thesis is developed to quantify the impact of sedimentation on calcareous algae samples in a _wet-lab_ experiment. The size and color change of the imaged samples over time can be quantified using a consumer camera and a color reference plate placed in the field of view for each recorded image. Extracting biologically relevant information from underwater images is only the first step for marine environmental monitoring. The extracted image information, like behavior or color change, needs to be related to other environmental parameters. Therefore, also data science methods are applied in this thesis to unveil some of the relations between individual species' information extracted semi-automatically from underwater images and other environmental parameters

Technology-based product market entries : managerial resources and decision-making process

Entering new product markets on the basis of an existing technological competence is an important possible source of growth for technology intensive companies. The resource-based view explains resource-based growth in general. Diversification research has studied patterns of technology-based new product market entries. More research is, however, needed on managerial and decision-making process factors impacting the success of technology-based product market entries. The findings of this dissertation contribute to the knowledge on technology-based new product market entries. A framework is developed and tested, which explains the impact of managerial resources and decision-making process characteristics on the success of technology-based product market entries. Hypotheses are derived from the resource-based view and decision-making research. The main research question is: Which levers does the management control that help promote technology-based growth into new product markets? This question is further broken down into the following three questions covering the three hypothesized main levers: Do the managerial resources invested in generating technology-based growth impact the success of technology-based product market entries? What is the impact of operational capabilities on the success of individual technology-based product market entries? What is the impact of decision-making process characteristics on the success of individual technology-based product market entries? The framework and the underlying hypotheses are examined in the light of data collected with a survey of 63 companies. The framework and the survey questionnaire were developed on the basis of interviews with nine companies as well as a literature review of previous empirical research on diversification and research on new product development. The empirical data support the main arguments of the framework. In order to further examine the conclusions from the analysis of the survey data, five of the survey respondent-companies were engaged in deepening case studies. Several theoretical contributions are identified and managerial implications are derived. Primarily, this dissertation contributes to the literature on technology-based product market entries by creating a better understanding of the managerial levers for promoting success. Chosen process, management and capability-related factors are studied. The dissertation contributes to diversification research by examining on the project level the process, organizational, and managerial components of diversification. The main contribution related to the resource-based view is further clarification concerning the role and support to the importance of managerial services in resource-based growth. Aspects of successful management of technology-based market entry projects are pointed out. The impact of familiarity with the new product markets on success of the entry receives support. Further insights are generated on the complex social phenomenon of participating in a project team with the challenging task of implementing a technology-based product market entry. The most important managerial implication is that certain managerial levers are more effective than others in promoting the success of a growth strategy consisting of technology-based product-market entries. The findings give an indication of certain risks and uncertainties related to individual leveraging projects, which managers should take into account.reviewe

RecoMIA - Recommendations for Marine Image Annotation: Lessons Learned and Future Directions

Marine imaging is transforming into a sensor technology applied for high throughput sampling. In the context of habitat mapping, imaging establishes thereby an important bridge technology regarding the spatial resolution and information content between physical sampling gear (e.g., box corer, multi corer) on the one end and hydro-acoustic sensors on the other end of the spectrum of sampling methods. In contrast to other scientific imaging domains, such as digital pathology, there are no protocols and reports available that guide users (often referred to as observers) in the non-trivial process of assigning semantic categories to whole images, regions, or objects of interest (OOI), which is referred to as annotation. These protocols are crucial to facilitate image analysis as a robust scientific method. In this article we will review the past observations in manual Marine Image Annotations (MIA) and provide (a) a guideline for collecting manual annotations, (b) definitions for annotation quality, and (c) a statistical framework to analyze the performance of human expert annotations and to compare those to computational approaches

RecoMIA - Recommendations for marine image annotation: Lessons learned and future directions

Schoening T, Osterloff J, Nattkemper TW. RecoMIA - Recommendations for marine image annotation: Lessons learned and future directions. Frontiers in Marine Science. 2016;3: 59.Marine imaging is transforming into a sensor technology applied for high throughput sampling. In the context of habitat mapping, imaging establishes thereby an important bridge technology regarding the spatial resolution and information content between physical sampling gear (e.g., box corer, multi corer) on the one end and hydro-acoustic sensors on the other end of the spectrum of sampling methods. In contrast to other scientific imaging domains, such as digital pathology, there are no protocols and reports available that guide users (often referred to as observers) in the non-trivial process of assigning semantic categories to whole images, regions, or objects of interest (OOI), which is referred to as annotation. These protocols are crucial to facilitate image analysis as a robust scientific method. In this article we will review the past observations in manual Marine Image Annotations (MIA) and provide (a) a guideline for collecting manual annotations, (b) definitions for annotation quality, and (c) a statistical framework to analyze the performance of human expert annotations and to compare those to computational approaches

Computational analysis of spatial species distribution for integrated stationary environmental monitoring

Osterloff J, Nilssen I, Möller T, Nattkemper TW. Computational analysis of spatial species distribution for integrated stationary environmental monitoring. Presented at the Geohab 2015, Salvador, Bahia, Brazil

Extracting Scalar Quantities from Underwater Images - a Toolbox for Image Data from Fixed Observatories.

Osterloff J, Möller T, Nilssen I, Nattkemper TW. Extracting Scalar Quantities from Underwater Images - a Toolbox for Image Data from Fixed Observatories. Presented at the Marine Imaging Workshop 2017, Kiel

Why a new volume on Non-Pollen Palynomorphs?

Here we introduce SP511, Applications of Non-Pollen Palynomorphs: from Palaeoenvironmental Reconstructions to Biostratigraphy. The study of Non-pollen palynomorphs (NPPs) has a long and rich history that is interwoven with that of pollen-based studies. NPPs are among the oldest fossils on record, and are instrumental in determining the origin and evolution of life, as well as studying origination and extinction events prior to the origin of pollen-producing angiosperms. This new volume on NPPs provides an up-to-date and seminal overview of the subject, linking deep-time and Quaternary study of the subject for the first time

Change Detection in underwater time laps videos from stationary observatories

Möller T, Nilssen I, Sumida P, Osterloff J, Nattkemper TW. Change Detection in underwater time laps videos from stationary observatories. Presented at the GEOHAB, Salvador, Brazil

Computer vision enables short- and long-term analysis of <i>Lophelia pertusa</i> polyp behaviour and colour from an underwater observatory

An array of sensors, including an HD camera mounted on a Fixed Underwater Observatory (FUO) were used to monitor a cold-water coral (Lophelia pertusa) reef in the Lofoten-Vesterålen area from April to November 2015. Image processing and deep learning enabled extraction of time series describing changes in coral colour and polyp activity (feeding). The image data was analysed together with data from the other sensors from the same period, to provide new insights into the short- and long-term dynamics in polyp features. The results indicate that diurnal variations and tidal current influenced polyp activity, by controlling the food supply. On a longer time-scale, the coral’s tissue colour changed from white in the spring to slightly red during the summer months, which can be explained by a seasonal change in food supply. Our work shows, that using an effective integrative computational approach, the image time series is a new and rich source of information to understand and monitor the dynamics in underwater environments due to the high temporal resolution and coverage enabled with FUOs