Ensembles of wrappers for automated feature selection in fish age classification

In feature selection, the most important features must be chosen so as to decrease the number thereof while retaining their discriminatory information. Within this context, a novel feature selection method based on an ensemble of wrappers is proposed and applied for automatically select features in fish age classification. The effectiveness of this procedure using an Atlantic cod database has been tested for different powerful statistical learning classifiers. The subsets based on few features selected, e.g. otolith weight and fish weight, are particularly noticeable given current biological findings and practices in fishery research and the classification results obtained with them outperforms those of previous studies in which a manual feature selection was performed.Peer ReviewedPostprint (author's final draft

Digital imaging techniques in otolith data capture, analysis and interpretation

Otoliths or ear-stones are hard, calcium carbonate structures located within the inner ear of bony fishes. Counts of rings and measurements of seasonal growth increments from otoliths are important metrics for assessment and management of fish stocks, and the preparation and microscopic analysis of otoliths forms an essential part of the routine work undertaken by fisheries scientists worldwide. Otolith analysis is a skilled task requiring accuracy and precision, but it is laborious, time-consuming to perform, and represents a significant cost to fisheries management. In the last 2 decades, several attempts to apply ‘computer vision’ (systems that perform high-level tasks and exhibit intelligent behaviour) in otolith analysis have been reported. Although considerable progress has been made and several prototype systems developed, laboratories have been reluctant to adopt image-based computer-assisted age and growth estimation (CAAGE) systems. This paper surveys applications of CAAGE, focusing on their utility for automated ageing using images of otolith macrostructure. A cost-benefit analysis of CAAGE of cod, plaice and anchovy shows that computer vision performs relatively poorly compared with morphometric techniques. However, there is evidence that information from visual features can boost the performance of morphometric CAAGE, and further work is needed to develop effective frameworks for this integrated approach. The cost benefit of these systems might be attractive to smaller laboratories that are already using age-length keys derived from otolith morphometrics for management of smaller artisanal fisheries

DeepOtolith v1.0: An Open-Source AI Platform for Automating Fish Age Reading from Otolith or Scale Images

Every year, marine scientists around the world read thousands of otolith or scale images to determine the age structure of commercial fish stocks. This knowledge is important for fisheries and conservation management. However, the age-reading procedure is time-consuming and costly to perform due to the specialized expertise and labor needed to identify annual growth zones in otoliths. Effective automated systems are needed to increase throughput and reduce cost. DeepOtolith is an open-source artificial intelligence (AI) platform that addresses this issue by providing a web system with a simple interface that automatically estimates fish age by combining otolith images with convolutional neural networks (CNNs), a class of deep neural networks that has been a dominant method in computer vision tasks. Users can upload otolith image data for selective fish species, and the platform returns age estimates. The estimates of multiple images can be exported to conduct conclusions or further age-related research. DeepOtolith currently contains classifiers/regressors for three fish species; however, more species will be included as related work on ageing will be tested and published soon. Herein, the architecture and functionality of the platform are presented. Current limitations and future directions are also discussed. Overall, DeepOtolith should be considered as the first step towards building a community of marine ecologists, machine learning experts, and stakeholders that will collaborate to support the conservation of fishery resources.publishedVersio

Automatic fish ageing from otolith images using statistical learning.

International audienceWe investigate the use of statistical learning techniques for fish age estimation from otolith images. The core of this study lies in the definition of relevant image-related features. We rely on the characterization of a 1D signal summing up the image content within a predefined area of interest. Fish age estimation is then viewed as a multi-class classification issue using neural networks and SVMs. A procedure based on demodulation and remodulation of fish growth patterns is used to improve the generalization properties of the trained classifiers. We also investigate the combination of additional biological and shape features to the image-related ones. The performances are evaluated for a database of several hundred of plaice otoliths

Automated fish age estimation from otolith images using statistical learning

International audienceThe acquisition of age and growth data is of key importance for fisheries research (assessment, marine ecology issues, etc.). Consequently, automating this task is of great interest. In this paper, we investigate the use of statistical learning techniques for fish age estimation. The core of this study lies in the definition of relevant image-related features. We rely on the computation of a 1D representation summing up the content of otolith images within a predefined area of interest. Features are then extracted from this non-stationary representation depicting the alternation of seasonal growth rings. Thus, fish age estimation can be viewed as a multi-class classification issue using statistical learning strategies. In particular, a procedure based on demodulation and remodulation of fish growth patterns is used to improve the generalization properties of the trained classifiers. The experimental evaluation is carried out over a dataset of 320 plaice otolith images from age groups 1–6. We analyze both, the performances of several statistical classifiers, namely SVMs (support vector machines) and neural networks, and the relevance of the proposed image-based feature sets. In addition, the combination of additional biological and shape features to the image-related ones is considered. We reach a rate of correct age estimation of 88% w.r.t. the expert ground truth. This demonstrates the relevance of the proposed approach for the automation of routine aging and for computer-assisted aging

Semi-local extraction of ring structures in images of biological hard tissues: application to the Bayesian interpretation of fish otoliths for age and growth estimation

International audienceThis paper deals with the analysis of images of biological tissue that involves ring structures, such as tree trunks, bivalve seashells or fish otoliths, with a view to automating the acquisition of age and growth data. A bottom-up template-based scheme extracts meaningfulridge and valley curve data using growth-adapted time-frequency filtering. Age and growth estimation is then stated as the Bayesian selection of a subset of ring curves, combining ameasure of curve significativity and ana prioristatistical growth model. Experiments on realsamples demonstrate the efficiency of the proposed extraction stage. Our Bayesian frameworkis shown to significantly outperform previous methods for the interpretation of a dataset of200 plaice otoliths and compares favorably to inter-expert agreement rates (88% of agreement to expert interpretations)

On the automatic detection of otolith features for fish species identification and their age estimation

This thesis deals with the automatic detection of features in signals, either extracted from photographs or captured by means of electronic sensors, and its possible application in the detection of morphological structures in fish otoliths so as to identify species and estimate their age at death. From a more biological perspective, otoliths, which are calcified structures located in the auditory system of all teleostean fish, constitute one of the main elements employed in the study and management of marine ecology. In this sense, the application of Fourier descriptors to otolith images, combined with component analysis, is habitually a first and a key step towards characterizing their morphology and identifying fish species. However, some of the main limitations arise from the poor interpretation that can be obtained with this representation and the use that is made of the coefficients, as generally they are selected manually for classification purposes, both in quantity and representativity. The automatic detection of irregularities in signals, and their interpretation, was first addressed in the so-called Best-Basis paradigm. In this sense, Saito's Local discriminant Bases algorithm (LDB) uses the Discrete Wavelet Packet Transform (DWPT) as the main descriptive tool for positioning the irregularities in the time-frequency space, and an energy-based discriminant measure to guide the automatic search of relevant features in this domain. Current density-based proposals have tried to overcome the limitations of the energy-based functions with relatively little success. However, other measure strategies more consistent with the true classification capability, and which can provide generalization while reducing the dimensionality of features, are yet to be developed. The proposal of this work focuses on a new framework for one-dimensional signals. An important conclusion extracted therein is that such generalization involves a mesure system of bounded values representing the density where no class overlaps. This determines severely the selection of features and the vector size that is needed for proper class identification, which must be implemented not only based on global discriminant values but also on the complementary information regarding the provision of samples in the domain. The new tools have been used in the biological study of different hake species, yielding good classification results. However, a major contribution lies on the further interpretation of features the tool performs, including the structure of irregularities, time-frequency position, extension support and degree of importance, which is highlighted automatically on the same images or signals. As for aging applications, a new demodulation strategy for compensating the nonlinear growth effect on the intensity profile has been developed. Although the method is, in principle, able to adapt automatically to the specific growth of individual specimens, preliminary results with LDB-based techniques suggest to study the effect of lighting conditions on the otoliths in order to design more reliable techniques for reducing image contrast variation. In the meantime, a new theoretic framework for otolith-based fish age estimation has been presented. This theory suggests that if the true fish growth curve is known, the regular periodicity of age structures in the demodulated profile is related to the radial length the original intensity profile is extracted from. Therefore, if this periodicity can be measured, it is possible to infer the exact fish age omitting feature extractors and classifiers. This could have important implications in the use of computational resources anc current aging approaches.El eje principal de esta tesis trata sobre la detección automática de singularidades en señales, tanto si se extraen de imágenes fotográ cas como si se capturan de sensores electrónicos, así como su posible aplicación en la detección de estructuras morfológicas en otolitos de peces para identi car especies, y realizar una estimación de la edad en el momento de su muerte. Desde una vertiente más biológica, los otolitos, que son estructuras calcáreas alojadas en el sistema auditivo de todos los peces teleósteos, constituyen uno de los elementos principales en el estudio y la gestión de la ecología marina. En este sentido, el uso combinado de descriptores de Fourier y el análisis de componentes es el primer paso y la clave para caracterizar su morfología e identi car especies marinas. Sin embargo, una de las limitaciones principales de este sistema de representación subyace en la interpretación limitada que se puede obtener de las irregularidades, así como el uso que se hace de los coe cientes en tareas de clasi cación que, por lo general, acostumbra a seleccionarse manualmente tanto por lo que respecta a la cantidad y a su importancia. La detección automática de irregularidades en señales, y su interpretación, se abordó por primera bajo el marco del Best-Basis paradigm. En este sentido, el algoritmo Local Discriminant Bases (LDB) de N. Saito utiliza la Transformada Wavelet Discreta (DWT) para describir el posicionamiento de características en el espacio tiempo-frecuencia, y una medida discriminante basada en la energía para guiar la búsqueda automática de características en dicho dominio. Propuestas recientes basadas en funciones de densidad han tratado de superar las limitaciones que presentaban las medidas de energía con un éxito relativo. No obstante, todavía están por desarrollar nuevas estrategias más consistentes con la capacidad real de clasi cación y que ofrezcan mayor generalización al reducir la dimensión de los datos de entrada. La propuesta de este trabajo se centra en un nuevo marco para señales unidimensionales. Una conclusión principal que se extrae es que dicha generalización pasa por un marco de medidas de valores acotados que re ejen la densidad donde las clases no se solapan. Esto condiciona severamente el proceso de selección de características y el tamaño del vector necesario para identi car las clases correctamente, que se ha de establecer no sólo en base a valores discriminantes globales sino también en la información complementaria sobre la disposición de las muestras en el dominio. Las nuevas herramientas han sido utilizadas en el estudio biológico de diferentes especies de merluza, donde se han conseguido buenos resultados de identi cación. No obstante, la contribución principal subyace en la interpretación que dicha herramienta hace de las características seleccionadas, y que incluye la estructura de las irregularidades, su posición temporal-frecuencial, extensión en el eje y grado de relevancia, el cual, se resalta automáticamente sobre la misma imagen o señal. Por lo que respecta a la determinación de la edad, se ha planteado una nueva estrategia de demodulación para compensar el efecto del crecimiento no lineal en los per les de intensidad. Inicialmente, aunque el método implementa un proceso de optimización capaz de adaptarse al crecimiento individual de cada pez automáticamente, resultados preliminares obtenidos con técnicas basadas en el LDB sugieren estudiar el efecto de las condiciones lumínicas sobre los otolitos con el n de diseñar algoritmos que reduzcan la variación del contraste de la imagen más ablemente. Mientras tanto, se ha planteado una nueva teoría para estimar la edad de los peces en base a otolitos. Esta teoría sugiere que si la curva de crecimiento real del pez se conoce, el período regular de los anillos en el per l demodulado está relacionado con la longitud total del radio donde se extrae el per l original. Por tanto, si dicha periodicidad es medible, es posible determinar la edad exacta sin necesidad de utilizar extractores de características o clasi cadores, lo cual tendría implicaciones importantes en el uso de recursos computacionales y en las técnicas actuales de estimación de la edad.L'eix principal d'aquesta tesi tracta sobre la detecció automàtica d'irregularitats en senyals, tant si s'extreuen de les imatges fotogrà ques com si es capturen de sensors electrònics, així com la seva possible aplicació en la detecció d'estructures morfològiques en otòlits de peixos per identi car espècies, i realitzar una estimació de l'edat en el moment de la seva mort. Des de la vesant més biològica, els otòlits, que son estructures calcàries que es troben en el sistema auditiu de tots els peixos teleostis, constitueixen un dels elements principals en l'estudi i la gestió de l'ecologia marina. En aquest sentit, l'ús combinat de descriptors de Fourier i l'anàlisi de components es el primer pas i la clau per caracteritzar la seva morfologia i identi car espècies marines. No obstant, una de les limitacions principals d'aquest sistema de representació consisteix en la interpretació limitada de les irregularitats que pot desenvolupar, així com l'ús que es realitza dels coe cients en tasques de classi cació, els quals, acostumen a ser seleccionats manualment tant pel que respecta a la quantitat com la seva importància. La detecció automàtica d'irregularitats en senyals, així com la seva interpretació, es va tractar per primera vegada sota el marc del Best-Basis paradigm. En aquest sentit, l'algorisme Local Discriminant Bases (LDB) de N. Saito es basa en la Transformada Wavelet Discreta (DWT) per descriure el posicionament de característiques dintre de l'espai temporal-freqüencial, i en una mesura discriminant basada en l'energia per guiar la cerca automàtica de característiques dintre d'aquest domini. Propostes més recents basades en funcions de densitat han tractat de superar les limitacions de les mesures d'energia amb un èxit relatiu. No obstant, encara s'han de desenvolupar noves estratègies que siguin més consistents amb la capacitat real de classi cació i ofereixin més generalització al reduir la dimensió de les dades d'entrada. La proposta d'aquest treball es centra en un nou marc per senyals unidimensionals. Una de las conclusions principals que s'extreu es que aquesta generalització passa per establir un marc de mesures acotades on els valors re ecteixin la densitat on cap classe es solapa. Això condiciona bastant el procés de selecció de característiques i la mida del vector necessari per identi car les classes correctament, que s'han d'establir no només en base a valors discriminants globals si no també en informació complementària sobre la disposició de les mostres en el domini. Les noves eines s'han utilitzat en diferents estudis d'espècies de lluç, on s'han obtingut bons resultats d'identi cació. No obstant, l'aportació principal consisteix en la interpretació que l'eina extreu de les característiques seleccionades, i que inclou l'estructura de les irregularitats, la seva posició temporal-freqüencial, extensió en l'eix i grau de rellevància, el qual, es ressalta automàticament sobre les mateixa imatge o senyal. En quan a l'àmbit de determinació de l'edat, s'ha plantejat una nova estratègia de demodulació de senyals per compensar l'efecte del creixement no lineal en els per ls d'intensitat. Tot i que inicialment aquesta tècnica desenvolupa un procés d'optimització capaç d'adaptar-se automàticament al creixement individual de cada peix, els resultats amb el LDB suggereixen estudiar l'efecte de les condicions lumíniques sobre els otòlits amb la nalitat de dissenyar algorismes que redueixin la variació del contrast de les imatges més ablement. Mentrestant s'ha plantejat una nova teoria per realitzar estimacions d'edat en peixos en base als otòlits. Aquesta teoria suggereix que si la corba de creixement és coneguda, el període regular dels anells en el per l d'intensitat demodulat està relacionat amb la longitud total de radi d'on s'agafa el per l original. Per tant, si la periodicitat es pot mesurar, es possible conèixer l'edat exacta del peix sense usar extractors de característiques o classi cadors, la qual cosa tindria implicacions importants en l'ús de recursos computacionals i en les tècniques actuals d'estimació de l'edat.Postprint (published version

Extraction and interpretation of ring structures in images of biological hard tissues: application to fish age and growth estimation.

International audienceThis paper presents a general framework for the automated estimation of age and growth from images of biological materials depicting concentric ring-like structures such as tree trunks, corals, bivalve seashells, fish scales or otoliths. This interpretation task can be seen as a ring segmentation issue, where growth rings are associated to image ridge and valley structures. This is stated as the Bayesian selection of a subset of partial ring curves extracted using a semi-local template-based growth-adapted scheme. The application to fish otolith interpretation provides a consistent and convincing validation of the proposed framework

Automatic Fish Age Determination across Different Otolith Image Labs Using Domain Adaptation

The age determination of fish is fundamental to marine resource management. This task is commonly done by analysis of otoliths performed manually by human experts. Otolith images from Greenland halibut acquired by the Institute of Marine Research (Norway) were recently used to train a convolutional neural network (CNN) for automatically predicting fish age, opening the way for requiring less human effort and availability of expertise by means of deep learning (DL). In this study, we demonstrate that applying a CNN model trained on images from one lab (in Norway) does not lead to a suitable performance when predicting fish ages from otolith images from another lab (in Iceland) for the same species. This is due to a problem known as dataset shift, where the source data, i.e., the dataset the model was trained on have different characteristics from the dataset at test stage, here denoted as target data. We further demonstrate that we can handle this problem by using domain adaptation, such that an existing model trained in the source domain is adapted to perform well in the target domain, without requiring extra annotation effort. We investigate four different approaches: (i) simple adaptation via image standardization, (ii) adversarial generative adaptation, (iii) adversarial discriminative adaptation and (iv) self-supervised adaptation. The results show that the performance varies substantially between the methods, with adversarial discriminative and self-supervised adaptations being the best approaches. Without using a domain adaptation approach, the root mean squared error (RMSE) and coefficient of variation (CV) on the Icelandic dataset are as high as 5.12 years and 28.6%, respectively, whereas by using the self-supervised domain adaptation, the RMSE and CV are reduced to 1.94 years and 11.1%. We conclude that careful consideration must be given before DL-based predictors are applied to perform large scale inference. Despite that, domain adaptation is a promising solution for handling problems of dataset shift across image labs.publishedVersio

Deep learning applied to fish otolith images

This thesis is concerned with classification and regression using deep learning applied to fish otolith images. Otoliths (earstones) are calcified structures in the inner ear of vertebrates, and are used, for instance, in fish stock assessment and fish age determination. We use convolutional neural networks – a class of deep learning models - on two specific problems: discrimination between Northeast Arctic Cod and Norwegian Coastal Cod, and age determination of Greenland halibut. In relation to classification and regression, we are also concerned with the usage of cross-validation procedures such as k*l-fold cross-validation, to obtain reliable test results. We obtain test results for all available data, and we argue for the usage of cross-validation on the bases of variations in test results. Furthermore, feature relevance attribution methods are discussed and compared, which aims at explaining outputs from deep learning models by attributing relevance scores to the input. These comparisons are conducted using image input heatmaps produced by methods such as gradient saliency maps, guided backpropagation, and integrated gradients, along with two proposed variations of those techniques