Transform Learning - Registration of medical images using self organization

A network model is introduced that allows a multimodal registration of two images. It can be used for a image-model or a model-model registration. The application of the network to registering tomographic to 3D ultrasonic data is introduced. Results on artificial and real ultrasound image data sets are discussed

Vessel Extracting Gas - Using self organization in the extraction of vascular trees

A network model is introduced that allows the extraction of the topological structure of a set of input vectors corresponding to image voxels from a 3D doppler or contrast enhanced ultrasound. This extraction is a precondition for many medical image registration algorithms. Results on artificial and real ultrasound image data sets are discussed

Transform Learning - Registration of medical images using self organization

A network model is introduced that allows a multimodal registration of two images. It can be used for a image-model or a model-model registration. The application of the network to registering tomographic to 3D ultrasonic data is introduced. Results on artificial and real ultrasound image data sets are discussed

ConvGenVisMo: Evaluation of Conversational Generative Vision Models

Conversational generative vision models (CGVMs) like Visual ChatGPT (Wu et al., 2023) have recently emerged from the synthesis of computer vision and natural language processing techniques. These models enable more natural and interactive communication between humans and machines, because they can understand verbal inputs from users and generate responses in natural language along with visual outputs. To make informed decisions about the usage and deployment of these models, it is important to analyze their performance through a suitable evaluation framework on realistic datasets. In this paper, we present ConvGenVisMo, a framework for the novel task of evaluating CGVMs. ConvGenVisMo introduces a new benchmark evaluation dataset for this task, and also provides a suite of existing and new automated evaluation metrics to evaluate the outputs. All ConvGenVisMo assets, including the dataset and the evaluation code, will be made available publicly on GitHub

Vector quantization based learning algorithms for mixed data types and their application in cognitive support systems for biomedical research

Computermodel ondersteunt bij onderzoek naar kanker Meerlaagse modellen van complexe relaties worden steeds belangrijker in biomedisch onderzoek. De hoge complexiteit maakt het gebruik van computers noodzakelijk bij het analyseren van deze relaties. Omdat er meestal geen duidelijke hypotheses over de verwachte relaties bestaan, zijn traditionele biostatistische methodes ongeschikt. Dit proefschrift introduceert een framework dat de groepering van meerlaagse objecten optimaliseert. Met behulp daarvan kunnen de objecten volgens gegeven classificaties worden gegroepeerd. Tijdens het promotieonderzoek werd het framework toegepast op borstkankeronderzoek. Daaruit werd duidelijk dat het kan dienen als cognitief ondersteuningssysteem voor biomedisch onderzoek. Het bleek mogelijk om op intuïtieve manier met de modellen om te gaan. Het ondersteuningssysteem maakt het mogelijk om hypotheses te genereren over biomedische relaties, die vervolgens gecontroleerd kunnen worden met traditionele biostatistische methoden. Multi-layer models are of increasing importance in biomedical research. By representing objects as ensembles of heterogeneous partial aspects they allow modeling complex relations. Due to this high complexity the aid of computers is needed in investigating these relations. As often there are no clear hypotheses on the expected relations, traditional bio-statistical approaches are unsuitable for this task. This thesis introduces a framework that optimizes the intrinsic grouping (clustering) of multi-layer objects as well as the grouping of these objects according to a set of given class assignments (classification). It identifies prototypical representatives of the groups. Adequate corresponding distance measures take account for the heterogeneity of the partial aspects. Additionally, the framework allows analyzing the relevance of single aspects in the models in their joined context. In the integrative analysis either preselected aspects or the whole ensemble are used for a suitable grouping. Thereby single aspects are weighted according to their influence on the grouping. These weights can under certain constraints be interpreted as relevance values. Applying the framework for heterogeneous data in breast cancer research during the thesis it could be shown that if handled suitably it succeeds as a cognitive support system for biomedical research. The identification of prototypes as well as the determination of relevance values relates to cognitive models of human expert thinking. They can be handled intuitively by the domain experts. The support system thus enables the generation of hypotheses concerning biomedical relations that are then testable using traditional bio-statistical approaches.