Super Neurons

Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generation neural network models with nonlinear learning units, i.e., the generative neurons that yield an elegant level of diversity; however, like its predecessor, conventional Convolutional Neural Networks (CNNs), they still have a common drawback: localized (fixed) kernel operations. This severely limits the receptive field and information flow between layers and thus brings the necessity for deep and complex models. It is highly desired to improve the receptive field size without increasing the kernel dimensions. This requires a significant upgrade over the generative neurons to achieve the “non-localized kernel operations” for each connection between consecutive layers. In this article, we present superior (generative) neuron models (or super neurons in short) that allow random or learnable kernel shifts and thus can increase the receptive field size of each connection. The kernel localization process varies among the two super-neuron models. The first model assumes randomly localized kernels within a range and the second one learns (optimizes) the kernel locations during training. An extensive set of comparative evaluations against conventional and deformable convolutional, along with the generative neurons demonstrates that super neurons can empower Self-ONNs to achieve a superior learning and generalization capability with a minimal computational complexity burden. PyTorch implementation of Self-ONNs with super-neurons is now publically shared.Peer reviewe

Improving Content-Based Image Indexing and Retrieval Performance

Latest image technology improvements along with the Internet growth have led to a huge amount of digital multimedia during the recent decades. Various methods, algorithms and systems have been proposed addressing image storage and management problems. Such studies revealed the indexing and retrieval concepts, which have further evolved to Content-Based Image Retrieval (CBIR). CBIR systems often analyze image content via the so-called low-level features for indexing and retrieval, such as color, texture and shape. In order to achieve significantly higher semantic performance, recent systems seek to combine low-level with high-level features that contain perceptual information for human. However, such combinations increase the feature extraction processing time and the memory requirements as well as the retrieval complexity. Performance improvements of indexing and retrieval play an important role for providing advanced CBIR services on every hardware platform. In this thesis, we propose novel techniques for improving the overall performance of CBIR. We define general CBIR challenges as memory and disk space requirements, computational complexity, semantic retrieval performance, and usability. Bringing generic and feasible solutions to these challenges is the main contribution of this thesis. A novel system for feature selection is introduced for enhancing semantic image retrieval results, decreasing retrieval process complexity, and improving the overall system usability for end-users of CBIR systems. Three feature selection criteria and a decision method construct the proposed feature selection system. A majority voting based method is adapted for efficient selection of features and feature combinations. The performance of the proposed criteria is assessed over a large image database and a number of features, and compared against other techniques from the literature. Experiments show that the proposed feature selection system improves semantic performance results in image retrieval systems. We introduce a novel Transform-Based Layered Query (TLQ) Scheme designed for efficient handling of visual media retrieval, which mainly aims at decreasing processing time and run-time memory consumption without degrading semantic retrieval results. The proposed scheme is based on abstract layers in indexing and retrieval phases, where each indexing layer of TLQ corresponds to a retrieval layer. The layers are independent from the underlying indexing and retrieval methods, and mainly constructed using multimedia and feature data transformations for reducing data dimensions. A two-layer TLQ scheme is implemented and integrated into MUVIS content-based multimedia indexing and retrieval framework. A new feature dimension reduction method referred to as Mapping by Adaptive Threshold (MAT) is also proposed as a solution for memory requirements and computational complexity of retrieval processes. Theoretical and practical advantages of TLQ over existing methods are validated experimentally on image databases using the MAT method for feature data. Experimental studies also show that the proposed MAT method is a fast feature transformation for successfully reducing the dimension of feature data without degrading semantic retrieval performance significantly. We also studied the effects of image downscaling techniques on semantic retrieval performance via dedicated experiments in order to utilize the downscaling methods in TLQ scheme. The evaluation results show that image downscaling does not have significant impact on color and moderately affects texture-based retrieval in general, while it degrades edge-based retrieval performance significantly. In order to accomplish the primary objective of the thesis, a novel study on system profiles and adaptation of parameters for CBIR applications is presented. The main aim of the study is to improve the overall CBIR system performance in different hardware platforms having different technical capabilities and conditions. We define CBIR system profiles in terms of hardware and system platform properties and propose CBIR parameters for each defined system profile. The performances of the proposed parameters for each system profile are assessed over a large set of experiments. Experimental studies show that the proposed parameters for each system profile improve semantic performance, while reducing computational complexity and storage requirement

Improving Content-Based Image Indexing and Retrieval Performance

Latest image technology improvements along with the Internet growth have led to a huge amount of digital multimedia during the recent decades. Various methods, algorithms and systems have been proposed addressing image storage and management problems. Such studies revealed the indexing and retrieval concepts, which have further evolved to Content-Based Image Retrieval (CBIR). CBIR systems often analyze image content via the so-called low-level features for indexing and retrieval, such as color, texture and shape. In order to achieve significantly higher semantic performance, recent systems seek to combine low-level with high-level features that contain perceptual information for human. However, such combinations increase the feature extraction processing time and the memory requirements as well as the retrieval complexity. Performance improvements of indexing and retrieval play an important role for providing advanced CBIR services on every hardware platform. In this thesis, we propose novel techniques for improving the overall performance of CBIR. We define general CBIR challenges as memory and disk space requirements, computational complexity, semantic retrieval performance, and usability. Bringing generic and feasible solutions to these challenges is the main contribution of this thesis. A novel system for feature selection is introduced for enhancing semantic image retrieval results, decreasing retrieval process complexity, and improving the overall system usability for end-users of CBIR systems. Three feature selection criteria and a decision method construct the proposed feature selection system. A majority voting based method is adapted for efficient selection of features and feature combinations. The performance of the proposed criteria is assessed over a large image database and a number of features, and compared against other techniques from the literature. Experiments show that the proposed feature selection system improves semantic performance results in image retrieval systems. We introduce a novel Transform-Based Layered Query (TLQ) Scheme designed for efficient handling of visual media retrieval, which mainly aims at decreasing processing time and run-time memory consumption without degrading semantic retrieval results. The proposed scheme is based on abstract layers in indexing and retrieval phases, where each indexing layer of TLQ corresponds to a retrieval layer. The layers are independent from the underlying indexing and retrieval methods, and mainly constructed using multimedia and feature data transformations for reducing data dimensions. A two-layer TLQ scheme is implemented and integrated into MUVIS content-based multimedia indexing and retrieval framework. A new feature dimension reduction method referred to as Mapping by Adaptive Threshold (MAT) is also proposed as a solution for memory requirements and computational complexity of retrieval processes. Theoretical and practical advantages of TLQ over existing methods are validated experimentally on image databases using the MAT method for feature data. Experimental studies also show that the proposed MAT method is a fast feature transformation for successfully reducing the dimension of feature data without degrading semantic retrieval performance significantly. We also studied the effects of image downscaling techniques on semantic retrieval performance via dedicated experiments in order to utilize the downscaling methods in TLQ scheme. The evaluation results show that image downscaling does not have significant impact on color and moderately affects texture-based retrieval in general, while it degrades edge-based retrieval performance significantly. In order to accomplish the primary objective of the thesis, a novel study on system profiles and adaptation of parameters for CBIR applications is presented. The main aim of the study is to improve the overall CBIR system performance in different hardware platforms having different technical capabilities and conditions. We define CBIR system profiles in terms of hardware and system platform properties and propose CBIR parameters for each defined system profile. The performances of the proposed parameters for each system profile are assessed over a large set of experiments. Experimental studies show that the proposed parameters for each system profile improve semantic performance, while reducing computational complexity and storage requirement