Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks

Deeper and wider Convolutional Neural Networks (CNNs) achieve superior performance but bring expensive computation cost. Accelerating such over-parameterized neural network has received increased attention. A typical pruning algorithm is a three-stage pipeline, i.e., training, pruning, and retraining. Prevailing approaches fix the pruned filters to zero during retraining, and thus significantly reduce the optimization space. Besides, they directly prune a large number of filters at first, which would cause unrecoverable information loss. To solve these problems, we propose an Asymptotic Soft Filter Pruning (ASFP) method to accelerate the inference procedure of the deep neural networks. First, we update the pruned filters during the retraining stage. As a result, the optimization space of the pruned model would not be reduced but be the same as that of the original model. In this way, the model has enough capacity to learn from the training data. Second, we prune the network asymptotically. We prune few filters at first and asymptotically prune more filters during the training procedure. With asymptotic pruning, the information of the training set would be gradually concentrated in the remaining filters, so the subsequent training and pruning process would be stable. Experiments show the effectiveness of our ASFP on image classification benchmarks. Notably, on ILSVRC-2012, our ASFP reduces more than 40% FLOPs on ResNet-50 with only 0.14% top-5 accuracy degradation, which is higher than the soft filter pruning (SFP) by 8%.Comment: Extended Journal Version of arXiv:1808.0686

Cognitive Algorithms and digitized Tissue – based Diagnosis

Aims: To analyze the nature and impact of cognitive algorithms and programming on digitized tissue – based diagnosis. Definitions: Digitized tissue – based diagnosis includes all computerized tissue investigations that contribute to the most appropriate description and forecast of the actual patient’s disease [1]. Cognitive algorithms are programs that encompass machine learning, reasoning, and human – computer interaction [2]. Theoretical considerations: Digitized blood data, objective clinical findings, microscopic, gross, radiological images and gene alterations are analyzed by specialized image analysis methods, and transferred in numbers and vectors. These are analyzed by statistical procedures. They include higher order statistics such as multivariate analysis, neural networks and ‘black box’ strategies, for example ‘deep learning’ or ‘Watson’ approaches. These algorithms can be applied at different cognitive ‘levels’, to reach a digital decision for different procedures which should assist the patient’s health condition. These levels can be grouped in self learning, self promoting, self targeting, and self exploring algorithms. Each of them requires a memory and neighbourhood condition. Self targeting and exploring algorithms are circumscribed mechanisms with singularities and repair procedures. They develop self recognition.   Consecutives: Medical doctors including pathologists are commonly not trained to understand the basic principles and workflow of applied or potential future procedures. At present, basic medical data only serve for simple cognitive algorithms. Most of the investigations focus on ‘deep learning’ procedures. The applied learning and decision algorithms might be modified and themselves be used for ‘next order cognitive algorithms’. Such systems will develop their own strategies, and become independent from potential human interactions. The basic strategy of such IT systems is described herein. Perspectives: Medical doctors including pathologists should be aware about the abilities to enhance their work by supporting tools. In some case the users may not be able to fully understand these tools. Furthermore, these tools will probably become self learning, and, therefore, seem to propose the daily workflow probably without any medical control or even interaction

Deep learning methods for solving linear inverse problems: Research directions and paradigms

The linear inverse problem is fundamental to the development of various scientific areas. Innumerable attempts have been carried out to solve different variants of the linear inverse problem in different applications. Nowadays, the rapid development of deep learning provides a fresh perspective for solving the linear inverse problem, which has various well-designed network architectures results in state-of-the-art performance in many applications. In this paper, we present a comprehensive survey of the recent progress in the development of deep learning for solving various linear inverse problems. We review how deep learning methods are used in solving different linear inverse problems, and explore the structured neural network architectures that incorporate knowledge used in traditional methods. Furthermore, we identify open challenges and potential future directions along this research line

Tensor Regression

Regression analysis is a key area of interest in the field of data analysis and machine learning which is devoted to exploring the dependencies between variables, often using vectors. The emergence of high dimensional data in technologies such as neuroimaging, computer vision, climatology and social networks, has brought challenges to traditional data representation methods. Tensors, as high dimensional extensions of vectors, are considered as natural representations of high dimensional data. In this book, the authors provide a systematic study and analysis of tensor-based regression models and their applications in recent years. It groups and illustrates the existing tensor-based regression methods and covers the basics, core ideas, and theoretical characteristics of most tensor-based regression methods. In addition, readers can learn how to use existing tensor-based regression methods to solve specific regression tasks with multiway data, what datasets can be selected, and what software packages are available to start related work as soon as possible. Tensor Regression is the first thorough overview of the fundamentals, motivations, popular algorithms, strategies for efficient implementation, related applications, available datasets, and software resources for tensor-based regression analysis. It is essential reading for all students, researchers and practitioners of working on high dimensional data.Comment: 187 pages, 32 figures, 10 table

Estudio y mejora de la técnica en representación de información tridimensional y bidimensional sobre display

En esta tesis se abordan diferentes cuestiones relacionadas con el procesamiento digital de imágenes condicionadas a la representación y visualización de las mismas. En primer lugar, se ha estudiado la representación y análisis de la función de light-field sobre displays automultiscópicos. Esto se consigue presentando a cada ojo un conjunto de imágenes similares, pero diferenciables en tanto a la disparidad espacial obtenida en relación a la distancia real de los objetos presentes en la escena. Específicamente, esta parte del trabajo se ha elaborado sobre el concepto de display multicapa, siendo éste, esencialmente, un dispositivo compuesto por múltiples capas en las que es posible representar diferentes imágenes, y cuya superposición permite proyectar direccionalmente información que cumpla los requisitos anteriormente descritos. La contribución en este sentido es la mejora de los algoritmos ya existentes para la síntesis de la función de light-field. Los métodos numéricos comúnmente empleados son muy sensibles al mal condicionamiento del problema, y por tanto, dependen enormemente de la solución inicial del problema. Se analizan diferentes métodos, y se presentan dos alternativas que mejoran los resultados previos sobre el método. como generalización del algoritmo WNMF, en tomografía de la atmósfera. Además se ha estudiado un método que permite mejorar la resolución de las imágenes mostradas por encima del límite que el ancho de banda del dispositivo determina. Esto se ha conseguido aplicando un desenfoque artificial al sistema observado y pre-compensando la imagen representada teniendo en cuenta el diámetro del círculo de confusión. Finalmente, se presenta una técnica basada en la medida y caracterización de un display con microlentes para mejorar la percepción de un observador no emétrope