Fine-tuning or top-tuning? Transfer learning with pretrained features and fast kernel methods

The impressive performances of deep learning architectures is associated to massive increase of models complexity. Millions of parameters need be tuned, with training and inference time scaling accordingly. But is massive fine-tuning necessary? In this paper, focusing on image classification, we consider a simple transfer learning approach exploiting pretrained convolutional features as input for a fast kernel method. We refer to this approach as top-tuning, since only the kernel classifier is trained. By performing more than 2500 training processes we show that this top-tuning approach provides comparable accuracy w.r.t. fine-tuning, with a training time that is between one and two orders of magnitude smaller. These results suggest that top-tuning provides a useful alternative to fine-tuning in small/medium datasets, especially when training efficiency is crucial

Efficient Unsupervised Learning for Plankton Images

Monitoring plankton populations in situ is fundamental to preserve the aquatic ecosystem. Plankton microorganisms are in fact susceptible of minor environmental perturbations, that can reflect into consequent morphological and dynamical modifications. Nowadays, the availability of advanced automatic or semi-automatic acquisition systems has been allowing the production of an increasingly large amount of plankton image data. The adoption of machine learning algorithms to classify such data may be affected by the significant cost of manual annotation, due to both the huge quantity of acquired data and the numerosity of plankton species. To address these challenges, we propose an efficient unsupervised learning pipeline to provide accurate classification of plankton microorganisms. We build a set of image descriptors exploiting a two-step procedure. First, a Variational Autoencoder (VAE) is trained on features extracted by a pre-trained neural network. We then use the learnt latent space as image descriptor for clustering. We compare our method with state-of-the-art unsupervised approaches, where a set of pre-defined hand-crafted features is used for clustering of plankton images. The proposed pipeline outperforms the benchmark algorithms for all the plankton datasets included in our analysis, providing better image embedding properties

Efficient machine learning with resources constraints

Since machine learning techniques spread in the scientific community and in real-world scenarios, their usage has been justified by the impossibility of traditional techniques to deal with simple problems that require the retrieval of specific task-related information. In the beginning, neural networks were made of a very reduced amount of layers, with a limited capacity to solve complicated problems. However, in the last years, the set of methodologies we usually refer to as \textit{deep learning} became the de-facto standard in a large variety of fields. Their astonishing ability to solve different kinds of problems has been proven, from very simple and specific tasks to more general problems, such as image recognition, object detection, video recognition, and natural language processing. In the last two years a new approach, referred to as transformers, has been proposed showing state-of-the-art performances in similar contexts to the ones covered by convolutional neural networks. The huge improvement in performances obtained by recent models came at a cost from different points of view. The number of learnable parameters involved moved from tens of millions to hundreds of billions in less than ten years coupled with an increase from a few hundred to millions of PFLOPS needed to train better models in terms of performance. Overall, the amount of energy needed to train the more recent architectures increased drastically in the last few years showing a problematic situation in terms of resources needed to obtain the next state-of-the-art performance. In this thesis, we will see different methodologies to alleviate the computational costs of some typical machine learning problems. First, we will focus on image classification, considering a simple transfer learning approach that exploits pre-trained convolutional features as input for a fast kernel method. By performing more than three thousand training processes, we will show that this fast-kernel approach provides comparable accuracy w.r.t. fine-tuning, with a training time that is between one and two orders of magnitude smaller. Then we will introduce and discuss an unsupervised pipeline that projects input images to a latent space with reduced dimension, making the clustering operation doable. We will show the pipeline effectiveness in a plankton monitoring context where operating in an unsupervised manner is crucial. Indeed, studying plankton population in situ is paramount to protect marine ecosystems as they can be regarded as biosensors. Lastly, we will discuss different methodologies to compare two or more image datasets. Indeed, each dataset can be seen as a set of points sampled by an unknown distribution that we can estimate and analyze. We will introduce different methodologies to study such distributions. We will show that, even on simple tasks involving images, the concept of dataset distance is elusive and very complicated to quantify. It is possible to obtain information on different image datasets, via good partitioning, as long as we analyze a small datasets subset. Overall, in this thesis, we will consider a set of techniques that can alleviate machine learning computational costs, in order to keep them computationally accessible to the scientific community