On Building Generalizable Learning Agents

It has been a long-standing goal in Artificial Intelligence (AI) to build machines that can solve tasks that humans can. Thanks to the recent rapid progress in data-driven methods, which train agents to solve tasks by learning from massive training data, there have been many successes in applying such learning approaches to handle and even solve a number of extremely challenging tasks, including image classification, language generation, robotics control, and several multi-player games. The key factor for all these data-driven successes is that the trained agents can generalize to test scenarios that are unseen during training. This generalization capability is the foundation for building any practical AI system. This thesis studies generalization, the fundamental challenge in AI, and proposes solutions to improve the generalization performances of learning agents in a variety of problems. We start by providing a formal formulation of the generalization problem in the context of reinforcement learning and proposing 4 principles within this formulation to guide the design of training techniques for improved generalization. We validate the effectiveness of our proposed principles by considering 4 different domains, from simple to complex, and developing domain-specific techniques following these principles. Particularly, we begin with the simplest domain, i.e., path-finding on graphs (Part I), and then consider visual navigation in a 3D world (Part II) and competition in complex multi-agent games (Part III), and lastly tackle some natural language processing tasks (Part IV). Empirical evidences demonstrate that the proposed principles can generally lead to much improved generalization performances in a wide range of problems

A PCNN Framework for Blood Cell Image Segmentation

This research presents novel methods for segmenting digital blood cell images under a Pulse Coupled Neural Network (PCNN) framework. A blood cell image contains different types of blood cells found in the peripheral blood stream such as red blood cells (RBCs), white blood cells (WBCs), and platelets. WBCs can be classified into five normal types – neutrophil, monocyte, lymphocyte, eosinophil, and basophil – as well as abnormal types such as lymphoblasts and others. The focus of this research is on identifying and counting RBCs, normal types of WBCs, and lymphoblasts. The total number of RBCs and WBCs, along with classification of WBCs, has important medical significance which includes providing a physician with valuable information for diagnosis of diseases such as leukemia. The approach comprises two phases – segmentation and cell separation – followed by classification of WBC types including detection of lymphoblasts. The first phase presents two methods based on PCNN and region growing to segment followed by a separate method that combines Circular Hough Transform (CHT) with a separation algorithm to find and separate each RBC and WBC object into separate images. The first method uses a standard PCNN to segment. The second method uses a region growing PCNN with a maximum region size to segment. The second phase presents a WBC classification method based on PCNN. It uses a PCNN to capture the texture features of an image as a sequence of entropy values known as a texture vector. First, the parameters of the texture vector PCNN are defined. This is then used to produce texture vectors for the training images. Each cell type is represented by several texture vectors across its instances. Then, given a test image to be classified, the texture vector PCNN is used to capture its texture vector, which is compared to the texture vectors for classification. This two-phase approach yields metrics based on the RBC and WBC counts, WBC classification, and identification of lymphoblasts. Both the standard and region growing PCNNs were successful in segmenting RBC and WBC objects, with better accuracy when using the standard PCNN. The separate method introduced with this research provided accurate WBC counts but less accurate RBC counts. The WBC subimages created with the separate method facilitated cell counting and WBC classification. Using a standard PCNN as a WBC classifier, introduced with this research, proved to be a successful classifier and lymphoblast detector. While RBC accuracy was low, WBC accuracy for total counts, WBC classification, and lymphoblast detection were overall above 96%

Novel neural network-based algorithms for urban classification and change detection from satellite imagery

L`attività umana sta cambiando radicalmente l`ecosistema ambientale, unito anche alla rapida espansione demografica dei sistemi urbani. Benche` queste aree rappresentano solo una minima frazione della Terra, il loro impatto sulla richiesta di energia, cibo, acqua e materiali primi, e` enorme. Per cui, una informazione accurata e tempestiva risulta essere essenziale per gli enti di protezione civile in caso, ad esempio, di catastrofi ambientali. Negli ultimi anni il forte sviluppo di sistemi satellitari, sia dal punto di vista della risoluzione spaziale che di quella radiometrica e temporale, ha permesso una sempre piu` accurato monitoraggio della Terra, sia con sistemi ottici che con quelli RADAR. Ad ogni modo, una piu` alta risoluzione (sia spaziale, che spettrale o temporale) presenta tanti vantaggi e miglioramenti quanti svantaggi e limitazioni. In questa tesi sono discussi in dettaglio i diversi aspetti e tecniche per la classificazione e monitoraggio dei cambiamenti di aree urbane, utilizzando sia sistemi ottici che RADAR. Particolare enfasi e` data alla teoria ed all`uso di reti neurali.Human activity dominates the Earth's ecosystems with structural modifications. The rapid population growth over recent decades and the concentration of this population in and around urban areas have significantly impacted the environment. Although urban areas represent a small fraction of the land surface, they affect large areas due to the magnitude of the associated energy, food, water, and raw material demands. Reliable information in populated areas is essential for urban planning and strategic decision making, such as civil protection departments in cases of emergency. Remote sensing is increasingly being used as a timely and cost-effective source of information in a wide number of applications, from environment monitoring to location-aware systems. However, mapping human settlements represents one of the most challenging areas for the remote sensing community due to its high spatial and spectral diversity. From the physical composition point of view, several different materials can be used for the same man-made element (for example, building roofs can be made of clay tiles, metal, asphalt, concrete, plastic, grass or stones). On the other hand, the same material can be used for different purposes (for example, concrete can be found in paved roads or building roofs). Moreover, urban areas are often made up of materials present in the surrounding region, making them indistinguishable from the natural or agricultural areas (examples can be unpaved roads and bare soil, clay tiles and bare soil, or parks and vegetated open spaces) [1]. During the last two decades, significant progress has been made in developing and launching satellites with instruments, in both the optical/infrared and microwave regions of the spectra, well suited for Earth observation with an increasingly finer spatial, spectral and temporal resolution. Fine spatial sensors with metric or sub-metric resolution allow the detection of small-scale objects, such as elements of residential housing, commercial buildings, transportation systems and utilities. Multi-spectral and hyper-spectral remote sensing systems provide additional discriminative features for classes that are spectrally similar, due to their higher spectral resolution. The temporal component, integrated with the spectral and spatial dimensions, provides essential information, for example on vegetation dynamics. Moreover, the delineation of temporal homogeneous patches reduces the effect of local spatial heterogeneity that often masks larger spatial patterns. Nevertheless, higher resolution (spatial, spectral or temporal) imagery comes with limits and challenges that equal the advantages and improvements, and this is valid for both optical and synthetic aperture radar data [2]. This thesis addresses the different aspects of mapping and change detection of human settlements, discussing the main issues related to the use of optical and synthetic aperture radar data. Novel approaches and techniques are proposed and critically discussed to cope with the challenges of urban areas, including data fusion, image information mining, and active learning. The chapters are subdivided into three main parts. Part I addresses the theoretical aspects of neural networks, including their different architectures, design, and training. The proposed neural networks-based algorithms, their applications to classification and change detection problems, and the experimental results are described in Part II and Part III