Closed-Loop Learning of Visual Control Policies

In this paper we present a general, flexible framework for learning mappings from images to actions by interacting with the environment. The basic idea is to introduce a feature-based image classifier in front of a reinforcement learning algorithm. The classifier partitions the visual space according to the presence or absence of few highly informative local descriptors that are incrementally selected in a sequence of attempts to remove perceptual aliasing. We also address the problem of fighting overfitting in such a greedy algorithm. Finally, we show how high-level visual features can be generated when the power of local descriptors is insufficient for completely disambiguating the aliased states. This is done by building a hierarchy of composite features that consist of recursive spatial combinations of visual features. We demonstrate the efficacy of our algorithms by solving three visual navigation tasks and a visual version of the classical Car on the Hill control problem

Segmentation Methods in Biomedical Image Processing

Disertační práce pojednává o moderních metodách a přístupech ke zpracování obrazů, konkrétně k jejich segmentaci, klasifikaci a vyhodnocování parametrů. Jedná se především o zpracování medicínských snímků měkkých tkání pořízených metodou magnetické rezonance (MR) a dále mikroskopických obrazů tkání. Ze segmentovaných obrazů lze jednoduše popsat hranice hledaných objektů. Tyto nalezené hranice mohou sloužit k dalšímu zpracování jako výpočet obvodů, obsahů, povrchů, objemů nebo dokonce k trojrozměrné rekonstrukci zobrazovaného objektu. Popsaná navržená řešení lze použít pro klasifikaci zdravých či postižených tkání snímaných metodami MR či jinými. V disertační práci jsou uvedeny příklady aplikací, ve kterých byly navržené segmentační metody použity. V oblasti segmentace obrazů se práce zaměřuje na metody založené na řešení parciálních diferenciálních rovnic. Jedná se o moderní přístupy zpracování obrazů, zvané též aktivní kontury. Tento přístup ke zpracování obrazů je velmi výhodný u segmentace reálného obrazu, který je zatížený šumem, má neostré hrany a přechody mezi objekty. Výsledkem disertační práce jsou navržené metody pro automatickou segmentaci obrazů a klasifikaci objektů.The PhD thesis deals with modern methods of image processing, especially image segmentation, classification and evaluation of parameters. It is focused primarily on processing medical images of soft tissues obtained by magnetic resonance tomography (MR) and microscopic images of tissues. It is easy to describe edges of the sought objects using of segmented images. The edges found can be useful for further processing of monitored object such as calculating the perimeter, surface and volume evaluation or even three-dimensional shape reconstruction. The proposed solutions can be used for the classification of healthy/unhealthy tissues in MR or other imaging. Application examples of the proposed segmentation methods are shown in this thesis. Research in the area of image segmentation is focused on methods based on solving partial differential equations. This is a modern method for image processing, often called the active contour method. It is of great advantage in the segmentation of real images degraded by noise with fuzzy edges and transitions between objects. The results of the thesis are methods proposed for automatic image segmentation and classification.

Reinforced Segmentation of Images Containing One Object of Interest

In many image-processing applications, one object of interest must be segmented. The techniques used for segmentation vary depending on the particular situation and the specifications of the problem at hand. In methods that rely on a learning process, the lack of a sufficient number of training samples is usually an obstacle, especially when the samples need to be manually prepared by an expert. The performance of some other methods may suffer from frequent user interactions to determine the critical segmentation parameters. Also, none of the existing approaches use online (permanent) feedback, from the user, in order to evaluate the generated results. Considering the above factors, a new multi-stage image segmentation system, based on Reinforcement Learning (RL) is introduced as the main contribution of this research. In this system, the RL agent takes specific actions, such as changing the tasks parameters, to modify the quality of the segmented image. The approach starts with a limited number of training samples and improves its performance in the course of time. In this system, the expert knowledge is continuously incorporated to increase the segmentation capabilities of the method. Learning occurs based on interactions with an offline simulation environment, and later online through interactions with the user. The offline mode is performed using a limited number of manually segmented samples, to provide the segmentation agent with basic information about the application domain. After this mode, the agent can choose the appropriate parameter values for different processing tasks, based on its accumulated knowledge. The online mode, consequently, guarantees that the system is continuously training and can increase its accuracy, the more the user works with it. During this mode, the agent captures the user preferences and learns how it must change the segmentation parameters, so that the best result is achieved. By using these two learning modes, the RL agent allows us to optimally recognize the decisive parameters for the entire segmentation process

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field