An algorithm for learning from hints

To take advantage of prior knowledge (hints) about the function one wants to learn, we introduce a method that generalizes learning from examples to learning from hints. A canonical representation of hints is defined and illustrated. All hints are represented to the learning process by examples, and examples of the function are treated on equal footing with the rest of the hints. During learning, examples from different hints are selected for processing according to a given schedule. We present two types of schedules; fixed schedules that specify the relative emphasis of each hint, and adaptive schedules that are based on how well each hint has been learned so far. Our learning method is compatible with any descent technique

Hints and the VC Dimension

Learning from hints is a generalization of learning from examples that allows for a variety of information about the unknown function to be used in the learning process. In this paper, we use the VC dimension, an established tool for analyzing learning from examples, to analyze learning from hints. In particular, we show how the VC dimension is affected by the introduction of a hint. We also derive a new quantity that defines a VC dimension for the hint itself. This quantity is used to estimate the number of examples needed to "absorb" the hint. We carry out the analysis for two types of hints, invariances and catalysts. We also describe how the same method can be applied to other types of hints

A Method for Learning from Hints

We address the problem of learning an unknown function by putting together several pieces of information (hints) that we know about the function. We introduce a method that generalizes learning from examples to learning from hints. A canonical representation of hints is defined and illustrated for new types of hints. All the hints are represented to the learning process by examples, and examples of the function are treated on equal footing with the rest of the hints. During learning, examples from different hints are selected for processing according to a given schedule. We present two types of schedules; fixed schedules that specify the relative emphasis of each hint, and adaptive schedules that are based on how well each hint has been learned so far. Our learning method is compatible with any descent technique that we may choose to use

Hints

The systematic use of hints in the learning-from-examples paradigm is the subject of this review. Hints are the properties of the target function that are known to us independently of the training examples. The use of hints is tantamount to combining rules and data in learning, and is compatible with different learning models, optimization techniques, and regularization techniques. The hints are represented to the learning process by virtual examples, and the training examples of the target function are treated on equal footing with the rest of the hints. A balance is achieved between the information provided by the different hints through the choice of objective functions and learning schedules. The Adaptive Minimization algorithm achieves this balance by relating the performance on each hint to the overall performance. The application of hints in forecasting the very noisy foreign-exchange markets is illustrated. On the theoretical side, the information value of hints is contrasted to the complexity value and related to the VC dimension

Natural Language Processing (Almost) from Scratch

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements

Heurística para TSP-2d euclideo y simétrico basadas en la triangulación de Delaunay y sus subgrafos

El objetivo de esta tesis es el desarrollo de nuevas heurísticas para el Traveling Salesman Problem, TSP en adelante, mediante el estudio de estructuras geométricas discretas basadas en la triangulación de Delaunay y sus subgrafos. Dichas heurísticas deberán proporcionar soluciones factibles a grandes instancias euclideas del TSP en el plano. Las mismas poseerán baja complejidad computacional y las soluciones que encuentren serán comparadas empíricamente con las encontradas por otros algoritmos existentes en la literatura. Para llevar a cabo esta tarea se incursionará en temas de complejidad computacional, teoría de grafos, geometría computacional y estructuras de datos, convergiendo estos en la más amplia y multidisciplinaria optimización combinatoria.Facultad de Ciencias Exacta

Heurística para TSP-2d euclideo y simétrico basadas en la triangulación de Delaunay y sus subgrafos

El objetivo de esta tesis es el desarrollo de nuevas heurísticas para el Traveling Salesman Problem, TSP en adelante, mediante el estudio de estructuras geométricas discretas basadas en la triangulación de Delaunay y sus subgrafos. Dichas heurísticas deberán proporcionar soluciones factibles a grandes instancias euclideas del TSP en el plano. Las mismas poseerán baja complejidad computacional y las soluciones que encuentren serán comparadas empíricamente con las encontradas por otros algoritmos existentes en la literatura. Para llevar a cabo esta tarea se incursionará en temas de complejidad computacional, teoría de grafos, geometría computacional y estructuras de datos, convergiendo estos en la más amplia y multidisciplinaria optimización combinatoria.Facultad de Ciencias Exacta