A NeuroGenetic Approach for Multiprocessor Scheduling

This chapter presents a NeuroGenetic approach for solving a family of multiprocessor scheduling problems. We address primarily the Job-Shop scheduling problem, one of the hardest of the various scheduling problems. We propose a new approach, the NeuroGenetic approach, which is a hybrid metaheuristic that combines augmented-neural-networks (AugNN) and genetic algorithms-based search methods. The AugNN approach is a nondeterministic iterative local-search method which combines the benefits of a heuristic search and iterative neural-network search. Genetic algorithms based search is particularly good at global search. An interleaved approach between AugNN and GA combines the advantages of local search and global search, thus providing improved solutions compared to AugNN or GA search alone. We discuss the encoding and decoding schemes for switching between GA and AugNN approaches to allow interleaving. The purpose of this study is to empirically test the extent of improvement obtained by using the interleaved hybrid approach instead of applied using a single approach on the job-shop scheduling problem. We also describe the AugNN formulation and a Genetic Algorithm approach for the JobShop problem. We present the results of AugNN, GA and the NeuroGentic approach on some benchmark job-shop scheduling problems

Augmented neural networks and problem-structure based heuristics for the bin-packing problem

In this paper, we apply the Augmented-neural-networks (AugNN) approach for solving the classical bin-packing problem (BPP). AugNN is a metaheuristic that combines a priority- rule heuristic with the iterative search approach of neural networks to generate good solutions fast. This is the first time this approach has been applied to the BPP. We also propose a decomposition approach for solving harder BPP, in which sub problems are solved using a combination of AugNN approach and heuristics that exploit the problem structure. We discuss the characteristics of problems on which such problem-structure based heuristics could be applied. We empirically show the effectiveness of the AugNN and the decomposition approach on many benchmark problems in the literature. For the 1210 benchmark problems tested, 917 problems were solved to optimality and the average gap between the obtained solution and the upper bound for all the problems was reduced to under 0.66% and computation time averaged below 33 seconds per problem. We also discuss the computational complexity of our approach

The Task Scheduling Problem: A NeuroGenetic Approach

This paper addresses the task scheduling problem which involves minimizing the makespan in scheduling n tasks on m machines (resources) where the tasks follow a precedence relation and preemption is not allowed.  The machines (resources) are all identical and a task needs only one machine for processing.  Like most scheduling problems, this one is NP-hard in nature, making it difficult to find exact solutions for larger problems in reasonable computational time.  Heuristic and metaheuristic approaches are therefore needed to solve this type of problem.   This paper proposes a metaheuristic approach - called NeuroGenetic - which is a combination of an augmented neural network and a genetic algorithm.  The augmented neural network approach is itself a hybrid of a heuristic approach and a neural network approach.  The NeuroGenetic approach is tested against some popular test problems from the literature, and the results indicate that the NeuroGenetic approach performs significantly better than either the augmented neural network or the genetic algorithms alone.

The min-conflicts heuristic: Experimental and theoretical results

This paper describes a simple heuristic method for solving large-scale constraint satisfaction and scheduling problems. Given an initial assignment for the variables in a problem, the method operates by searching through the space of possible repairs. The search is guided by an ordering heuristic, the min-conflicts heuristic, that attempts to minimize the number of constraint violations after each step. We demonstrate empirically that the method performs orders of magnitude better than traditional backtracking techniques on certain standard problems. For example, the one million queens problem can be solved rapidly using our approach. We also describe practical scheduling applications where the method has been successfully applied. A theoretical analysis is presented to explain why the method works so well on certain types of problems and to predict when it is likely to be most effective

Curriculum Dropout

Dropout is a very effective way of regularizing neural networks. Stochastically "dropping out" units with a certain probability discourages over-specific co-adaptations of feature detectors, preventing overfitting and improving network generalization. Besides, Dropout can be interpreted as an approximate model aggregation technique, where an exponential number of smaller networks are averaged in order to get a more powerful ensemble. In this paper, we show that using a fixed dropout probability during training is a suboptimal choice. We thus propose a time scheduling for the probability of retaining neurons in the network. This induces an adaptive regularization scheme that smoothly increases the difficulty of the optimization problem. This idea of "starting easy" and adaptively increasing the difficulty of the learning problem has its roots in curriculum learning and allows one to train better models. Indeed, we prove that our optimization strategy implements a very general curriculum scheme, by gradually adding noise to both the input and intermediate feature representations within the network architecture. Experiments on seven image classification datasets and different network architectures show that our method, named Curriculum Dropout, frequently yields to better generalization and, at worst, performs just as well as the standard Dropout method.Comment: Accepted at ICCV (International Conference on Computer Vision) 201