112,905 research outputs found
SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization
Computer vision is experiencing an AI renaissance, in which machine learning
models are expediting important breakthroughs in academic research and
commercial applications. Effectively training these models, however, is not
trivial due in part to hyperparameters: user-configured values that control a
model's ability to learn from data. Existing hyperparameter optimization
methods are highly parallel but make no effort to balance the search across
heterogeneous hardware or to prioritize searching high-impact spaces. In this
paper, we introduce a framework for massively Scalable Hardware-Aware
Distributed Hyperparameter Optimization (SHADHO). Our framework calculates the
relative complexity of each search space and monitors performance on the
learning task over all trials. These metrics are then used as heuristics to
assign hyperparameters to distributed workers based on their hardware. We first
demonstrate that our framework achieves double the throughput of a standard
distributed hyperparameter optimization framework by optimizing SVM for MNIST
using 150 distributed workers. We then conduct model search with SHADHO over
the course of one week using 74 GPUs across two compute clusters to optimize
U-Net for a cell segmentation task, discovering 515 models that achieve a lower
validation loss than standard U-Net.Comment: 10 pages, 6 figure
Online Planner Selection with Graph Neural Networks and Adaptive Scheduling
Automated planning is one of the foundational areas of AI. Since no single
planner can work well for all tasks and domains, portfolio-based techniques
have become increasingly popular in recent years. In particular, deep learning
emerges as a promising methodology for online planner selection. Owing to the
recent development of structural graph representations of planning tasks, we
propose a graph neural network (GNN) approach to selecting candidate planners.
GNNs are advantageous over a straightforward alternative, the convolutional
neural networks, in that they are invariant to node permutations and that they
incorporate node labels for better inference.
Additionally, for cost-optimal planning, we propose a two-stage adaptive
scheduling method to further improve the likelihood that a given task is solved
in time. The scheduler may switch at halftime to a different planner,
conditioned on the observed performance of the first one. Experimental results
validate the effectiveness of the proposed method against strong baselines,
both deep learning and non-deep learning based.
The code is available at \url{https://github.com/matenure/GNN_planner}.Comment: AAAI 2020. Code is released at
https://github.com/matenure/GNN_planner. Data set is released at
https://github.com/IBM/IPC-graph-dat
AADLib, A Library of Reusable AADL Models
The SAE Architecture Analysis and Design Language is now a well-established language for the description of critical embedded systems, but also cyber-physical ones. A wide range of analysis tools is already available, either as part of the OSATE tool chain, or separate ones.
A key missing elements of AADL is a set of reusable building blocks to help learning AADL concepts, but also experiment already existing tool chains on validated real-life examples.
In this paper, we present AADLib, a library of reusable model elements. AADLib is build on two pillars: 1/ a set of ready-to- use examples so that practitioners can learn more about the AADL language itself, but also experiment with existing tools. Each example comes with a full description of available analysis and expected results. This helps reducing the learning curve of the language. 2/ a set of reusable model elements that cover typical building blocks of critical systems: processors, networks, devices with a high level of fidelity so that the cost to start a new project is reduced.
AADLib is distributed under a Free/Open Source License to further disseminate the AADL language. As such, AADLib provides a convenient way to discover AADL concepts and tool chains, and learn about its features
The Emergence of Norms via Contextual Agreements in Open Societies
This paper explores the emergence of norms in agents' societies when agents
play multiple -even incompatible- roles in their social contexts
simultaneously, and have limited interaction ranges. Specifically, this article
proposes two reinforcement learning methods for agents to compute agreements on
strategies for using common resources to perform joint tasks. The computation
of norms by considering agents' playing multiple roles in their social contexts
has not been studied before. To make the problem even more realistic for open
societies, we do not assume that agents share knowledge on their common
resources. So, they have to compute semantic agreements towards performing
their joint actions. %The paper reports on an empirical study of whether and
how efficiently societies of agents converge to norms, exploring the proposed
social learning processes w.r.t. different society sizes, and the ways agents
are connected. The results reported are very encouraging, regarding the speed
of the learning process as well as the convergence rate, even in quite complex
settings
- …