715 research outputs found
Shifting Data Collection from a Fixed to an Adaptive Sampling Paradigm
For domains where data are difficult to obtain due to human or resource limitations, an emphasis is needed to efficiently explore the dimensions of information spaces to acquire any given response of interest. Many disciplines are still making the transition from brute force, dense, full factorial exploration of their information spaces to a more efficient design of experiments approach; the latter being in use successfully for many decades in agricultural and automotive applications. Although this transition is still incomplete, groundwork must be laid for incorporating the next generation of algorithms to adaptively explore the information space in response to data collected, as well as any resulting empirical models (i.e., metamodels). The methodology in the present work was to compare metamodel quality using a fixed sampling technique compared to an adaptive sampling technique based on metamodel variance. In order to quantify metamodeling errors, a delta method was used to provide quantitative model variance estimates. The present methodology was applied to a design space with an air-breathing engine performance response. It was shown that competitive metamodel quality with lower associated error could be achieved for an adaptive sampling technique for the same level of effort as a fixed, a priori sampling technique
Probabilistic Metamodels for an Efficient Characterization of Complex Driving Scenarios
To validate the safety of automated vehicles (AV), scenario-based testing
aims to systematically describe driving scenarios an AV might encounter. In
this process, continuous inputs such as velocities result in an infinite number
of possible variations of a scenario. Thus, metamodels are used to perform
analyses or to select specific variations for examination. However, despite the
safety criticality of AV testing, metamodels are usually seen as a part of an
overall approach, and their predictions are not questioned. This paper analyzes
the predictive performance of Gaussian processes (GP), deep Gaussian processes,
extra-trees, and Bayesian neural networks (BNN), considering four scenarios
with 5 to 20 inputs. Building on this, an iterative approach is introduced and
evaluated, which allows to efficiently select test cases for common analysis
tasks. The results show that regarding predictive performance, the appropriate
selection of test cases is more important than the choice of metamodels.
However, the choice of metamodels remains crucial: Their great flexibility
allows BNNs to benefit from large amounts of data and to model even the most
complex scenarios. In contrast, less flexible models like GPs convince with
higher reliability. Hence, relevant test cases are best explored using scalable
virtual test setups and flexible models. Subsequently, more realistic test
setups and more reliable models can be used for targeted testing and
validation.Comment: 10 pages, 14 figures, 1 table, associated dataset at
https://github.com/wnklmx/DSIO
Thematic issue on evolutionary algorithms in water resources
Special Issue on Evolutionary Algorithms.H.R. Maier, Z. Kapelan, J. Kasprzyk, L.S. Matot
On the Value of Quality Attributes for Refactoring Model Transformations Using a Multi-Objective Algorithm
Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/152454/1/QMOOD_for_ATL__Copy_.pd
SIMULATION METAMODELING AND OPTIMIZATION WITH AN ADDITIVE GLOBAL AND LOCAL GAUSSIAN PROCESS MODEL FOR STOCHASTIC SYSTEMS
Ph.DDOCTOR OF PHILOSOPH
From examples to knowledge in model-driven engineering : a holistic and pragmatic approach
Le Model-Driven Engineering (MDE) est une approche de développement logiciel qui
propose d’élever le niveau d’abstraction des langages afin de déplacer l’effort de
conception et de compréhension depuis le point de vue des programmeurs vers celui des
décideurs du logiciel. Cependant, la manipulation de ces représentations abstraites, ou
modèles, est devenue tellement complexe que les moyens traditionnels ne suffisent plus à
automatiser les différentes tâches.
De son côté, le Search-Based Software Engineering (SBSE) propose de reformuler
l’automatisation des tâches du MDE comme des problèmes d’optimisation. Une fois
reformulé, la résolution du problème sera effectuée par des algorithmes métaheuristiques.
Face à la pléthore d’études sur le sujet, le pouvoir d’automatisation du SBSE n’est plus à
démontrer.
C’est en s’appuyant sur ce constat que la communauté du Example-Based MDE (EBMDE)
a commencé à utiliser des exemples d’application pour alimenter la reformulation
SBSE du problème d’apprentissage de tâche MDE. Dans ce contexte, la concordance de la
sortie des solutions avec les exemples devient un baromètre efficace pour évaluer l’aptitude
d’une solution à résoudre une tâche. Cette mesure a prouvé être un objectif sémantique de
choix pour guider la recherche métaheuristique de solutions.
Cependant, s’il est communément admis que la représentativité des exemples a un
impact sur la généralisabilité des solutions, l'étude de cet impact souffre d’un manque de
considération flagrant. Dans cette thèse, nous proposons une formulation globale du
processus d'apprentissage dans un contexte MDE incluant une méthodologie complète pour
caractériser et évaluer la relation qui existe entre la généralisabilité des solutions et deux
propriétés importantes des exemples, leur taille et leur couverture.
Nous effectuons l’analyse empirique de ces deux propriétés et nous proposons un plan
détaillé pour une analyse plus approfondie du concept de représentativité, ou d’autres
représentativités.Model-Driven Engineering (MDE) is a software development approach that proposes to
raise the level of abstraction of languages in order to shift the design and understanding
effort from a programmer point of view to the one of decision makers. However, the
manipulation of these abstract representations, or models, has become so complex that
traditional techniques are not enough to automate its inherent tasks.
For its part, the Search-Based Software Engineering (SBSE) proposes to reformulate
the automation of MDE tasks as optimization problems. Once reformulated, the problem will
be solved by metaheuristic algorithms. With a plethora of studies on the subject, the power
of automation of SBSE has been well established.
Based on this observation, the Example-Based MDE community (EB-MDE) started
using application examples to feed the reformulation into SBSE of the MDE task learning
problem. In this context, the concordance of the output of the solutions with the examples
becomes an effective barometer for evaluating the ability of a solution to solve a task. This
measure has proved to be a semantic goal of choice to guide the metaheuristic search for
solutions.
However, while it is commonly accepted that the representativeness of the examples
has an impact on the generalizability of the solutions, the study of this impact suffers from a
flagrant lack of consideration. In this thesis, we propose a thorough formulation of the
learning process in an MDE context including a complete methodology to characterize and
evaluate the relation that exists between two important properties of the examples, their size
and coverage, and the generalizability of the solutions.
We perform an empirical analysis, and propose a detailed plan for further investigation
of the concept of representativeness, or of other representativities
- …