16 research outputs found
Perceived Trustworthiness of Natural Language Generators
Natural Language Generation tools, such as chatbots that can generate
human-like conversational text, are becoming more common both for personal and
professional use. However, there are concerns about their trustworthiness and
ethical implications. The paper addresses the problem of understanding how
different users (e.g., linguists, engineers) perceive and adopt these tools and
their perception of machine-generated text quality. It also discusses the
perceived advantages and limitations of Natural Language Generation tools, as
well as users' beliefs on governance strategies. The main findings of this
study include the impact of users' field and level of expertise on the
perceived trust and adoption of Natural Language Generation tools, the users'
assessment of the accuracy, fluency, and potential biases of machine-generated
text in comparison to human-written text, and an analysis of the advantages and
ethical risks associated with these tools as identified by the participants.
Moreover, this paper discusses the potential implications of these findings for
enhancing the AI development process. The paper sheds light on how different
user characteristics shape their beliefs on the quality and overall
trustworthiness of machine-generated text. Furthermore, it examines the
benefits and risks of these tools from the perspectives of different users.Comment: 16 pages, 5 figures, First International Symposium on Trustworthy
Autonomous Systems (TAS '23
RE-centric Recommendations for the Development of Trustworthy(er) Autonomous Systems
Complying with the EU AI Act (AIA) guidelines while developing and
implementing AI systems will soon be mandatory within the EU. However,
practitioners lack actionable instructions to operationalise ethics during AI
systems development. A literature review of different ethical guidelines
revealed inconsistencies in the principles addressed and the terminology used
to describe them. Furthermore, requirements engineering (RE), which is
identified to foster trustworthiness in the AI development process from the
early stages was observed to be absent in a lot of frameworks that support the
development of ethical and trustworthy AI. This incongruous phrasing combined
with a lack of concrete development practices makes trustworthy AI development
harder. To address this concern, we formulated a comparison table for the
terminology used and the coverage of the ethical AI principles in major ethical
AI guidelines. We then examined the applicability of ethical AI development
frameworks for performing effective RE during the development of trustworthy AI
systems. A tertiary review and meta-analysis of literature discussing ethical
AI frameworks revealed their limitations when developing trustworthy AI. Based
on our findings, we propose recommendations to address such limitations during
the development of trustworthy AI.Comment: Accepted at [TAS '23]{First International Symposium on Trustworthy
Autonomous Systems
Systematic Evaluation of Applying Space-Filling Curves to Automotive Maneuver Detection
Identifying driving maneuvers plays an essential role on-board vehicles to
monitor driving and driver states, as well as off-board to train and evaluate
machine learning algorithms for automated driving for example. Maneuvers can be
characterized by vehicle kinematics or data from its surroundings including
other traffic participants. Extracting relevant maneuvers therefore requires
analyzing time-series of (i) structured, multi-dimensional kinematic data, and
(ii) unstructured, large data samples for video, radar, or LiDAR sensors.
However, such data analysis requires scalable and computationally efficient
approaches, especially for non-annotated data. In this paper, we are presenting
a maneuver detection approach based on two variants of space-filling curves
(Z-order and Hilbert) to detect maneuvers when passing roundabouts that do not
use GPS data. We systematically evaluate their respective performance by
including permutations of selections of kinematic signals at varying
frequencies and compare them with two alternative baselines: All manually
identified roundabouts, and roundabouts that are marked by geofences. We find
that encoding just longitudinal and lateral accelerations sampled at 10Hz using
a Hilbert space-filling curve is already successfully identifying roundabout
maneuvers, which allows to avoid the use of potentially sensitive signals such
as GPS locations to comply with data protection and privacy regulations like
GDPR.Comment: 7 pages, 4 figure
A Shift In Artistic Practices through Artificial Intelligence
The explosion of content generated by Artificial Intelligence models has
initiated a cultural shift in arts, music, and media, where roles are changing,
values are shifting, and conventions are challenged. The readily available,
vast dataset of the internet has created an environment for AI models to be
trained on any content on the web. With AI models shared openly, and used by
many, globally, how does this new paradigm shift challenge the status quo in
artistic practices? What kind of changes will AI technology bring into music,
arts, and new media?Comment: Submitted to Leonardo Journa
Generative Artificial Intelligence for Software Engineering -- A Research Agenda
Generative Artificial Intelligence (GenAI) tools have become increasingly
prevalent in software development, offering assistance to various managerial
and technical project activities. Notable examples of these tools include
OpenAIs ChatGPT, GitHub Copilot, and Amazon CodeWhisperer. Although many recent
publications have explored and evaluated the application of GenAI, a
comprehensive understanding of the current development, applications,
limitations, and open challenges remains unclear to many. Particularly, we do
not have an overall picture of the current state of GenAI technology in
practical software engineering usage scenarios. We conducted a literature
review and focus groups for a duration of five months to develop a research
agenda on GenAI for Software Engineering. We identified 78 open Research
Questions (RQs) in 11 areas of Software Engineering. Our results show that it
is possible to explore the adoption of GenAI in partial automation and support
decision-making in all software development activities. While the current
literature is skewed toward software implementation, quality assurance and
software maintenance, other areas, such as requirements engineering, software
design, and software engineering education, would need further research
attention. Common considerations when implementing GenAI include industry-level
assessment, dependability and accuracy, data accessibility, transparency, and
sustainability aspects associated with the technology. GenAI is bringing
significant changes to the field of software engineering. Nevertheless, the
state of research on the topic still remains immature. We believe that this
research agenda holds significance and practical value for informing both
researchers and practitioners about current applications and guiding future
research
AI for Agile development: a Meta-Analysis
This study explores the benefits and challenges of integrating Artificial
Intelligence with Agile software development methodologies, focusing on
improving continuous integration and delivery. A systematic literature review
and longitudinal meta-analysis of the retrieved studies was conducted to
analyse the role of Artificial Intelligence and it's future applications within
Agile software development. The review helped identify critical challenges,
such as the need for specialised socio-technical expertise. While Artificial
Intelligence holds promise for improved software development practices, further
research is needed to better understand its impact on processes and
practitioners, and to address the indirect challenges associated with its
implementation.Comment: 8 pages, 2 figures, 24th International Conference on Agile Software
Development. AI-Assisted Agile Software Development Research Worksho
Automating crowd simulation: from parameter tuning to dynamic context-to-policy adaptation
Computer-generated crowds are becoming more and more frequent in films, video games and safety assessment applications. Many crowd simulation algorithms exist to address the needs of this diverse range of industries. Even though the underlying principles are similar, there are large differences between the resulting synthetic trajectories. Each algorithm has strengths and weaknesses that need to be weighted, and appropriate parameter values for them must be selected as well. These are not easy tasks and Machine Learning algorithms are often used to guide these decisions. In this work we study three of these tasks: parameter tuning, trajectory evaluation, and character policy selection and adaptation. Our results show the usefulness of the proposed methods to evaluate previously unseen synthetic trajectories to find appropriate parameter values for the algorithms without directly relying on real data. Moreover, by classifying the context of characters, we propose a policy adaptation strategy to improve crowd simulations.Les multituds simulades per ordinador sĂłn cada cop mĂ©s habituals en cinema, vĂdeo jocs i en aplicacions relacionades amb la seguretat. Existeixen molts algoritmes per simular multituds per adreçar tal varietat d’indĂşstries. Tot i que els principis subjacents sĂłn similars, hi ha diferències entre les simulacions resultants. Cada algoritme tĂ© avantatges i inconvenients que s’han de valorar, i, a mĂ©s a mĂ©s, cal trobar valors pels seus parĂ metres. Aquestes no sĂłn tasques senzilles i, sovint, es fan servir algoritmes d’aprenentatge automĂ tic per guiar aquestes decisions. Estudiem tres d’aquestes tasques: donar valor als parĂ metres, avaluar trajectòries, i adaptar les polĂtiques. Els resultats demostren la utilitat dels mètodes proposats per avaluar trajectòries noves per tal de trobar valors apropiats pels parĂ metres dels algorismes sense fer servir dades reals directament. A mĂ©s a mĂ©s, proposem una estratègia per adaptar la polĂtica de cada agent a travĂ©s del reconeixement del context, millorant les simulacions
DARTER: Digital twins for Accessible Real Testing grounds for automotive Engineers and Researchers
<p>The DARTER dataset contains a sample (for now) of driving data (camera, LiDAR, IMU, steering wheel angle, etc.) without labels gathered at AstaZero's test track using the Chalmers ReVeRe lab's SnowFox test vehicle. DARTER stands for Digital twins for Accessible Real Testing grounds for automotive Engineers and Researchers. This research was possible thanks to the funds of a SAFER pre-study grant.</p>
<p> </p>
<p><strong>About DARTER:</strong></p>
<p>Verification and validation (V&V) of Intelligent Transport Solutions and their components in real traffic is difficult (costs, passers-by, etc.) and running tests under all possible conditions (weather, traffic, etc.) is impossible. For these reasons, controlled proving grounds and simulations are being used to safely increase the coverage of V&V. </p>
<p>The DARTER project, a SAFER pre-study, addresses two issues connected to these alternatives. On the one hand, the limited access to real proving grounds and their corresponding high-fidelity simulations of academic researchers, key in evaluating the benefits and social and environmental harms that technological advances can pose. On the other hand, the fidelity gap between the virtual and the real world, that prevents the usage of simulations at vehicle integration test level and needs to be understood and measured. </p>
<p>SAFER pre-studies https://www.saferresearch.com/content/safer-pre-studies </p>
Cross-Entropy method for Kullback-Leibler control in multi-agent systems
Supervisor: Dr. Vicenç Gómez Cerdà ; Co-Supervisor: Dr. Mario CeresaTreball fi de mà ster de: Master in Intelligent Interactive SystemsWe consider the problem of computing optimal control policies in large-scale multiagent
systems, for which the standard approach via the Bellman equation is intractable.
Our formulation is based on the Kullback-Leibler control framework, also
known as Linearly-Solvable Markov Decision Problems. In this setting, adaptive
importance sampling methods have been derived that, when combined with function
approximation, can be effective for high-dimensional systems. Our approach
iteratively learns an importance sampler from which the optimal control can be
extracted and requires to simulate and reweight agents’ trajectories in the world
multiple times. We illustrate our approach through a modified version of the popular
stag-hunt game; in this scenario, there is a multiplicity of optimal policies
depending on the “temperature” parameter of the environment. The system is built
inside Pandora, a multi-agent-based modeling framework and toolbox for parallelization,
freeing us from dealing with memory management when running multiple
simulations. By using function approximation and assuming some particular factorization
of the system dynamics, we are able to scale-up our method to problems
with M = 12 agents moving in two-dimensional grids of size N = 21Ă—21, improving
on existing methods that perform approximate inference on a temporal probabilistic
graphical model
A perceptually-validated metric for crowd trajectory quality evaluation
Simulating crowds requires controlling a very large number of trajectories and is usually performed using crowd motion algorithms for which appropriate parameter values need to be found. The study of the relation between parametric values for simulation techniques and the quality of the resulting trajectories has been studied either through perceptual experiments or by comparison with real crowd trajectories. In this paper, we integrate both strategies. A quality metric, QF, is proposed to abstract from reference data while capturing the most salient features that affect the perception of trajectory realism. QF weights and combines cost functions that are based on several individual, local and global properties of trajectories. These trajectory features are selected from the literature and from interviews with experts. To validate the capacity of QF to capture perceived trajectory quality, we conduct an online experiment that demonstrates the high agreement between the automatic quality score and non-expert users. To further demonstrate the usefulness of QF, we use it in a data-free parameter tuning application able to tune any parametric microscopic crowd simulation model that outputs independent trajectories for characters. The learnt parameters for the tuned crowd motion model maintain the influence of the reference data which was used to weight the terms of QF.With partial support of the EU funded project PRESENT, H2020-ICT-2018-3-856879. As Serra HĂşnter Fellow, Ricardo Marques acknowledges the support of the Serra HĂşnter Programme to this work