369,350 research outputs found
Test Maintenance for Machine Learning Systems: A Case Study in the Automotive Industry
Machine Learning (ML) systems have seen widespread use for automated decision making. Testing is essential to ensure the quality of these systems, especially safety-critical autonomous systems in the automotive domain. ML systems introduce new challenges with the potential to affect test maintenance, the process of updating test cases to match the evolving system. We conducted an exploratory case study in the automotive domain to identify factors that affect test maintenance for ML systems, as well as to make recommendations to improve the maintenance process. Based on interview and artifact analysis, we identified 14 factors affecting maintenance, including five especially relevant for ML systems—with the most important relating to non-determinism and large input spaces. We also proposed ten recommendations for improving test maintenance, including four targeting ML systems—in particular, emphasizing the use of test oracles tolerant to acceptable non-determinism. The study’s findings expand our knowledge of test maintenance for an emerging class of systems, benefiting the practitioners testing these systems
Conceptual biology, hypothesis discovery, and text mining: Swanson's legacy
Innovative biomedical librarians and information specialists who want to expand their roles as expert searchers need to know about profound changes in biology and parallel trends in text mining. In recent years, conceptual biology has emerged as a complement to empirical biology. This is partly in response to the availability of massive digital resources such as the network of databases for molecular biologists at the National Center for Biotechnology Information. Developments in text mining and hypothesis discovery systems based on the early work of Swanson, a mathematician and information scientist, are coincident with the emergence of conceptual biology. Very little has been written to introduce biomedical digital librarians to these new trends. In this paper, background for data and text mining, as well as for knowledge discovery in databases (KDD) and in text (KDT) is presented, then a brief review of Swanson's ideas, followed by a discussion of recent approaches to hypothesis discovery and testing. 'Testing' in the context of text mining involves partially automated methods for finding evidence in the literature to support hypothetical relationships. Concluding remarks follow regarding (a) the limits of current strategies for evaluation of hypothesis discovery systems and (b) the role of literature-based discovery in concert with empirical research. Report of an informatics-driven literature review for biomarkers of systemic lupus erythematosus is mentioned. Swanson's vision of the hidden value in the literature of science and, by extension, in biomedical digital databases, is still remarkably generative for information scientists, biologists, and physicians. © 2006Bekhuis; licensee BioMed Central Ltd
A Rapidly Deployable Classification System using Visual Data for the Application of Precision Weed Management
In this work we demonstrate a rapidly deployable weed classification system
that uses visual data to enable autonomous precision weeding without making
prior assumptions about which weed species are present in a given field.
Previous work in this area relies on having prior knowledge of the weed species
present in the field. This assumption cannot always hold true for every field,
and thus limits the use of weed classification systems based on this
assumption. In this work, we obviate this assumption and introduce a rapidly
deployable approach able to operate on any field without any weed species
assumptions prior to deployment. We present a three stage pipeline for the
implementation of our weed classification system consisting of initial field
surveillance, offline processing and selective labelling, and automated
precision weeding. The key characteristic of our approach is the combination of
plant clustering and selective labelling which is what enables our system to
operate without prior weed species knowledge. Testing using field data we are
able to label 12.3 times fewer images than traditional full labelling whilst
reducing classification accuracy by only 14%.Comment: 36 pages, 14 figures, published Computers and Electronics in
Agriculture Vol. 14
Design, Application and Evaluation of a Multi Agent System in the Logistics Domain
The increasing demand for flexibility of automated production systems also
affects the automated material flow systems (aMFS) they contain and demands
reconfigurable systems. However, the centralized control concept usually
applied in aMFS hinders an easy adaptation, as the entire control software has
to be re-tested, when manually changing sub-parts of the control. As adaption
and subsequent testing are a time-consuming task, concepts for splitting the
control from one centralized to multiple, decentralized control nodes are
required. Therefore, this paper presents a holistic agent-based control concept
for aMFS, whereby the system is divided into so-called automated material flow
modules (aMFM), each being controlled by a dedicated module agent. The concept
allows the reconfiguration of aMFS, consisting of heterogeneous, stationary
aMFM, during runtime. Furthermore, it includes aspects such as uniform agent
knowledge bases through metamodel-based development, a communication ontology
considering different information types and properties, strategic route
optimization in decentralized control architecture and a visualization concept
to make decisions of the module agents comprehensible to operators and
maintenance staff. The evaluation of the concept is performed by means of
material flow simulations as well as a prototypical implementation on a
lab-sized demonstrator.Comment: 13 pages, https://ieeexplore.ieee.org/abstract/document/9042827
Recommended from our members
The Application of Natural Language Processing and Automated Scoring in Second Language Assessment
Natural language processing (NLP) is an area of research that is used to investigate the application of natural language and is the foundation of machine translation, natural language text processing, natural language generation, multilingual and cross language information retrieval, speech recognition, parsing, and expert systems. To understand natural language in order to build or select appropriate algorithms for processing, three major issues are called into attention: humans’ thought processes, the meaning of linguistic input in context, and world knowledge. These considerations have led to the development of various types of NLP tools for lexical and morphological analysis, semantic and discourse analysis, as well as knowledge-based approaches (c.f., Chowdhury, 2003). After decades of evolution and advancement, the current stage of NLP, as Xi (2010) pointed out, has allowed language testing researchers to apply its techniques in developing automated scoring systems for the purpose of language learning and assessment
Assessing safety functionalities in the design and validation of driving automation
This paper aims to contribute to the comprehensive and systematic safety assessment of Automated Driving Systems (ADSs) by identifying unknown hazardous areas of operation. The current methodologies employed in this domain typically involve estimating the distributions of situational variables based on human-centered field test, crash databases, or expert knowledge of critical values. However, due to the lack of a-priori knowledge regarding the influential factors, their critical ranges, and their distributions, these approaches may not be entirely suitable for the assessment of emerging automated driving technologies. To deal with this challenging problem, here we propose a testing methodology incorporating realistic yet unobserved driving conditions, distinguished by numerous situational variables, so to encompass unknown unsafe conditions comprehensively. Our methodology utilizes stochastic simulation and uncertainty modeling techniques to account for the variability of realistic driving conditions and their impact on ADSs' performances. By doing so, we aim to identify unsafe operational regions and triggering conditions that can lead to hazardous behaviors, thus improving the development and safety of automated driving functions. For our purposes, the Latin Hypercube Sampling technique and the recently proposed PAWN density-based sensitivity analysis method are employed. We apply this methodology for the first time in the specific field of ADSs design and validation, using an exemplificative use case. We discuss and compare the results obtained from our approach with those obtained from a traditional approach
Guarantee of quality of electronic testirvaniya in the system of the additional vocational education
Today becomes urgent the creation of the information texnologies, based on the systems approach to the estimation of test tasks. For this it is expedient to develop test programs for the checking of the standard of knowledge of listeners. Undoubtedly, the automated evaluation of knowledge occupies here fundamental importance. We developed innovation approach to the form of the idea of the results of electronic testing. The special importance takes the form of the idea of the result of testing, which gives advantage during the guarantee of quality of educational services. It is shown that the test technologies increase the quality of instruction in the system of additional vocational educationСегодня становится актуальным создание информационных технологий, основанных на системном подходе к оценке тестовых заданий. Для этого целесообразно разработать тест-программы для контроля уровня знаний слушателей. Несомненно, автоматизированная оценка знаний занимает здесь первостепенное значение. Мы разработали инновационный подход к форме представления результатов электронного тестирования. Особое значение имеет вид представления результата тестирования, которое дает преимущество при обеспечении качества образовательных услуг. Показано, что тестовые технологии повышают качество обучения в системе дополнительного профессионального образовани
MODELS OF INTEGRATION OF INFORMATION SYSTEMS IN HIGHER EDUCATION INSTITUTIONS
At present a lot of automated systems are developing and implementing to support the educational and research processes in the universities. Often these systems duplicate some functions, databases, and also there are problems of compatibility of these systems. The most common educational systems are systems for creating electronic libraries, access to scientific and educational information, a program for detecting plagiarism, testing knowledge, etc. In this article, models and solutions for the integration of such educational automated systems as the information library system (ILS) and the anti-plagiarism system are examined. Integration of systems is based on the compatibility of databases, if more precisely in the metadata of different information models. At the same time, Cloud technologies are used - data processing technology, in which computer resources are provided to the user of the integrated system as an online service. ILS creates e-library of graduation papers and dissertations on the main server. During the creation of the electronic catalog, the communication format MARC21 is used. The database development is distributed for each department. The subsystem of anti-plagiarism analyzes the full-text database for the similarity of texts (dissertations, diploma works and others). Also it identifies the percentage of coincidence, creates the table of statistical information on the coincidence of tests for each author and division, indicating similar fields. The integrated system was developed and tested at the Tashkent University of Information Technologies to work in the corporate mode of various departments (faculties, departments, TUIT branches)
Testing automated driving systems to calibrate drivers’ trust
Automated Driving Systems (ADSs) offer many potential benefits like improved safety, reduced traffic congestion and lower emissions. However, such benefits can only be realised if drivers trust and make use of such systems. The two challenges explored in this thesis are: 1) How to increase trust in ADSs? 2) How to identify the test scenarios to establish the true capabilities and limitations of ADSs?
Firstly, drivers’ trust needs to be calibrated to the “appropriate” level to prevent misuse (due to over trust) or disuse (due to under trust) of the system. In this research, a method to calibrate drivers’ trust to the appropriate level has been created. This method involves providing knowledge of the capabilities and limitations of the ADSs to the driver.
However, there is a need to establish the capabilities and limitations of the ADSs which form the knowledge to be imparted to the driver. Therefore, the next research contribution lies in the development of a novel method to establish the knowledge of capabilities and limitations of ADSs (used to calibrate trust) in a reliable manner. This knowledge can be created by testing ADSs. However, in literature, an unanswered research question remains: How to identify test scenarios which highlight the limitations of ADSs? In order to identify such test scenarios, a novel hazard based testing approach to establish the capabilities and limitations of ADSs is presented by extending STPA (a hazard identification method) to create test scenarios. To ensure reliability of the hazard classification (and of the knowledge), the author created a novel objective approach for risk classification by creating a rule-set for risk ratings.
The contribution of this research lies in developing a method to increase trust in ADSs by creating reliable knowledge using hazard based testing approach which identifies how an ADS can fail
Simulating Operational Concepts for Autonomous Robotic Space Exploration Systems: A Framework for Early Design Validation
During mission design, the concept of operations (ConOps) describes how the system operates during various life cycle phases to meet stakeholder expectations. ConOps is sometimes declined in a simple evaluation of the power consumption or data generation per mode. Different operational timelines are typically developed based on expert knowledge. This approach is robust when designing an automated system or a system with a low level of autonomy. However, when studying highly autonomous systems, designers may be interested in understanding how the system would react in an operational scenario when provided with knowledge about its actions and operational environment. These considerations can help verify and validate the proposed ConOps architecture, highlight shortcomings in both physical and functional design, and help better formulate detailed requirements. Hence, this study aims to provide a framework for the simulation and validation of operational scenarios for autonomous robotic space exploration systems during the preliminary design phases. This study extends current efforts in autonomy technology for planetary systems by focusing on testing their operability and assessing their performances in different scenarios early in the design process. The framework uses Model-Based Systems Engineering (MBSE) as the knowledge base for the studied system and its operations. It then leverages a Markov Decision Process (MDP) to simulate a set of system operations in a relevant scenario. It then outputs a feasible plan with the associated variation of a set of considered resources as step functions. This method was applied to simulate the operations of a small rover exploring an unknown environment to observe and sample a set of targets
- …