How Usability Defects Defer from Non-Usability Defects? : A Case Study on Open Source Projects

Usability is one of the software qualities attributes that is subjective and often considered as a less critical defect to be fixed. One of the reasons was due to the vague defect descriptions that could not convince developers about the validity of usability issues. Producing a comprehensive usability defect description can be a challenging task, especially in reporting relevant and important information. Prior research in improving defect report comprehension has often focused on defects in general or studied various aspects of software quality improvement such as triaging defect reports, metrics and predictions, automatic defect detection and fixing.  In this paper, we studied 2241 usability and non-usability defects from three open-source projects - Mozilla Thunderbird, Firefox for Android, and Eclipse Platform. We examined the presence of eight defect attributes - steps to reproduce, impact, software context, expected output, actual output, assume cause, solution proposal, and supplementary information, and used various statistical tests to answer the research questions. In general, we found that usability defects are resolved slower than non-usability defects, even for non-usability defect reports that have less information. In terms of defect report content, usability defects often contain output details and software context while non-usability defects are preferably explained using supplementary information, such as stack traces and error logs. Our research findings extend the body of knowledge of software defect reporting, especially in understanding the characteristics of usability defects. The promising results also may be valuable to improve software development practitioners' practice

MLGuard: Defend Your Machine Learning Model!

Machine Learning (ML) is used in critical highly regulated and high-stakes fields such as finance, medicine, and transportation. The correctness of these ML applications is important for human safety and economic benefit. Progress has been made on improving ML testing and monitoring of ML. However, these approaches do not provide i) pre/post conditions to handle uncertainty, ii) defining corrective actions based on probabilistic outcomes, or iii) continual verification during system operation. In this paper, we propose MLGuard, a new approach to specify contracts for ML applications. Our approach consists of a) an ML contract specification defining pre/post conditions, invariants, and altering behaviours, b) generated validation models to determine the probability of contract violation, and c) an ML wrapper generator to enforce the contract and respond to violations. Our work is intended to provide the overarching framework required for building ML applications and monitoring their safety.Comment: Accepted in SE4SafeML'2

Technical Report: Anomaly Detection for a Critical Industrial System using Context, Logs and Metrics

Recent advances in contextual anomaly detection attempt to combine resource metrics and event logs to un- cover unexpected system behaviors and malfunctions at run- time. These techniques are highly relevant for critical software systems, where monitoring is often mandated by international standards and guidelines. In this technical report, we analyze the effectiveness of a metrics-logs contextual anomaly detection technique in a middleware for Air Traffic Control systems. Our study addresses the challenges of applying such techniques to a new case study with a dense volume of logs, and finer monitoring sampling rate. We propose an automated abstraction approach to infer system activities from dense logs and use regression analysis to infer the anomaly detector. We observed that the detection accuracy is impacted by abrupt changes in resource metrics or when anomalies are asymptomatic in both resource metrics and event logs. Guided by our experimental results, we propose and evaluate several actionable improvements, which include a change detection algorithm and the use of time windows on contextual anomaly detection. This technical report accompanies the paper “Contextual Anomaly Detection for a Critical Industrial System based on Logs and Metrics” [1] and provides further details on the analysis method, case study and experimental results

P-gram: positional N-gram for the clustering of machine-generated messages

An IT system generates messages for other systems or users to consume, through direct interaction or as system logs. Automatically identifying the types of these machine-generated messages has many applications, such as intrusion detection and system behavior discovery. Among various heuristic methods for automatically identifying message types, the clustering methods based on keyword extraction have been quite effective. However, these methods still suffer from keyword misidentification problems, i.e., some keyword occurrences are wrongly identified as payload and some strings in the payload are wrongly identified as keyword occurrences, leading to the misidentification of the message types. In this paper, we propose a new machine language processing (MLP) approach, called P-gram, specifically designed for identifying keywords in, and subsequently clustering, machine-generated messages. First, we introduce a novel concept and technique, positional n-gram, for message keywords extraction. By associating the position as meta-data with each n-gram, we can more accurately discern which n-grams are keywords of a message and which n-grams are parts of the payload information. Then, the positional keywords are used as features to cluster the messages, and an entropy-based positional weighting method is devised to measure the importance or weight of the positional keywords to each message. Finally, a general centroid clustering method, K-Medoids, is used to leverage the importance of the keywords and cluster messages into groups reflecting their types. We evaluate our method on a range of machine-generated (text and binary) messages from the real-world systems and show that our method achieves higher accuracy than the current state-of-the-art tools

The F4/AS01B HIV-1 Vaccine Candidate Is Safe and Immunogenic, But Does Not Show Viral Efficacy in Antiretroviral Therapy-Naive, HIV-1-Infected Adults: A Randomized Controlled Trial

The impact of the investigational human immunodeficiency virus type 1 (HIV-1) F4/AS01(B) vaccine on HIV-1 viral load (VL) was evaluated in antiretroviral therapy (ART)-naive HIV-1 infected adults.This phase IIb, observer-blind study (NCT01218113), included ART-naive HIV-1 infected adults aged 18 to 55 years. Participants were randomized to receive 2 (F4/AS01(B)_2 group, N=64) or 3 (F4/AS01(B)_3 group, N=62) doses of F4/AS01(B) or placebo (control group, N=64) at weeks 0, 4, and 28. Efficacy (HIV-1 VL, CD4(+) T-cell count, ART initiation, and HIV-related clinical events), safety, and immunogenicity (antibody and T-cell responses) were evaluated during 48 weeks.At week 48, based on a mixed model, no statistically significant difference in HIV-1 VL change from baseline was demonstrated between F4/AS01(B)_2 and control group (0.073 log(10)copies/mL [97.5% confidence interval (CI): -0.088; 0.235]), or F4/AS01(B)_3 and control group (-0.096 log(10)copies/mL [97.5% CI: -0.257; 0.065]). No differences between groups were observed in HIV-1 VL change, CD4(+) T-cell count, ART initiation, or HIV-related clinical events at intermediate timepoints. Among F4/AS01(B) recipients, the most frequent solicited symptoms were pain at injection site (252/300 doses), fatigue (137/300 doses), myalgia (105/300 doses), and headache (90/300 doses). Twelve serious adverse events were reported in 6 participants; 1 was considered vaccine-related (F4/AS01(B)_2 group: angioedema). F4/AS01(B) induced polyfunctional F4-specific CD4(+) T-cells, but had no significant impact on F4-specific CD8(+) T-cell and anti-F4 antibody levels.F4/AS01(B) had a clinically acceptable safety profile, induced F4-specific CD4(+) T-cell responses, but did not reduce HIV-1 VL, impact CD4(+) T-cells count, delay ART initiation, or prevent HIV-1 related clinical events

GeneFarm, structural and functional annotation of Arabidopsis gene and protein families by a network of experts

Genomic projects heavily depend on genome annotations and are limited by the current deficiencies in the published predictions of gene structure and function. It follows that, improved annotation will allow better data mining of genomes, and more secure planning and design of experiments. The purpose of the GeneFarm project is to obtain homogeneous, reliable, documented and traceable annotations for Arabidopsis nuclear genes and gene products, and to enter them into an added-value database. This re-annotation project is being performed exhaustively on every member of each gene family. Performing a family-wide annotation makes the task easier and more efficient than a gene-by-gene approach since many features obtained for one gene can be extrapolated to some or all the other genes of a family. A complete annotation procedure based on the most efficient prediction tools available is being used by 16 partner laboratories, each contributing annotated families from its field of expertise. A database, named GeneFarm, and an associated user-friendly interface to query the annotations have been developed. More than 3000 genes distributed over 300 families have been annotated and are available at http://genoplante-info.infobiogen.fr/Genefarm/. Furthermore, collaboration with the Swiss Institute of Bioinformatics is underway to integrate the GeneFarm data into the protein knowledgebase Swiss-Prot

GeneFarm, structural and functional annotation of Arabidopsis gene and protein families by a network of experts

Genomic projects heavily depend on genome annotations and are limited by the current deficiencies in the published predictions of gene structure and function. It follows that, improved annotation will allow better data mining of genomes, and more secure planning and design of experiments. The purpose of the GeneFarm project is to obtain homogeneous, reliable, documented and traceable annotations for Arabidopsis nuclear genes and gene products, and to enter them into an added-value database. This re-annotation project is being performed exhaustively on every member of each gene family. Performing a family-wide annotation makes the task easier and more efficient than a gene-by-gene approach since many features obtained for one gene can be extrapolated to some or all the other genes of a family. A complete annotation procedure based on the most efficient prediction tools available is being used by 16 partner laboratories, each contributing annotated families from its field of expertise. A database, named GeneFarm, and an associated user-friendly interface to query the annotations have been developed. More than 3000 genes distributed over 300 families have been annotated and are available at http://genoplante-info.infobiogen.fr/Genefarm/. Furthermore, collaboration with the Swiss Institute of Bioinformatics is underway to integrate the GeneFarm data into the protein knowledgebase Swiss-Pro