EvLog: Evolving Log Analyzer for Anomalous Logs Identification

Software logs record system activities, aiding maintainers in identifying the underlying causes for failures and enabling prompt mitigation actions. However, maintainers need to inspect a large volume of daily logs to identify the anomalous logs that reveal failure details for further diagnosis. Thus, how to automatically distinguish these anomalous logs from normal logs becomes a critical problem. Existing approaches alleviate the burden on software maintainers, but they are built upon an improper yet critical assumption: logging statements in the software remain unchanged. While software keeps evolving, our empirical study finds that evolving software brings three challenges: log parsing errors, evolving log events, and unstable log sequences. In this paper, we propose a novel unsupervised approach named Evolving Log analyzer (EvLog) to mitigate these challenges. We first build a multi-level representation extractor to process logs without parsing to prevent errors from the parser. The multi-level representations preserve the essential semantics of logs while leaving out insignificant changes in evolving events. EvLog then implements an anomaly discriminator with an attention mechanism to identify the anomalous logs and avoid the issue brought by the unstable sequence. EvLog has shown effectiveness in two real-world system evolution log datasets with an average F1 score of 0.955 and 0.847 in the intra-version setting and inter-version setting, respectively, which outperforms other state-of-the-art approaches by a wide margin. To our best knowledge, this is the first study on tackling anomalous logs over software evolution. We believe our work sheds new light on the impact of software evolution with the corresponding solutions for the log analysis community

Performance Issue Identification in Cloud Systems with Relational-Temporal Anomaly Detection

Performance issues permeate large-scale cloud service systems, which can lead to huge revenue losses. To ensure reliable performance, it's essential to accurately identify and localize these issues using service monitoring metrics. Given the complexity and scale of modern cloud systems, this task can be challenging and may require extensive expertise and resources beyond the capacity of individual humans. Some existing methods tackle this problem by analyzing each metric independently to detect anomalies. However, this could incur overwhelming alert storms that are difficult for engineers to diagnose manually. To pursue better performance, not only the temporal patterns of metrics but also the correlation between metrics (i.e., relational patterns) should be considered, which can be formulated as a multivariate metrics anomaly detection problem. However, most of the studies fall short of extracting these two types of features explicitly. Moreover, there exist some unlabeled anomalies mixed in the training data, which may hinder the detection performance. To address these limitations, we propose the Relational- Temporal Anomaly Detection Model (RTAnomaly) that combines the relational and temporal information of metrics. RTAnomaly employs a graph attention layer to learn the dependencies among metrics, which will further help pinpoint the anomalous metrics that may cause the anomaly effectively. In addition, we exploit the concept of positive unlabeled learning to address the issue of potential anomalies in the training data. To evaluate our method, we conduct experiments on a public dataset and two industrial datasets. RTAnomaly outperforms all the baseline models by achieving an average F1 score of 0.929 and Hit@3 of 0.920, demonstrating its superiority

Prism: Revealing Hidden Functional Clusters from Massive Instances in Cloud Systems

Ensuring the reliability of cloud systems is critical for both cloud vendors and customers. Cloud systems often rely on virtualization techniques to create instances of hardware resources, such as virtual machines. However, virtualization hinders the observability of cloud systems, making it challenging to diagnose platform-level issues. To improve system observability, we propose to infer functional clusters of instances, i.e., groups of instances having similar functionalities. We first conduct a pilot study on a large-scale cloud system, i.e., Huawei Cloud, demonstrating that instances having similar functionalities share similar communication and resource usage patterns. Motivated by these findings, we formulate the identification of functional clusters as a clustering problem and propose a non-intrusive solution called Prism. Prism adopts a coarse-to-fine clustering strategy. It first partitions instances into coarse-grained chunks based on communication patterns. Within each chunk, Prism further groups instances with similar resource usage patterns to produce fine-grained functional clusters. Such a design reduces noises in the data and allows Prism to process massive instances efficiently. We evaluate Prism on two datasets collected from the real-world production environment of Huawei Cloud. Our experiments show that Prism achieves a v-measure of ~0.95, surpassing existing state-of-the-art solutions. Additionally, we illustrate the integration of Prism within monitoring systems for enhanced cloud reliability through two real-world use cases.Comment: The paper was accepted by the 38th IEEE/ACM International Conference on Automated Software Engineering (ASE 2023

A Large-scale Benchmark for Log Parsing

Log data is pivotal in activities like anomaly detection and failure diagnosis in the automated maintenance of software systems. Due to their unstructured format, log parsing is often required to transform them into a structured format for automated analysis. A variety of log parsers exist, making it vital to benchmark these tools to comprehend their features and performance. However, existing datasets for log parsing are limited in terms of scale and representativeness, posing challenges for studies that aim to evaluate or develop log parsers. This problem becomes more pronounced when these parsers are evaluated for production use. To address these issues, we introduce a new collection of large-scale annotated log datasets, named LogPub, which more accurately mirrors log data observed in real-world software systems. LogPub comprises 14 datasets, each averaging 3.6 million log lines. Utilizing LogPub, we re-evaluate 15 log parsers in a more rigorous and practical setting. We also propose a new evaluation metric to lessen the sensitivity of current metrics to imbalanced data distribution. Furthermore, we are the first to scrutinize the detailed performance of log parsers on logs that represent rare system events and offer comprehensive information for system troubleshooting. Parsing such logs accurately is vital yet challenging. We believe that our work could shed light on the design and evaluation of log parsers in more realistic settings, thereby facilitating their implementation in production systems

BnERF114.A1, a Rapeseed Gene Encoding APETALA2/ETHYLENE RESPONSE FACTOR, Regulates Plant Architecture through Auxin Accumulation in the Apex in Arabidopsis

Plant architecture is crucial for rapeseed breeding. Here, we demonstrate the involvement of BnERF114.A1, a transcription factor for ETHYLENE RESPONSE FACTOR (ERF), in the regulation of plant architecture in Brassica napus. BnERF114.A1 is a member of the ERF family group X-a, encoding a putative 252-amino acid (aa) protein, which harbours the AP2/ERF domain and the conserved CMX-1 motif. BnERF114.A1 is localised to the nucleus and presents transcriptional activity, with the functional region located at 142–252 aa of the C-terminus. GUS staining revealed high BnERF114.A1 expression in leaf primordia, shoot apical meristem, leaf marginal meristem, and reproductive organs. Ectopic BnERF114.A1 expression in Arabidopsis reduced plant height, increased branch and silique number per plant, and improved seed yield per plant. Furthermore, in Arabidopsis, BnERF114.A1 overexpression inhibited indole-3-acetic acid (IAA) efflux, thus promoting auxin accumulation in the apex and arresting apical dominance. Therefore, BnERF114.A1 probably plays an important role in auxin-dependent plant architecture regulation

Adaptive Performance Anomaly Detection for Online Service Systems via Pattern Sketching

To ensure the performance of online service systems, their status is closely monitored with various software and system metrics. Performance anomalies represent the performance degradation issues (e.g., slow response) of the service systems. When performing anomaly detection over the metrics, existing methods often lack the merit of interpretability, which is vital for engineers and analysts to take remediation actions. Moreover, they are unable to effectively accommodate the ever-changing services in an online fashion. To address these limitations, in this paper, we propose ADSketch, an interpretable and adaptive performance anomaly detection approach based on pattern sketching. ADSketch achieves interpretability by identifying groups of anomalous metric patterns, which represent particular types of performance issues. The underlying issues can then be immediately recognized if similar patterns emerge again. In addition, an adaptive learning algorithm is designed to embrace unprecedented patterns induced by service updates or user behavior changes. The proposed approach is evaluated with public data as well as industrial data collected from a representative online service system in Huawei Cloud. The experimental results show that ADSketch outperforms state-of-the-art approaches by a significant margin, and demonstrate the effectiveness of the online algorithm in new pattern discovery. Furthermore, our approach has been successfully deployed in industrial practice.Comment: Accepted by The 44th International Conference on Software Engineering (ICSE 2022

BZR1 Physically Interacts with SPL9 to Regulate the Vegetative Phase Change and Cell Elongation in Arabidopsis

As sessile organisms, the precise development phase transitions are very important for the success of plant adaptability, survival and reproduction. The transition from juvenile to the adult phase—referred to as the vegetative phase change—is significantly influenced by numbers of endogenous and environmental signals. Here, we showed that brassinosteroid (BR), a major growth-promoting steroid hormone, positively regulates the vegetative phase change in Arabidopsis thaliana. The BR-deficient mutant det2-1 and BR-insensitive mutant bri1-301 displayed the increased ratio of leaf width to length and reduced blade base angle. The plant specific transcription factors SQUAMOSA PROMOTER BINDING PROTEIN-LIKE (SPL) are key masters for the vegetative phase transition in plants. The expression levels of SPL9, SPL10 and SPL15 were significantly induced by BR treatment, but reduced in bri1-116 mutant compared to wild-type plants. The gain-of-function pSPL9:rSPL9 transgenic plants displayed the BR hypersensitivity on hypocotyl elongation and partially suppressed the delayed vegetative phase change of det2-1 and bri1-301. Furthermore, we showed that BRASSINAZOLE-RESISTANT 1 (BZR1), the master transcription factor of BR signaling pathway, interacted with SPL9 to cooperatively regulate the expression of downstream genes. Our findings reveal an important role for BRs in promoting vegetative phase transition through regulating the activity of SPL9 at transcriptional and post-transcriptional levels

<i>BnERF114.A1</i>, a Rapeseed Gene Encoding APETALA2/ETHYLENE RESPONSE FACTOR, Regulates Plant Architecture through Auxin Accumulation in the Apex in <i>Arabidopsis</i>

Plant architecture is crucial for rapeseed breeding. Here, we demonstrate the involvement of BnERF114.A1, a transcription factor for ETHYLENE RESPONSE FACTOR (ERF), in the regulation of plant architecture in Brassica napus. BnERF114.A1 is a member of the ERF family group X-a, encoding a putative 252-amino acid (aa) protein, which harbours the AP2/ERF domain and the conserved CMX-1 motif. BnERF114.A1 is localised to the nucleus and presents transcriptional activity, with the functional region located at 142–252 aa of the C-terminus. GUS staining revealed high BnERF114.A1 expression in leaf primordia, shoot apical meristem, leaf marginal meristem, and reproductive organs. Ectopic BnERF114.A1 expression in Arabidopsis reduced plant height, increased branch and silique number per plant, and improved seed yield per plant. Furthermore, in Arabidopsis, BnERF114.A1 overexpression inhibited indole-3-acetic acid (IAA) efflux, thus promoting auxin accumulation in the apex and arresting apical dominance. Therefore, BnERF114.A1 probably plays an important role in auxin-dependent plant architecture regulation