10 research outputs found
Log-based software monitoring: a systematic mapping study
Modern software development and operations rely on monitoring to understand
how systems behave in production. The data provided by application logs and
runtime environment are essential to detect and diagnose undesired behavior and
improve system reliability. However, despite the rich ecosystem around
industry-ready log solutions, monitoring complex systems and getting insights
from log data remains a challenge.
Researchers and practitioners have been actively working to address several
challenges related to logs, e.g., how to effectively provide better tooling
support for logging decisions to developers, how to effectively process and
store log data, and how to extract insights from log data. A holistic view of
the research effort on logging practices and automated log analysis is key to
provide directions and disseminate the state-of-the-art for technology
transfer.
In this paper, we study 108 papers (72 research track papers, 24 journals,
and 12 industry track papers) from different communities (e.g., machine
learning, software engineering, and systems) and structure the research field
in light of the life-cycle of log data.
Our analysis shows that (1) logging is challenging not only in open-source
projects but also in industry, (2) machine learning is a promising approach to
enable a contextual analysis of source code for log recommendation but further
investigation is required to assess the usability of those tools in practice,
(3) few studies approached efficient persistence of log data, and (4) there are
open opportunities to analyze application logs and to evaluate state-of-the-art
log analysis techniques in a DevOps context
Introductory Computer Forensics
INTERPOL (International Police) built cybercrime programs to keep up with emerging cyber threats, and aims to coordinate and assist international operations for ?ghting crimes involving computers. Although signi?cant international efforts are being made in dealing with cybercrime and cyber-terrorism, ?nding effective, cooperative, and collaborative ways to deal with complicated cases that span multiple jurisdictions has proven dif?cult in practic
Improving Salience Retention and Identification in the Automated Filtering of Event Log Messages
Event log messages are currently the only genuine interface through which computer systems
administrators can effectively monitor their systems and assemble a mental perception
of system state. The popularisation of the Internet and the accompanying meteoric
growth of business-critical systems has resulted in an overwhelming volume of event log
messages, channeled through mechanisms whose designers could not have envisaged the
scale of the problem. Messages regarding intrusion detection, hardware status, operating
system status changes, database tablespaces, and so on, are being produced at the rate
of many gigabytes per day for a significant computing environment.
Filtering technologies have not been able to keep up. Most messages go unnoticed; no
filtering whatsoever is performed on them, at least in part due to the difficulty of implementing
and maintaining an effective filtering solution. The most commonly-deployed
filtering alternatives rely on regular expressions to match pre-defi ned strings, with 100%
accuracy, which can then become ineffective as the code base for the software producing
the messages 'drifts' away from those strings. The exactness requirement means all possible
failure scenarios must be accurately anticipated and their events catered for with
regular expressions, in order to make full use of this technique.
Alternatives to regular expressions remain largely academic. Data mining, automated
corpus construction, and neural networks, to name the highest-profi le ones, only produce
probabilistic results and are either difficult or impossible to alter in any deterministic way.
Policies are therefore not supported under these alternatives.
This thesis explores a new architecture which utilises rich metadata in order to avoid the
burden of message interpretation. The metadata itself is based on an intention to improve
end-to-end communication and reduce ambiguity. A simple yet effective filtering scheme
is also presented which fi lters log messages through a short and easily-customisable set
of rules. With such an architecture, it is envisaged that systems administrators could
signi ficantly improve their awareness of their systems while avoiding many of the false-positives
and -negatives which plague today's fi ltering solutions
Cross-domain Recommendations based on semantically-enhanced User Web Behavior
Information seeking in the Web can be facilitated by recommender systems that guide the users in a personalized manner to relevant resources in the large space of the possible options in the Web. This work investigates how to model people\u27s Web behavior at multiple sites and learn to predict future preferences, in order to generate relevant cross-domain recommendations. This thesis contributes with novel techniques for building cross-domain recommender systems in an open Web setting
The Meaning of Logs
While logging events is becoming increasingly common in computing, in communication and in collaborative environments, log systems need to satisfy increasingly challenging (if not conflicting) requirements. In this paper we propose a high-level framework for modeling log systems, and reasoning about them. This framework allows one to give a high-level representation of a log system and to check whether it satisfies given audit and privacy properties which in turn can be expressed in standard logic. In particular, the framework can be used for comparing and assessing log systems. We validate our proposal by formalizing a number of standard log properties and by using it to review a number of existing systems. Despite the growing pervasiveness of log systems, we believe this is the first framework of this sort
The meaning of logs
While logging events is becoming increasingly common in computing, in communication and in collaborative environments, log systems need to satisfy increasingly challenging (if not conflicting) requirements. In this paper we propose a high-level framework for modeling log systems, and reasoning about them. This framework allows one to give a high-level representation of a log system and to check whether it satisfies given audit and privacy properties which in turn can be expressed in standard logic. In particular, the framework can be used for comparing and assessing log systems. We validate our proposal by formalizing a number of standard log properties and by using it to review a number of existing systems. Despite the growing pervasiveness of log systems, we believe this is the first framework of this sort