1,270 research outputs found
Recommender System Based on Process Mining
Automation of repetitive tasks can be achieved with Robotic Process Automation (RPA) using scripts that encode fine-grained interactions with software applications on desktops and the web. Automating these processes can be achieved through several applications. It is possible for users to record desktop activity, including metadata, with these tools. The very fine-grained steps in the processes contain details about very small steps that the user takes. Several steps are involved in this process, including clicking on buttons, typing text, selecting the text, and changing the focus. Automating these processes requires connectors connecting them to the appropriate applications. Currently, users choose these connectors manually rather than automatically being linked to processes.
In this thesis, we propose a method for recommending the top-k suitable connectors based on event logs for each process. This method indicates that we can use process discovery, create the process models of the train processes with identified connectors, and calculate
the conformance checking between the process models and test event logs (unknown connectors). Then we select top-k maximum values of the conformance checking results and observe that we have the suitable connector with 80% accuracy among the top-3 recommended connectors. This solution can be configurable by changing the parameters and the methods of process discovery and conformance checking.Automation of repetitive tasks can be achieved with Robotic Process Automation (RPA) using scripts that encode fine-grained interactions with software applications on desktops and the web. Automating these processes can be achieved through several applications. It is possible for users to record desktop activity, including metadata, with these tools. The very fine-grained steps in the processes contain details about very small steps that the user takes. Several steps are involved in this process, including clicking on buttons, typing text, selecting the text, and changing the focus. Automating these processes requires connectors connecting them to the appropriate applications. Currently, users choose these connectors manually rather than automatically being linked to processes.
In this thesis, we propose a method for recommending the top-k suitable connectors based on event logs for each process. This method indicates that we can use process discovery, create the process models of the train processes with identified connectors, and calculate the conformance checking between the process models and test event logs (unknown connectors). Then we select top-k maximum values of the conformance checking results and observe that we have the suitable connector with 80% accuracy among the top-3 recommended connectors. This solution can be configurable by changing the parameters and the methods of process discovery and conformance checking
Towards Automated Circuit Discovery for Mechanistic Interpretability
Recent work in mechanistic interpretability has reverse-engineered nontrivial
behaviors of transformer models. These contributions required considerable
effort and researcher intuition, which makes it difficult to apply the same
methods to understand the complex behavior that current models display. At
their core however, the workflow for these discoveries is surprisingly similar.
Researchers create a data set and metric that elicit the desired model
behavior, subdivide the network into appropriate abstract units, replace
activations of those units to identify which are involved in the behavior, and
then interpret the functions that these units implement. By varying the data
set, metric, and units under investigation, researchers can understand the
functionality of each neural network region and the circuits they compose. This
work proposes a novel algorithm, Automatic Circuit DisCovery (ACDC), to
automate the identification of the important units in the network. Given a
model's computational graph, ACDC finds subgraphs that explain a behavior of
the model. ACDC was able to reproduce a previously identified circuit for
Python docstrings in a small transformer, identifying 6/7 important attention
heads that compose up to 3 layers deep, while including 91% fewer the
connections
Computer-aided HAZOP of batch processes
The modern batch chemical processing plants have a tendency of increasing
technological complexity and flexibility which make it difficult to control the
occurrence of accidents. Social and legal pressures have increased the demands
for verifying the safety of chemical plants during their design and operation.
Complete identification and accurate assessment of the hazard potential in the
early design stages is therefore very important so that preventative or protective
measures can be integrated into future design without adversely affecting
processing and control complexity or capital and operational costs. Hazard and
Operability Study (HAZOP) is a method of systematically identifying every
conceivable process deviation, its abnormal causes and adverse hazardous
consequences in the chemical plants. [Continues.
Post Hoc Explanations of Language Models Can Improve Language Models
Large Language Models (LLMs) have demonstrated remarkable capabilities in
performing complex tasks. Moreover, recent research has shown that
incorporating human-annotated rationales (e.g., Chain-of- Thought prompting)
during in-context learning can significantly enhance the performance of these
models, particularly on tasks that require reasoning capabilities. However,
incorporating such rationales poses challenges in terms of scalability as this
requires a high degree of human involvement. In this work, we present a novel
framework, Amplifying Model Performance by Leveraging In-Context Learning with
Post Hoc Explanations (AMPLIFY), which addresses the aforementioned challenges
by automating the process of rationale generation. To this end, we leverage
post hoc explanation methods which output attribution scores (explanations)
capturing the influence of each of the input features on model predictions.
More specifically, we construct automated natural language rationales that
embed insights from post hoc explanations to provide corrective signals to
LLMs. Extensive experimentation with real-world datasets demonstrates that our
framework, AMPLIFY, leads to prediction accuracy improvements of about 10-25%
over a wide range of tasks, including those where prior approaches which rely
on human-annotated rationales such as Chain-of-Thought prompting fall short.
Our work makes one of the first attempts at highlighting the potential of post
hoc explanations as valuable tools for enhancing the effectiveness of LLMs.
Furthermore, we conduct additional empirical analyses and ablation studies to
demonstrate the impact of each of the components of AMPLIFY, which, in turn,
lead to critical insights for refining in-context learning
Circuit Component Reuse Across Tasks in Transformer Language Models
Recent work in mechanistic interpretability has shown that behaviors in
language models can be successfully reverse-engineered through circuit
analysis. A common criticism, however, is that each circuit is task-specific,
and thus such analysis cannot contribute to understanding the models at a
higher level. In this work, we present evidence that insights (both low-level
findings about specific heads and higher-level findings about general
algorithms) can indeed generalize across tasks. Specifically, we study the
circuit discovered in Wang et al. (2022) for the Indirect Object Identification
(IOI) task and 1.) show that it reproduces on a larger GPT2 model, and 2.) that
it is mostly reused to solve a seemingly different task: Colored Objects
(Ippolito & Callison-Burch, 2023). We provide evidence that the process
underlying both tasks is functionally very similar, and contains about a 78%
overlap in in-circuit attention heads. We further present a proof-of-concept
intervention experiment, in which we adjust four attention heads in middle
layers in order to 'repair' the Colored Objects circuit and make it behave like
the IOI circuit. In doing so, we boost accuracy from 49.6% to 93.7% on the
Colored Objects task and explain most sources of error. The intervention
affects downstream attention heads in specific ways predicted by their
interactions in the IOI circuit, indicating that this subcircuit behavior is
invariant to the different task inputs. Overall, our results provide evidence
that it may yet be possible to explain large language models' behavior in terms
of a relatively small number of interpretable task-general algorithmic building
blocks and computational components
Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla
\emph{Circuit analysis} is a promising technique for understanding the
internal mechanisms of language models. However, existing analyses are done in
small models far from the state of the art. To address this, we present a case
study of circuit analysis in the 70B Chinchilla model, aiming to test the
scalability of circuit analysis. In particular, we study multiple-choice
question answering, and investigate Chinchilla's capability to identify the
correct answer \emph{label} given knowledge of the correct answer \emph{text}.
We find that the existing techniques of logit attribution, attention pattern
visualization, and activation patching naturally scale to Chinchilla, allowing
us to identify and categorize a small set of `output nodes' (attention heads
and MLPs).
We further study the `correct letter' category of attention heads aiming to
understand the semantics of their features, with mixed results. For normal
multiple-choice question answers, we significantly compress the query, key and
value subspaces of the head without loss of performance when operating on the
answer labels for multiple-choice questions, and we show that the query and key
subspaces represent an `Nth item in an enumeration' feature to at least some
extent. However, when we attempt to use this explanation to understand the
heads' behaviour on a more general distribution including randomized answer
labels, we find that it is only a partial explanation, suggesting there is more
to learn about the operation of `correct letter' heads on multiple choice
question answering
Enabling Auditing and Intrusion Detection of Proprietary Controller Area Networks
The goal of this dissertation is to provide automated methods for security researchers to overcome ‘security through obscurity’ used by manufacturers of proprietary Industrial Control Systems (ICS). `White hat\u27 security analysts waste significant time reverse engineering these systems\u27 opaque network configurations instead of performing meaningful security auditing tasks. Automating the process of documenting proprietary protocol configurations is intended to improve independent security auditing of ICS networks. The major contributions of this dissertation are a novel approach for unsupervised lexical analysis of binary network data flows and analysis of the time series data extracted as a result. We demonstrate the utility of these methods using Controller Area Network (CAN) data sampled from passenger vehicles
Recommended from our members
Discovering Models of Software Processes from Event-Based Data ; CU-CS-819-96
Many software process methods and tools presuppose the existence of a formal model of a process. Unfortunately, developing a formal model for an on-going, complex process can be dicult, costly, and error prone. This presents a practical barrier to the adoption of process technologies, which would be lowered by automated assistance in creating formal models. To this end, we have developed a data analysis technique that we term process discovery. Under this technique, data describing process events are rst captured from an on-going process and then used to generate a formal model of the behavior of that process. In this paper we describe a Markov method that we developed specically for process discovery, as well as describe two additional methods that we adopted from other domains and augmented for our purposes. The three methods range from the purely algorithmic to the purely statistical. We compare the methods and discuss their application in an industrial case study
- …