125 research outputs found
Conceptual Model Interpreter for Large Language Models
Large Language Models (LLMs) recently demonstrated capabilities for
generating source code in common programming languages. Additionally,
commercial products such as ChatGPT 4 started to provide code interpreters,
allowing for the automatic execution of generated code fragments, instant
feedback, and the possibility to develop and refine in a conversational
fashion. With an exploratory research approach, this paper applies code
generation and interpretation to conceptual models. The concept and prototype
of a conceptual model interpreter is explored, capable of rendering visual
models generated in textual syntax by state-of-the-art LLMs such as Llama~2 and
ChatGPT 4. In particular, these LLMs can generate textual syntax for the
PlantUML and Graphviz modeling software that is automatically rendered within a
conversational user interface. The first result is an architecture describing
the components necessary to interact with interpreters and LLMs through APIs or
locally, providing support for many commercial and open source LLMs and
interpreters. Secondly, experimental results for models generated with ChatGPT
4 and Llama 2 are discussed in two cases covering UML and, on an instance
level, graphs created from custom data. The results indicate the possibility of
modeling iteratively in a conversational fashion.Comment: ER Forum 2023, 42nd International Conference on Conceptual Modeling
(ER 2023), November 6-9, 2023, Lisbon, P
A Theory of Emergent In-Context Learning as Implicit Structure Induction
Scaling large language models (LLMs) leads to an emergent capacity to learn
in-context from example demonstrations. Despite progress, theoretical
understanding of this phenomenon remains limited. We argue that in-context
learning relies on recombination of compositional operations found in natural
language data. We derive an information-theoretic bound showing how in-context
learning abilities arise from generic next-token prediction when the
pretraining distribution has sufficient amounts of compositional structure,
under linguistically motivated assumptions. A second bound provides a
theoretical justification for the empirical success of prompting LLMs to output
intermediate steps towards an answer. To validate theoretical predictions, we
introduce a controlled setup for inducing in-context learning; unlike previous
approaches, it accounts for the compositional nature of language. Trained
transformers can perform in-context learning for a range of tasks, in a manner
consistent with the theoretical results. Mirroring real-world LLMs in a
miniature setup, in-context learning emerges when scaling parameters and data,
and models perform better when prompted to output intermediate steps. Probing
shows that in-context learning is supported by a representation of the input's
compositional structure. Taken together, these results provide a step towards
theoretical understanding of emergent behavior in large language models
Faith and Fate: Limits of Transformers on Compositionality
Transformer large language models (LLMs) have sparked admiration for their
exceptional performance on tasks that demand intricate multi-step reasoning.
Yet, these models simultaneously show failures on surprisingly trivial
problems. This begs the question: Are these errors incidental, or do they
signal more substantial limitations? In an attempt to demystify Transformers,
we investigate the limits of these models across three representative
compositional tasks -- multi-digit multiplication, logic grid puzzles, and a
classic dynamic programming problem. These tasks require breaking problems down
into sub-steps and synthesizing these steps into a precise answer. We formulate
compositional tasks as computation graphs to systematically quantify the level
of complexity, and break down reasoning steps into intermediate sub-procedures.
Our empirical findings suggest that Transformers solve compositional tasks by
reducing multi-step compositional reasoning into linearized subgraph matching,
without necessarily developing systematic problem-solving skills. To round off
our empirical study, we provide theoretical arguments on abstract multi-step
reasoning problems that highlight how Transformers' performance will rapidly
decay with increased task complexity.Comment: 10 pages + appendix (21 pages
Large Language Models for Software Engineering: Survey and Open Problems
This paper provides a survey of the emerging area of Large Language Models
(LLMs) for Software Engineering (SE). It also sets out open research challenges
for the application of LLMs to technical problems faced by software engineers.
LLMs' emergent properties bring novelty and creativity with applications right
across the spectrum of Software Engineering activities including coding,
design, requirements, repair, refactoring, performance improvement,
documentation and analytics. However, these very same emergent properties also
pose significant technical challenges; we need techniques that can reliably
weed out incorrect solutions, such as hallucinations. Our survey reveals the
pivotal role that hybrid techniques (traditional SE plus LLMs) have to play in
the development and deployment of reliable, efficient and effective LLM-based
SE
An Identification System for Head Mounted Displays
Personalized devices often require a form of user identification to provide customized performance and rudimentary privacy between a limited amount of users. Because of the personal nature of head mounted devices, the new and growing industry of head mounted displays requires a method to identify users to increase customizability and usability of such devices. This project introduces a system that accurately identifies users with common sensors included on head mounted displays. The proposed system records user blink behavior, head position and head movement and then uses high dimensional machine learning algorithms to identify users based on trends in their collected data. The system demonstrated over 98% accuracy, demonstrating its ability to identify users
Some optimization problems in power system reliability analysis
This dissertation aims to address two optimization problems involving power system reliabilty analysis, namely multi-area power system adequacy planning and transformer maintenance optimization. A new simulation method for power system reliability evaluation is proposed. The proposed method provides reliability indexes and distributions which can be used for risk assessment. Several solution methods for the planning problem are also proposed. The first method employs sensitivity analysis with Monte Carlo simulation. The procedure is simple yet effective and can be used as a guideline to quantify effectiveness of additional capacity. The second method applies scenario analysis with a state-space decomposition approach called global decomposition. The algorithm requires less memory usage and converges with fewer stages of decomposition. A system reliability equation is derived that leads to the development of the third method using dynamic programming. The main contribution of the third method is the approximation of reliability equation. The fourth method is the stochastic programming framework. This method offers modeling flexibility. The implementation of the solution techniques is presented and discussed. Finally, a probabilistic maintenance model of the transformer is proposed where mathematical equations relating maintenance practice and equipment lifetime and cost are derived. The closed-form expressions insightfully explain how the transformer parameters relate to reliability. This mathematical model facilitates an optimum, cost-effective maintenance scheme for the transformer
The Digital Humanities and Literary Studies
This book delivers an introduction and overview of developing intersections between digital methods and literary studies. The Digital Humanities and Literary Studies serves as a starting place for those who wish to learn more about the possibilities, and the limitations, of the oft-touted digital humanities in the literary space. The volume engages with the proponents of digital humanities and its detractors alike, aiming to offer a fair and balanced perspective on this controversial topic. The book combines a survey and background approach with original literary research and, therefore, straddles the divide between seasoned digital experts and interested newcomers
The Digital Humanities and Literary Studies
This book delivers an introduction and overview of developing intersections between digital methods and literary studies. The Digital Humanities and Literary Studies serves as a starting place for those who wish to learn more about the possibilities, and the limitations, of the oft-touted digital humanities in the literary space. The volume engages with the proponents of digital humanities and its detractors alike, aiming to offer a fair and balanced perspective on this controversial topic. The book combines a survey and background approach with original literary research and, therefore, straddles the divide between seasoned digital experts and interested newcomers
- …