Diversity Measures: Domain-Independent Proxies for Failure in Language Model Queries

Error prediction in large language models often relies on domain-specific information. In this paper, we present measures for quantification of error in the response of a large language model based on the diversity of responses to a given prompt - hence independent of the underlying application. We describe how three such measures - based on entropy, Gini impurity, and centroid distance - can be employed. We perform a suite of experiments on multiple datasets and temperature settings to demonstrate that these measures strongly correlate with the probability of failure. Additionally, we present empirical results demonstrating how these measures can be applied to few-shot prompting, chain-of-thought reasoning, and error detection

Categorization of unorganized text corpora for better domain-specific language modeling

This paper describes the process of categorization of unorganized text data gathered from the Internet to the in-domain and out-of-domain data for better domain-specific language modeling and speech recognition. An algorithm for text categorization and topic detection based on the most frequent key phrases is presented. In this scheme, each document entered into the process of text categorization is represented by a vector space model with term weighting based on computing the term frequency and inverse document frequency. Text documents are then classified to the in-domain and out-of-domain data automatically with predefined threshold using one of the selected distance/similarity measures comparing to the list of key phrases. The experimental results of the language modeling and adaptation to the judicial domain show significant improvement in the model perplexity about 19 % and decreasing of the word error rate of the Slovak transcription and dictation system about 5,54 %, relatively

Modeling self-organization in pedestrians and animal groups from macroscopic and microscopic viewpoints

This paper is concerned with mathematical modeling of intelligent systems, such as human crowds and animal groups. In particular, the focus is on the emergence of different self-organized patterns from non-locality and anisotropy of the interactions among individuals. A mathematical technique by time-evolving measures is introduced to deal with both macroscopic and microscopic scales within a unified modeling framework. Then self-organization issues are investigated and numerically reproduced at the proper scale, according to the kind of agents under consideration.Comment: 24 pages, 13 figure

Complex Independent Component Analysis of Frequency-Domain Electroencephalographic Data

Independent component analysis (ICA) has proven useful for modeling brain and electroencephalographic (EEG) data. Here, we present a new, generalized method to better capture the dynamics of brain signals than previous ICA algorithms. We regard EEG sources as eliciting spatio-temporal activity patterns, corresponding to, e.g., trajectories of activation propagating across cortex. This leads to a model of convolutive signal superposition, in contrast with the commonly used instantaneous mixing model. In the frequency-domain, convolutive mixing is equivalent to multiplicative mixing of complex signal sources within distinct spectral bands. We decompose the recorded spectral-domain signals into independent components by a complex infomax ICA algorithm. First results from a visual attention EEG experiment exhibit (1) sources of spatio-temporal dynamics in the data, (2) links to subject behavior, (3) sources with a limited spectral extent, and (4) a higher degree of independence compared to sources derived by standard ICA.Comment: 21 pages, 11 figures. Added final journal reference, fixed minor typo

Inheritance-Based Diversity Measures for Explicit Convergence Control in Evolutionary Algorithms

Diversity is an important factor in evolutionary algorithms to prevent premature convergence towards a single local optimum. In order to maintain diversity throughout the process of evolution, various means exist in literature. We analyze approaches to diversity that (a) have an explicit and quantifiable influence on fitness at the individual level and (b) require no (or very little) additional domain knowledge such as domain-specific distance functions. We also introduce the concept of genealogical diversity in a broader study. We show that employing these approaches can help evolutionary algorithms for global optimization in many cases.Comment: GECCO '18: Genetic and Evolutionary Computation Conference, 2018, Kyoto, Japa

A model of factors influencing deck officers' cyber risk perception in offshore operations

Offshore operations onboard vessels are increasingly reliant on digitalization, integration, automation, and networked-based systems, which creates new dimensions of cyber risks. The causes of cyber incidents often include complex relationships between humans and technology, and in offshore operations, the onboard crew can be both a cyber security risk and a vital resource in strengthening the cyber security. This makes the behaviour of the decisionmakers onboard important in both preventing and handling cyber risks at sea. By use of in-depth interviews and the constant comparative analysis (CCA), this paper investigates factors influencing deck officers’ cyber risk perception in offshore operations and presents a contextual model of these factors. The model indicates that deck officers’ cyber risk perception can be affected by a feeling of distance towards cyber risks, being more restricted in their working environment because of digitalization, and trust in their reliable cyber-physical systems and suppliers. Further, targeted cyber risk mitigation measures should be implemented on multiple levels in shipping companies. The measures may benefit from focusing on increased risk communication, operational training, awareness campaigns, vessel-specific procedures, and policies, in addition to increased communication from management regarding the demand for digitalization. With this approach, the contextual model can contribute to the ongoing work of developing targeted measures for cyber risk mitigation in the maritime domain and can be used as a point of departure for further studies to discover additional nuances and factors within cyber risk perception in this domain.publishedVersio

Advances on Time Series Analysis using Elastic Measures of Similarity

A sequence is a collection of data instances arranged in a structured manner. When this arrangement is held in the time domain, sequences are instead referred to as time series. As such, each observation in a time series represents an observation drawn from an underlying process, produced at a specific time instant. However, other type of data indexing structures, such as space- or threshold-based arrangements are possible. Data points that compose a time series are often correlated with each other. To account for this correlation in data mining tasks, time series are usually studied as a whole data object rather than as a collection of independent observations. In this context, techniques for time series analysis aim at analyzing this type of data structures by applying specific approaches developed to leverage intrinsic properties of the time series for a wide range of problems, such as classification, clustering and other tasks alike. The development of monitoring and storage devices has made time se- ries analysis proliferate in numerous application fields, including medicine, economics, manufacturing and telecommunications, among others. Over the years, the community has gathered efforts towards the development of new data-based techniques for time series analysis suited to address the problems and needs of such application fields. In the related literature, such techniques can be divided in three main groups: feature-, model- and distance-based methods. The first group (feature-based) transforms time series into a collection of features, which are then used by conventional learning algorithms to provide solutions to the task under consideration. In contrast, methods belonging to the second group (model-based) assume that each time series is drawn from a generative model, which is then har- nessed to elicit knowledge from data. Finally, distance-based techniques operate directly on raw time series. To this end, these methods resort to specially defined measures of distance or similarity for comparing time series, without requiring any further processing. Among them, elastic sim- ilarity measures (e.g., dynamic time warping and edit distance) compute the closeness between two sequences by finding the best alignment between them, disregarding differences in time, and thus focusing exclusively on shape differences. This Thesis presents several contributions to the field of distance-based techniques for time series analysis, namely: i) a novel multi-dimensional elastic similarity learning method for time series classification; ii) an adap- tation of elastic measures to streaming time series scenarios; and iii) the use of distance-based time series analysis to make machine learning meth- ods for image classification robust against adversarial attacks. Throughout the Thesis, each contribution is framed within its related state of the art, explained in detail and empirically evaluated. The obtained results lead to new insights on the application of distance-based time series methods for the considered scenarios, and motivates research directions that highlight the vibrant momentum of this research area