2,071 research outputs found
Recommended from our members
Synaptic plasticity and memory addressing in biological and artificial neural networks
Biological brains are composed of neurons, interconnected by synapses to create large complex networks. Learning and memory occur, in large part, due to synaptic plasticity -- modifications in the efficacy of information transmission through these synaptic connections. Artificial neural networks model these with neural "units" which communicate through synaptic weights. Models of learning and memory propose synaptic plasticity rules that describe and predict the weight modifications. An equally important but under-evaluated question is the selection of \textit{which} synapses should be updated in response to a memory event. In this work, we attempt to separate the questions of synaptic plasticity from that of memory addressing.
Chapter 1 provides an overview of the problem of memory addressing and a summary of the solutions that have been considered in computational neuroscience and artificial intelligence, as well as those that may exist in biology. Chapter 2 presents in detail a solution to memory addressing and synaptic plasticity in the context of familiarity detection, suggesting strong feedforward weights and anti-Hebbian plasticity as the respective mechanisms. Chapter 3 proposes a model of recall, with storage performed by addressing through local third factors and neo-Hebbian plasticity, and retrieval by content-based addressing. In Chapter 4, we consider the problem of concurrent memory consolidation and memorization. Both storage and retrieval are performed by content-based addressing, but the plasticity rule itself is implemented by gradient descent, modulated according to whether an item should be stored in a distributed manner or memorized verbatim. However, the classical method for computing gradients in recurrent neural networks, backpropagation through time, is generally considered unbiological. In Chapter 5 we suggest a more realistic implementation through an approximation of recurrent backpropagation.
Taken together, these results propose a number of potential mechanisms for memory storage and retrieval, each of which separates the mechanism of synaptic updating -- plasticity -- from that of synapse selection -- addressing. Explicit studies of memory addressing may find applications not only in artificial intelligence but also in biology. In artificial networks, for example, selectively updating memories in large language models can help improve user privacy and security. In biological ones, understanding memory addressing can help with health outcomes and treating memory-based illnesses such as Alzheimers or PTSD
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks
Transformer is a deep neural network that employs a self-attention mechanism
to comprehend the contextual relationships within sequential data. Unlike
conventional neural networks or updated versions of Recurrent Neural Networks
(RNNs) such as Long Short-Term Memory (LSTM), transformer models excel in
handling long dependencies between input sequence elements and enable parallel
processing. As a result, transformer-based models have attracted substantial
interest among researchers in the field of artificial intelligence. This can be
attributed to their immense potential and remarkable achievements, not only in
Natural Language Processing (NLP) tasks but also in a wide range of domains,
including computer vision, audio and speech processing, healthcare, and the
Internet of Things (IoT). Although several survey papers have been published
highlighting the transformer's contributions in specific fields, architectural
differences, or performance evaluations, there is still a significant absence
of a comprehensive survey paper encompassing its major applications across
various domains. Therefore, we undertook the task of filling this gap by
conducting an extensive survey of proposed transformer models from 2017 to
2022. Our survey encompasses the identification of the top five application
domains for transformer-based models, namely: NLP, Computer Vision,
Multi-Modality, Audio and Speech Processing, and Signal Processing. We analyze
the impact of highly influential transformer-based models in these domains and
subsequently classify them based on their respective tasks using a proposed
taxonomy. Our aim is to shed light on the existing potential and future
possibilities of transformers for enthusiastic researchers, thus contributing
to the broader understanding of this groundbreaking technology
EQUI-VOCAL: Synthesizing Queries for Compositional Video Events from Limited User Interactions [Technical Report]
We introduce EQUI-VOCAL: a new system that automatically synthesizes queries
over videos from limited user interactions. The user only provides a handful of
positive and negative examples of what they are looking for. EQUI-VOCAL
utilizes these initial examples and additional ones collected through active
learning to efficiently synthesize complex user queries. Our approach enables
users to find events without database expertise, with limited labeling effort,
and without declarative specifications or sketches. Core to EQUI-VOCAL's design
is the use of spatio-temporal scene graphs in its data model and query language
and a novel query synthesis approach that works on large and noisy video data.
Our system outperforms two baseline systems -- in terms of F1 score, synthesis
time, and robustness to noise -- and can flexibly synthesize complex queries
that the baselines do not support.Comment: This is an extended technical report for the following paper: "Enhao
Zhang, Maureen Daum, Dong He, Brandon Haynes, Ranjay Krishna, and Magdalena
Balazinska. EQUI-VOCAL: Synthesizing Queries for Compositional Video Events
from Limited User Interactions. PVLDB, 16(11): 2714-2727, 2023.
doi:10.14778/3611479.3611482
Development of an Event Management Web Application For Students: A Focus on Back-end
Managing schedules can be challenging for students, with different calendars on various platforms leading to confusion and missed events. To address this problem, this thesis presents the development of an event management website designed to help students stay organized and motivated. With a focus on the application's back-end, this thesis explores the technology stack used to build the website and the implementation details of each chosen technology. By providing a detailed case study of the website development process, this thesis serves as a helpful resource for future developers looking to build their web applications
Providing Private and Fast Data Access for Cloud Systems
Cloud storage and computing systems have become the backbone of many applications such as streaming (Netflix, YouTube), storage (Dropbox, Google Drive), and computing (Amazon Elastic Computing, Microsoft Azure). To address the ever growing demand for storage and computing requirements of these applications, cloud services are typically im-plemented over a large-scale distributed data storage system. Cloud systems are expected to provide the following two pivotal services for the users: 1) private content access and 2) fast content access. The goal of this thesis is to understand and address some of the challenges that need to be overcome to provide these two services.
The first part of this thesis focuses on private data access in distributed systems. In particular, we contribute to the areas of Private Information Retrieval (PIR) and Private Computation (PC). In the PIR problem, there is a user who wishes to privately retrieve a subset of files belonging to a database stored on a single or multiple remote server(s). In the PC problem, the user wants to privately compute functions of a subset of files in the database. The PIR and PC problems seek the most efficient solutions with the minimum download cost that enable the user to retrieve or compute what it wants privately.
We establish fundamental bounds on the minimum download cost required for guaran-teeing the privacy requirement in some practical and realistic settings of the PIR and PC problems and develop novel and efficient privacy-preserving algorithms for these settings. In particular, we study the single-server and multi-server settings of PIR in which the user initially has a random linear combination of a subset of files in the database as side in-formation, referred to as PIR with coded side information. We also study the multi-server setting of the PC in which the user wants to privately compute multiple linear combinations of a subset of files in the database, referred to as Private Linear Transformation.
The second part of this thesis focuses on fast content access in distributed systems. In particular, we study the use of erasure coding to handle data access requests in distributed storage and computing systems. Service rate region is an important performance metric for coded distributed systems, which expresses the set of all data access request rates that can be simultaneously served by the system. In this context, two classes of problems arise: 1) characterizing the service rate region of a given storage scheme and finding the optimal request allocation, and 2) designing the underlying erasure code to handle a given desired service rate region.
As contributions along the first class of problems, we characterize the service rate region of systems with some common coding schemes such as Simplex codes and Reed-Muller codes by introducing two novel techniques: 1) fractional matching and vertex cover on graph representation of codes, and 2) geometric representations of codes. Moreover, along the second class of code design, we establish some lower bounds on the minimum storage required to handle a desired service rate region for a coded distributed system and in some regimes, we design efficient storage schemes that provide the desired service rate region while minimizing the storage requirements
Analýza a klasifikace nabíjecích dat pro mikro sítě
This thesis aims to develop a classification model for electric vehicles (EVs) based on data from
EV charging stations.The study utilizes a dataset of 6 charging sessions from EV charging station
and implements two deep learning algorithm including LSTM and Auto-Encoders to classify EVs.
The performance of the classification model is evaluated based on accuracy rates & precision.The
study also identifies key charging characteristics that are most significant in distinguishing between
different types of EVs, including charging time, energy consumption, and charging patterns.The
findings of this research have significant implications for the development of EV charging infras-
tructure and services. The classification model developed in this thesis can be used to optimize
charging station operations, improve charging services, and develop EV adoption strategies. The
study also highlights the importance of utilizing data from EV charging stations in understanding
the EV market and improving the efficiency of charging infrastructure.Tato práce si klade za cíl vyvinout model klasifikace elektrických vozidel (EV) založený na datech
z nabíjecích stanic pro elektromobily. Studie využívá datovou sadu 6 nabíjecích relací z nabíjecí
stanice pro elektromobily a implementuje dva algoritmy hlubokého učení, včetně LSTM a Auto-
Encoders pro klasifikaci EV. . Výkon klasifikačního modelu je hodnocen na základě míry přesnosti
a preciznosti. Studie také identifikuje klíčové charakteristiky nabíjení, které jsou nejvýznamnější při
rozlišování mezi různými typy elektromobilů, včetně doby nabíjení, spotřeby energie a vzorců nabí-
jení. Zjištění tohoto výzkumu mají významné důsledky pro rozvoj infrastruktury a služeb nabíjení
elektromobilů. Klasifikační model vyvinutý v této práci lze použít k optimalizaci provozu nabíjecích
stanic, zlepšení nabíjecích služeb a rozvoji strategií přijetí elektromobilů. Studie také zdůrazňuje
důležitost využití dat z nabíjecích stanic pro elektromobily pro pochopení trhu s elektromobily a
zlepšení efektivity nabíjecí infrastruktury450 - Katedra kybernetiky a biomedicínského inženýrstvívýborn
A Business Intelligence Solution, based on a Big Data Architecture, for processing and analyzing the World Bank data
The rapid growth in data volume and complexity has needed the adoption of advanced technologies to extract valuable insights for decision-making. This project aims to address this need by developing a comprehensive framework that combines Big Data processing, analytics, and visualization techniques to enable effective analysis of World Bank data. The problem addressed in this study is the need for a scalable and efficient Business Intelligence solution that can handle the vast amounts of data generated by the World Bank. Therefore, a Big Data architecture is implemented on a real use case for the International Bank of Reconstruction and Development. The findings of this project demonstrate the effectiveness of the proposed solution. Through the integration of Apache Spark and Apache Hive, data is processed using Extract, Transform and Load techniques, allowing for efficient data preparation. The use of Apache Kylin enables the construction of a multidimensional model, facilitating fast and interactive queries on the data. Moreover, data visualization techniques are employed to create intuitive and informative visual representations of the analysed data. The key conclusions drawn from this project highlight the advantages of a Big Data-driven Business Intelligence solution in processing and analysing World Bank data. The implemented framework showcases improved scalability, performance, and flexibility compared to traditional approaches. In conclusion, this bachelor thesis presents a Business Intelligence solution based on a Big Data architecture for processing and analysing the World Bank data. The project findings emphasize the importance of scalable and efficient data processing techniques, multidimensional modelling, and data visualization for deriving valuable insights. The application of these techniques contributes to the field by demonstrating the potential of Big Data Business Intelligence solutions in addressing the challenges associated with large-scale data analysis
- …