Chunking Patterns Reflect Effector-dependent Representation of Motor Sequence

Sequential organization is central to much of human intelligent behavior ranging from everyday skills such as lacing shoes to using a computer. It is well known that such sequential skills involve chaining a number of primitive actions together. A robust representation of skills can be formed by chunking together several elements of a sequence. We demonstrate, using a 2x6 finger movement task, that during the process of acquiring visuomotor skills the chunking patterns remained unaltered when utilizing an effector dependent representation of the sequence. In the 2x6 task, subjects learned a sequence of 12 visual cues displayed as six sets of two elements each and performed finger movements on a keypad. Two experiments Normal-Motor and Normal-Visual were conducted on nine subjects and two observations were collected from each subject. Each experiment consisted of a Normal and a Rotated condition. In the Rotated (Motor and Visual) conditions, subjects were required to rotate the visual cues by 180 degrees and press the corresponding keys. The display sequence was also rotated for the Motor condition, requiring an identical set of effector movements to be performed as in the Normal condition. Chunking patterns were identified using the response times (RTs) for individual sets of the sequence. A pause between set RTs demarcates an ensuing chunk. We demonstrate that usage of an effector dependent representation is supported by the observation of identical chunking patterns between the Normal and Motor conditions, and the lack of similarity in chunking patterns between the Normal and Visual conditions

Efficient Lock-free Binary Search Trees

In this paper we present a novel algorithm for concurrent lock-free internal binary search trees (BST) and implement a Set abstract data type (ADT) based on that. We show that in the presented lock-free BST algorithm the amortized step complexity of each set operation - {\sc Add}, {\sc Remove} and {\sc Contains} - is $O(H(n) + c)$, where, $H(n)$ is the height of BST with $n$ number of nodes and $c$ is the contention during the execution. Our algorithm adapts to contention measures according to read-write load. If the situation is read-heavy, the operations avoid helping pending concurrent {\sc Remove} operations during traversal, and, adapt to interval contention. However, for write-heavy situations we let an operation help pending {\sc Remove}, even though it is not obstructed, and so adapt to tighter point contention. It uses single-word compare-and-swap (\texttt{CAS}) operations. We show that our algorithm has improved disjoint-access-parallelism compared to similar existing algorithms. We prove that the presented algorithm is linearizable. To the best of our knowledge this is the first algorithm for any concurrent tree data structure in which the modify operations are performed with an additive term of contention measure.Comment: 15 pages, 3 figures, submitted to POD

Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees

Asynchronous distributed algorithms are a popular way to reduce synchronization costs in large-scale optimization, and in particular for neural network training. However, for nonsmooth and nonconvex objectives, few convergence guarantees exist beyond cases where closed-form proximal operator solutions are available. As most popular contemporary deep neural networks lead to nonsmooth and nonconvex objectives, there is now a pressing need for such convergence guarantees. In this paper, we analyze for the first time the convergence of stochastic asynchronous optimization for this general class of objectives. In particular, we focus on stochastic subgradient methods allowing for block variable partitioning, where the shared-memory-based model is asynchronously updated by concurrent processes. To this end, we first introduce a probabilistic model which captures key features of real asynchronous scheduling between concurrent processes; under this model, we establish convergence with probability one to an invariant set for stochastic subgradient methods with momentum. From the practical perspective, one issue with the family of methods we consider is that it is not efficiently supported by machine learning frameworks, as they mostly focus on distributed data-parallel strategies. To address this, we propose a new implementation strategy for shared-memory based training of deep neural networks, whereby concurrent parameter servers are utilized to train a partitioned but shared model in single- and multi-GPU settings. Based on this implementation, we achieve on average 1.2x speed-up in comparison to state-of-the-art training methods for popular image classification tasks without compromising accuracy

Detection of Cognitive States from fMRI data using Machine Learning Techniques

Over the past decade functional Magnetic Resonance Imaging (fMRI) has emerged as a powerful technique to locate activity of human brain while engaged in a particular task or cognitive state. We consider the inverse problem of detecting the cognitive state of a human subject based on the fMRI data. We have explored classification techniques such as Gaussian Naive Bayes, k-Nearest Neighbour and Support Vector Machines. In order to reduce the very high dimensional fMRI data, we have used three feature selection strategies. Discriminating features and activity based features were used to select features for the problem of identifying the instantaneous cognitive state given a single fMRI scan and correlation based features were used when fMRI data from a single time interval was given. A case study of visuo-motor sequence learning is presented. The set of cognitive states we are interested in detecting are whether the subject has learnt a sequence, and if the subject is paying attention only towards the position or towards both the color and position of the visual stimuli. We have successfully used correlation based features to detect position-color related cognitive states with 80% accuracy and the cognitive states related to learning with 62.5% accuracy