Pseudorehearsal in value function approximation

Catastrophic forgetting is of special importance in reinforcement learning, as the data distribution is generally non-stationary over time. We study and compare several pseudorehearsal approaches for Q-learning with function approximation in a pole balancing task. We have found that pseudorehearsal seems to assist learning even in such very simple problems, given proper initialization of the rehearsal parameters

SMAPs: Short Message Authentication Protocols

There is a long history of authentication protocols designed for ease of human use, which rely on users copying a short string of digits. Historical examples include telex test keys and early nuclear firing codes; familiar modern examples include prepayment meter codes and the 3-digit card verification values used in online shopping. In this paper, we show how security protocols that are designed for human readability and interaction can fail to provide adequate protection against simple attacks. To illustrate the problem, we discuss an offline payment protocol and explain various problems. We work through multiple iterations, or 'evolutions', of the protocol in order to get better tradeoffs between security and usability. We discuss the limitation of verifying such protocols using BAN logic. Our aim is to develop usable human-friendly protocols that can be used in constrained offline environments. We conclude that protocol designers need to be good curators of security state, and also pay attention to the interaction between online and offline functions. In fact, we suggest that delay-tolerant networking might be a future direction of evolution for protocol research

Benefit of enactment over oral repetition of verbal instruction does not require additional working memory during encoding

For this research, we used a dual-task approach to investigate the involvement of working memory in following written instructions. In two experiments, participants read instructions to perform a series of actions on objects and then recalled the instructions either by spoken repetition or performance of the action sequence. Participants engaged in concurrent articulatory suppression, backward-counting, and spatial-tapping tasks during the presentation of the instructions, in order to disrupt the phonological-loop, central-executive, and visuospatial-sketchpad components of working memory, respectively. Recall accuracy was substantially disrupted by all three concurrent tasks, indicating that encoding and retaining verbal instructions depends on multiple components of working memory. The accuracy of recalling the instructions was greater when the actions were performed than when the instructions were repeated, and this advantage was unaffected by the concurrent tasks, suggesting that the benefit of enactment over oral repetition does not cost additional working memory resource

Positive Interactions between Desert Granivores: Localized Facilitation of Harvester Ants by Kangaroo Rats

Facilitation, when one species enhances the environment or performance of another species, can be highly localized in space. While facilitation in plant communities has been intensely studied, the role of facilitation in shaping animal communities is less well understood. In the Chihuahuan Desert, both kangaroo rats and harvester ants depend on the abundant seeds of annual plants. Kangaroo rats, however, are hypothesized to facilitate harvester ants through soil disturbance and selective seed predation rather than competing with them. I used a spatially explicit approach to examine whether a positive or negative interaction exists between banner-tailed kangaroo rat (Dipodomys spectabilis) mounds and rough harvester ant (Pogonomyrmex rugosus) colonies. The presence of a scale-dependent interaction between mounds and colonies was tested by comparing fitted spatial point process models with and without interspecific effects. Also, the effect of proximity to a mound on colony mortality and spatial patterns of surviving colonies was examined. The spatial pattern of kangaroo rat mounds and harvester ant colonies was consistent with a positive interspecific interaction at small scales (<10 m). Mortality risk of vulnerable, recently founded harvester ant colonies was lower when located close to a kangaroo rat mound and proximity to a mound partly predicted the spatial pattern of surviving colonies. My findings support localized facilitation of harvester ants by kangaroo rats, likely mediated through ecosystem engineering and foraging effects on plant cover and composition. The scale-dependent effect of kangaroo rats on abiotic and biotic factors appears to result in greater founding and survivorship of young colonies near mounds. These results suggest that soil disturbance and foraging by rodents can have subtle impacts on the distribution and demography of other species

Adaptive Filtering Enhances Information Transmission in Visual Cortex

Sensory neuroscience seeks to understand how the brain encodes natural environments. However, neural coding has largely been studied using simplified stimuli. In order to assess whether the brain's coding strategy depend on the stimulus ensemble, we apply a new information-theoretic method that allows unbiased calculation of neural filters (receptive fields) from responses to natural scenes or other complex signals with strong multipoint correlations. In the cat primary visual cortex we compare responses to natural inputs with those to noise inputs matched for luminance and contrast. We find that neural filters adaptively change with the input ensemble so as to increase the information carried by the neural response about the filtered stimulus. Adaptation affects the spatial frequency composition of the filter, enhancing sensitivity to under-represented frequencies in agreement with optimal encoding arguments. Adaptation occurs over 40 s to many minutes, longer than most previously reported forms of adaptation.Comment: 20 pages, 11 figures, includes supplementary informatio

Individual differences in explicit and implicit visuomotor learning and working memory capacity

The theoretical basis for the association between high working memory capacity (WMC) and enhanced visuomotor adaptation is unknown. Visuomotor adaptation involves interplay between explicit and implicit systems. We examined whether the positive association between adaptation and WMC is specific to the explicit component of adaptation. Experiment 1 replicated the positive correlation between WMC and adaptation, but revealed this was specific to the explicit component of adaptation, and apparently driven by a sub-group of participants who did not show any explicit adaptation in the correct direction. A negative correlation was observed between WMC and implicit learning. Experiments 2 and 3 showed that when the task restricted the development of an explicit strategy, high WMC was no longer associated with enhanced adaptation. This work reveals that the benefit of high WMC is specifically linked to an individual’s capacity to use an explicit strategy. It also reveals an important contribution of individual differences in determining how adaptation is performed

Verbal thinking and inner speech use in autism spectrum disorder

The extent to which cognition is verbally mediated in neurotypical individuals is the subject of debate in cognitive neuropsychology, as well as philosophy and psychology. Studying “verbal thinking” in developmental/neuropsychological disorders provides a valuable opportunity to inform theory building, as well as clinical practice. In this paper, we provide a comprehensive, critical review of such studies among individuals with autism spectrum disorder (ASD). ASD involves severe social-communication deficits and limitations in cognitive/behavioural flexibility. The prevailing view in the field is that neither cognition nor behaviour is mediated verbally in ASD, and that this contributes to diagnostic features. However, our review suggests that, on the contrary, most studies to date actually find that among people with ASD cognitive task performance is either a) mediated verbally in a typical fashion, or b) not mediated verbally, but at no obvious cost to overall task performance. Overall though, these studies have methodological limitations and thus clear-cut conclusions are not possible at this stage. The aim of the review is to take stock of existing empirical findings, as well as to help develop the directions for future research that will resolve the many outstanding issues in this field

Incidence of re-amputation following partial first ray amputation associated with diabetes mellitus and peripheral sensory neuropathy: a systematic review.

Diabetes mellitus with peripheral sensory neuropathy frequently results in forefoot ulceration. Ulceration at the first ray level tends to be recalcitrant to local wound care modalities and off-loading techniques. If healing does occur, ulcer recurrence is common. When infection develops, partial first ray amputation in an effort to preserve maximum foot length is often performed. However, the survivorship of partial first ray amputations in this patient population and associated re-amputation rate remain unknown. Therefore, in an effort to determine the actual re-amputation rate following any form of partial first ray amputation in patients with diabetes mellitus and peripheral neuropathy, the authors conducted a systematic review. Only studies involving any form of partial first ray amputation associated with diabetes mellitus and peripheral sensory neuropathy but without critical limb ischemia were included. Our search yielded a total of 24 references with 5 (20.8%) meeting our inclusion criteria involving 435 partial first ray amputations. The weighted mean age of patients was 59 years and the weighted mean follow-up was 26 months. The initial amputation level included the proximal phalanx base 167 (38.4%) times; first metatarsal head resection 96 (22.1%) times; first metatarsal-phalangeal joint disarticulation 53 (12.2%) times; first metatarsal mid-shaft 39 (9%) times; hallux fillet flap 32 (7.4%) times; first metatarsal base 29 (6.7%) times; and partial hallux 19 (4.4%) times. The incidence of re-amputation was 19.8% (86/435). The end stage, most proximal level, following re-amputation was an additional digit 32 (37.2%) times; transmetatarsal 28 (32.6%) times; below-knee 25 (29.1%) times; and LisFranc 1 (1.2%) time. The results of our systematic review reveal that one out of every five patients undergoing any version of a partial first ray amputation will eventually require more proximal re-amputation. These results reveal that partial first ray amputation for patients with diabetes and peripheral sensory neuropathy may not represent a durable, functional, or predictable foot-sparing amputation and that a more proximal amputation, such as a balanced transmetatarsal amputation, as the index amputation may be more beneficial to the patient. However, this remains a matter for conjecture due to the limited data available and, therefore, additional prospective investigations are warranted

The Neuronal Correlates of Digits Backward Are Revealed by Voxel-Based Morphometry and Resting-State Functional Connectivity Analyses

Digits backward (DB) is a widely used neuropsychological measure that is believed to be a simple and effective index of the capacity of the verbal working memory. However, its neural correlates remain elusive. The aim of this study is to investigate the neural correlates of DB in 299 healthy young adults by combining voxel-based morphometry (VBM) and resting-state functional connectivity (rsFC) analyses. The VBM analysis showed positive correlations between the DB scores and the gray matter volumes in the right anterior superior temporal gyrus (STG), the right posterior STG, the left inferior frontal gyrus and the left Rolandic operculum, which are four critical areas in the auditory phonological loop of the verbal working memory. Voxel-based correlation analysis was then performed between the positive rsFCs of these four clusters and the DB scores. We found that the DB scores were positively correlated with the rsFCs within the salience network (SN), that is, between the right anterior STG, the dorsal anterior cingulate cortex and the right fronto-insular cortex. We also found that the DB scores were negatively correlated with the rsFC within an anti-correlation network of the SN, between the right posterior STG and the left posterior insula. Our findings suggest that DB performance is related to the structural and functional organizations of the brain areas that are involved in the auditory phonological loop and the SN