Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition

The success of self-attention in NLP has led to recent applications in end-to-end encoder-decoder architectures for speech recognition. Separately, connectionist temporal classification (CTC) has matured as an alignment-free, non-autoregressive approach to sequence transduction, either by itself or in various multitask and decoding frameworks. We propose SAN-CTC, a deep, fully self-attentional network for CTC, and show it is tractable and competitive for end-to-end speech recognition. SAN-CTC trains quickly and outperforms existing CTC models and most encoder-decoder models, with character error rates (CERs) of 4.7% in 1 day on WSJ eval92 and 2.8% in 1 week on LibriSpeech test-clean, with a fixed architecture and one GPU. Similar improvements hold for WERs after LM decoding. We motivate the architecture for speech, evaluate position and downsampling approaches, and explore how label alphabets (character, phoneme, subword) affect attention heads and performance.Comment: Accepted to ICASSP 201

Factors Influencing Decision For Contralateral Prophylactic Mastectomy Versus Unilateral Mastectomy

Background: Rates of contralateral prophylactic mastectomy (CPM) are increasing despite falling rates of contralateral breast cancer and controversy over survival benefit of the procedure. We investigated why and how patients choose to undergo CPM, with focus on the roles of financial burden, decision-making resources, and active participation in decision-making. Methods:Female unilateral breast cancer patients older than 18 years old who had undergone mastectomy (with or without CPM) coming in to the Yale Breast Center for a follow-up or post-operative visit between June and August 2017 were approached with an opportunity to participate in this survey study. Chart review was completed for each patient to collect clinicopathologic data. The survey included both author-generated questions as well as the validated Satisfaction With Decision survey and Functional Assessment of Cancer Therapy – Breast, a quality of life survey. Results:109 eligible patients were approached, and 101 completed the survey (response rate 93%). 55 CPM patients (54.5%) and 46 unilateral mastectomy (UM) patients (45.5%) were included in this study. Both CPM and UM patients were highly satisfied with their respective decisions, with mean Satisfaction With Decision (SWD) scores of 4.72 and 4.85, respectively, out of 5.00 where 5.00 is the highest possible satisfaction (p=0.078). CPM patients were found on bivariate analysis to more often anticipate a “very large” financial burden compared with UM patients (25.5% vs. 8.7%, p=0.037), but the effect was dampened upon multivariate analysis controlling for other factors associated with financial burden and CPM (OR 3.65, 95% CI 0.848-15.698, p=0.082). CPM and UM patients had similar rates of reporting anticipated financial burden at least somewhat affecting surgical decision (19.6% vs. 12.7%, p=0.417). There was no association between actual financial burden and satisfaction or QoL across both CPM and UM patients (p\u3e0.05). Use of cancer patients’ experiences was independently associated with above average satisfaction (OR 9.40, 95% CI 1.01-87.58, p=0.049). Those who reported engaging in active participation in decision-making more often chose CPM compared with those who received a recommendation from their surgeon (68.3% vs. 30.8%, p Conclusion:CPM patients were just as satisfied with their surgical decision as were UM patients, and both groups reported similar rates of having anticipated financial burden affecting surgical decision-making. Use of other cancer patients’ experiences while choosing between CPM and UM predicted higher satisfaction. Patients who engaged in active participation in decision-making tended to undergo CPM, with the majority of all patients preferring to take an active role with their surgeon in considering CPM vs. UM

Parallel Five-Cycle Counting Algorithms

Counting the frequency of subgraphs in large networks is a classic research question that reveals the underlying substructures of these networks for important applications. However, subgraph counting is a challenging problem, even for subgraph sizes as small as five, due to the combinatorial explosion in the number of possible occurrences. This paper focuses on the five-cycle, which is an important special case of five-vertex subgraph counting and one of the most difficult to count efficiently. We design two new parallel five-cycle counting algorithms and prove that they are work-efficient and achieve polylogarithmic span. Both algorithms are based on computing low out-degree orientations, which enables the efficient computation of directed two-paths and three-paths, and the algorithms differ in the ways in which they use this orientation to eliminate double-counting. We develop fast multicore implementations of the algorithms and propose a work scheduling optimization to improve their performance. Our experiments on a variety of real-world graphs using a 36-core machine with two-way hyper-threading show that our algorithms achieves 10-46x self-relative speed-up, outperform our serial benchmarks by 10-32x, and outperform the previous state-of-the-art serial algorithm by up to 818x

PECANN: Parallel Efficient Clustering with Graph-Based Approximate Nearest Neighbor Search

This paper studies density-based clustering of point sets. These methods use dense regions of points to detect clusters of arbitrary shapes. In particular, we study variants of density peaks clustering, a popular type of algorithm that has been shown to work well in practice. Our goal is to cluster large high-dimensional datasets, which are prevalent in practice. Prior solutions are either sequential, and cannot scale to large data, or are specialized for low-dimensional data. This paper unifies the different variants of density peaks clustering into a single framework, PECANN, by abstracting out several key steps common to this class of algorithms. One such key step is to find nearest neighbors that satisfy a predicate function, and one of the main contributions of this paper is an efficient way to do this predicate search using graph-based approximate nearest neighbor search (ANNS). To provide ample parallelism, we propose a doubling search technique that enables points to find an approximate nearest neighbor satisfying the predicate in a small number of rounds. Our technique can be applied to many existing graph-based ANNS algorithms, which can all be plugged into PECANN. We implement five clustering algorithms with PECANN and evaluate them on synthetic and real-world datasets with up to 1.28 million points and up to 1024 dimensions on a 30-core machine with two-way hyper-threading. Compared to the state-of-the-art FASTDP algorithm for high-dimensional density peaks clustering, which is sequential, our best algorithm is 45x-734x faster while achieving competitive ARI scores. Compared to the state-of-the-art parallel DPC-based algorithm, which is optimized for low dimensions, we show that PECANN is two orders of magnitude faster. As far as we know, our work is the first to evaluate DPC variants on large high-dimensional real-world image and text embedding datasets

(In)visible Cities: What Generative Algorithms Tell Us About Our Collective Memory Schema

The last decade has witnessed a turn in AI technologies working with differentiable neural network architectures learning the embedded functions between data points and performing generative operations synthesising unseen data. The move to a continuous and generative AI paradigm aligns with ideas in the field of cognition and psychology, where a growing body of authors are beginning to conceptualise memory and our representation of the past as a dynamic, malleable and ultimately generative field. So, how effective are generative algorithms in supporting and enabling this creative process of remembrance? To answer this research question, we propose an experiment on how the spatial movement and exploration of maps of real and imagined images can help our brain reconstruct its memories in a dynamic yet accurate manner. We develop an application allowing visitors to dynamically explore real and AI-generated images of a given site clustered by similarity in a virtual 3D space. Analysing visitor paths and observed images helps us understand visitors’ perspectives on real and AI-generated data such as an increased preference for synthetic images by visitors familiarised with the site. We conclude with recommendations on how to approach visitor experience in generative AI-powered applications for engagement with historical and archival data

BIPHASIC PATTERN OF THYMUS REGENERATION AFTER WHOLE-BODY IRRADIATION

Whole-body irradiation of mice with 300 or 400 R causes a precipitous fall in thymus weight, followed by an increase in the mitotic index and an almost complete restoration of thymus mass. This phase is followed by a secondary fall in thymus weight and gradual recovery. This secondary fall can be prevented by intravenous injection of bone marrow or shielding of the hind limbs during irradiation. The hypothesis is proposed that the thymus depends on the migration of cells from the bone marrow to the thymus for the maintenance of its cell population. Bone marrow cells with chromosome markers injected intravenously into normal or lightly irradiated (150 R) animals do not populate the host bone marrow to any significant degree. After whole-body irradiation with heavy doses (400 R), donor cells dominate the marrow. There may be a competition between dividing cells in the bone marrow which regulates proliferation of hemic cells. Bone marrow cells with marker chromosomes do not repopulate the thymus in irradiated animals until long after repopulating the bone marrow. It is possible that these cells have to pass through the marrow or the blood-marrow barrier to acquire characteristics needed for entering the thymus. After whole-body irradiation with 500 R or more, the first phase of regeneration of the thymus, represented by an increase in the mitotic index, does not occur to a significant degree. Apparently cells in the thymus capable of proliferation have been largely eliminated, and restoration of organ mass depends chiefly on seeding from other sources, probably the bone marrow. After whole-body irradiation with 200 R, only the first phase of thymus weight loss and regeneration takes place. Probably bone marrow injury is too small to interfere with the supply of cells repopulating the thymus

Evidence for mechanisms underlying the functional benefits of a myocardial matrix hydrogel for post-MI treatment

Background There is increasing need for better therapies to prevent the development of heart failure after myocardial infarction (MI). An injectable hydrogel derived from decellularized porcine ventricular myocardium has been shown to halt the post-infarction progression of negative left ventricular remodeling and decline in cardiac function in both small and large animal models. Objectives This study sought to elucidate the tissue-level mechanisms underlying the therapeutic benefits of myocardial matrix injection. Methods Myocardial matrix or saline was injected into infarcted myocardium 1 week after ischemia-reperfusion in Sprague-Dawley rats. Cardiac function was evaluated by magnetic resonance imaging and hemodynamic measurements at 5 weeks after injection. Whole transcriptome microarrays were performed on RNA isolated from the infarct at 3 days and 1 week after injection. Quantitative polymerase chain reaction and histologic quantification confirmed expression of key genes and their activation in altered pathways. Results Principal component analysis of the transcriptomes showed that samples collected from myocardial matrix-injected infarcts are distinct and cluster separately from saline-injected control subjects. Pathway analysis indicated that these differences are due to changes in several tissue processes that may contribute to improved cardiac healing after MI. Matrix-injected infarcted myocardium exhibits an altered inflammatory response, reduced cardiomyocyte apoptosis, enhanced infarct neovascularization, diminished cardiac hypertrophy and fibrosis, altered metabolic enzyme expression, increased cardiac transcription factor expression, and progenitor cell recruitment, along with improvements in global cardiac function and hemodynamics. Conclusions These results indicate that the myocardial matrix alters several key pathways after MI creating a pro-regenerative environment, further demonstrating its promise as a potential post-MI therapy