Telling BERT's full story: from Local Attention to Global Aggregation

We take a deep look into the behavior of self-attention heads in the transformer architecture. In light of recent work discouraging the use of attention distributions for explaining a model's behavior, we show that attention distributions can nevertheless provide insights into the local behavior of attention heads. This way, we propose a distinction between local patterns revealed by attention and global patterns that refer back to the input, and analyze BERT from both angles. We use gradient attribution to analyze how the output of an attention attention head depends on the input tokens, effectively extending the local attention-based analysis to account for the mixing of information throughout the transformer layers. We find that there is a significant discrepancy between attention and attribution distributions, caused by the mixing of context inside the model. We quantify this discrepancy and observe that interestingly, there are some patterns that persist across all layers despite the mixing.Comment: Accepted at EACL 202

Force- and length-dependent catastrophe activities explain interphase microtubule organization in fission yeast.

The cytoskeleton is essential for the maintenance of cell morphology in eukaryotes. In fission yeast, for example, polarized growth sites are organized by actin, whereas microtubules (MTs) acting upstream control where growth occurs. Growth is limited to the cell poles when MTs undergo catastrophes there and not elsewhere on the cortex. Here, we report that the modulation of MT dynamics by forces as observed in vitro can quantitatively explain the localization of MT catastrophes in Schizosaccharomyces pombe. However, we found that it is necessary to add length-dependent catastrophe rates to make the model fully consistent with other previously measured traits of MTs. We explain the measured statistical distribution of MT-cortex contact times and re-examine the curling behavior of MTs in unbranched straight tea1Delta cells. Importantly, the model demonstrates that MTs together with associated proteins such as depolymerizing kinesins are, in principle, sufficient to mark the cell poles

Tubulin Dimers Oligomerize before Their Incorporation into Microtubules

In the presence of GTP, purified dimers of α- and β-tubulin will interact longitudinally and laterally to self-assemble into microtubules (MTs). This property provides a powerful in vitro experimental system to describe MT dynamic behavior at the micrometer scale and to study effects and functioning of a large variety of microtubule associated proteins (MAPs). Despite the plethora of such data produced, the molecular mechanisms of MT assembly remain disputed. Electron microscopy (EM) studies suggested that tubulin dimers interact longitudinally to form short oligomers which form a tube by lateral interaction and which contribute to MT elongation. This idea is however challenged: Based on estimated association constants it was proposed that single dimers represent the major fraction of free tubulin. This view was recently supported by measurements suggesting that MTs elongate by addition of single tubulin dimers. To solve this discrepancy, we performed a direct measurement of the longitudinal interaction energy for tubulin dimers. We quantified the size distribution of tubulin oligomers using EM and fluorescence correlation spectroscopy (FCS). From the distribution we derived the longitudinal interaction energy in the presence of GDP and the non-hydrolysable GTP analog GMPCPP. Our data suggest that MT elongation and nucleation involves interactions of short tubulin oligomers rather than dimers. Our approach provides a solid experimental framework to better understand the role of MAPs in MT nucleation and growth

Spatial transcriptomics combined with single-cell RNA-sequencing unravels the complex inflammatory cell network in atopic dermatitis

BackgroundAtopic dermatitis (AD) is the most common chronic inflammatory skin disease with complex pathogenesis for which the cellular and molecular crosstalk in AD skin has not been fully understood.MethodsSkin tissues examined for spatial gene expression were derived from the upper arm of 6 healthy control (HC) donors and 7 AD patients (lesion and nonlesion). We performed spatial transcriptomics sequencing to characterize the cellular infiltrate in lesional skin. For single‐cell analysis, we analyzed the single‐cell data from suction blister material from AD lesions and HC skin at the antecubital fossa skin (4 ADs and 5 HCs) and full‐thickness skin biopsies (4 ADs and 2 HCs). The multiple proximity extension assays were performed in the serum samples from 36 AD patients and 28 HCs.ResultsThe single‐cell analysis identified unique clusters of fibroblasts, dendritic cells, and macrophages in the lesional AD skin. Spatial transcriptomics analysis showed the upregulation of COL6A5, COL4A1, TNC, and CCL19 in COL18A1‐expressing fibroblasts in the leukocyte‐infiltrated areas in AD skin. CCR7‐expressing dendritic cells (DCs) showed a similar distribution in the lesions. Additionally, M2 macrophages expressed CCL13 and CCL18 in this area. Ligand–receptor interaction analysis of the spatial transcriptome identified neighboring infiltration and interaction between activated COL18A1‐expressing fibroblasts, CCL13‐ and CCL18‐expressing M2 macrophages, CCR7‐ and LAMP3‐expressing DCs, and T cells. As observed in skin lesions, serum levels of TNC and CCL18 were significantly elevated in AD, and correlated with clinical disease severity.ConclusionIn this study, we show the unknown cellular crosstalk in leukocyte‐infiltrated area in lesional skin. Our findings provide a comprehensive in‐depth knowledge of the nature of AD skin lesions to guide the development of better treatments

Simulation Study of Photon-to-Digital Converter (PDC) Timing Specifications for LoLX Experiment

The Light only Liquid Xenon (LoLX) experiment is a prototype detector aimed to study liquid xenon (LXe) light properties and various photodetection technologies. LoLX is also aimed to quantify LXe's time resolution as a potential scintillator for 10~ps time-of-flight (TOF) PET. Another key goal of LoLX is to perform a time-based separation of Cerenkov and scintillation photons for new background rejection methods in LXe experiments. To achieve this separation, LoLX is set to be equipped with photon-to-digital converters (PDCs), a photosensor type that provides a timestamp for each observed photon. To guide the PDC design, we explore requirements for time-based Cerenkov separation. We use a PDC simulator, whose input is the light information from the Geant4-based LoLX simulation model, and evaluate the separation quality against time-to-digital converter (TDC) parameters. Simulation results with TDC parameters offer possible configurations supporting a good separation. Compared with the current filter-based approach, simulations show Cerenkov separation level increases from 54% to 71% when using PDC and time-based separation. With the current photon time profile of LoLX simulation, the results also show 71% separation is achievable with just 4 TDCs per PDC. These simulation results will lead to a specification guide for the PDC as well as expected results to compare against future PDC-based experimental measurements. In the longer term, the overall LoLX results will assist large LXe-based experiments and motivate the assembly of a LXe-based TOF-PET demonstrator system.Comment: 5 pages, 7 figure

The Fission Yeast XMAP215 Homolog Dis1p Is Involved in Microtubule Bundle Organization

Microtubules are essential for a variety of fundamental cellular processes such as organelle positioning and control of cell shape. Schizosaccharomyces pombe is an ideal organism for studying the function and organization of microtubules into bundles in interphase cells. Using light microscopy and electron tomography we analyzed the bundle organization of interphase microtubules in S. pombe. We show that cells lacking ase1p and klp2p still contain microtubule bundles. In addition, we show that ase1p is the major determinant of inter-microtubule spacing in interphase bundles since ase1 deleted cells have an inter-microtubule spacing that differs from that observed in wild-type cells. We then identified dis1p, a XMAP215 homologue, as factor that promotes the stabilization of microtubule bundles. In wild-type cells dis1p partially co-localized with ase1p at regions of microtubule overlap. In cells deleted for ase1 and klp2, dis1p accumulated at the overlap regions of interphase microtubule bundles. In cells lacking all three proteins, both microtubule bundling and inter-microtubule spacing were further reduced, suggesting that Dis1p contributes to interphase microtubule bundling

BHPR research: qualitative1. Complex reasoning determines patients' perception of outcome following foot surgery in rheumatoid arhtritis

Background: Foot surgery is common in patients with RA but research into surgical outcomes is limited and conceptually flawed as current outcome measures lack face validity: to date no one has asked patients what is important to them. This study aimed to determine which factors are important to patients when evaluating the success of foot surgery in RA Methods: Semi structured interviews of RA patients who had undergone foot surgery were conducted and transcribed verbatim. Thematic analysis of interviews was conducted to explore issues that were important to patients. Results: 11 RA patients (9 ♂, mean age 59, dis dur = 22yrs, mean of 3 yrs post op) with mixed experiences of foot surgery were interviewed. Patients interpreted outcome in respect to a multitude of factors, frequently positive change in one aspect contrasted with negative opinions about another. Overall, four major themes emerged. Function: Functional ability & participation in valued activities were very important to patients. Walking ability was a key concern but patients interpreted levels of activity in light of other aspects of their disease, reflecting on change in functional ability more than overall level. Positive feelings of improved mobility were often moderated by negative self perception ("I mean, I still walk like a waddling duck”). Appearance: Appearance was important to almost all patients but perhaps the most complex theme of all. Physical appearance, foot shape, and footwear were closely interlinked, yet patients saw these as distinct separate concepts. Patients need to legitimize these feelings was clear and they frequently entered into a defensive repertoire ("it's not cosmetic surgery; it's something that's more important than that, you know?”). Clinician opinion: Surgeons' post operative evaluation of the procedure was very influential. The impact of this appraisal continued to affect patients' lasting impression irrespective of how the outcome compared to their initial goals ("when he'd done it ... he said that hasn't worked as good as he'd wanted to ... but the pain has gone”). Pain: Whilst pain was important to almost all patients, it appeared to be less important than the other themes. Pain was predominately raised when it influenced other themes, such as function; many still felt the need to legitimize their foot pain in order for health professionals to take it seriously ("in the end I went to my GP because it had happened a few times and I went to an orthopaedic surgeon who was quite dismissive of it, it was like what are you complaining about”). Conclusions: Patients interpret the outcome of foot surgery using a multitude of interrelated factors, particularly functional ability, appearance and surgeons' appraisal of the procedure. While pain was often noted, this appeared less important than other factors in the overall outcome of the surgery. Future research into foot surgery should incorporate the complexity of how patients determine their outcome Disclosure statement: All authors have declared no conflicts of interes

Reversible solidification of fission yeast cytoplasm after prolonged nutrient starvation

Cells depend on a highly ordered organisation of their content and must develop strategies to maintain the anisotropic distribution of organelles during periods of nutrient shortage. One of these strategies is to solidify the cytoplasm, which was observed in bacteria and yeast cells with acutely interrupted energy production. Here, we describe a different type of cytoplasm solidification fission yeast cells switch to, after having run out of nutrients during multiple days in culture. It provides the most profound reversible cytoplasmic solidification of yeast cells described to date. Our data exclude the previously proposed mechanisms for cytoplasm solidification in yeasts and suggest a mechanism that immobilises cellular components in a size-dependent manner. We provide experimental evidence that, in addition to time, cells use intrinsic nutrients and energy sources to reach this state. Such cytoplasmic solidification may provide a robust means to protect cellular architecture in dormant cells