FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Attention, as a core layer of the ubiquitous Transformer architecture, is the bottleneck for large language models and long-context applications. FlashAttention elaborated an approach to speed up attention on GPUs through minimizing memory reads/writes. However, it has yet to take advantage of new capabilities present in recent hardware, with FlashAttention-2 achieving only 35% utilization on the H100 GPU. We develop three main techniques to speed up attention on Hopper GPUs: exploiting asynchrony of the Tensor Cores and TMA to (1) overlap overall computation and data movement via warp-specialization and (2) interleave block-wise matmul and softmax operations, and (3) block quantization and incoherent processing that leverages hardware support for FP8 low-precision. We demonstrate that our method, FlashAttention-3, achieves speedup on H100 GPUs by 1.5-2.0$\times$ with FP16 reaching up to 740 TFLOPs/s (75% utilization), and with FP8 reaching close to 1.2 PFLOPs/s. We validate that FP8 FlashAttention-3 achieves 2.6$\times$ lower numerical error than a baseline FP8 attention

Differential inhibition of invasion and proliferation by bisphosphonates: Anti-metastatic potential of zoledronic acid in prostate cancer

Objectives: To determine the mode of action of Zoledronic acid in the inhibition of metastasis in prostate cancer and the reduction of prostate cancer bone metastases. Methods: Benign and malignant primary prostatic epithelial cells (PEC) and the PC-3 prostate cancer cell line were studied in co-culture using human bone marrow stroma in the presence of escalating doses of EDTA, Clodronate, Pamidronate and Zoledronic acid. PEC binding and colony growth in bone marrow stroma was measured using standardised quantitative techniques. PEC cellular invasion through Matrigel and an endothelial monolayer was measured either in invasion chambers or by the measurement of endothelial monolayer permeability to fluorescent dextran. Co-culture supernatants were assayed for specific cytokine levels. Bone marrow cellular toxicity was assessed using a standard Mix assay. Results: Treatment of PEC with up to 100 μM bisphosphonate did not affect their ability to bind to bone marrow endothelium or stroma. Bone marrow endothelial permeability was reduced by 100 μM Zoledronic acid by 3.8% (p=0.03856). Both Pamidronate (40% at 100 μM, p≤0.05) and Zoledronic acid inhibited PEC invasion, with Zoledronic acid being the most potent (40% at 10 μM, p≤0.05 rising to 91% at 100 μM, p≤0.001). Zoledronic acid inhibits malignant PEC proliferation in bone marrow stroma co-culture (26.5% at 10 μM rising to 66.5% at 40 μM). This was accompanied by changes within the cytokine milieu with a >800% rise in TIMP-2. Conclusion: Zoledronic acid is a potent inhibitor of PEC invasion across bone marrow endothelium and colony formation with the bone marrow stroma, affecting the MMP: TIMP-2 balance to favour MMP inhibition. © 2004 Elsevier B.V. All rights reserved

Novel Method for the Isolation and Characterisation of the Putative Prostatic Stem Cell

Background: Prostate stem cells, responsible for the development, maturation, and function of the prostate, have been implicated in the aetiology of both benign prostate hyperplasia (BPH) and prostate cancer (CaP). However, research has been hampered by the lack of a definitive stem cell marker. We have adapted the protocol for differential Hoechst 33342 uptake by hemopoietic stem cells to enable isolation of putative stem cells from the prostate. Methods: Prostate epithelial cells isolated from prostate tissue obtained from patients with BPH after transurethral resection of the prostate were stained with Hoechst 33342. The Hoechst 33342 Red/Blue flow cytometry profile was then determined. Hoechst 33342 and Pyronin Y staining was used to determined the cell cycle status. Results: A verapamil-sensitive side population (SP) can be isolated from primary prostate tissue accounting for 1.38% ± 0.07% of prostate epithelial cells. Cell cycle analysis of this SP population revealed that the majority of SP cells are in either G0 (12.38 ± 0.31%) or G1 (63.19 ± 2.13%). Conclusions: The Hoechst 33342 dye efflux protocol can be adapted for the isolation of a SP from primary prostate tissue. © 2003 Wiley-Liss, Inc

Hoechst 33342 side population identification is a conserved and unified mechanism in urological cancers

Mutation within the adult human stem cell (SC) compartment has been proposed as a factor in the initiation and promotion of carcinogenesis. Isolation of these cancer stem cells (CSCs) has proven difficult, limiting their subsequent phenotypic, functional, and genetic characterization. We have used the Hoechst 33342 dye efflux technique to isolate an epithelial side population (SP) from genitourinary (GU) cancers, which is enriched for cells with SC traits. With informed consent, samples were taken from patients with primary tumors and undergoing surgery for prostatic (CaP), invasive bladder transitional cell (TCC), and renal cell carcinomas (RCC). Single cell epithelial suspensions were extracted from these and incubated with Hoechst 33342. Hoechst SP/non-SP profiles were then generated by flow cytometry using standardized protocols. SP/non-SP cell cycle status was established by Hoechst 33342 and Pyronin Y staining. Immunocytochemistry staining was performed for markers suggested as stem markers as well as lineage-specific markers. Functionality was determined using colony-forming assays and long-term monolayer culture. A characteristic verapamil-sensitive SP was isolated from all 3 urological malignancies and represented 0.57% ± 0.11% (CaP), 0.52% ± 0.49% (TCC), and 5.9% ± 0.9% (RCC) of the total epithelial population. Cell cycle analysis showed that the SP had enhanced numbers of cells in G0 as compared to the total cell population (CaP 12.4% ± 3.2 vs. 3.8% ± 1.0, RCC 23.2% ± 3.4 vs. 1.8% ± 0.9, and TCC 28.5% ± 4.9 vs. 4% ± 1.3). Immunocytochemistry demonstrated an increased expression of proliferative and putative stem markers within the SP fraction. Cultures confirmed significant enhancement of colony-forming ability and proliferative capacity of the SP fraction. A characteristic SP enriched for stem-like cells has been isolated from the 3 most common urological malignancies. This provides strong evidence that Hoechst 33342 efflux is a conserved and unified mechanism in GU cancer. © 2009 Mary Ann Liebert, Inc

Aging activates escape of the silent X chromosome in the female mouse hippocampus.

Women live longer than men and exhibit less cognitive aging. The X chromosome contributes to sex differences, as females harbor an inactive X (Xi) and active X (Xa), in contrast to males with only an Xa. Thus, reactivation of silent Xi genes may contribute to sex differences. We use allele-specific, single-nucleus RNA sequencing to show that aging remodels transcription of the Xi and Xa across hippocampal cell types. Aging preferentially changed gene expression on the Xs relative to autosomes. Select genes on the Xi underwent activation, with new escape across cells including in the dentate gyrus, critical to learning and memory. Expression of the Xi escapee Plp1, a myelin component, was increased in the aging hippocampus of female mice and parahippocampus of women. AAV-mediated Plp1 elevation in the dentate gyrus of aging male and female mice improved cognition. Understanding how the Xi may confer female advantage could lead to novel targets that counter brain aging and disease in both sexes

HMGB1 deforms nucleosomal DNA to generate a dynamic chromatin environment counteracting the effects of linker histone

The essential architectural protein HMGB1 increases accessibility of nucleosomal DNA and counteracts the effects of linker histone H1. However, HMGB1 is less abundant than H1 and binds nucleosomes more weakly, raising the question of how it competes with H1. Here, we find that HMGB1 increases nucleosomal DNA accessibility without displacing H1. HMGB1 also increases the dynamics of condensed, H1-bound chromatin. Unexpectedly, cryo-electron microscopy structures show HMGB1 bound at internal locations on nucleosomes and local DNA distortion. These sites are away from where H1 binds, explaining how HMGB1 and H1 can co-occupy a nucleosome. Our findings suggest a model where HMGB1 counteracts the effects of H1 by distorting nucleosomal DNA and disrupting interactions of the H1 carboxyl-terminal tail with DNA. Compared to mutually exclusive binding, co-occupancy by HMGB1 and H1 allows greater diversity in dynamic chromatin states. More generally, these results explain how architectural proteins acting at the nucleosome scale can have large effects on chromatin dynamics at the mesoscale

Agroforestry Systems for Soil Health Improvement and Maintenance

Agroforestry integrates woody perennials with arable crops, livestock, or fodder in the same piece of land, promoting the more efficient utilization of resources as compared to monocropping via the structural and functional diversification of components. This integration of trees provides various soil-related ecological services such as fertility enhancements and improvements in soil physical, biological, and chemical properties, along with food, wood, and fodder. By providing a particular habitat, refugia for epigenic organisms, microclimate heterogeneity, buffering action, soil moisture, and humidity, agroforestry can enhance biodiversity more than monocropping. Various studies confirmed the internal restoration potential of agroforestry. Agroforestry reduces runoff, intercepts rainfall, and binds soil particles together, helping in erosion control. This trade-off between various non-cash ecological services and crop production is not a serious constraint in the integration of trees on the farmland and also provides other important co-benefits for practitioners. Tree-based systems increase livelihoods, yields, and resilience in agriculture, thereby ensuring nutrition and food security. Agroforestry can be a cost-effective and climate-smart farming practice, which will help to cope with the climate-related extremities of dryland areas cultivated by smallholders through diversifying food, improving and protecting soil, and reducing wind erosion. This review highlighted the role of agroforestry in soil improvements, microclimate amelioration, and improvements in productivity through agroforestry, particularly in semi-arid and degraded areas under careful consideration of management practices

Genomic evaluation of multiparametric magnetic resonance imaging-visible and -nonvisible lesions in clinically localised prostate cancer

Background: The prostate cancer (PCa) diagnostic pathway is undergoing a radical change with the introduction of multiparametric magnetic resonance imaging (mpMRI), genomic testing, and different prostate biopsy techniques. It has been proposed that these tests should be used in a sequential manner to optimise risk stratification. Objective: To characterise the genomic, epigenomic, and transcriptomic features of mpMRI-visible and -nonvisible PCa in clinically localised disease. Design, setting, and participants: Multicore analysis of fresh prostate tissue sampled immediately after radical prostatectomy was performed for intermediate- to high-risk PCa. Intervention: Low-pass whole-genome, exome, methylation, and transcriptome profiling of patient tissue cores taken from microscopically benign and cancerous areas in the same prostate. Circulating free and germline DNA was assessed from the blood of five patients. Outcome measurement and statistical analysis: Correlations between preoperative mpMRI and genomic characteristics of tumour and benign prostate samples were assessed. Gene profiles for individual tumour cores were correlated with existing genomic classifiers currently used for prognostication. Results and limitations: A total of 43 prostate cores (22 tumour and 21 benign) were profiled from six whole prostate glands. Of the 22 tumour cores, 16 were tumours visible and six were tumours nonvisible on mpMRI. Intratumour genomic, epigenomic, and transcriptomic heterogeneity was found within mpMRI-visible lesions. This could potentially lead to misclassification of patients using signatures based on copy number or RNA expression. Moreover, three of the six cores obtained from mpMRI-nonvisible tumours harboured one or more genetic alterations commonly observed in metastatic castration-resistant PCa. No circulating free DNA alterations were found. Limitations include the small cohort size and lack of follow-up. Conclusions: Our study supports the continued use of systematic prostate sampling in addition to mpMRI, as avoidance of systematic biopsies in patients with negative mpMRI may mean that clinically significant tumours harbouring genetic alterations commonly seen in metastatic PCa are missed. Furthermore, there is inconsistency in individual genomics when genomic classifiers are applied. Patient summary: Our study shows that tumour heterogeneity within prostate tumours visible on multiparametric magnetic resonance imaging (mpMRI) can lead to misclassification of patients if only one core is used for genomic analysis. In addition, some cancers that were missed by mpMRI had genomic aberrations that are commonly seen in advanced metastatic prostate cancer. Avoiding biopsies in mpMRI-negative cases may mean that such potentially lethal cancers are missed