Causal Interpretation of Self-Attention in Pre-Trained Transformers

We propose a causal interpretation of self-attention in the Transformer neural network architecture. We interpret self-attention as a mechanism that estimates a structural equation model for a given input sequence of symbols (tokens). The structural equation model can be interpreted, in turn, as a causal structure over the input symbols under the specific context of the input sequence. Importantly, this interpretation remains valid in the presence of latent confounders. Following this interpretation, we estimate conditional independence relations between input symbols by calculating partial correlations between their corresponding representations in the deepest attention layer. This enables learning the causal structure over an input sequence using existing constraint-based algorithms. In this sense, existing pre-trained Transformers can be utilized for zero-shot causal-discovery. We demonstrate this method by providing causal explanations for the outcomes of Transformers in two tasks: sentiment classification (NLP) and recommendation.Comment: 37th Conference on Neural Information Processing Systems (NeurIPS 2023). arXiv admin note: text overlap with arXiv:2210.1062

From Temporal to Contemporaneous Iterative Causal Discovery in the Presence of Latent Confounders

We present a constraint-based algorithm for learning causal structures from observational time-series data, in the presence of latent confounders. We assume a discrete-time, stationary structural vector autoregressive process, with both temporal and contemporaneous causal relations. One may ask if temporal and contemporaneous relations should be treated differently. The presented algorithm gradually refines a causal graph by learning long-term temporal relations before short-term ones, where contemporaneous relations are learned last. This ordering of causal relations to be learnt leads to a reduction in the required number of statistical tests. We validate this reduction empirically and demonstrate that it leads to higher accuracy for synthetic data and more plausible causal graphs for real-world data compared to state-of-the-art algorithms.Comment: Proceedings of the 40-th International Conference on Machine Learning (ICML), 202

Private Education in the Absence of a Public Option: The Cases of the United Arab Emirates and Qatar

In the face of rising demand for private schooling in the United Arab Emirates (UAE) and Qatar, a lack of affordable schooling options, monopolistic behavior of private education providers, and unpredictable government regulations have created a complex and unequal education sector. This research employs a mixed methods comparative approach to explore the ways in which private education providers navigate the regulatory schooling environments and assess the impact on education stakeholders in the UAE and Qatar. The study finds that there are considerable socioeconomic differences in terms of who has access to schooling and that a growing for-profit education sector may be deepening existing inequities in both countries, leaving poorer expatriate families only able to access low-quality education or in the worst cases, unable to access education at all. The promise of non-profit providers as a viable alternative to ensure access is explored

Modern Breast Cancer Detection: A Technological Review

Breast cancer is a serious threat worldwide and is the number two killer of women in the United States. The key to successful management is screening and early detection. What follows is a description of the state of the art in screening and detection for breast cancer as well as a discussion of new and emerging technologies. This paper aims to serve as a starting point for those who are not acquainted with this growing field

Consanguinity decreases risk of breast cancer – cervical cancer unaffected

Marriages between third-degree and more distant relatives are common in many parts of the world. Offspring of consanguineous parents have increased morbidity and mortality related to recessive gene disorders. In a population with a high frequency of consanguinity, we examined the frequency of breast cancer (related in part to tumour genes) and cervical cancers (related to virus infection) among offspring of consanguineous and non-consanguineous parents. Study was done prospectively in the United Arab Emirates. Selected were married female citizens, ages 40–65, who attended 12 primary health care clinics for whatever reason. In a face-to-face interview, subjects were asked: (a) about consanguineous marriages in family; (b) if they have or have had breast or cervical cancer; (c) about family history of cancer, cancer screening and other parameters. Tumour diagnosis was confirmed by review of medical records. Of 1750 women invited into study, 1445 (79%) could be used in analysis. Among 579 (40%) women of consanguineous and 866 (60%) of non-consanguineous parents there were 24 and 54 with breast cancer, respectively (RR = 0.66, CI 0.42 – 1.06). In the 40 to 50 age group, breast cancer reported 13 of 446 women of consanguineous and 37 of 633 of non-consanguineous parents (RR = 0.50, Cl 0.27 – 0.93). Cervical cancer had 15 women in consanguineous and 32 in non-consanguineous group (RR = 0.70, Cl 0.38 – 1.28). Number of families with history of breast cancer in consanguineous and non-consanguineous group was 21 and 23, respectively (P = 0.29). The cancer screening rates and other variable values had fairly balanced distribution between the 2 groups. Having consanguineous parents decreases the risk of breast cancer especially in younger women, risk of cervical cancer being unaffected. © 2001 Cancer Research Campaign http://www.bjcancer.co

Expression of Osteopontin in oesophageal squamous cell carcinoma

Osteopontin is a multifunctional 34 kDa extracellular matrix protein with a cell-binding domain. It is involved cell adhesion and cell migration and is therefore considered to influence tumorigenesis and/or metastasis. The purpose of the present study was to evaluate the clinical significance of Osteopontin expression in oesophageal squamous cell carcinoma (ESCC). In the present study, we immunohistochemically investigated the relationship between Osteopontin expression and clinicopathological factors including prognosis in surgical specimens of primary tumours in 175 patients with ESCC. Osteopontin was expressed in 48% of 175 patients. Osteopontin expression was significantly correlated with lymph node metastasis, lymphatic invasion, and stage (P=0.0015, 0.037 and 0.033, respectively). Tumours with expressing Osteopontin exhibited more lymph node metastasis, lymphatic invasion and advanced stage than the tumour with negative Osteopontin expression. Five-year survival rate was better in patients with negative Osteopontin expression than in those with positive Osteopontin expression (P=0.035). However, multivariate analysis revealed that Osteopontin expression was not an independent prognostic factor. As our findings suggest that Osteopontin may play an important role in progress of ESCC, the evaluation of Osteopontin expression is useful for predicting the malignant properties of ESCC