Unified Normalization for Accelerating and Stabilizing Transformers

Solid results from Transformers have made them prevailing architectures in various natural language and vision tasks. As a default component in Transformers, Layer Normalization (LN) normalizes activations within each token to boost the robustness. However, LN requires on-the-fly statistics calculation in inference as well as division and square root operations, leading to inefficiency on hardware. What is more, replacing LN with other hardware-efficient normalization schemes (e.g., Batch Normalization) results in inferior performance, even collapse in training. We find that this dilemma is caused by abnormal behaviors of activation statistics, including large fluctuations over iterations and extreme outliers across layers. To tackle these issues, we propose Unified Normalization (UN), which can speed up the inference by being fused with other linear operations and achieve comparable performance on par with LN. UN strives to boost performance by calibrating the activation and gradient statistics with a tailored fluctuation smoothing strategy. Meanwhile, an adaptive outlier filtration strategy is applied to avoid collapse in training whose effectiveness is theoretically proved and experimentally verified in this paper. We demonstrate that UN can be an efficient drop-in alternative to LN by conducting extensive experiments on language and vision tasks. Besides, we evaluate the efficiency of our method on GPU. Transformers equipped with UN enjoy about 31% inference speedup and nearly 18% memory reduction. Code will be released at https://github.com/hikvision-research/Unified-Normalization.Comment: ACM MM'2

The social determinants of chronic disease management: perspectives of elderly patients with hypertension from low socio-economic background in Singapore

Background In Singapore, the burden of hypertension disproportionately falls on the elderly population of low socio-economic status. Despite availability of effective treatment, studies have shown high prevalence of sub-optimal blood pressure control in this group. Poor hypertension management can be attributed to a number of personal factors including awareness, management skills and overall adherence to treatment. However, these factors are also closely linked to a broader range of community and policy factors. This paper explores the perceived social and physical environments of low socio-economic status and elderly patients with hypertension; and how the interplay of factors within these environments influences their ability to mobilise resources for hypertension management. Methods In-depth interviews were conducted in English, Chinese, Chinese dialects and Malay with 20 hypertensive patients of various ethnic backgrounds. Purposive sampling was adopted for recruitment of participants from a previous community health screening campaign. Interviews were translated into English and transcribed verbatim. We deductively analysed leveraging on the Social Model of Health to identify key themes, while inductive analysis was used simultaneously to allow sub-themes to emerge. Results and discussion Our finding shows that financing is an overarching topic embedded in most themes. Despite the availability of multiple safety nets, some patients were left out and lacked capital to navigate systems effectively, which resulted in delayed treatment or debt. The built environment played a significant role in enabling patients to access care easily and lead a more active lifestyle. A closer look is needed to enhance the capacity of patients with mobility challenges to enjoy equitable access. Furthermore, the establishment of community based elderly centres has enabled patients to engage in meaningful and healthy social activities. In contrast, participants' descriptions showed that their communication with healthcare professionals remained brief, and that personalised and meaningful interactions that are context and culturally specific are essential to advocate for patients' overall treatment adherence and lifestyle modification. Conclusion Elderly patients with hypertension from lower socio-economic background have various unmet needs in managing their hypertension and other comorbidities. These needs are closely related to broader societal factors such as socio-demographic characteristics, support systems, urban planning and public policies, and health systems factors. Policy decisions to address these needs require an integrated multi-sectoral approach grounded in the principles of health equity.This research was supported by MOE TIER - 1 FUND R-608-000-133-112 and the National University Health System (NUHS) Singapore Population Health Improvement Centre (SPHERiC)

Jejunojejunal intussusception secondary to inflammatory fibroid polyp: A rare cause of small bowel obstruction

Background: Adult intussusception is a relatively rare clinical entity. The majority of cases of intussusception in adults are due to a pathologic condition that serves as a lead point and requires surgery. Small bowel intussusception is usually caused by benign or malignant neoplasms appearing at the head of the invagination. Inflammatory fibroid polyp (IFP) of the small bowel is an unusual benign neoplastic lesion that has been rarely reported to cause intussusception, especially in the jejunum. Case presentation: We present a rare case of adult intussusception who presented with a triad of intestinal obstruction. Computed tomography revealed small bowel intussusception with bowel ischemia. Intraoperatively, she required resection of the small bowel and primary anastomosis. Macroscopic examination revealed a single pedunculated polyp, which is the lead point of intestinal obstruction and confirmed histologically. Conclusion: Inflammatory fibroid polyp should be considered as a cause of intussusception among adults with small bowel obstruction

High-speed measurement-device-independent quantum key distribution with integrated silicon photonics

Measurement-device-independent quantum key distribution (MDI-QKD) removes all detector side channels and enables secure QKD with an untrusted relay. It is suitable for building a star-type quantum access network, where the complicated and expensive measurement devices are placed in the central untrusted relay and each user requires only a low-cost transmitter, such as an integrated photonic chip. Here, we experimentally demonstrate a 1.25 GHz silicon photonic chip-based MDI-QKD system using polarization encoding. The photonic chip transmitters integrate the necessary encoding components for a standard QKD source. We implement random modulations of polarization states and decoy intensities, and demonstrate a finite-key secret rate of 31 bps over 36 dB channel loss (or 180 km standard fiber). This key rate is higher than state-of-the-art MDI-QKD experiments. The results show that silicon photonic chip-based MDI-QKD, benefiting from miniaturization, low-cost manufacture and compatibility with CMOS microelectronics, is a promising solution for future quantum secure networks.Comment: 30 pages, 12 figure

Depression, anxiety and insomnia in Chinese older adults and their family caregivers during the COVID-19 pandemic: an actor-partner interdependence model approach

AimsThis study aimed to explore the dyadic effects of depression and anxiety on insomnia symptoms in Chinese older adults and their caregivers living in a community setting.MethodsData were collected from 1,507 pairs of older adults and their caregivers who were in the Guangdong Mental Health Survey in China. The 9-item Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder module 7 (GAD-7), and Insomnia Severity Index (ISI) were used to measure depression, anxiety, and insomnia symptoms. Actor-Partner Interdependence Models (APIM) were used to determine whether anxiety or depression symptoms predicted individual or dyadic insomnia.ResultsOlder adults' and caregivers' depression and anxiety had significant positive correlations with their own and their caregivers' insomnia symptoms (all P < 0.001). Actor effects were found between depression and insomnia symptoms in both older adults and caregivers (B = 0.695, P < 0.001; B = 0.547, P < 0.001, respectively), with one significant partner effects (B = 0.080, P = 0.007). Actor effects were also found between anxiety and insomnia symptoms in both older adults and caregivers (B = 0.825, P < 0.001; B = 0.751, P < 0.001, respectively), with one significant partner effects (B = 0.097, P = 0.004). However, the caregivers' depression and anxiety were not associated with older adults' insomnia symptoms in the APIM analyses.ConclusionsOlder adults and their caregivers had an interrelationship between psychological distress and insomnia. Consequently, healthcare providers might consider involving dyads when designing programs to reduce insomnia and improve psychological distress for family caregivers