XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model

We present XMem, a video object segmentation architecture for long videos with unified feature memory stores inspired by the Atkinson-Shiffrin memory model. Prior work on video object segmentation typically only uses one type of feature memory. For videos longer than a minute, a single feature memory model tightly links memory consumption and accuracy. In contrast, following the Atkinson-Shiffrin model, we develop an architecture that incorporates multiple independent yet deeply-connected feature memory stores: a rapidly updated sensory memory, a high-resolution working memory, and a compact thus sustained long-term memory. Crucially, we develop a memory potentiation algorithm that routinely consolidates actively used working memory elements into the long-term memory, which avoids memory explosion and minimizes performance decay for long-term prediction. Combined with a new memory reading mechanism, XMem greatly exceeds state-of-the-art performance on long-video datasets while being on par with state-of-the-art methods (that do not work on long videos) on short-video datasets. Code is available at https://hkchengrex.github.io/XMemComment: Accepted to ECCV 2022. Project page: https://hkchengrex.github.io/XMe

Inter-individual deep image reconstruction via hierarchical neural code conversion

The sensory cortex is characterized by general organizational principles such as topography and hierarchy. However, measured brain activity given identical input exhibits substantially different patterns across individuals. Although anatomical and functional alignment methods have been proposed in functional magnetic resonance imaging (fMRI) studies, it remains unclear whether and how hierarchical and fine-grained representations can be converted between individuals while preserving the encoded perceptual content. In this study, we trained a method of functional alignment called neural code converter that predicts a target subject’s brain activity pattern from a source subject given the same stimulus, and analyzed the converted patterns by decoding hierarchical visual features and reconstructing perceived images. The converters were trained on fMRI responses to identical sets of natural images presented to pairs of individuals, using the voxels on the visual cortex that covers from V1 through the ventral object areas without explicit labels of the visual areas. We decoded the converted brain activity patterns into the hierarchical visual features of a deep neural network using decoders pre-trained on the target subject and then reconstructed images via the decoded features. Without explicit information about the visual cortical hierarchy, the converters automatically learned the correspondence between visual areas of the same levels. Deep neural network feature decoding at each layer showed higher decoding accuracies from corresponding levels of visual areas, indicating that hierarchical representations were preserved after conversion. The visual images were reconstructed with recognizable silhouettes of objects even with relatively small numbers of data for converter training. The decoders trained on pooled data from multiple individuals through conversions led to a slight improvement over those trained on a single individual. These results demonstrate that the hierarchical and fine-grained representation can be converted by functional alignment, while preserving sufficient visual information to enable inter-individual visual image reconstruction

Putting the Object Back into Video Object Segmentation

We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentation result. Recent works on VOS employ bottom-up pixel-level memory reading which struggles due to matching noise, especially in the presence of distractors, resulting in lower performance in more challenging data. In contrast, Cutie performs top-down object-level memory reading by adapting a small set of object queries for restructuring and interacting with the bottom-up pixel features iteratively with a query-based object transformer (qt, hence Cutie). The object queries act as a high-level summary of the target object, while high-resolution feature maps are retained for accurate segmentation. Together with foreground-background masked attention, Cutie cleanly separates the semantics of the foreground object from the background. On the challenging MOSE dataset, Cutie improves by 8.7 J&F over XMem with a similar running time and improves by 4.2 J&F over DeAOT while running three times as fast. Code is available at: https://hkchengrex.github.io/CutieComment: Project page: https://hkchengrex.github.io/Cuti

Nonlocal Particles as Strings

We find nonlocal particle theories with two dimensional conformal symmetry, including examples equivalent to the bosonic open string and closed string. This work provides a new approach to construct solvable consistent backgrounds in string theory.Comment: 25 pages, Latex, minor change

Associations among stressors, perceived stress, and psychological distress in nursing students: a mixed methods longitudinal study of a Hong Kong sample

BackgroundNursing students are at risk for high-stress levels and psychological distress. Limited longitudinal studies have been conducted examining factors associated with stress levels and psychological distress of nursing students in their course of study.PurposeThe purpose of this study was to examine the levels of stress and corresponding stressors, particularly those predicting psychological distress, among nursing students over their 5 years of study.MethodsA longitudinal design, using questionnaires and focus group interviews of a single cohort of nursing students in Hong Kong and following them over their 5 years of training. The Stressors in Nursing Students Scale-Chinese version and the Chinese version of General Health Questionnaire-12 were used to assess stress levels and psychological distress, respectively.ResultsNinety-seven participants completed the questionnaires 5 times. Quantitative findings revealed that the overall stress levels of the nursing students increased over 5 years (from mean = 3.08 to 3.33), with the highest levels in the second wave (mean = 3.33). Nursing students experienced higher stress during years 2 (p = 0.006) and 4 (p = 0.037). Psychological distress was the highest in year 3 (sum score = 18.47) (p = 0.002) but declined from year 4 (p < 0.001). Thematic analysis revealed that academic performance issues, coping challenges, unfavorable learning environments, relationships were identified as the stressors. However, nursing students also used positive coping strategies to pursue success and seek support.ConclusionThis study suggests that the year of study is a significant predictor of stress levels among nursing students, especially during the first and senior years due to heavy academic workload. Psychological distress was observed among nursing students, and those who worked more part-time jobs tended to report higher levels of distress. The junior year was associated with higher levels of distress related to financial and time-related stress, while academic and personal problems were more prevalent during the senior year

Association of Genetic Variants Related to Combined Exposure to Higher Body Mass Index and Waist-to-Hip Ratio on Lifelong Cardiovascular Risk in UK Biobank

OBJECTIVE: This study examines the individual and combined association of body mass index (BMI) and 7 waist-to-hip ratio (WHR) with cardiovascular diseases (CVD) risk using genetic scores of the 8 obesity measurements as proxies. DESIGN: A 2×2 factorial analysis approach was applied, with participants divided into four groups of lifetime exposure to low BMI and WHR, high BMI, high WHR, and high BMI and WHR based on weighted genetic risk scores. The difference in CVD risk across groups was evaluated using multivariable logistic regression. SETTING: Cohort study. PARTICIPANTS: A total of 408,003 participants were included from the prospective observational UK Biobank study. RESULTS: A total of 58,429 of CVD events were recorded. Compared to the low BMI and WHR genetic scores group, higher BMI or higher WHR genetic scores were associated with an increase in CVD risk (high BMI: odds ratio (OR), 1.07; 95%CI, 1.04-1.10; high WHR: OR, 1.12; 95%CI, 1.09-1.16). A weak additive effect on CVD risk was found between BMI and WHR (high BMI and WHR: OR, 1.16; 95%CI, 1.12-1.19). Subgroup analysis showed similar patterns between different sex, age (<65, ≥65 years old), smoking status, Townsend deprivation index, fasting glucose level and medication uses, but lower systolic blood pressure was associated with higher CVD risk in obese participants. CONCLUSIONS: High BMI or WHR were associated with increased CVD risk, and their effects are weakly additive. Even though there were overlapping of effect, both BMI and WHR are important in assessing the CVD risk in the general population

Risk of thyroid dysfunction associated with mRNA and inactivated COVID-19 vaccines: a population-based study of 2.3 million vaccine recipients

Background: In view of accumulating case reports of thyroid dysfunction following COVID-19 vaccination, we evaluated the risks of incident thyroid dysfunction following inactivated (CoronaVac) and mRNA (BNT162b2) COVID-19 vaccines using a population-based dataset. / Methods: We identified people who received COVID-19 vaccination between 23 February and 30 September 2021 from a population-based electronic health database in Hong Kong, linked to vaccination records. Thyroid dysfunction encompassed anti-thyroid drug (ATD)/levothyroxine (LT4) initiation, biochemical picture of hyperthyroidism/hypothyroidism, incident Graves’ disease (GD), and thyroiditis. A self-controlled case series design was used to estimate the incidence rate ratio (IRR) of thyroid dysfunction in a 56-day post-vaccination period compared to the baseline period (non-exposure period) using conditional Poisson regression. / Results: A total of 2,288,239 people received at least one dose of COVID-19 vaccination (57.8% BNT162b2 recipients and 42.2% CoronaVac recipients). 94.3% of BNT162b2 recipients and 92.2% of CoronaVac recipients received the second dose. Following the first dose of COVID-19 vaccination, there was no increase in the risks of ATD initiation (BNT162b2: IRR 0.864, 95% CI 0.670–1.114; CoronaVac: IRR 0.707, 95% CI 0.549–0.912), LT4 initiation (BNT162b2: IRR 0.911, 95% CI 0.716–1.159; CoronaVac: IRR 0.778, 95% CI 0.618–0.981), biochemical picture of hyperthyroidism (BNT162b2: IRR 0.872, 95% CI 0.744–1.023; CoronaVac: IRR 0.830, 95% CI 0.713–0.967) or hypothyroidism (BNT162b2: IRR 1.002, 95% CI 0.838–1.199; CoronaVac: IRR 0.963, 95% CI 0.807–1.149), GD, and thyroiditis. Similarly, following the second dose of COVID-19 vaccination, there was no increase in the risks of ATD initiation (BNT162b2: IRR 0.972, 95% CI 0.770–1.227; CoronaVac: IRR 0.879, 95%CI 0.693–1.116), LT4 initiation (BNT162b2: IRR 1.019, 95% CI 0.833–1.246; CoronaVac: IRR 0.768, 95% CI 0.613–0.962), hyperthyroidism (BNT162b2: IRR 1.039, 95% CI 0.899–1.201; CoronaVac: IRR 0.911, 95% CI 0.786–1.055), hypothyroidism (BNT162b2: IRR 0.935, 95% CI 0.794–1.102; CoronaVac: IRR 0.945, 95% CI 0.799–1.119), GD, and thyroiditis. Age- and sex-specific subgroup and sensitivity analyses showed consistent neutral associations between thyroid dysfunction and both types of COVID-19 vaccines. / Conclusions: Our population-based study showed no evidence of vaccine-related increase in incident hyperthyroidism or hypothyroidism with both BNT162b2 and CoronaVac