Enhancing Transformers without Self-supervised Learning: A Loss Landscape Perspective in Sequential Recommendation

Transformer and its variants are a powerful class of architectures for sequential recommendation, owing to their ability of capturing a user's dynamic interests from their past interactions. Despite their success, Transformer-based models often require the optimization of a large number of parameters, making them difficult to train from sparse data in sequential recommendation. To address the problem of data sparsity, previous studies have utilized self-supervised learning to enhance Transformers, such as pre-training embeddings from item attributes or contrastive data augmentations. However, these approaches encounter several training issues, including initialization sensitivity, manual data augmentations, and large batch-size memory bottlenecks. In this work, we investigate Transformers from the perspective of loss geometry, aiming to enhance the models' data efficiency and generalization in sequential recommendation. We observe that Transformers (e.g., SASRec) can converge to extremely sharp local minima if not adequately regularized. Inspired by the recent Sharpness-Aware Minimization (SAM), we propose SAMRec, which significantly improves the accuracy and robustness of sequential recommendation. SAMRec performs comparably to state-of-the-art self-supervised Transformers, such as S$^3$Rec and CL4SRec, without the need for pre-training or strong data augmentations

Senescence atlas reveals an aged-like inflamed niche that blunts muscle regeneration.

Tissue regeneration requires coordination between resident stem cells and local niche cells1,2. Here we identify that senescent cells are integral components of the skeletal muscle regenerative niche that repress regeneration at all stages of life. The technical limitation of senescent-cell scarcity3 was overcome by combining single-cell transcriptomics and a senescent-cell enrichment sorting protocol. We identified and isolated different senescent cell types from damaged muscles of young and old mice. Deeper transcriptome, chromatin and pathway analyses revealed conservation of cell identity traits as well as two universal senescence hallmarks (inflammation and fibrosis) across cell type, regeneration time and ageing. Senescent cells create an aged-like inflamed niche that mirrors inflammation associated with ageing (inflammageing4) and arrests stem cell proliferation and regeneration. Reducing the burden of senescent cells, or reducing their inflammatory secretome through CD36 neutralization, accelerates regeneration in young and old mice. By contrast, transplantation of senescent cells delays regeneration. Our results provide a technique for isolating in vivo senescent cells, define a senescence blueprint for muscle, and uncover unproductive functional interactions between senescent cells and stem cells in regenerative niches that can be overcome. As senescent cells also accumulate in human muscles, our findings open potential paths for improving muscle repair throughout life.We thank M. Jardí, A. Navarro, J. M. Ballestero, K. Slobodnyuk, M. González, J. López and M. Raya for their technical contributions; A. Harada and K. Tanaka for assistance in ATAC-seq; all of the members of the P.M.-C. laboratory for discussions; J. Campisi for p16-3MR mice; J. A. Fernández-Blanco (PRBB Animal Facility); O. Fornas (UPF/CRG FACS Facility); E. Rebollo (IBMB Molecular Imaging Platform); V. A. Raker for manuscript editing; and the members of the Myoage network (A. Maier) for human material. We acknowledge funding from MINECO-Spain (RTI2018-096068, to P.M.-C. and E.P.); ERC-2016-AdG-741966, LaCaixa-HEALTHHR17-00040, MDA, UPGRADE-H2020-825825, AFM, DPP-Spain, Fundació La MaratóTV3-80/19- 202021 and MWRF to P.M.-C.; Fundació La MaratóTV3-137/38-202033 to A.L.S.; Maria-de-Maeztu ́ Program for Units of Excellence to UPF (MDM-2014-0370) and Severo-Ochoa Program for Centers of Excellence to CNIC (SEV-2015-0505). This work was also supported by JST-CREST JPMJCR16G1 and MEXT/JSPS JP20H00456/18H05527 to Y.O.; the Strategic Priority Research Program of the Chinese Academy of Sciences (XDA16030502) to M.A.E.; V.M. and A.C. were supported by FPI and Maria-de-Maeztu predoctoral fellowships, respectively, and V.S. by a Marie Skłodowska-Curie individual fellowship. Parts of the figures were drawn using pictures from Servier Medical Art. Servier Medical Art by Servier is licensed under a Creative Commons Attribution 3.0 Unported License (https://creativecommons.org/licences/by/3.0/).S

Spatiotemporal transcriptomic atlas of mouse organogenesis using DNA nanoball-patterned arrays.

Spatially resolved transcriptomic technologies are promising tools to study complex biological processes such as mammalian embryogenesis. However, the imbalance between resolution, gene capture, and field of view of current methodologies precludes their systematic application to analyze relatively large and three-dimensional mid- and late-gestation embryos. Here, we combined DNA nanoball (DNB)-patterned arrays and in situ RNA capture to create spatial enhanced resolution omics-sequencing (Stereo-seq). We applied Stereo-seq to generate the mouse organogenesis spatiotemporal transcriptomic atlas (MOSTA), which maps with single-cell resolution and high sensitivity the kinetics and directionality of transcriptional variation during mouse organogenesis. We used this information to gain insight into the molecular basis of spatial cell heterogeneity and cell fate specification in developing tissues such as the dorsal midbrain. Our panoramic atlas will facilitate in-depth investigation of longstanding questions concerning normal and abnormal mammalian development.This work is part of the ‘‘SpatioTemporal Omics Consortium’’ (STOC) paper package. A list of STOC members is available at: http://sto-consortium.org. We would like to thank the MOTIC China Group, Rongqin Ke (Huaqiao University, Xiamen, China), Jiazuan Ni (Shenzhen University, Shenzhen, China), Wei Huang (Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China), and Jonathan S. Weissman (Whitehead Institute, Boston, USA) for their help. This work was supported by the grant of Top Ten Foundamental Research Institutes of Shenzhen, the Shenzhen Key Laboratory of Single-Cell Omics (ZDSYS20190902093613831), and the Guangdong Provincial Key Laboratory of Genome Read and Write (2017B030301011); Longqi Liu was supported by the National Natural Science Foundation of China (31900466) and Miguel A. Esteban’s laboratory at the Guangzhou Institutes of Biomedicine and Health by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDA16030502), National Natural Science Foundation of China (92068106), and the Guangdong Basic and Applied Basic Research Foundation (2021B1515120075).S

Cell transcriptomic atlas of the non-human primate Macaca fascicularis.

Studying tissue composition and function in non-human primates (NHPs) is crucial to understand the nature of our own species. Here we present a large-scale cell transcriptomic atlas that encompasses over 1 million cells from 45 tissues of the adult NHP Macaca fascicularis. This dataset provides a vast annotated resource to study a species phylogenetically close to humans. To demonstrate the utility of the atlas, we have reconstructed the cell-cell interaction networks that drive Wnt signalling across the body, mapped the distribution of receptors and co-receptors for viruses causing human infectious diseases, and intersected our data with human genetic disease orthologues to establish potential clinical associations. Our M. fascicularis cell atlas constitutes an essential reference for future studies in humans and NHPs.We thank W. Liu and L. Xu from the Huazhen Laboratory Animal Breeding Centre for helping in the collection of monkey tissues, D. Zhu and H. Li from the Bioland Laboratory (Guangzhou Regenerative Medicine and Health Guangdong Laboratory) for technical help, G. Guo and H. Sun from Zhejiang University for providing HCL and MCA gene expression data matrices, G. Dong and C. Liu from BGI Research, and X. Zhang, P. Li and C. Qi from the Guangzhou Institutes of Biomedicine and Health for experimental advice or providing reagents. This work was supported by the Shenzhen Basic Research Project for Excellent Young Scholars (RCYX20200714114644191), Shenzhen Key Laboratory of Single-Cell Omics (ZDSYS20190902093613831), Shenzhen Bay Laboratory (SZBL2019062801012) and Guangdong Provincial Key Laboratory of Genome Read and Write (2017B030301011). In addition, L.L. was supported by the National Natural Science Foundation of China (31900466), Y. Hou was supported by the Natural Science Foundation of Guangdong Province (2018A030313379) and M.A.E. was supported by a Changbai Mountain Scholar award (419020201252), the Strategic Priority Research Program of the Chinese Academy of Sciences (XDA16030502), a Chinese Academy of Sciences–Japan Society for the Promotion of Science joint research project (GJHZ2093), the National Natural Science Foundation of China (92068106, U20A2015) and the Guangdong Basic and Applied Basic Research Foundation (2021B1515120075). M.L. was supported by the National Key Research and Development Program of China (2021YFC2600200).S

Impact and Post-Impact Performance of Sandwich Wall Boards with GFRP Face Sheets and a Web-Foam Core: The Effects of Impact Location

In recent years, load-bearing exterior sandwich wall boards have been adopted in civil engineering. The exterior walls of structures are often exposed to low velocity impacts such as stones, tools, and windborne debris, etc. The ultimate loading capacity, deformation, and ductility of sandwich walls are weakened by impact loads. In this study, the sandwich wall boards consisted of glass fiber reinforced plastic (GFRP) face sheets and a web-foam core. The core of wall boards was not the isotropic material. There was no doubt that the mechanical performance was seriously influenced by the impact locations. Therefore, it is necessary to carry out an investigation on the impact and post-impact performance of exterior wall boards. A comprehensive testing program was conducted to evaluate the effects of impact locations and impact energies on the maximum contact load, deflection, and contact time. Meanwhile, the compression after impact (CAI) performance of wall boards were also studied. The results indicated that the impact location significantly affects the performance of wall boards. Compared with an un-damaged wall board, the residual ultimate loading capacity of damaged wall boards reduced seriously, which were not larger than 50% of the designed ultimate loading capacity

Phase Synchronization Stability of Non-Homogeneous Low-Voltage Distribution Networks with Large-Scale Distributed Generations

The ideal distributed network composed of distributed generations (DGs) has unweighted and undirected interactions which omit the impact of the power grid structure and actual demand. Apparently, the coupling relationship between DGs, which is determined by line impedance, node voltage, and droop coefficient, is generally non-homogeneous. Motivated by this, this paper investigates the phase synchronization of an islanded network with large-scale DGs in a non-homogeneous condition. Furthermore, we explicitly deduce the critical coupling strength formula for different weighting cases via the synchronization condition. On this basis, three cases of Gaussian distribution, power-law distribution, and frequency-weighted distribution are analyzed. A synthetical analysis is also presented, which helps to identify the order parameter. Finally, this paper employs the numerical simulation methods to test the effectiveness of the critical coupling strength formula and the superiority over the power-law distribution

Effectiveness of mHealth interventions for improving hypertension control in uncontrolled hypertensive patients: A meta‐analysis of randomized controlled trials

Abstract The benefits of mHealth interventions in uncontrolled hypertension are unclear. To determine whether mHealth effectively improves the control rate of uncontrolled hypertension. PubMed, Web of Science, EMBASE, Scopus, and Cochrane Library were searched for randomized controlled trials (RCTs) from January 2007 to September 2022. The intervention group consisted of mHealth intervention, and the control group was usual care. Random‐effects meta‐analysis models were used to assess pooled mHealth intervention effects and CIs. The primary outcome was the blood pressure (BP) control rate of uncontrolled hypertension. The secondary outcome was the change of BP. Thirteen RCTs were included in this meta‐analysis, of which eight reported the successful BP control rate, 13 reported the change of systolic blood pressure (SBP), and 11 reported the change in diastolic blood pressure (DBP). The mean age of trial participants ranged from 47.7 to 66.9 years old, with a female composition ratio of 40.0%–66.1%. The duration of follow‐up ranged from 3 to 18 months. This study showed a more robust effect size for improving BP control rate by mHealth interventions than usual care (57.5% vs. 40.8% of successful control rate; odds ratio [OR], 2.19 [95% CI, 1.32—3.62]). Furthermore, mHealth significantly reduced SBP by 4.45 mm Hg and DBP by 2.47 mm Hg, and subgroup analysis did not observe the major source of heterogeneity. This meta‐analysis found that mHealth could significantly improve the uncontrolled hypertension control rate and might be a feasible, acceptable, and effective tool for uncontrolled hypertension management

PARCS: A Deployment-Oriented AI System for Robust Parcel-Level Cropland Segmentation of Satellite Images

Cropland segmentation of satellite images is an essential basis for crop area and yield estimation tasks in the remote sensing and computer vision interdisciplinary community. Instead of common pixel-level segmentation results with salt-and-pepper effects, a parcel-level output conforming to human recognition is required according to the clients' needs during the model deployment. However, leveraging CNN-based models requires fine-grained parcel-level labels, which is an unacceptable annotation burden. To cure these practical pain points, in this paper, we present PARCS, a holistic deployment-oriented AI system for PARcel-level Cropland Segmentation. By consolidating multi-disciplinary knowledge, PARCS has two algorithm branches. The first branch performs pixel-level crop segmentation by learning from limited labeled pixel samples with an active learning strategy to avoid parcel-level annotation costs. The second branch aims at generating the parcel regions without a learning procedure. The final parcel-level segmentation result is achieved by integrating the outputs of these two branches in tandem. The robust effectiveness of PARCS is demonstrated by its outstanding performance on public and in-house datasets (an overall accuracy of 85.3% and an mIoU of 61.7% on the public PASTIS dataset, and an mIoU of 65.16% on the in-house dataset). We also include subjective feedback from clients and discuss the lessons learned from deployment

Integral experiments on thorium assemblies with D-T neutron source

To validate nuclear data and code in the neutronics design of a hybrid reactor with thorium, integral experiments in two kinds of benchmark thorium assemblies with a D-T fusion neutron source have been performed. The one kind of 1D assemblies consists of polyethylene and depleted uranium shells. The other kind of 2D assemblies consists of three thorium oxide cylinders. The capture reaction rates, fission reaction rates, and (n, 2n) reaction rates in 232Th in the assemblies are measured by ThO2 foils. The leakage neutron spectra from the ThO2 cylinders are measured by a liquid scintillation detector. The experimental uncertainties in all the results are analyzed. The measured results are compared to the calculated ones with MCNP code and ENDF/B-VII.0 library data