Scaling Law for Recommendation Models: Towards General-purpose User Representations

Recent advancement of large-scale pretrained models such as BERT, GPT-3, CLIP, and Gopher, has shown astonishing achievements across various task domains. Unlike vision recognition and language models, studies on general-purpose user representation at scale still remain underexplored. Here we explore the possibility of general-purpose user representation learning by training a universal user encoder at large scales. We demonstrate that the scaling law is present in user representation learning areas, where the training error scales as a power-law with the amount of computation. Our Contrastive Learning User Encoder (CLUE), optimizes task-agnostic objectives, and the resulting user embeddings stretch our expectation of what is possible to do in various downstream tasks. CLUE also shows great transferability to other domains and companies, as performances on an online experiment shows significant improvements in Click-Through-Rate (CTR). Furthermore, we also investigate how the model performance is influenced by the scale factors, such as training data size, model capacity, sequence length, and batch size. Finally, we discuss the broader impacts of CLUE in general.Comment: Accepted at AAAI 2023. This version includes the technical appendi

Deformable Graph Transformer

Transformer-based models have recently shown success in representation learning on graph-structured data beyond natural language processing and computer vision. However, the success is limited to small-scale graphs due to the drawbacks of full dot-product attention on graphs such as the quadratic complexity with respect to the number of nodes and message aggregation from enormous irrelevant nodes. To address these issues, we propose Deformable Graph Transformer (DGT) that performs sparse attention via dynamically sampled relevant nodes for efficiently handling large-scale graphs with a linear complexity in the number of nodes. Specifically, our framework first constructs multiple node sequences with various criteria to consider both structural and semantic proximity. Then, combining with our learnable Katz Positional Encodings, the sparse attention is applied to the node sequences for learning node representations with a significantly reduced computational cost. Extensive experiments demonstrate that our DGT achieves state-of-the-art performance on 7 graph benchmark datasets with 2.5 - 449 times less computational cost compared to transformer-based graph models with full attention.Comment: 16 pages, 3 figure

Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning

Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its versatile usability has been widely investigated in many domains, its applications to recommender systems still remain underexplored. We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. Also, leveraging additional task-agnostic user histories delivers significant performance benefits. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems, even on unseen domains and services.Comment: 14 pages, 5 figures, 9 table

Solution-Processed Cu2_2S Nanostructures for Solar Hydrogen Production

Cu$_2$S is a promising solar energy conversion material due to its suitable optical properties, high elemental earth abundance, and nontoxicity. In addition to the challenge of multiple stable secondary phases, the short minority carrier diffusion length poses an obstacle to its practical application. This work addresses the issue by synthesizing nanostructured Cu$_2$S thin films, which enables increased charge carrier collection. A simple solution-processing method involving the preparation of CuCl and CuCl$_2$ molecular inks in a thiol-amine solvent mixture followed by spin coating and low-temperature annealing was used to obtain phase-pure nanostructured (nanoplate and nanoparticle) Cu$_2$S thin films. The photocathode based on the nanoplate Cu$_2$S (FTO/Au/Cu$_2$S/CdS/TiO$_2$/RuO$_x$) reveals enhanced charge carrier collection and improved photoelectrochemical water-splitting performance compared to the photocathode based on the non-nanostructured Cu$_2$S thin film reported previously. A photocurrent density of 3.0 mA cm$^{–2}$ at −0.2 versus a reversible hydrogen electrode (V$_{RHE}$) with only 100 nm thickness of a nanoplate Cu$_2$S layer and an onset potential of 0.43 V$_{RHE}$ were obtained. This work provides a simple, cost-effective, and high-throughput method to prepare phase-pure nanostructured Cu$_2$S thin films for scalable solar hydrogen production

Developmental endothelial locus-1 as a potential biomarker for the incidence of acute exacerbation in patients with chronic obstructive pulmonary disease

Background Despite the high disease burden of chronic obstructive pulmonary disease (COPD) and risk of acute COPD exacerbation, few COPD biomarkers are available. As developmental endothelial locus-1 (DEL-1) has been proposed to possess beneficial effects, including anti-inflammatory effects, we hypothesized that DEL-1 could be a blood biomarker for COPD. Objective To elucidate the role of plasma DEL-1 as a biomarker of COPD in terms of pathogenesis and for predicting acute exacerbation. Methods Cigarette smoke extract (CSE) or saline was intratracheally administered to wild-type (WT) and DEL-1 knockout (KO) C57BL/6 mice. Subsequently, lung sections were obtained to quantify the degree of emphysema using the mean linear intercept (MLI). Additionally, plasma DEL-1 levels were compared between COPD and non-COPD participants recruited in ongoing prospective cohorts. Using negative binomial regression analysis, the association between the plasma DEL-1 level and subsequent acute exacerbation risk was evaluated in patients with COPD. Results In the in vivo study, DEL-1 KO induced emphysema (KO saline vs. WT saline; P = 0.003) and augmented CSE-induced emphysema (KO CSE vs. WT CSE; P < 0.001) in 29 mice. Among 537 participants, patients with COPD presented plasma log (DEL-1) levels lower than non-COPD participants (P = 0.04), especially non-COPD never smokers (P = 0.019). During 1.2 ± 0.3 years, patients with COPD in the lowest quartile of Log(DEL-1) demonstrated an increased risk of subsequent acute exacerbation, compared with those in the highest quartile of Log(DEL-1) (adjusted incidence rate ratio, 3.64; 95% confidence interval, 1.03–12.9). Conclusion Low DEL-1 levels are associated with COPD development and increased risk of subsequent COPD acute exacerbation. DEL-1 can be a useful biomarker in patients with COPD.This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1C1C1007918). This research was also supported by funds (2016ER670100, 2016ER670101, 2016ER670102 and 2018ER670100, 2018ER670101, 2018ER670102) from Research of Korea Centers for Disease Control and Prevention

The IPIN 2019 Indoor Localisation Competition—Description and Results

IPIN 2019 Competition, sixth in a series of IPIN competitions, was held at the CNR Research Area of Pisa (IT), integrated into the program of the IPIN 2019 Conference. It included two on-site real-time Tracks and three off-site Tracks. The four Tracks presented in this paper were set in the same environment, made of two buildings close together for a total usable area of 1000 m 2 outdoors and and 6000 m 2 indoors over three floors, with a total path length exceeding 500 m. IPIN competitions, based on the EvAAL framework, have aimed at comparing the accuracy performance of personal positioning systems in fair and realistic conditions: past editions of the competition were carried in big conference settings, university campuses and a shopping mall. Positioning accuracy is computed while the person carrying the system under test walks at normal walking speed, uses lifts and goes up and down stairs or briefly stops at given points. Results presented here are a showcase of state-of-the-art systems tested side by side in real-world settings as part of the on-site real-time competition Tracks. Results for off-site Tracks allow a detailed and reproducible comparison of the most recent positioning and tracking algorithms in the same environment as the on-site Tracks

1H NMR-Based Metabolomic Analysis of Intraerythrocytic Stages of P. falciparum and Antimalarial Treatments

1H NMR provides an indirect approach that can be used to study the metabolism of the malaria parasite, Plasmodium falciparum, in human erythrocytes. It allows for facile analysis of the metabolites that the parasite excretes into and consumes from the surrounding media to gain insight into this dynamic system. This study carefully processes a very large batch of NMR data using MestReNova through a rigorous procedure. This is followed by multivariate statistical analyses in SIMCA to metabolically understand how erythrocytes respond to parasitic infection and subsequent treatment with artemisinin, chloroquine, or a candidate antimalarial, tiamulin. Trajectories reveal separation between the infected and artemisinin sets, and the uninfected, chloroquine, and tiamulin sets. Coefficient plots provide an introduction to the specific metabolites, the concentrations of which change systematically in the media. This study conclusively demonstrates that tiamulin treatment results in highly similar metabolic effects to those of chloroquine, both in terms of overall time-evolution trajectories and metabolic exchange with the medium during each individual phase. It also appears to normalize cells to be similar to control uninfected cells. This cost effective and non-targeted 1H NMR approach to profile metabolites can be adopted for large-scale drug screenings

HSP70-Homolog DnaK of Pseudomonas aeruginosa Increases the Production of IL-27 through Expression of EBI3 via TLR4-Dependent NF-κB and TLR4-Independent Akt Signaling

IL-27, a heterodimeric cytokine composed of the p28 subunit and Epstein&ndash;Barr virus-induced gene 3 (EBI3), acts as a potent immunosuppressant and thus limits pathogenic inflammatory responses. IL-27 is upregulated upon Pseudomonas aeruginosa infection in septic mice, increasing susceptibility to the infection and decreasing clearance of the pathogen. However, it remains unclear which P. aeruginosa-derived molecules promote production of IL-27. In this study, we explored the mechanism by which P. aeruginosa DnaK, a heat shock protein 70-like protein, induces EBI3 expression, thereby promoting production of IL-27. Upregulation of EBI3 expression did not lead to an increase in IL-35, which consists of the p35 subunit and EBI3. The IL-27 production in response to DnaK was biologically active, as reflected by stimulation of IL-10 production. DnaK-mediated expression of EBI3 was driven by two distinct signaling pathways, NF-&kappa;B and Akt. However, NF-&kappa;B is linked to TLR4-associated signaling pathways, whereas Akt is not. Taken together, our results reveal that P. aeruginosa DnaK potently upregulates EBI3 expression, which in turn drives production of the prominent anti-inflammatory cytokine IL-27, as a consequence of TLR4-dependent activation of NF-&kappa;B and TLR4-independent activation of the Akt signaling pathway

Intervention study of Korean firefighters: a scoping review

Firefighters in Korea have a variety of health problems due to their complex job characteristics and organizational culture. However, to date, there is a lack of intervention studies that reflect the characteristics of firefighters. Therefore, this study aims to identify the current status and characteristics of intervention studies on firefighters in Korea and make recommendations for further research. This study was conducted in seven steps according to the methodological guidelines of the topic scoping literature review provided by The Joanna Briggs Institute (JBI), an international research organization. We also reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) checklist. The literature search included articles published in domestic and international journals up to January 25, 2024, and the data were extracted according to the analysis framework developed prior to this extraction and the Template for Intervention Description and Replication (TIDieR) checklist

Synthesis of discrete bottlebrush polymers via the iterative convergent growth technique and post-functionalization

Bottlebrush polymers (BBPs) have unique physical properties that arise from polymeric side chains that are densely grafted onto their polymer backbone. Despite their promising attributes, well-defined BBPs are challenging to prepare partly due to steric hindrance encountered during macromonomer polymerization or during the coupling of side-chain polymers with functional groups present in the repeating units of the backbone polymers. Here we report the synthesis of perfectly discrete BBPs with absolutely defined chemical structures. The copolyester backbone, composed of up to 64 repeating units of an alternating sequence of lactic acid and 2-hydroxy-4-pentenoic acid, was synthesized using an iterative convergent growth technique. Discrete side chains, such as discrete polylactides (LA(n)-SH), were introduced in the presence of the vinyl groups of the 2-hydroxy-4-pentenoic acid residues on the copolyester backbone using thiol-ene click chemistry promoted by UV light (lambda = 365 nm), which produced a fully grafted BBP, t-LP(LA(8))32-LA(8) (Molecular weight (MW = 34 kDa). We also synthesized an amphiphilic bottlebrush block copolymer (BBCP) (MW = 19 kDa) by introducing the hydrophobic dodecanethiol and discrete poly(ethylene glycol) as side chains. The resulting BBCP self-assembled into well-defined micelles in water. Discrete BBCPs may serve as model systems that help to elucidate the role of the branched architecture on the physical and chemical behavior of BBCPs in bulk and solution.N