GPT-4V(ision) as A Social Media Analysis Engine

Recent research has offered insights into the extraordinary capabilities of Large Multimodal Models (LMMs) in various general vision and language tasks. There is growing interest in how LMMs perform in more specialized domains. Social media content, inherently multimodal, blends text, images, videos, and sometimes audio. Understanding social multimedia content remains a challenging problem for contemporary machine learning frameworks. In this paper, we explore GPT-4V(ision)'s capabilities for social multimedia analysis. We select five representative tasks, including sentiment analysis, hate speech detection, fake news identification, demographic inference, and political ideology detection, to evaluate GPT-4V. Our investigation begins with a preliminary quantitative analysis for each task using existing benchmark datasets, followed by a careful review of the results and a selection of qualitative samples that illustrate GPT-4V's potential in understanding multimodal social media content. GPT-4V demonstrates remarkable efficacy in these tasks, showcasing strengths such as joint understanding of image-text pairs, contextual and cultural awareness, and extensive commonsense knowledge. Despite the overall impressive capacity of GPT-4V in the social media domain, there remain notable challenges. GPT-4V struggles with tasks involving multilingual social multimedia comprehension and has difficulties in generalizing to the latest trends in social media. Additionally, it exhibits a tendency to generate erroneous information in the context of evolving celebrity and politician knowledge, reflecting the known hallucination problem. The insights gleaned from our findings underscore a promising future for LMMs in enhancing our comprehension of social media content and its users through the analysis of multimodal information

Privacy-preserving design of graph neural networks with applications to vertical federated learning

The paradigm of vertical federated learning (VFL), where institutions collaboratively train machine learning models via combining each other's local feature or label information, has achieved great success in applications to financial risk management (FRM). The surging developments of graph representation learning (GRL) have opened up new opportunities for FRM applications under FL via efficiently utilizing the graph-structured data generated from underlying transaction networks. Meanwhile, transaction information is often considered highly sensitive. To prevent data leakage during training, it is critical to develop FL protocols with formal privacy guarantees. In this paper, we present an end-to-end GRL framework in the VFL setting called VESPER, which is built upon a general privatization scheme termed perturbed message passing (PMP) that allows the privatization of many popular graph neural architectures.Based on PMP, we discuss the strengths and weaknesses of specific design choices of concrete graph neural architectures and provide solutions and improvements for both dense and sparse graphs. Extensive empirical evaluations over both public datasets and an industry dataset demonstrate that VESPER is capable of training high-performance GNN models over both sparse and dense graphs under reasonable privacy budgets

Influence Pathway Discovery on Social Media

This paper addresses influence pathway discovery, a key emerging problem in today's online media. We propose a discovery algorithm that leverages recently published work on unsupervised interpretable ideological embedding, a mapping of ideological beliefs (done in a self-supervised fashion) into interpretable low-dimensional spaces. Computing the ideological embedding at scale allows one to analyze correlations between the ideological positions of leaders, influencers, news portals, or population segments, deriving potential influence pathways. The work is motivated by the importance of social media as the preeminent means for global interactions and collaborations on today's Internet, as well as their frequent (mis-)use to wield influence that targets social beliefs and attitudes of selected populations. Tools that enable the understanding and mapping of influence propagation through population segments on social media are therefore increasingly important. In this paper, influence is measured by the perceived ideological shift over time that is correlated with influencers' activity. Correlated shifts in ideological embeddings indicate changes, such as swings/switching (among competing ideologies), polarization (depletion of neutral ideological positions), escalation/radicalization (shifts to more extreme versions of the ideology), or unification/cooldown (shifts towards more neutral stances). Case-studies are presented to explore selected influence pathways (i) in a recent French election, (ii) during political discussions in the Philippines, and (iii) for some Russian messaging during the Russia/Ukraine conflict.Comment: This paper is accepted by IEEE CIC as an invited vision pape

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Content Variation and Potential Runoff Loss Risk of Nutrients in Surface Water of Saline-Alkali Paddy in Response to the Application of Different Nitrogen Fertilizer Types

As the saline-alkali paddy area continues to grow, the nutrient (e.g., nitrogen (N) and phosphorus (P)) runoff loss is becoming more serious in the world. The N-fertilizer application affects the nutrient runoff loss risk in paddy. Selecting suitable fertilizer types to reduce nutrient loss is beneficial to agricultural sustainability. However, the effects of N-fertilizer application in saline-alkali paddy are not clear. This study measured the N and P concentration of surface water in saline-alkali paddy, using various N—fertilizer treatments (i.e., urea (U), urea with urease—nitrification inhibitors (UI), organic–inorganic compound fertilizer (OCF), carbon—based slow—release fertilizer (CSF), and no N fertilization (CK)). Based on the structural equation model, both phosphate (PO43−-P) and total−P (TP) concentrations had a positive influence on total-N (TN) concentration regardless of N−fertilizer types applied. Potential risks of ammonia—N (NH4+—N) and nitrate—N (NO3−—N) runoff losses were reduced in UI treatment, but the TN and TP losses were increased. At the panicle-initiation fertilizer stage, the NO3−−N, TN, and TP concentrations in CSF and OCF treatments were lower than U. The CSF application can control the TP runoff loss risk during the rice-growing season. UI should not be suggested for the control of nutrient runoff loss in saline-alkali paddy

Ligand-directed Photocatalysts and Far-red Light Enable Catalytic Bioorthogonal Uncaging inside Live Cells

Described are ligand-directed catalysts for live-cell, photocatalytic activation of bioorthogonal chemistry. Catalytic groups are localized via a tethered ligand either to DNA or to tubulin, and red-light (660 nm) photocatalysis is used to initiate a cascade of DHTz-oxidation, intramolecular Diels-Alder reaction, and elimination to release phenolic compounds. Silarhodamine (SiR) dyes, more conventionally used as biological fluorophores, serve as photocatalysts that have high cytocompatibility and produce minimal singlet oxygen. Commercially-available conjugates of Hoechst dye (SiR-H) and Taxol (SiR-T) are used to localize SiR to the nucleus and tubulin, respectively. Computation was used to assist the design of a new class of redox-activated photocage to release either phenol or n-CA4, a microtubule-destabilizing agent. In model studies, uncaging is complete within 5 min using only 2 µM of SiR and 40 µM of the photocage. In situ spectroscopic studies support a mechanism involving rapid intramolecular Diels-Alder reaction and a rate determining elimination step. In cellular studies, this uncaging process is successful at low concentration of both the photocage (25 nM) and the SiR-H dye (500 nM). Uncaging n-CA4 causes microtubule depolymerization and an accompanying reduction in cell area. Control studies demonstrate that SiR-H catalyzes uncaging inside the cell, and not in the extracellular environment. With SiR-T, the same dye serves as photocatalyst and the fluorescent reporter for tubulin depolymerization, and with confocal microscopy it was possible to visualize tubulin depolymerization in real time as the result of photocatalytic uncaging in live cells

The association between dietary mineral intake and the risk of preeclampsia in Chinese pregnant women: a matched case–control study

Abstract Previous studies on the relationship between dietary minerals and preeclampsia (PE) have given inconsistent results. The aim of this study was to further clarify the relationship between dietary minerals intake and PE in Chinese pregnant women. In this study, 440 pairs of hospital–based preeclamptic and healthy women were matched 1:1. Dietary intake was obtained through a 78–item semi–quantitative food frequency questionnaire. Multivariate conditional logistic regression was used to estimate the odds ratios (ORs) and 95% confidence intervals (CIs). Restricted cubic splines were plotted to evaluate the dose–response relationship between dietary minerals intake and PE. This study found significant inverse associations for dietary intake of calcium, magnesium, phosphorus, iron, copper, manganese and zinc and the risk of PE in both univariate and multivariate models (all P- trend < 0.05). After adjusting for possible confounders, compared with the lowest quartile, the odds ratio of the highest quartile was 0.74 (95% CI 0.56–0.98) for calcium, 0.63 (95% CI 0.42–0.93) for magnesium, 0.45 (95% CI 0.31–0.65) for phosphorus, 0.44 (95% CI 0.30–0.65) for iron, 0.72 (95% CI 0.53–0.97) for copper, 0.66 (95% CI 0.48–0.91) for manganese and 0.38 (95% CI 0.25–0.57) for zinc. In addition, a reverse J–shaped relationship between dietary minerals intake and PE risk was observed (P–overall association < 0.05). In Chinese pregnant women, a higher intake of dietary minerals, including calcium, magnesium, phosphorus, copper, iron, manganese, and zinc was associated with a lower odds of PE