Differentially Private Bias-Term only Fine-tuning of Foundation Models

We study the problem of differentially private (DP) fine-tuning of large pre-trained models -- a recent privacy-preserving approach suitable for solving downstream tasks with sensitive data. Existing work has demonstrated that high accuracy is possible under strong privacy constraint, yet requires significant computational overhead or modifications to the network architecture. We propose differentially private bias-term fine-tuning (DP-BiTFiT), which matches the state-of-the-art accuracy for DP algorithms and the efficiency of the standard BiTFiT. DP-BiTFiT is model agnostic (not modifying the network architecture), parameter efficient (only training about $0.1\%$ of the parameters), and computation efficient (almost removing the overhead caused by DP, in both the time and space complexity). On a wide range of tasks, DP-BiTFiT is $2\sim 30\times$ faster and uses $2\sim 8\times$ less memory than DP full fine-tuning, even faster than the standard full fine-tuning. This amazing efficiency enables us to conduct DP fine-tuning on language and vision tasks with long-sequence texts and high-resolution images, which were computationally difficult using existing methods

On the accuracy and efficiency of group-wise clipping in differentially private optimization

Recent advances have substantially improved the accuracy, memory cost, and training speed of differentially private (DP) deep learning, especially on large vision and language models with millions to billions of parameters. In this work, we thoroughly study the per-sample gradient clipping style, a key component in DP optimization. We show that different clipping styles have the same time complexity but instantiate an accuracy-memory trade-off: while the all-layer clipping (of coarse granularity) is the most prevalent and usually gives the best accuracy, it incurs heavier memory cost compared to other group-wise clipping, such as the layer-wise clipping (of finer granularity). We formalize this trade-off through our convergence theory and complexity analysis. Importantly, we demonstrate that the accuracy gap between group-wise clipping and all-layer clipping becomes smaller for larger models, while the memory advantage of the group-wise clipping remains. Consequently, the group-wise clipping allows DP optimization of large models to achieve high accuracy and low peak memory simultaneously

A Novel Radar Sensor for the Non-Contact Detection of Speech Signals

Different speech detection sensors have been developed over the years but they are limited by the loss of high frequency speech energy, and have restricted non-contact detection due to the lack of penetrability. This paper proposes a novel millimeter microwave radar sensor to detect speech signals. The utilization of a high operating frequency and a superheterodyne receiver contributes to the high sensitivity of the radar sensor for small sound vibrations. In addition, the penetrability of microwaves allows the novel sensor to detect speech signals through nonmetal barriers. Results show that the novel sensor can detect high frequency speech energies and that the speech quality is comparable to traditional microphone speech. Moreover, the novel sensor can detect speech signals through a nonmetal material of a certain thickness between the sensor and the subject. Thus, the novel speech sensor expands traditional speech detection techniques and provides an exciting alternative for broader application prospects

High Altitude test of RPCs for the ARGO-YBJ experiment

A 50 m**2 RPC carpet was operated at the YangBaJing Cosmic Ray Laboratory (Tibet) located 4300 m a.s.l. The performance of RPCs in detecting Extensive Air Showers was studied. Efficiency and time resolution measurements at the pressure and temperature conditions typical of high mountain laboratories, are reported.Comment: 16 pages, 10 figures, submitted to Nucl. Instr. Met

Search for Gamma Ray Bursts with the Argo-YBJ Detector in Scaler Mode

We report on the search for Gamma Ray Bursts (GRBs) in the energy range 1-100 GeV in coincidence with the prompt emission detected by satellites using the Astrophysical Radiation with Ground-based Observatory at YangBaJing (ARGO-YBJ) air shower detector. Thanks to its mountain location (Yangbajing, Tibet, P.R. China, 4300 m a.s.l.), active surface (about 6700 m**2 of Resistive Plate Chambers), and large field of view (about 2 sr, limited only by the atmospheric absorption), the ARGO-YBJ air shower detector is particularly suitable for the detection of unpredictable and short duration events such as GRBs. The search is carried out using the "single particle technique", i.e. counting all the particles hitting the detector without measurement of the energy and arrival direction of the primary gamma rays. Between 2004 December 17 and 2009 April 7, 81 GRBs detected by satellites occurred within the field of view of ARGO-YBJ (zenith angle < 45 deg). It was possible to examine 62 of these for >1 GeV counterpart in the ARGO-YBJ data finding no statistically significant emission. With a lack of detected spectra in this energy range fluence upper limits are profitable, especially when the redshift is known and the correction for the extragalactic absorption can be considered. The obtained fluence upper limits reach values as low as 10**{-5} erg cm**{-2} in the 1-100 GeV energy region. Besides this individual search for a higher energy counterpart, a statistical study of the stack of all the GRBs both in time and in phase was made, looking for a common feature in the GRB high energy emission. No significant signal has been detected.Comment: accepted for publication in Ap

Measurement of the cosmic ray antiproton/proton flux ratio at TeV energies with the ARGO-YBJ detector

Cosmic ray antiprotons provide an important probe to study the cosmic ray propagation in the interstellar space and to investigate the existence of dark matter. Acting the Earth-Moon system as a magnetic spectrometer, paths of primary antiprotons are deflected in the opposite sense with respect to those of the protons in their way to the Earth. This effect allows, in principle, the search for antiparticles in the direction opposite to the observed deficit of cosmic rays due to the Moon (the so-called `Moon shadow'). The ARGO-YBJ experiment, located at the Yangbajing Cosmic Ray Laboratory (Tibet, P.R. China, 4300 m a.s.l., 606 g/cm$^2$), is particularly effective in measuring the cosmic ray antimatter content via the observation of the cosmic rays shadowing effect due to: (1) good angular resolution, pointing accuracy and long-term stability; (2) low energy threshold; (3) real sensitivity to the geomagnetic field. Based on all the data recorded during the period from July 2006 through November 2009 and on a full Monte Carlo simulation, we searched for the existence of the shadow cast by antiprotons in the TeV energy region. No evidence of the existence of antiprotons is found in this energy region. Upper limits to the $\bar{p}/p$ flux ratio are set to 5 % at a median energy of 1.4 TeV and 6 % at 5 TeV with a confidence level of 90%. In the TeV energy range these limits are the lowest available.Comment: Contact authors: G. Di Sciascio ([email protected]) and R. Iuppa ([email protected]), INFN Sezione di Roma Tor Vergata, Roma, Ital

Genetic Regulation of the Thymic Stromal Lymphopoietin (TSLP)/TSLP Receptor (TSLPR) Gene Expression and Influence of Epistatic Interactions Between IL-33 and the TSLP/TSLPR Axis on Risk of Coronary Artery Disease

The thymic stromal lymphopoietin (TSLP)/TSLP receptor (TSLPR) axis is involved in multiple inflammatory immune diseases, including coronary artery disease (CAD). To explore the causal relationship between this axis and CAD, we performed a three-stage case-control association analysis with 3,628 CAD cases and 3,776 controls using common variants in the genes TSLP, interleukin 7 receptor (IL7R), and TSLPR. Three common variants in the TSLP/TSLPR axis were significantly associated with CAD in a Chinese Han population [rs3806933T in TSLP, Padj = 4.35 × 10−5, odds ratio (OR) = 1.18; rs6897932T in IL7R, Padj = 1.13 × 10−7, OR = 1.31; g.19646A&gt;GA in TSLPR, Padj = 2.04 × 10−6, OR = 1.20]. Reporter gene analysis demonstrated that rs3806933 and rs6897932 could influence TSLP and IL7R expression, respectively. Furthermore, the “T” allele of rs3806933 might increase plasma TSLP levels (R2 = 0.175, P &lt; 0.01). In a stepwise procedure, the risk for CAD increased by nearly fivefold compared with the maximum effect of any single variant (Padj = 6.99 × 10−4, OR = 4.85). In addition, the epistatic interaction between TSLP and IL33 produced a nearly threefold increase in the risk of CAD in the combined model of rs3806933TT-rs7025417TT (Padj = 3.67 × 10−4, OR = 2.98). Our study illustrates that the TSLP/TSLPR axis might be involved in the pathogenesis of CAD through upregulation of mRNA or protein expression of the referenced genes and might have additive effects on the CAD risk when combined with IL-33 signaling

Management of granulomatous lobular mastitis: an international multidisciplinary consensus (2021 edition)

Granulomatous lobular mastitis (GLM) is a rare and chronic benign inflammatory disease of the breast. Difficulties exist in the management of GLM for many front-line surgeons and medical specialists who care for patients with inflammatory disorders of the breast. This consensus is summarized to establish evidence-based recommendations for the management of GLM. Literature was reviewed using PubMed from January 1, 1971 to July 31, 2020. Sixty-six international experienced multidisciplinary experts from 11 countries or regions were invited to review the evidence. Levels of evidence were determined using the American College of Physicians grading system, and recommendations were discussed until consensus. Experts discussed and concluded 30 recommendations on historical definitions, etiology and predisposing factors, diagnosis criteria, treatment, clinical stages, relapse and recurrence of GLM. GLM was recommended as a widely accepted definition. In addition, this consensus introduced a new clinical stages and management algorithm for GLM to provide individual treatment strategies. In conclusion, diagnosis of GLM depends on a combination of history, clinical manifestations, imaging examinations, laboratory examinations and pathology. The approach to treatment of GLM should be applied according to the different clinical stage of GLM. This evidence-based consensus would be valuable to assist front-line surgeons and medical specialists in the optimal management of GLM.Improving the Ability of Diagnosis and Treatment of Difficult Disease