Provable Guarantees for Neural Networks via Gradient Feature Learning

Neural networks have achieved remarkable empirical performance, while the current theoretical analysis is not adequate for understanding their success, e.g., the Neural Tangent Kernel approach fails to capture their key feature learning ability, while recent analyses on feature learning are typically problem-specific. This work proposes a unified analysis framework for two-layer networks trained by gradient descent. The framework is centered around the principle of feature learning from gradients, and its effectiveness is demonstrated by applications in several prototypical problems, such as mixtures of Gaussians and parity functions. The framework also sheds light on interesting network learning phenomena such as feature learning beyond kernels and the lottery ticket hypothesis.Comment: NeurIPS 2023, 71 page

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning

Foundation models have emerged as a powerful tool for many AI problems. Despite the tremendous success of foundation models, effective adaptation to new tasks, particularly those with limited labels, remains an open question and lacks theoretical understanding. An emerging solution with recent success in vision and NLP involves finetuning a foundation model on a selection of relevant tasks, before its adaptation to a target task with limited labeled samples. In this paper, we study the theoretical justification of this multitask finetuning approach. Our theoretical analysis reveals that with a diverse set of related tasks, this multitask finetuning leads to reduced error in the target task, in comparison to directly adapting the same pretrained model. We quantify the relationship between finetuning tasks and target tasks by diversity and consistency metrics, and further propose a practical task selection algorithm. We substantiate our theoretical claims with extensive empirical evidence. Further, we present results affirming our task selection algorithm adeptly chooses related finetuning tasks, providing advantages to the model performance on target tasks. We believe our study shed new light on the effective adaptation of foundation models to new tasks that lack abundant labels. Our code is available at https://github.com/OliverXUZY/Foudation-Model_Multitask.Comment: Published at ICLR 2024. 54 page

Towards Net-Zero Carbon Emissions in Network AI for 6G and Beyond

A global effort has been initiated to reduce the worldwide greenhouse gas (GHG) emissions, primarily carbon emissions, by half by 2030 and reach net-zero by 2050. The development of 6G must also be compliant with this goal. Unfortunately, developing a sustainable and net-zero emission systems to meet the users' fast growing demands on mobile services, especially smart services and applications, may be much more challenging than expected. Particularly, despite the energy efficiency improvement in both hardware and software designs, the overall energy consumption and carbon emission of mobile networks are still increasing at a tremendous speed. The growing penetration of resource-demanding AI algorithms and solutions further exacerbate this challenge. In this article, we identify the major emission sources and introduce an evaluation framework for analyzing the lifecycle of network AI implementations. A novel joint dynamic energy trading and task allocation optimization framework, called DETA, has been introduced to reduce the overall carbon emissions. We consider a federated edge intelligence-based network AI system as a case study to verify the effectiveness of our proposed solution. Experimental results based on a hardware prototype suggest that our proposed solution can reduce carbon emissions of network AI systems by up to 74.9%. Finally, open problems and future directions are discussed

Pivotal roles of CD8+ T cells restricted by MHC class I–like molecules in autoimmune diseases

Unlike T cells restricted by major histocompatibility complex (MHC) class Ia or class II molecules, T cells restricted by MHC class I–like molecules demonstrate properties of both innate and adaptive immunity and are therefore considered innate-like lymphocytes (ILLs). ILLs are believed to have immunoregulatory functions, but their roles in autoimmunity and defense against infections remain elusive. To study the properties of ILLs, we generated mice expressing only MHC class I–like molecules by crossing CIITA−/− with Kb−/−Db−/− mice. Surprisingly, these mice developed a lymphoproliferative syndrome and autoimmunity, most notably inflammatory bowel disease (IBD) and insulitis. The CD8+ ILLs in these mice exhibit a constitutively activated phenotype, and depletion of these cells abolished the autoimmune disorders. In addition, adoptive transfer of CD8+ ILLs from Kb−/−Db−/−CIITA−/− mice to Rag-1−/−pfn−/− mice also resulted in IBD and insulitis. These findings provide direct evidence that CD8+ ILLs are sufficient to initiate and mediate autoimmune diseases

Reasoning over the Air: A Reasoning-based Implicit Semantic-Aware Communication Framework

Semantic-aware communication is a novel paradigm that draws inspiration from human communication focusing on the delivery of the meaning of messages. It has attracted significant interest recently due to its potential to improve the efficiency and reliability of communication and enhance users' QoE. Most existing works focus on transmitting and delivering the explicit semantic meaning that can be directly identified from the source signal. This paper investigates the implicit semantic-aware communication in which the hidden information that cannot be directly observed from the source signal must be recognized and interpreted by the intended users. To this end, a novel implicit semantic-aware communication (iSAC) architecture is proposed for representing, communicating, and interpreting the implicit semantic meaning between source and destination users. A projection-based semantic encoder is proposed to convert the high-dimensional graphical representation of explicit semantics into a low-dimensional semantic constellation space for efficient physical channel transmission. To enable the destination user to learn and imitate the implicit semantic reasoning process of source user, a generative adversarial imitation learning-based solution, called G-RML, is proposed. Different from existing communication solutions, the source user in G-RML does not focus only on sending as much of the useful messages as possible; but, instead, it tries to guide the destination user to learn a reasoning mechanism to map any observed explicit semantics to the corresponding implicit semantics that are most relevant to the semantic meaning. Compared to the existing solutions, our proposed G-RML requires much less communication and computational resources and scales well to the scenarios involving the communication of rich semantic meanings consisting of a large number of concepts and relations.Comment: accepted at IEEE Transactions on Wireless Communication

Isolation of Mycobacterium tuberculosis complex (MTBC) from dairy cows in China

Eleven thousand five hundred and eighty non-blood samples from dairy cows were subjected to mycobacterium culture and genotyping. As a result, a total of 142 isolates of Mycobacterium tuberculosis complex (MBTC) were identified. Among them, 65 were Mycobacterium tuberculosis, while 77 Mycobacterium bovis. The genotype of M. tuberculosis strains was mainly Beijing family. In addition, the isolation rates of MTBC were 33.89% for lung lymph nodes, 2.81% for nasal swabs, and 3.95% for pharyngeal swabs from cattle positive to tuberculin skin test, respectively. This evidence implied that M. tuberculosis infection in cattle is a new risk to public health and should be paid more attention.Key words: Mycobacterium tuberculosis complex, cows, tuberculosis, zoonosis