Exploring the Influence of Information Entropy Change in Learning Systems

In this work, we explore the influence of entropy change in deep learning systems by adding noise to the inputs/latent features. The applications in this paper focus on deep learning tasks within computer vision, but the proposed theory can be further applied to other fields. Noise is conventionally viewed as a harmful perturbation in various deep learning architectures, such as convolutional neural networks (CNNs) and vision transformers (ViTs), as well as different learning tasks like image classification and transfer learning. However, this paper aims to rethink whether the conventional proposition always holds. We demonstrate that specific noise can boost the performance of various deep architectures under certain conditions. We theoretically prove the enhancement gained from positive noise by reducing the task complexity defined by information entropy and experimentally show the significant performance gain in large image datasets, such as the ImageNet. Herein, we use the information entropy to define the complexity of the task. We categorize the noise into two types, positive noise (PN) and harmful noise (HN), based on whether the noise can help reduce the complexity of the task. Extensive experiments of CNNs and ViTs have shown performance improvements by proactively injecting positive noise, where we achieved an unprecedented top 1 accuracy of over 95% on ImageNet. Both theoretical analysis and empirical evidence have confirmed that the presence of positive noise can benefit the learning process, while the traditionally perceived harmful noise indeed impairs deep learning models. The different roles of noise offer new explanations for deep models on specific tasks and provide a new paradigm for improving model performance. Moreover, it reminds us that we can influence the performance of learning systems via information entropy change.Comment: Information Entropy, CNN, Transforme

The nestin-expressing and non-expressing neurons in rat basal forebrain display different electrophysiological properties and project to hippocampus

<p>Abstract</p> <p>Background</p> <p>Nestin-immunoreactive (nestin-ir) neurons have been identified in the medial septal/diagonal band complex (MS/DBB) of adult rat and human, but the significance of nestin expression in functional neurons is not clear. This study investigated electrophysiological properties and neurochemical phenotypes of nestin-expressing (nestin+) neurons using whole-cell recording combined with single-cell RT-PCR to explore the significance of nestin expression in functional MS/DBB neurons. The retrograde labelling and immunofluorescence were used to investigate the nestin+ neuron related circuit in the septo-hippocampal pathway.</p> <p>Results</p> <p>The results of single-cell RT-PCR showed that 87.5% (35/40) of nestin+ cells expressed choline acetyltransferase mRNA (ChAT+), only 44.3% (35/79) of ChAT+ cells expressed nestin mRNA. Furthermore, none of the nestin+ cells expressed glutamic acid decarboxylases 67 (GAD₆₇) or vesicular glutamate transporters (VGLUT) mRNA. All of the recorded nestin+ cells were excitable and demonstrated slow-firing properties, which were distinctive from those of GAD₆₇or VGLUT mRNA-positive neurons. These results show that the MS/DBB cholinergic neurons could be divided into nestin-expressing cholinergic neurons (NEChs) and nestin non-expressing cholinergic neurons (NNChs). Interestingly, NEChs had higher excitability and received stronger spontaneous excitatory synaptic inputs than NNChs. Retrograde labelling combined with choline acetyltransferase and nestin immunofluorescence showed that both of the NEChs and NNChs projected to hippocampus.</p> <p>Conclusions</p> <p>These results suggest that there are two parallel cholinergic septo-hippocampal pathways that may have different functions. The significance of nestin expressing in functional neurons has been discussed.</p

Fine Carbohydrate Structure of Dietary Resistant Glucans Governs the Structure and Function of Human Gut Microbiota

Increased dietary fiber consumption has been shown to increase human gut microbial diversity, but the mechanisms driving this effect remain unclear. One possible explanation is that microbes are able to divide metabolic labor in consumption of complex carbohydrates, which are composed of diverse glycosidic linkages that require specific cognate enzymes for degradation. However, as naturally derived fibers vary in both sugar composition and linkage structure, it is challenging to separate out the impact of each of these variables. We hypothesized that fine differences in carbohydrate linkage structure would govern microbial community structure and function independently of variation in glycosyl residue composition. To test this hypothesis, we fermented commercially available soluble resistant glucans, which are uniformly composed of glucose linked in different structural arrangements, in vitro with fecal inocula from each of three individuals. We measured metabolic outputs (pH, gas, and short-chain fatty acid production) and community structure via 16S rRNA amplicon sequencing. We determined that community metabolic outputs from identical glucans were highly individual, emerging from divergent initial microbiome structures. However, specific operational taxonomic units (OTUs) responded similarly in growth responses across individuals’ microbiota, though in context-dependent ways; these data suggested that certain taxa were more efficient in competing for some structures than others. Together, these data support the hypothesis that variation in linkage structure, independent of sugar composition, governs compositional and functional responses of microbiota

MadDroid: Characterising and Detecting Devious Ad Content for Android Apps

Advertisement drives the economy of the mobile app ecosystem. As a key component in the mobile ad business model, mobile ad content has been overlooked by the research community, which poses a number of threats, e.g., propagating malware and undesirable contents. To understand the practice of these devious ad behaviors, we perform a large-scale study on the app contents harvested through automated app testing. In this work, we first provide a comprehensive categorization of devious ad contents, including five kinds of behaviors belonging to two categories: \emph{ad loading content} and \emph{ad clicking content}. Then, we propose MadDroid, a framework for automated detection of devious ad contents. MadDroid leverages an automated app testing framework with a sophisticated ad view exploration strategy for effectively collecting ad-related network traffic and subsequently extracting ad contents. We then integrate dedicated approaches into the framework to identify devious ad contents. We have applied MadDroid to 40,000 Android apps and found that roughly 6\% of apps deliver devious ad contents, e.g., distributing malicious apps that cannot be downloaded via traditional app markets. Experiment results indicate that devious ad contents are prevalent, suggesting that our community should invest more effort into the detection and mitigation of devious ads towards building a trustworthy mobile advertising ecosystem.Comment: To be published in The Web Conference 2020 (WWW'20

Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era

Explainable AI (XAI) refers to techniques that provide human-understandable insights into the workings of AI models. Recently, the focus of XAI is being extended towards Large Language Models (LLMs) which are often criticized for their lack of transparency. This extension calls for a significant transformation in XAI methodologies because of two reasons. First, many existing XAI methods cannot be directly applied to LLMs due to their complexity advanced capabilities. Second, as LLMs are increasingly deployed across diverse industry applications, the role of XAI shifts from merely opening the "black box" to actively enhancing the productivity and applicability of LLMs in real-world settings. Meanwhile, unlike traditional machine learning models that are passive recipients of XAI insights, the distinct abilities of LLMs can reciprocally enhance XAI. Therefore, in this paper, we introduce Usable XAI in the context of LLMs by analyzing (1) how XAI can benefit LLMs and AI systems, and (2) how LLMs can contribute to the advancement of XAI. We introduce 10 strategies, introducing the key techniques for each and discussing their associated challenges. We also provide case studies to demonstrate how to obtain and leverage explanations. The code used in this paper can be found at: https://github.com/JacksonWuxs/UsableXAI_LLM.Comment: 38 pages, 4 figure

ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT

Large language models (LLMs) such as ChatGPT have recently demonstrated significant potential in mathematical abilities, providing valuable reasoning paradigm consistent with human natural language. However, LLMs currently have difficulty in bridging perception, language understanding and reasoning capabilities due to incompatibility of the underlying information flow among them, making it challenging to accomplish tasks autonomously. On the other hand, abductive learning (ABL) frameworks for integrating the two abilities of perception and reasoning has seen significant success in inverse decipherment of incomplete facts, but it is limited by the lack of semantic understanding of logical reasoning rules and the dependence on complicated domain knowledge representation. This paper presents a novel method (ChatABL) for integrating LLMs into the ABL framework, aiming at unifying the three abilities in a more user-friendly and understandable manner. The proposed method uses the strengths of LLMs' understanding and logical reasoning to correct the incomplete logical facts for optimizing the performance of perceptual module, by summarizing and reorganizing reasoning rules represented in natural language format. Similarly, perceptual module provides necessary reasoning examples for LLMs in natural language format. The variable-length handwritten equation deciphering task, an abstract expression of the Mayan calendar decoding, is used as a testbed to demonstrate that ChatABL has reasoning ability beyond most existing state-of-the-art methods, which has been well supported by comparative studies. To our best knowledge, the proposed ChatABL is the first attempt to explore a new pattern for further approaching human-level cognitive ability via natural language interaction with ChatGPT

AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology

In this pioneering study, inspired by AutoGPT, the state-of-the-art open-source application based on the GPT-4 large language model, we develop a novel tool called AD-AutoGPT which can conduct data collection, processing, and analysis about complex health narratives of Alzheimer's Disease in an autonomous manner via users' textual prompts. We collated comprehensive data from a variety of news sources, including the Alzheimer's Association, BBC, Mayo Clinic, and the National Institute on Aging since June 2022, leading to the autonomous execution of robust trend analyses, intertopic distance maps visualization, and identification of salient terms pertinent to Alzheimer's Disease. This approach has yielded not only a quantifiable metric of relevant discourse but also valuable insights into public focus on Alzheimer's Disease. This application of AD-AutoGPT in public health signifies the transformative potential of AI in facilitating a data-rich understanding of complex health narratives like Alzheimer's Disease in an autonomous manner, setting the groundwork for future AI-driven investigations in global health landscapes.Comment: 20 pages, 4 figure

Distinct resting-state effective connectivity of large-scale networks in first-episode and recurrent major depression disorder: evidence from the REST-meta-MDD consortium

IntroductionPrevious studies have shown disrupted effective connectivity in the large-scale brain networks of individuals with major depressive disorder (MDD). However, it is unclear whether these changes differ between first-episode drug-naive MDD (FEDN-MDD) and recurrent MDD (R-MDD).MethodsThis study utilized resting-state fMRI data from 17 sites in the Chinese REST-meta-MDD project, consisting of 839 patients with MDD and 788 normal controls (NCs). All data was preprocessed using a standardized protocol. Then, we performed a granger causality analysis to calculate the effectivity connectivity (EC) within and between brain networks for each participant, and compared the differences between the groups.ResultsOur findings revealed that R-MDD exhibited increased EC in the fronto-parietal network (FPN) and decreased EC in the cerebellum network, while FEDN-MDD demonstrated increased EC from the sensorimotor network (SMN) to the FPN compared with the NCs. Importantly, the two MDD subgroups displayed significant differences in EC within the FPN and between the SMN and visual network. Moreover, the EC from the cingulo-opercular network to the SMN showed a significant negative correlation with the Hamilton Rating Scale for Depression (HAMD) score in the FEDN-MDD group.ConclusionThese findings suggest that first-episode and recurrent MDD have distinct effects on the effective connectivity in large-scale brain networks, which could be potential neural mechanisms underlying their different clinical manifestations