64 research outputs found
Exploring the Influence of Information Entropy Change in Learning Systems
In this work, we explore the influence of entropy change in deep learning
systems by adding noise to the inputs/latent features. The applications in this
paper focus on deep learning tasks within computer vision, but the proposed
theory can be further applied to other fields. Noise is conventionally viewed
as a harmful perturbation in various deep learning architectures, such as
convolutional neural networks (CNNs) and vision transformers (ViTs), as well as
different learning tasks like image classification and transfer learning.
However, this paper aims to rethink whether the conventional proposition always
holds. We demonstrate that specific noise can boost the performance of various
deep architectures under certain conditions. We theoretically prove the
enhancement gained from positive noise by reducing the task complexity defined
by information entropy and experimentally show the significant performance gain
in large image datasets, such as the ImageNet. Herein, we use the information
entropy to define the complexity of the task. We categorize the noise into two
types, positive noise (PN) and harmful noise (HN), based on whether the noise
can help reduce the complexity of the task. Extensive experiments of CNNs and
ViTs have shown performance improvements by proactively injecting positive
noise, where we achieved an unprecedented top 1 accuracy of over 95% on
ImageNet. Both theoretical analysis and empirical evidence have confirmed that
the presence of positive noise can benefit the learning process, while the
traditionally perceived harmful noise indeed impairs deep learning models. The
different roles of noise offer new explanations for deep models on specific
tasks and provide a new paradigm for improving model performance. Moreover, it
reminds us that we can influence the performance of learning systems via
information entropy change.Comment: Information Entropy, CNN, Transforme
The nestin-expressing and non-expressing neurons in rat basal forebrain display different electrophysiological properties and project to hippocampus
<p>Abstract</p> <p>Background</p> <p>Nestin-immunoreactive (nestin-ir) neurons have been identified in the medial septal/diagonal band complex (MS/DBB) of adult rat and human, but the significance of nestin expression in functional neurons is not clear. This study investigated electrophysiological properties and neurochemical phenotypes of nestin-expressing (nestin+) neurons using whole-cell recording combined with single-cell RT-PCR to explore the significance of nestin expression in functional MS/DBB neurons. The retrograde labelling and immunofluorescence were used to investigate the nestin+ neuron related circuit in the septo-hippocampal pathway.</p> <p>Results</p> <p>The results of single-cell RT-PCR showed that 87.5% (35/40) of nestin+ cells expressed choline acetyltransferase mRNA (ChAT+), only 44.3% (35/79) of ChAT+ cells expressed nestin mRNA. Furthermore, none of the nestin+ cells expressed glutamic acid decarboxylases 67 (GAD<sub>67</sub>) or vesicular glutamate transporters (VGLUT) mRNA. All of the recorded nestin+ cells were excitable and demonstrated slow-firing properties, which were distinctive from those of GAD<sub>67 </sub>or VGLUT mRNA-positive neurons. These results show that the MS/DBB cholinergic neurons could be divided into nestin-expressing cholinergic neurons (NEChs) and nestin non-expressing cholinergic neurons (NNChs). Interestingly, NEChs had higher excitability and received stronger spontaneous excitatory synaptic inputs than NNChs. Retrograde labelling combined with choline acetyltransferase and nestin immunofluorescence showed that both of the NEChs and NNChs projected to hippocampus.</p> <p>Conclusions</p> <p>These results suggest that there are two parallel cholinergic septo-hippocampal pathways that may have different functions. The significance of nestin expressing in functional neurons has been discussed.</p
Fine Carbohydrate Structure of Dietary Resistant Glucans Governs the Structure and Function of Human Gut Microbiota
Increased dietary fiber consumption has been shown to increase human gut microbial diversity, but the mechanisms driving this effect remain unclear. One possible explanation is that microbes are able to divide metabolic labor in consumption of complex carbohydrates, which are composed of diverse glycosidic linkages that require specific cognate enzymes for degradation. However, as naturally derived fibers vary in both sugar composition and linkage structure, it is challenging to separate out the impact of each of these variables. We hypothesized that fine differences in carbohydrate linkage structure would govern microbial community structure and function independently of variation in glycosyl residue composition. To test this hypothesis, we fermented commercially available soluble resistant glucans, which are uniformly composed of glucose linked in different structural arrangements, in vitro with fecal inocula from each of three individuals. We measured metabolic outputs (pH, gas, and short-chain fatty acid production) and community structure via 16S rRNA amplicon sequencing. We determined that community metabolic outputs from identical glucans were highly individual, emerging from divergent initial microbiome structures. However, specific operational taxonomic units (OTUs) responded similarly in growth responses across individuals’ microbiota, though in context-dependent ways; these data suggested that certain taxa were more efficient in competing for some structures than others. Together, these data support the hypothesis that variation in linkage structure, independent of sugar composition, governs compositional and functional responses of microbiota
MadDroid: Characterising and Detecting Devious Ad Content for Android Apps
Advertisement drives the economy of the mobile app ecosystem. As a key
component in the mobile ad business model, mobile ad content has been
overlooked by the research community, which poses a number of threats, e.g.,
propagating malware and undesirable contents. To understand the practice of
these devious ad behaviors, we perform a large-scale study on the app contents
harvested through automated app testing. In this work, we first provide a
comprehensive categorization of devious ad contents, including five kinds of
behaviors belonging to two categories: \emph{ad loading content} and \emph{ad
clicking content}. Then, we propose MadDroid, a framework for automated
detection of devious ad contents. MadDroid leverages an automated app testing
framework with a sophisticated ad view exploration strategy for effectively
collecting ad-related network traffic and subsequently extracting ad contents.
We then integrate dedicated approaches into the framework to identify devious
ad contents. We have applied MadDroid to 40,000 Android apps and found that
roughly 6\% of apps deliver devious ad contents, e.g., distributing malicious
apps that cannot be downloaded via traditional app markets. Experiment results
indicate that devious ad contents are prevalent, suggesting that our community
should invest more effort into the detection and mitigation of devious ads
towards building a trustworthy mobile advertising ecosystem.Comment: To be published in The Web Conference 2020 (WWW'20
ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT
Large language models (LLMs) such as ChatGPT have recently demonstrated
significant potential in mathematical abilities, providing valuable reasoning
paradigm consistent with human natural language. However, LLMs currently have
difficulty in bridging perception, language understanding and reasoning
capabilities due to incompatibility of the underlying information flow among
them, making it challenging to accomplish tasks autonomously. On the other
hand, abductive learning (ABL) frameworks for integrating the two abilities of
perception and reasoning has seen significant success in inverse decipherment
of incomplete facts, but it is limited by the lack of semantic understanding of
logical reasoning rules and the dependence on complicated domain knowledge
representation. This paper presents a novel method (ChatABL) for integrating
LLMs into the ABL framework, aiming at unifying the three abilities in a more
user-friendly and understandable manner. The proposed method uses the strengths
of LLMs' understanding and logical reasoning to correct the incomplete logical
facts for optimizing the performance of perceptual module, by summarizing and
reorganizing reasoning rules represented in natural language format. Similarly,
perceptual module provides necessary reasoning examples for LLMs in natural
language format. The variable-length handwritten equation deciphering task, an
abstract expression of the Mayan calendar decoding, is used as a testbed to
demonstrate that ChatABL has reasoning ability beyond most existing
state-of-the-art methods, which has been well supported by comparative studies.
To our best knowledge, the proposed ChatABL is the first attempt to explore a
new pattern for further approaching human-level cognitive ability via natural
language interaction with ChatGPT
AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology
In this pioneering study, inspired by AutoGPT, the state-of-the-art
open-source application based on the GPT-4 large language model, we develop a
novel tool called AD-AutoGPT which can conduct data collection, processing, and
analysis about complex health narratives of Alzheimer's Disease in an
autonomous manner via users' textual prompts. We collated comprehensive data
from a variety of news sources, including the Alzheimer's Association, BBC,
Mayo Clinic, and the National Institute on Aging since June 2022, leading to
the autonomous execution of robust trend analyses, intertopic distance maps
visualization, and identification of salient terms pertinent to Alzheimer's
Disease. This approach has yielded not only a quantifiable metric of relevant
discourse but also valuable insights into public focus on Alzheimer's Disease.
This application of AD-AutoGPT in public health signifies the transformative
potential of AI in facilitating a data-rich understanding of complex health
narratives like Alzheimer's Disease in an autonomous manner, setting the
groundwork for future AI-driven investigations in global health landscapes.Comment: 20 pages, 4 figure
Distinct resting-state effective connectivity of large-scale networks in first-episode and recurrent major depression disorder: evidence from the REST-meta-MDD consortium
IntroductionPrevious studies have shown disrupted effective connectivity in the large-scale brain networks of individuals with major depressive disorder (MDD). However, it is unclear whether these changes differ between first-episode drug-naive MDD (FEDN-MDD) and recurrent MDD (R-MDD).MethodsThis study utilized resting-state fMRI data from 17 sites in the Chinese REST-meta-MDD project, consisting of 839 patients with MDD and 788 normal controls (NCs). All data was preprocessed using a standardized protocol. Then, we performed a granger causality analysis to calculate the effectivity connectivity (EC) within and between brain networks for each participant, and compared the differences between the groups.ResultsOur findings revealed that R-MDD exhibited increased EC in the fronto-parietal network (FPN) and decreased EC in the cerebellum network, while FEDN-MDD demonstrated increased EC from the sensorimotor network (SMN) to the FPN compared with the NCs. Importantly, the two MDD subgroups displayed significant differences in EC within the FPN and between the SMN and visual network. Moreover, the EC from the cingulo-opercular network to the SMN showed a significant negative correlation with the Hamilton Rating Scale for Depression (HAMD) score in the FEDN-MDD group.ConclusionThese findings suggest that first-episode and recurrent MDD have distinct effects on the effective connectivity in large-scale brain networks, which could be potential neural mechanisms underlying their different clinical manifestations
ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data
Radiology report generation, as a key step in medical image analysis, is
critical to the quantitative analysis of clinically informed decision-making
levels. However, complex and diverse radiology reports with cross-source
heterogeneity pose a huge generalizability challenge to the current methods
under massive data volume, mainly because the style and normativity of
radiology reports are obviously distinctive among institutions, body regions
inspected and radiologists. Recently, the advent of large language models (LLM)
offers great potential for recognizing signs of health conditions. To resolve
the above problem, we collaborate with the Second Xiangya Hospital in China and
propose ChatRadio-Valuer based on the LLM, a tailored model for automatic
radiology report generation that learns generalizable representations and
provides a basis pattern for model adaptation in sophisticated analysts' cases.
Specifically, ChatRadio-Valuer is trained based on the radiology reports from a
single institution by means of supervised fine-tuning, and then adapted to
disease diagnosis tasks for human multi-system evaluation (i.e., chest,
abdomen, muscle-skeleton, head, and maxillofacial neck) from six different
institutions in clinical-level events. The clinical dataset utilized in this
study encompasses a remarkable total of \textbf{332,673} observations. From the
comprehensive results on engineering indicators, clinical efficacy and
deployment cost metrics, it can be shown that ChatRadio-Valuer consistently
outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and
GPT-4 et al., in terms of the diseases diagnosis from radiology reports.
ChatRadio-Valuer provides an effective avenue to boost model generalization
performance and alleviate the annotation workload of experts to enable the
promotion of clinical AI applications in radiology reports
- …