9,241 research outputs found
On information captured by neural networks: connections with memorization and generalization
Despite the popularity and success of deep learning, there is limited
understanding of when, how, and why neural networks generalize to unseen
examples. Since learning can be seen as extracting information from data, we
formally study information captured by neural networks during training.
Specifically, we start with viewing learning in presence of noisy labels from
an information-theoretic perspective and derive a learning algorithm that
limits label noise information in weights. We then define a notion of unique
information that an individual sample provides to the training of a deep
network, shedding some light on the behavior of neural networks on examples
that are atypical, ambiguous, or belong to underrepresented subpopulations. We
relate example informativeness to generalization by deriving nonvacuous
generalization gap bounds. Finally, by studying knowledge distillation, we
highlight the important role of data and label complexity in generalization.
Overall, our findings contribute to a deeper understanding of the mechanisms
underlying neural network generalization.Comment: PhD thesi
Is attention all you need in medical image analysis? A review
Medical imaging is a key component in clinical diagnosis, treatment planning
and clinical trial design, accounting for almost 90% of all healthcare data.
CNNs achieved performance gains in medical image analysis (MIA) over the last
years. CNNs can efficiently model local pixel interactions and be trained on
small-scale MI data. The main disadvantage of typical CNN models is that they
ignore global pixel relationships within images, which limits their
generalisation ability to understand out-of-distribution data with different
'global' information. The recent progress of Artificial Intelligence gave rise
to Transformers, which can learn global relationships from data. However, full
Transformer models need to be trained on large-scale data and involve
tremendous computational complexity. Attention and Transformer compartments
(Transf/Attention) which can well maintain properties for modelling global
relationships, have been proposed as lighter alternatives of full Transformers.
Recently, there is an increasing trend to co-pollinate complementary
local-global properties from CNN and Transf/Attention architectures, which led
to a new era of hybrid models. The past years have witnessed substantial growth
in hybrid CNN-Transf/Attention models across diverse MIA problems. In this
systematic review, we survey existing hybrid CNN-Transf/Attention models,
review and unravel key architectural designs, analyse breakthroughs, and
evaluate current and future opportunities as well as challenges. We also
introduced a comprehensive analysis framework on generalisation opportunities
of scientific and clinical impact, based on which new data-driven domain
generalisation and adaptation methods can be stimulated
Generating Efficient Training Data via LLM-based Attribute Manipulation
In this paper, we propose a novel method, Chain-of-Thoughts Attribute
Manipulation (CoTAM), to guide few-shot learning by carefully crafted data from
Large Language Models (LLMs). The main idea is to create data with changes only
in the attribute targeted by the task. Inspired by facial attribute
manipulation, our approach generates label-switched data by leveraging LLMs to
manipulate task-specific attributes and reconstruct new sentences in a
controlled manner. Instead of conventional latent representation controlling,
we implement chain-of-thoughts decomposition and reconstruction to adapt the
procedure to LLMs. Extensive results on text classification and other tasks
verify the advantage of CoTAM over other LLM-based text generation methods with
the same number of training examples. Analysis visualizes the attribute
manipulation effectiveness of CoTAM and presents the potential of LLM-guided
learning with even less supervision
An empirical investigation of the relationship between integration, dynamic capabilities and performance in supply chains
This research aimed to develop an empirical understanding of the relationships between integration,
dynamic capabilities and performance in the supply chain domain, based on which, two conceptual
frameworks were constructed to advance the field. The core motivation for the research was that, at
the stage of writing the thesis, the combined relationship between the three concepts had not yet
been examined, although their interrelationships have been studied individually.
To achieve this aim, deductive and inductive reasoning logics were utilised to guide the qualitative
study, which was undertaken via multiple case studies to investigate lines of enquiry that would
address the research questions formulated. This is consistent with the author’s philosophical
adoption of the ontology of relativism and the epistemology of constructionism, which was considered
appropriate to address the research questions. Empirical data and evidence were collected, and
various triangulation techniques were employed to ensure their credibility. Some key features of
grounded theory coding techniques were drawn upon for data coding and analysis, generating two
levels of findings. These revealed that whilst integration and dynamic capabilities were crucial in
improving performance, the performance also informed the former. This reflects a cyclical and
iterative approach rather than one purely based on linearity. Adopting a holistic approach towards
the relationship was key in producing complementary strategies that can deliver sustainable supply
chain performance.
The research makes theoretical, methodological and practical contributions to the field of supply
chain management. The theoretical contribution includes the development of two emerging
conceptual frameworks at the micro and macro levels. The former provides greater specificity, as it
allows meta-analytic evaluation of the three concepts and their dimensions, providing a detailed
insight into their correlations. The latter gives a holistic view of their relationships and how they are
connected, reflecting a middle-range theory that bridges theory and practice. The methodological
contribution lies in presenting models that address gaps associated with the inconsistent use of
terminologies in philosophical assumptions, and lack of rigor in deploying case study research
methods. In terms of its practical contribution, this research offers insights that practitioners could
adopt to enhance their performance. They can do so without necessarily having to forgo certain
desired outcomes using targeted integrative strategies and drawing on their dynamic capabilities
Machine Learning Meets Mental Training -- A Proof of Concept Applied to Memory Sports
This work aims to combine these two fields together by presenting a practical
implementation of machine learning to the particular form of mental training
that is the art of memory, taken in its competitive version called "Memory
Sports". Such a fusion, on the one hand, strives to raise awareness about both
realms, while on the other it seeks to encourage research in this mixed field
as a way to, ultimately, drive forward the development of this seemingly
underestimated sport.Comment: 75 pages, 47 figures, 2 tables, 26 code excerpt
Soundscape in Urban Forests
This Special Issue of Forests explores the role of soundscapes in urban forested areas. It is comprised of 11 papers involving soundscape studies conducted in urban forests from Asia and Africa. This collection contains six research fields: (1) the ecological patterns and processes of forest soundscapes; (2) the boundary effects and perceptual topology; (3) natural soundscapes and human health; (4) the experience of multi-sensory interactions; (5) environmental behavior and cognitive disposition; and (6) soundscape resource management in forests
Towards Understanding the Effect of Pretraining Label Granularity
In this paper, we study how pretraining label granularity affects the
generalization of deep neural networks in image classification tasks. We focus
on the "fine-to-coarse" transfer learning setting where the pretraining label
is more fine-grained than that of the target problem. We experiment with this
method using the label hierarchy of iNaturalist 2021, and observe a 8.76%
relative improvement of the error rate over the baseline. We find the following
conditions are key for the improvement: 1) the pretraining dataset has a strong
and meaningful label hierarchy, 2) its label function strongly aligns with that
of the target task, and most importantly, 3) an appropriate level of
pretraining label granularity is chosen. The importance of pretraining label
granularity is further corroborated by our transfer learning experiments on
ImageNet. Most notably, we show that pretraining at the leaf labels of
ImageNet21k produces better transfer results on ImageNet1k than pretraining at
other coarser granularity levels, which supports the common practice.
Theoretically, through an analysis on a two-layer convolutional ReLU network,
we prove that: 1) models trained on coarse-grained labels only respond strongly
to the common or "easy-to-learn" features; 2) with the dataset satisfying the
right conditions, fine-grained pretraining encourages the model to also learn
rarer or "harder-to-learn" features well, thus improving the model's
generalization
Application of improved you only look once model in road traffic monitoring system
The present research focuses on developing an intelligent traffic management solution for tracking the vehicles on roads. Our proposed work focuses on a much better you only look once (YOLOv4) traffic monitoring system that uses the CSPDarknet53 architecture as its foundation. Deep-sort learning methodology for vehicle multi-target detection from traffic video is also part of our research study. We have included features like the Kalman filter, which estimates unknown objects and can track moving targets. Hungarian techniques identify the correct frame for the object. We are using enhanced object detection network design and new data augmentation techniques with YOLOv4, which ultimately aids in traffic monitoring. Until recently, object identification models could either perform quickly or draw conclusions quickly. This was a big improvement, as YOLOv4 has an astoundingly good performance for a very high frames per second (FPS). The current study is focused on developing an intelligent video surveillance-based vehicle tracking system that tracks the vehicles using a neural network, image-based tracking, and YOLOv4. Real video sequences of road traffic are used to test the effectiveness of the method that has been suggested in the research. Through simulations, it is demonstrated that the suggested technique significantly increases graphics processing unit (GPU) speed and FSP as compared to baseline algorithms
Rational development of stabilized cyclic disulfide redox probes and bioreductive prodrugs to target dithiol oxidoreductases
Countless biological processes allow cells to develop, survive, and proliferate. Among these, tightly balanced regulatory enzymatic pathways that can respond rapidly to external impacts maintain dynamic physiological homeostasis. More specifically, redox homeostasis broadly affects cellular metabolism and proliferation, with major contributions by thiol/disulfide oxidoreductase systems, in particular, the Thioredoxin Reductase Thioredoxin (TrxR/Trx) and the Glutathione Reductase-Glutathione-Glutaredoxin (GR/GSH/Grx) systems.
These cascades drive vital cellular functions in many ways through signaling, regulating other proteins' activity by redox switches, and by stoichiometric reductant transfers in metabolism and antioxidant systems. Increasing evidence argues that there is a persistent alteration of the redox environment in certain pathological states, such as cancer, that heavily involve the Trx system: upregulation and/or overactivity of the Trx system may support or drive cancer progression, making both TrxR and Trx promising targets for anti-cancer drug development.
Understanding the biochemical mechanisms and connections between certain redox cascades requires research tools that interact with them. The state-of-the-art genetic tools are mostly ratiometric reporters that measure reduced:oxidized ratios of selected redox pairs or the general thiol pool. However, the precise cellular roles of the central oxidoreductase systems, including TrxR and Trx, remain inaccessible due to the lack of probes to selectively measure turnover by either of these proteins. However, such probes would allow measuring their effective reductive activity apart from expression levels in native systems, including in cells, animals, or patient samples. They are also of high interest to identify chemical inhibitors for TrxR/Trx in cells and to validate their potential use as anti-cancer agents (to date, there is no selective cellular Trx inhibitor, and most known TrxR inhibitors were not comprehensively evaluated considering selectivity and potential off-targets). However, small molecule redox imaging tools are underdeveloped: their protein specificity, spectral properties, and applicability remain poorly precedented.
This work aimed to address this opportunity gap and develop novel, small molecule diagnostic and therapeutic tools to selectively target the Trx system based on a modular trigger cargo design: artificial cyclic disulfide substrates (trigger) for oxidoreductases are tethered to molecular agents (cargo) such that the cargo’s activity is masked and is re-established only through reduction by a target protein.
The rational design of these novel reduction sensors to target the cell's strongest disulfide-reducing enzymes was driven by the following principles: (i) cyclic disulfide triggers with stabilized ring systems were used to gain low reduction potentials that should resist reduction except by the strongest cellular reductases, such as Trx; and (ii) the cyclic topology also offers the potential for kinetic reversibility that should select for dithiol-type redox proteins over the cellular monothiol background. Creating imaging agents based on such two-component designs to selectively measure redox protein activity in native cells required to combine the correct trigger reducibility, probe activation kinetics, and imaging modalities and to consider the overall molecular architecture.
The major prior art in this field has applied cyclic 5-membered disulfides (1,2 dithiolanes) as substrates for TrxR in a similar way to create such tools. However, this motif was described elsewhere as thermodynamically instable and was due to widely used for dynamic covalent cascade reactions. By comparing a novel 1,2 dithiolane-based probe to the state-of-the-art probes, including commercial TrxR sensors, by screening a conclusive assay panel of cellular TrxR modulations, I clarified that 1,2 dithiolanes are not selective substrates for TrxR in biological settings (Nat Commun 2022).
Instead, aiming for more stable ring systems and thus more robust redox probes, during this work, I developed bicyclic 6 membered disulfides (piperidine fused 1,2 dithianes) with remarkably low reduction potentials. I showed that molecular probes using them as reduction sensors can be mostly processed by thioredoxins while being stable against reduction by GSH. The thermodynamically stabilized decalin like topology of the cis-annelated 1,2 dithianes requires particularly strong reductants to be cleaved. They also select for dithiol type redox proteins, like Trx, based on kinetic reversibility and offer fast cyclization due to the preorganization by annelation (JACS 2021).
This work further expanded the system’s modularity with structural cores based on piperazine-fused 1,2 dithianes with the two amines allowing independent derivatization. Diagnostic tools using them as reduction sensors proved equally robust but with highly improved activation kinetics and were thus cellularly activated. Cellular studies evolved that they are substrates for both Trxs and their protein cousins Grxs, so measuring the cellular dithiol protein pool rather than solely Trx activity (preprint 2023).
Finally, a trigger based on a slightly adapted reduction sensor, a desymmetrized 1,2 thiaselenane, was designed for selective reduction by TrxR’s selenol/thiol active site, then combined with a precipitating large Stokes’ shift fluorophore and a solubilizing group, to evolve the first selective probe RX1 to measure cellular TrxR activity, which even allowed high throughput inhibitor screening (Chem 2022).
The central principle of this work was further advanced to therapeutic prodrugs based on the duocarmycin cargo (CBI) with tunable potency (JACS Au 2022) that can be used to create off-to-on therapeutic prodrugs. Such CBI prodrugs employing stabilized 1,2 dichalcogenide triggers proved to be cytotoxins that depend on Trx system activity in cells. They could further be exploited for cell-line dependent reductase activity profiling by screening their redox activation indices, the reduction-dependent part of total prodrug activation, in 177 cell lines. Beyond that, these prodrugs were well-tolerated in animals and showed anti-cancer efficacy in vivo in two distinct mouse tumor models (preprint 2022).
Taken together, I introduced unique monothiol-resistant reducible motifs to target the cellular Trx system with chemocompatible units for each for TrxR and Trx/Grx, where the cyclic nature of the dichalcogenides avoids activation by GSH. By using them with distinct molecular cargos, I developed novel selective fluorescent reporter probes; and introduced a new class of bioreductive therapeutic constructs based on a common modular design. These were either applied to selectively measure cellular reductase activity or to deliver cytotoxic anti cancer agents in vivo. Ongoing work aims to differentiate between the two major redox effector proteins Trx and Grx, requiring additional layers of selectivity that may be addressed by tuned molecular recognition. The flexible use of various molecular cargos allows harnessing the same cellular redox machinery by either probes or prodrugs. This allows predictive conclusions from diagnostics to be directly translated into therapy and offers great potential for future adaptation to other enzyme classes and therapeutic venues.Die zelluläre Redox-Homöostase hängt von Thiol/Disulfid-Oxidoreduktasen ab, die den Stoffwechsel, die Proliferation und die antioxidative Antwort von Zellen beeinflussen. Die wichtigsten Netzwerke sind die Thioredoxin Reduktase-Thioredoxin (TrxR/Trx) und Glutathion Reduktase-Glutathion-Glutaredoxin (GR/GSH/Grx) Systeme, die über Redox-Schalter in Substratproteinen lebenswichtige zelluläre Funktionen steuern und so an der Redox-Regulation und -Signalübertragung beteiligt sind. Persistente Veränderungen des Redoxmilieus in pathologischen Zuständen, wie z. B. bei Krebs, sind in hohem Maße mit dem Trx-System verbunden. Eine Hochregulierung und/oder Überaktivität des Trx-Systems, die bei vielen Krebsarten auftreten, unterstützt zudem das Fortschreiten des Krebswachstums, was TrxR/Trx zu vielversprechenden Zielproteinen für die Entwicklung neuer Krebsmedikamente macht.
Um die biochemischen Prozesse dahinter zu erforschen, sind spezielle Techniken zur Visualisierung und Messung enzymatischer Aktivität nötig. Die hierzu geeigneten, meist genetischen Sensoren messen ratiometrisch das Verhältnis reduzierter/oxidierter Spezies in zellulärem Umfeld oder spezifisch ausgewählte Redoxpaare. Die weitere Erforschung der exakten Funktion von TrxR/Trx und deren Substrate ist jedoch durch mangelnde Nachweismethoden limitiert. Diese sind außerdem zur Validierung chemischer Hemmstoffe für TrxR/Trx in Zellen und deren potenziellen Verwendung als Krebsmittel von großem Interesse. Bislang gibt es keinen selektiven zellulären Trx-Inhibitor und potenzielle Off-Target-Effekte der bekannten TrxR-Inhibitoren wurden nicht abschließend bewertet.
Ziel dieser Arbeit ist die Entwicklung niedermolekularer, diagnostischer und therapeutischer Werkzeuge, die selektiv auf das Trx-System abzielen und auf einem modularen Trigger-Cargo Design basieren. Hierzu werden zyklische Disulfid-Substrate (Trigger) für Oxidoreduktasen so mit molekularen Wirkstoffen (Cargo) verknüpft, dass dabei die Wirkstoffaktivität maskiert, und erst nach Reduktion durch ein Zielprotein wiederhergestellt wird. Diese neuartigen, synthetischen Reduktionssensoren basieren auf den folgenden Grundprinzipien: (i) Zyklische Disulfide sind thermodynamisch stabilisiert und können nur durch die stärksten Reduktasen gespalten werden; und (ii) die zyklische Topologie ermöglicht die kinetische Reversibilität der zwei Thiol-Disulfid-Austauschreaktionen, die eine erste Reaktion mit Monothiolen, wie z. B. GSH, sofort umkehrt und so eine vollständige Reduktion verhindert.
Die meisten früheren Arbeiten auf diesem Gebiet verwendeten ein zyklisches, fünfgliedriges Disulfid (1,2 Dithiolan) als Substrat für TrxR. Das gleiche Strukturmotiv wurde jedoch an anderer Stelle als thermodynamisch instabil beschrieben und aufgrund dieser Eigenschaft explizit für dynamische Kaskadenreaktionen verwendet. Deshalb vergleicht diese Arbeit zu Beginn einen neuen 1,2 Dithiolan basierten fluorogenen Indikator mit bestehenden, z. T. kommerziellen, Redox Sonden für TrxR in einer Reihe von Zellkultur-Experimenten unter Modulation der zellulären TrxR Aktivität und stellt so einen Widerspruch in der Literatur klar: 1,2 Dithiolane eignen sich nicht als selektive Substrate für TrxR, da sie labil sowohl gegen die Reduktion durch andere Redoxproteine, als auch gegen den Monothiol Hintergrund in Zellen sind (Nat. Commun. 2022).
Als alternatives Strukturmotiv wird in dieser Arbeit ein bizyklisches sechsgliedriges Disulfid (anneliertes 1,2 Dithian) etabliert. Durch sein niedriges Reduktionspotenzial, also seine hohe Resistenz gegen Reduktion, werden molekulare Sonden basierend auf diesem 1,2 Dithian als Reduktionssensor fast ausschließlich von Trx aktiviert, nicht aber von TrxR oder GSH (JACS 2021). Dieses Kernmotiv bestimmt dabei die Reduzierbarkeit, und damit die Enzymspezifität, durch seine zyklische Natur und die Annelierung, auch unter Verwendung unterschiedlicher Farb-/Wirkstoffe. Auf dieser Grundlage konnte die molekulare Struktur durch einen weiteren Modifikationspunkt für die flexible Verwendung weiterer funktioneller Einheiten ergänzt werden. Obwohl zelluläre Studien ergaben, dass diese neuartigen 1,2 Dithian Einheiten in Zellen sowohl Trx als auch das strukturell verwandte Grx adressieren, sind die daraus resultierenden diagnostischen Moleküle wertvoll, um den katalytischen Umsatz zellulärer Dithiol-Reduktasen, der sogenannten Trx Superfamilie, selektiv anzuzeigen (Preprint 2023).
Begünstigt durch das modulare Moleküldesign stellt diese Arbeit zudem das erste Reportersystem RX1 zum selektiven Nachweis der TrxR-Aktivität in Zellen vor. Es basiert auf der Verwendung eines zyklischen, unsymmetrischen Selenenylsulfid-Sensors (1,2 Thiaselenan), der selektiv von dem einzigartigen Selenolat der TrxR angegriffen wird, und dadurch letztlich nur von TrxR reduziert werden kann. RX1 eignete sich zudem für eine Hochdurchsatz-Validierung bestehender TrxR Inhibitoren und unterstreicht dadurch den kommerziellen Nutzen derartiger Diagnostika (Chem 2022).
Das zentrale Trigger-Cargo Konzept dieser Arbeit wurde für therapeutische Zwecke weiterentwickelt und nutzt dabei den einzigartigen Wirkmechanismus der Duocarmycin-Naturstoffklasse (CBI) (JACS Au 2022) zur Entwicklung reduktiv aktivierbarer Therapeutika. CBI Prodrugs basierend auf stabilisierten Redox-Schaltern (1,2 Dithiane für Trx; 1,2 Thiaselenan für TrxR) reagierten signifikant auf TrxR-Modulation in Zellen. Sie wurden darüber hinaus durch das Referenzieren ihrer Aktivität gegenüber nicht-reduzierbaren Kontrollmoleküle für die Erstellung zelllinienabhängiger Profile der Reduktaseaktivität in 177 Zelllinien genutzt. Schließlich waren diese neuen Krebsmittel im Tiermodell gut verträglich und zeigten in zwei verschiedenen Mausmodellen eine krebshemmende Wirkung (Preprint 2022b).
Zusammenfassend präsentiert diese Dissertation monothiol-resistente reduzierbare Trigger-Einheiten für das zelluläre Trx-System zur Entwicklung neuartiger, selektiver Reporter-Sonden, sowie eine neue Klasse reduktiv aktivierbarer Krebsmittel auf Basis eines adaptierbaren Trigger-Cargo Designs. Diese fanden entweder zur selektiven Messung zellulärer Proteinaktivität oder zum Einsatz als Antikrebsmittel Verwendung. Es wurden chemokompatible Motive sowohl für TrxR als auch für Trx/Grx identifiziert, wobei deren zyklische Natur eine Aktivierung durch GSH verhindert. Eine weitere Differenzierung zwischen den beiden Redox-Proteinen Trx und Grx und anderen Proteinen der Trx-Superfamilie erfordert eine zusätzliche Ebene der Selektierung, z. B. durch molekulare Erkennung, und ist Gegenstand laufender Arbeiten.
Die flexible Verwendung verschiedener molekularer Wirkstoffe ermöglicht dabei die „Pipeline-Entwicklung“ von Diagnostika und Therapeutika, die von der zellulären Redox-Maschinerie analog umgesetzt werden, und dadurch Schlussfolgerungen aus der Diagnostik direkt auf eine Therapie übertragbar machen. Dies birgt großes Potenzial für künftige Entwicklungen bei einer potenziellen Übertragung des modularen Konzepts auf andere Enzymklassen und therapeutische Einsatzgebiete
Synthetic Aperture Radar (SAR) Meets Deep Learning
This reprint focuses on the application of the combination of synthetic aperture radars and depth learning technology. It aims to further promote the development of SAR image intelligent interpretation technology. A synthetic aperture radar (SAR) is an important active microwave imaging sensor, whose all-day and all-weather working capacity give it an important place in the remote sensing community. Since the United States launched the first SAR satellite, SAR has received much attention in the remote sensing community, e.g., in geological exploration, topographic mapping, disaster forecast, and traffic monitoring. It is valuable and meaningful, therefore, to study SAR-based remote sensing applications. In recent years, deep learning represented by convolution neural networks has promoted significant progress in the computer vision community, e.g., in face recognition, the driverless field and Internet of things (IoT). Deep learning can enable computational models with multiple processing layers to learn data representations with multiple-level abstractions. This can greatly improve the performance of various applications. This reprint provides a platform for researchers to handle the above significant challenges and present their innovative and cutting-edge research results when applying deep learning to SAR in various manuscript types, e.g., articles, letters, reviews and technical reports
- …