Network Parameterisation and Activation Functions in Deep Learning

Deep learning, the study of multi-layered artificial neural networks, has received tremendous attention over the course of the last few years. Neural networks are now able to outperform humans in a growing variety of tasks and increasingly have an impact on our day-to-day lives. There is a wide range of potential directions to advance deep learning, two of which we investigate in this thesis:(1) One of the key components of a network are its activation functions. The activations have a big impact on the overall mathematical form of the network. The \textit{first paper} studies generalisation of neural networks with rectified linear activations units (“ReLUs”). Such networks partition the input space into so-called linear regions, which are the maximally connected subsets on which the network is affine. In contrast to previous work, which focused on obtaining estimates of the number of linear regions, we proposed a tropical algebra-based algorithm called TropEx to extract coefficients of the linear regions. Applied to fully-connected and convolutional neural networks, TropEx shows significant differences between the linear regions of these network types. The \textit{second paper} proposes a parametric rational activation function called ERA, which is learnable during network training. Although ERA only adds about ten parameters per layer, the activation significantly increases network expressivity and makes small architectures have a performance close to large ones. ERA outperforms previous activations when used in small architectures. This is relevant because neural networks keep growing larger and larger and the computational resources they require result in greater costs and electricity usage (which in turn increases the CO2 footprint).(2) For a given network architecture, each parameter configuration gives rise to a mathematical function. This functional realisation is far from unique and many different parameterisations can give rise to the same function. Changes to the parameterisation that do not change the function are called symmetries. The \textit{third paper} theoretically studies and classifies all the symmetries of 2-layer networks using the ReLU activation. Finally, the \textit{fourth paper} studies the effect of network parameterisation on network training. We provide a theoretical analysis of the effect that scaling layers have on the gradient updates. This provides a motivation for us to propose a Cooling method, which automatically scales the network parameters during training. Cooling reduces the reliance of the network on specific tricks, in particular the use of a learning rate schedule

The biochemical characterisation of respiratory mucus & mucins in normal, asthma & COPD

Includes abstract.Includes bibliographical references (leaves 78-87).Airway mucus hyper-secretion is the main cause of mortality and morbidity in respiratory diseases such as asthma and chronic obstructive pulmonary disease (COPD) and is the leading cause of death in South Africa. Mucus is a viscid, slimy visco-elastic gel-like material, which coats the epithelial tissue of gastrointestinal, reproductive and respiratory tracts. Mucus has defined rheological properties that enable it to be transported out of the lungs by mucociliary clearance

In situ syntheses of semiconducting nanoparticles in conjugated polymer matrices and their application in photovoltaics

Hybrid solar cells based on conjugated polymers and inorganic semiconducting nanoparticles combine beneficial properties of organic and inorganic semiconductors and are, therefore, an exciting alternative to pure organic or inorganic solar cell technologies. Several approaches for the fabrication of hybrid solar cells are already elaborated and explored. In the last years routes have emerged, where the nanoparticles are prepared directly in the matrix of the conjugated polymer. Here, the conjugated polymer prevents the nanoparticles from excessive growth and thereby makes additional capping agents obsolete. This review focuses on in situ preparation methods of inorganic semiconducting nanoparticles in conjugated polymers in view of applications in hybrid solar cells. The details, advantages and disadvantages of the different in situ methods are critically examined and put in comparison to the classical route where pre-synthesized nanoparticles are used. Various key factors influencing the solar cell performance as well as future strategies for increasing the overall efficiency of hybrid solar cells prepared via in situ routes are discussed

Advanced neuroimaging techniques in epilepsy

PURPOSE OF REVIEW: We review significant advances in epilepsy imaging in recent years. RECENT FINDINGS: Structural MRI at 7T with optimization of acquisition and postacquisition image processing increases the diagnostic yield but artefactual findings remain a challenge. MRI analysis from multiple sites indicates different atrophy patterns and white matter diffusion abnormalities in temporal lobe and generalized epilepsies, with greater abnormalities close to the presumed seizure source. Structural and functional connectivity relate to seizure spread and generalization; longitudinal studies are needed to clarify the causal relationship of these associations. Diffusion MRI may help predict surgical outcome and network abnormalities extending beyond the epileptogenic zone. Three-dimensional multimodal imaging can increase the precision of epilepsy surgery, improve seizure outcome and reduce complications. Language and memory fMRI are useful predictors of postoperative deficits, and lead to risk minimization. FDG PET is useful for clinical studies and specific ligands probe the pathophysiology of neurochemical fluxes and receptor abnormalities. SUMMARY: Improved structural MRI increases detection of abnormalities that may underlie epilepsy. Diffusion, structural and functional MRI indicate the widespread associations of epilepsy syndromes. These can assist stratification of surgical outcome and minimize risk. PET has continued utility clinically and for research into the pathophysiology of epilepsies

Zwischen Tradition und Spiegelung wirtschaftlicher und politischer Tendenzen

Meine Arbeit gibt einen Überblick über die Entwicklung der Wiener Festwochen. Dabei werden drei Untersuchungsgegenstände in Kontext der Wiener Festwochen gestellt: Tradition, Politik und Wirtschaft. Der erste Abschnitt zeigt die Geschichte des Festivals von der Entstehung und Gründung bis in die Jetztzeit. In den einzelnen Abschnitten wird das Festival programmkonzeptionell untersucht d.h. es wird versucht die Intentionen der Programmverantwortlichen wiederzugeben. Der nächste Abschnitt befasst sich mit dem politischen Hintergrund während der Geschichte der Wiener Festwochen. Von einer einleitenden Definition staatlicher Kulturpolitik werden nach und nach die Regierungsperioden und ihre Programme untersucht. Dabei handelt es sich vorrangig um kulturpolitische Maßnahmen und Entscheidungen. „Subvention“ ist ein wichtiges Schlagwort in diesem Zusammenhang, da mit staatlichen Förderungsmitteln politisch entschieden wird, welche Institutionen unterstützt werden können. Das letzte große Kapitel versucht einen Bogen zwischen Wirtschaft und Kultur zu spannen. Dabei werden Fragen wie: Wo liegt der Zusammenhang zwischen Kultur und Wirtschaft?, oder welche Bedeutung hat Kulturökonomie?, oder welche Beweggründe gibt es für Subventionierung?, beantwortet. Weiters wird die wirtschaftliche Eigenleistung der Wiener Festwochen behandelt.My essay shows an overview of the development of the Wiener Festwochen. The first part of this paper is about the history of the festival. It starts with the foundation of the Wiener Festwochen in 1951 and ends in the present time. In every single section I tried to examine the ideas of the person in charge and the resulting programs. The next chapter is about the political background. First there is an introductorily definition of governmental cultural and educational policy. After that there is a focus on the political programs of the last fifty years and how they affect the culture and specific the Wiener Festwochen. In this discussion "grant" is a keyword because national bounties were given by political decisions. The last chapter is about the importance of economic measures for cultural operations. I tried to answer questions like: "Where is the connection between culture and economy?", "Which denotation has cultural economics?", "Why there is financial aid by giving grants?". Also the commercial achievement of the Wiener Festwochen will be discussed

Transnationale Mobilitätsbarrieren

In dieser Arbeit wird der Frage nachgegangen, welche Faktoren für die physische Grenzen übergreifende Mobilität für transnationale Migranten eine Rolle spielen. Dafür werden einerseits persönliche Aspekte, wie biographische Erfahrungen, erworbene Kapitalien, Zugehörigkeitskonsrtuktionen und der Zugang zu sozialen Räumen und Netzen, und andererseits staatliche und gesellschaftliche Mechanismen, wie Migrations- und Staatsbürgerschaftspolitiken, oder Inklusion und Exklusion, als Faktoren analysiert. Die Arbeit basiert auf Literaturrecherche.This paper examines factors for physical mobility of transnational migrants over state borders. First it shows personal aspects like individual expiriences, different sorts of capital, opportunities to act in social fields and networks and memebership constructions, and second it analysis state and society mechanisms like migration- and citizenship politics, or inclusion and exclusion, as mobility factors. The paper is literary based

Qualität beim Spielfilm in Theorie und Praxis

Die vorliegende Arbeit widmet sich der Besprechung von Maßstäben, die in der derzeitigen österreichischen Filmförderpraxis zur Überprüfung der künstlerischen Qualität von Spielfilmen in Anwendung sind. Dabei sind zwei Bereiche zu unterscheiden, nämlich die Projektförderung und die Referenzfilmförderung. Das Forschungsziel ist es, zu eruieren, ob die jeweiligen Maßstäbe adäquat sind. Dazu wird ausgehend von entsprechender Literatur festgelegt, was die Qualität eines Films ausmacht. Ein Leitfaden, der sich großteils auf diese Erkenntnisse stützt, dient als Grundlage für Experteninterviews. Die Befragung richtet sich an Personen, die Erfahrung als Jury- oder Kommissionsmitglieder vorweisen können. Im Zuge von persönlichen Gesprächen wird die Beurteilungspraxis im Filmbereich erörtert und der Stellenwert der einzelnen Qualitätskriterien für den jeweiligen Befragten untersucht. Es zeigt sich, dass die Maßstäbe durchaus nicht unproblematisch in der Anwendung sind. Besonders im Bereich der Referenzfilmförderung sind Mängel feststellbar. Diese Ergebnisse legen nahe, dass seitens der österreichischen Filmpolitik eine Beschäftigung mit der Thematik angebracht wäre mit dem Ziel, den Status Quo zu optimieren. Da eine weitere Erhöhung des Budgets für das österreichische Filminstitut in Aussicht gestellt wurde, ist der Zeitpunkt ein guter, um präzisere Maßnahmen zur Überprüfung der künstlerischen Qualität von Filmen zu implementieren

Inspiratory oscillatory flow with a portable ventilator: a bench study

INTRODUCTION: We observed an oscillatory flow while ventilating critically ill patients with the Dräger Oxylog 3000™ transport ventilator during interhospital transfer. The phenomenon occurred in paediatric patients or in adult patients with severe airway obstruction ventilated in the pressure-regulated or pressure-controlled mode. As this had not been described previously, we conducted a bench study to investigate the phenomenon. METHODS: An Oxylog 3000™ intensive care unit ventilator and a Dräger Medical Evita-4 NeoFlow™ intensive care unit ventilator were connected to a Dräger Medical LS800™ lung simulator. Data were registered by a Datex-S5™ Monitor with a D-fend™ flow and pressure sensor, and were analysed with a laptop using S5-Collect™ software. Clinical conditions were simulated using various ventilatory modes, using various ventilator settings, using different filters and endotracheal tubes, and by changing the resistance and compliance. Data were recorded for 258 combinations of patient factors and respirator settings to detect thresholds for the occurrence of the phenomenon and methods to overcome it. RESULTS: Under conditions with high resistance in pressure-regulated ventilation with the Oxylog 3000™, an oscillatory flow during inspiration produced rapid changes of the airway pressure. The phenomenon resulted in a jerky inspiration with high peak airway pressures, higher than those set on the ventilator. Reducing the inspiratory flow velocity was effective to terminate the phenomenon, but resulted in reduced tidal volumes. CONCLUSION: Oscillatory flow with potentially harmful effects may occur during ventilation with the Dräger Oxylog 3000™, especially in conditions with high resistance such as small airways in children (endotracheal tube internal diameter <6 mm) or severe obstructive lung diseases or airway diseases in adult patients