Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Bayesian optimization has become a successful tool for hyperparameter optimization of machine learning algorithms, such as support vector machines or deep neural networks. Despite its success, for large datasets, training and validating a single configuration often takes hours, days, or even weeks, which limits the achievable performance. To accelerate hyperparameter optimization, we propose a generative model for the validation error as a function of training set size, which is learned during the optimization process and allows exploration of preliminary configurations on small subsets, by extrapolating to the full dataset. We construct a Bayesian optimization procedure, dubbed Fabolas, which models loss and training time as a function of dataset size and automatically trades off high information gain about the global optimum against computational cost. Experiments optimizing support vector machines and deep neural networks show that Fabolas often finds high-quality solutions 10 to 100 times faster than other state-of-the-art Bayesian optimization methods or the recently proposed bandit strategy Hyperband

Network Structure and User Roles of a Crowdsourcing Community – The Context of Social Innovations for a Development Project

The principles of crowdsourcing are increasingly applied in social contexts like development projects. In this study we explore a crowdsourcing community, which aims to enhance conditions in low income communities. We investigate the network structures of the community and detect behavioral pattern and user roles based on participation behavior for this specific context. Overall, the observed community shows a high level of collaboration and reciprocal dialogue. On the individual level we located four different user roles distinct in their interaction and contribution behavior. So called “collaborators” are considered as unique user role in an online community within a social context. We contribute to the theory of crowdsourcing by illustrating that context and purpose of crowdsourcing initiatives may influence the behavioral pattern of users. Further we add insights to the junctures between crowdsourcing and social innovation in the context of open development

Particle image velocimetry in a foam-like porous structure using refractive index matching: a method to characterize the hydrodynamic performance of porous structures

We present a method to measure two-dimensional velocity fields inside an artificial foam-like porous structure using particle image velocimetry and a refractive index matching technique to avoid optical distortion. The porous structure is manufactured by stereolithography with the epoxy resin WaterShed® XC 11122 as solid material, and anisole is used as refractive index-matched fluid. It was found that the direction of build-up of the stereolithographic structure plays an important role for the quality of the recorded images. The velocity fields measured in this study and the turbulent statistics derived thereof allow to characterize the hydrodynamic performance of the artificial foam-like structure and clarify the mechanisms of mixing. Results from this study compare well to results from a large eddy simulation reported by Hutter et al. (Chem Eng Sci 66:519–529, 2011b) and hence reinforce these simulations.Switzerland. Commission for Technology and Innovation (CTI) (DSM Nutritional Products and Premex Reactor AG

Particle image velocimetry in a foam-like porous structure using refractive index matching: a method to characterize the hydrodynamic performance of porous structures

We present a method to measure two-dimensional velocity fields inside an artificial foam-like porous structure using particle image velocimetry and a refractive index matching technique to avoid optical distortion. The porous structure is manufactured by stereolithography with the epoxy resin WaterShed® XC 11122 as solid material, and anisole is used as refractive index-matched fluid. It was found that the direction of build-up of the stereolithographic structure plays an important role for the quality of the recorded images. The velocity fields measured in this study and the turbulent statistics derived thereof allow to characterize the hydrodynamic performance of the artificial foam-like structure and clarify the mechanisms of mixing. Results from this study compare well to results from a large eddy simulation reported by Hutter etal. (Chem Eng Sci 66:519-529, 2011b) and hence reinforce these simulation

Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars

The discovery of neural architectures from simple building blocks is a long-standing goal of Neural Architecture Search (NAS). Hierarchical search spaces are a promising step towards this goal but lack a unifying search space design framework and typically only search over some limited aspect of architectures. In this work, we introduce a unifying search space design framework based on context-free grammars that can naturally and compactly generate expressive hierarchical search spaces that are 100s of orders of magnitude larger than common spaces from the literature. By enhancing and using their properties, we effectively enable search over the complete architecture and can foster regularity. Further, we propose an efficient hierarchical kernel design for a Bayesian Optimization search strategy to efficiently search over such huge spaces. We demonstrate the versatility of our search space design framework and show that our search strategy can be superior to existing NAS approaches. Code is available at https://github.com/automl/hierarchical_nas_construction

The Rhetoric of Solidarity: Nature and Measurement of Social Cohesion in the Self-representation of Civil Society Organizations

Scholars have called to study how social cohesion is discursively negotiated and produced in communication behavior. However, empirical evidence remains scarce. In this study, we investigate to what extent and how civil society organizations (CSOs), part of the backbone of social integration in modern democracies, make references to social cohesion in their public self-portrayals. We develop a standardized measure for content analyzing the manifestation of social cohesion along three theoretical dimensions: social relations, connectedness, and orientation towards the common good. We apply our innovative content measure to the external communication of an original sample of nearly 800 CSOs in Germany, using their websites. Subsequently, we use data from an accompanying organizational survey of these institutions to investigate whether and how certain organizational features help explain variance in social cohesion rhetoric. Findings suggest that CSOs’ external communications employ themes from all key dimensions of social cohesion, revealing a fair amount of variation on all three subdimensions and a summary index of the overall strength social cohesion rhetoric. These different emphases are contingent upon various organizational characteristics, namely the spheres in which CSOs are primarily active, their locations, and their target groups. Whereas culturally and media-oriented organizations as well as sports clubs are largely reluctant to make references to social cohesion, politically active CSOs and those addressing socially disadvantaged communities tend to push more in this direction. The latter tend to operate in more professionalized structures, indicating that referencing social cohesion legitimizes these groups’ political and social purposes in the public sphere

Fighting the wicked problem of plastic pollution and its consequences for developing regions with expert and crowd solutions

The wicked problem of plastic pollution is one of the key global challenges. Finding adequate solutions to this complex problem requires cross-cultural and inter-organizational collaboration among diverse sets of stakeholders. In this context, the Ellen Mac Arthur Foundation approaches the problem of plastic pollution not only by involving experts into innovation processes but also by integrating the general public in form of an IT enabled crowdsourcing initiative. In this study, we analyze the outcomes of these actions with the help of automated text mining techniques. Our analysis demonstrates significant differences between the solutions given by experts and the crowd along various criteria. Further, this study provides guidance for practitioners on how to integrate diverse sets of individuals in problem solving processes with the help of information systems technologies. Especially for sustainability issues affecting both, developed and developing regions

Exploring conformational equilibria of a heterodimeric ABC transporter

ABC exporters pump substrates across the membrane by coupling ATP-driven movements of nucleotide binding domains (NBDs) to the transmembrane domains (TMDs), which switch between inward- and outward-facing (IF, OF) orientations. DEER measurements on the heterodimeric ABC exporter TM287/288 from Thermotoga maritima, which contains a non-canonical ATP binding site, revealed that in the presence of nucleotides the transporter exists in an IF/OF equilibrium. While ATP binding was sufficient to partially populate the OF state, nucleotide trapping in the pre- or post-hydrolytic state was required for a pronounced conformational shift. At physiologically high temperatures and in the absence of nucleotides, the NBDs disengage asymmetrically while the conformation of the TMDs remains unchanged. Nucleotide binding at the degenerate ATP site prevents complete NBD separation, a molecular feature differentiating heterodimeric from homodimeric ABC exporters. Our data suggest hydrolysis-independent closure of the NBD dimer, which is further stabilized as the consensus site nucleotide is committed to hydrolysis

Gated Linear Networks

This paper presents a new family of backpropagation-free neural architectures, Gated Linear Networks (GLNs). What distinguishes GLNs from contemporary neural networks is the distributed and local nature of their credit assignment mechanism; each neuron directly predicts the target, forgoing the ability to learn feature representations in favor of rapid online learning. Individual neurons can model nonlinear functions via the use of data-dependent gating in conjunction with online convex optimization. We show that this architecture gives rise to universal learning capabilities in the limit, with effective model capacity increasing as a function of network size in a manner comparable with deep ReLU networks. Furthermore, we demonstrate that the GLN learning mechanism possesses extraordinary resilience to catastrophic forgetting, performing comparably to a MLP with dropout and Elastic Weight Consolidation on standard benchmarks. These desirable theoretical and empirical properties position GLNs as a complementary technique to contemporary offline deep learning methods.Comment: arXiv admin note: substantial text overlap with arXiv:1712.0189

Crisis, what crisis? Regulation and the academic orthodoxy

What can criminology or socio-legal studies tell us about the causes of the financial crisis – a failure of regulation, at the very least – or ways in which further such crises might be prevented, mitigated, responded to? The article begins by setting out the emergence and dimensions of the academic orthodoxy on regulation – a series of shared assumptions regarding feasible and desirable forms of regulation. Then it undertakes quantitative and qualitative content analysis of work on regulation and the crisis to assess the extent to which this orthodoxy has been reassessed in the light of events since 2007