Neural Architecture Search as Program Transformation Exploration

Improving the performance of deep neural networks (DNNs) is important to both the compiler and neural architecture search (NAS) communities. Compilers apply program transformations in order to exploit hardware parallelism and memory hierarchy. However, legality concerns mean they fail to exploit the natural robustness of neural networks. In contrast, NAS techniques mutate networks by operations such as the grouping or bottlenecking of convolutions, exploiting the resilience of DNNs. In this work, we express such neural architecture operations as program transformations whose legality depends on a notion of representational capacity. This allows them to be combined with existing transformations into a unified optimization framework. This unification allows us to express existing NAS operations as combinations of simpler transformations. Crucially, it allows us to generate and explore new tensor convolutions. We prototyped the combined framework in TVM and were able to find optimizations across different DNNs, that significantly reduce inference time - over 3$\times$ in the majority of cases. Furthermore, our scheme dramatically reduces NAS search time. Code is available at~\href{https://github.com/jack-willturner/nas-as-program-transformation-exploration}{this https url}

Controlling spatiotemporal dynamics of flame fronts

In certain mixtures of fuel and oxidizer, propagating flame fronts may exhibit both stable and unstable cellular structures. Such flames represent spatially extended chemical systems, with coupling from diffusion of heat and reactants. A new algorithm is proposed that allows the stabilization and tracking of a steady, two-cell front through a bifurcation sequence that eventually leads to chaotic behavior. Periodic modes of the front can also be stabilized and tracked. The system is stabilized by monitoring one experimentally accessible variable and perturbing one boundary condition. No knowledge of the detailed dynamics of the system (i.e., the underlying governing equations) is required to implement the tracking method. The algorithm automatically provides information about the locations of the unstable steady states and periodic orbits and the magnitudes of the associated eigenvalues and Ploquet multiplier

Observing a Dynamical Skeleton of Turbulence in Taylor-Couette Flow Experiments

Recent work suggests unstable recurrent solutions of the equations governing fluid flow can play an important role in structuring the dynamics of turbulence. Here we present a method for detecting intervals of time where turbulence "shadows" (spatially and temporally mimics) recurrent solutions. We find that shadowing occurs frequently and repeatedly in both numerical and experimental observations of counter-rotating Taylor-Couette flow, despite the relatively small number of known recurrent solutions in this system. Our results set the stage for experimentally-grounded dynamical descriptions of turbulence in a variety of wall-bounded shear flows, enabling applications to forecasting and control

Optimizing Grouped Convolutions on Edge Devices

When deploying a deep neural network on constrained hardware, it is possible to replace the network's standard convolutions with grouped convolutions. This allows for substantial memory savings with minimal loss of accuracy. However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. In this paper we propose Grouped Spatial Pack Convolutions (GSPC), a new implementation of grouped convolutions that outperforms existing solutions. We implement GSPC in TVM, which provides state-of-the-art performance on edge devices. We analyze a set of networks utilizing different types of grouped convolutions and evaluate their performance in terms of inference time on several edge devices. We observe that our new implementation scales well with the number of groups and provides the best inference times in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4x, 8x and 4x on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/Comment: Camera ready version to be published at ASAP 2020 - The 31st IEEE International Conference on Application-specific Systems, Architectures and Processors. 8 pages, 6 figure

Phaselocked patterns and amplitude death in a ring of delay coupled limit cycle oscillators

We study the existence and stability of phaselocked patterns and amplitude death states in a closed chain of delay coupled identical limit cycle oscillators that are near a supercritical Hopf bifurcation. The coupling is limited to nearest neighbors and is linear. We analyze a model set of discrete dynamical equations using the method of plane waves. The resultant dispersion relation, which is valid for any arbitrary number of oscillators, displays important differences from similar relations obtained from continuum models. We discuss the general characteristics of the equilibrium states including their dependencies on various system parameters. We next carry out a detailed linear stability investigation of these states in order to delineate their actual existence regions and to determine their parametric dependence on time delay. Time delay is found to expand the range of possible phaselocked patterns and to contribute favorably toward their stability. The amplitude death state is studied in the parameter space of time delay and coupling strength. It is shown that death island regions can exist for any number of oscillators N in the presence of finite time delay. A particularly interesting result is that the size of an island is independent of N when N is even but is a decreasing function of N when N is odd.Comment: 23 pages, 12 figures (3 of the figures in PNG format, separately from TeX); minor additions; typos correcte

Exotic torus manifolds and equivariant smooth structures on quasitoric manifolds

In 2006 Masuda and Suh asked if two compact non-singular toric varieties having isomorphic cohomology rings are homeomorphic. In the first part of this paper we discuss this question for topological generalizations of toric varieties, so-called torus manifolds. For example we show that there are homotopy equivalent torus manifolds which are not homeomorphic. Moreover, we characterize those groups which appear as the fundamental groups of locally standard torus manifolds. In the second part we give a classification of quasitoric manifolds and certain six-dimensional torus manifolds up to equivariant diffeomorphism. In the third part we enumerate the number of conjugacy classes of tori in the diffeomorphism group of torus manifolds. For torus manifolds of dimension greater than six there are always infinitely many conjugacy classes. We give examples which show that this does not hold for six-dimensional torus manifolds.Comment: 21 pages, 2 figures, results about quasitoric manifolds adde