A Survey on Continuous Time Computations

We provide an overview of theories of continuous time computation. These theories allow us to understand both the hardness of questions related to continuous time dynamical systems and the computational power of continuous time analog models. We survey the existing models, summarizing results, and point to relevant references in the literature

Empirical Bounds on Linear Regions of Deep Rectifier Networks

We can compare the expressiveness of neural networks that use rectified linear units (ReLUs) by the number of linear regions, which reflect the number of pieces of the piecewise linear functions modeled by such networks. However, enumerating these regions is prohibitive and the known analytical bounds are identical for networks with same dimensions. In this work, we approximate the number of linear regions through empirical bounds based on features of the trained network and probabilistic inference. Our first contribution is a method to sample the activation patterns defined by ReLUs using universal hash functions. This method is based on a Mixed-Integer Linear Programming (MILP) formulation of the network and an algorithm for probabilistic lower bounds of MILP solution sets that we call MIPBound, which is considerably faster than exact counting and reaches values in similar orders of magnitude. Our second contribution is a tighter activation-based bound for the maximum number of linear regions, which is particularly stronger in networks with narrow layers. Combined, these bounds yield a fast proxy for the number of linear regions of a deep neural network.Comment: AAAI 202

Delay-Based Controller Design for Continuous-Time and Hybrid Applications

Motivated by the availability of different types of delays in embedded systems and biological circuits, the objective of this work is to study the benefits that delay can provide in simplifying the implementation of controllers for continuous-time systems. Given a continuous-time linear time-invariant (LTI) controller, we propose three methods to approximate this controller arbitrarily precisely by a simple controller composed of delay blocks, a few integrators and possibly a unity feedback. Different problems associated with the approximation procedures, such as finding the optimal number of delay blocks or studying the robustness of the designed controller with respect to delay values, are then investigated. We also study the design of an LTI continuous-time controller satisfying given control objectives whose delay-based implementation needs the least number of delay blocks. A direct application of this work is in the sampled-data control of a real-time embedded system, where the sampling frequency is relatively high and/or the output of the system is sampled irregularly. Based on our results on delay-based controller design, we propose a digital-control scheme that can implement every continuous-time stabilizing (LTI) controller. Unlike a typical sampled-data controller, the hybrid controller introduced here -— consisting of an ideal sampler, a digital controller, a number of modified second-order holds and possibly a unity feedback -— is robust to sampling jitter and can operate at arbitrarily high sampling frequencies without requiring expensive, high-precision computation

Exploratory Analysis of Functional Data via Clustering and Optimal Segmentation

We propose in this paper an exploratory analysis algorithm for functional data. The method partitions a set of functions into $K$ clusters and represents each cluster by a simple prototype (e.g., piecewise constant). The total number of segments in the prototypes, $P$, is chosen by the user and optimally distributed among the clusters via two dynamic programming algorithms. The practical relevance of the method is shown on two real world datasets