Practical recommendations for gradient-based training of deep architectures

Learning algorithms related to artificial neural networks and in particular for Deep Learning may seem to involve many bells and whistles, called hyper-parameters. This chapter is meant as a practical guide with recommendations for some of the most commonly used hyper-parameters, in particular in the context of learning algorithms based on back-propagated gradient and gradient-based optimization. It also discusses how to deal with the fact that more interesting results can be obtained when allowing one to adjust many hyper-parameters. Overall, it describes elements of the practice used to successfully and efficiently train and debug large-scale and often deep multi-layer neural networks. It closes with open questions about the training difficulties observed with deeper architectures

On Verifying Complex Properties using Symbolic Shape Analysis

One of the main challenges in the verification of software systems is the analysis of unbounded data structures with dynamic memory allocation, such as linked data structures and arrays. We describe Bohne, a new analysis for verifying data structures. Bohne verifies data structure operations and shows that 1) the operations preserve data structure invariants and 2) the operations satisfy their specifications expressed in terms of changes to the set of objects stored in the data structure. During the analysis, Bohne infers loop invariants in the form of disjunctions of universally quantified Boolean combinations of formulas. To synthesize loop invariants of this form, Bohne uses a combination of decision procedures for Monadic Second-Order Logic over trees, SMT-LIB decision procedures (currently CVC Lite), and an automated reasoner within the Isabelle interactive theorem prover. This architecture shows that synthesized loop invariants can serve as a useful communication mechanism between different decision procedures. Using Bohne, we have verified operations on data structures such as linked lists with iterators and back pointers, trees with and without parent pointers, two-level skip lists, array data structures, and sorted lists. We have deployed Bohne in the Hob and Jahob data structure analysis systems, enabling us to combine Bohne with analyses of data structure clients and apply it in the context of larger programs. This report describes the Bohne algorithm as well as techniques that Bohne uses to reduce the ammount of annotations and the running time of the analysis

Explicit formula for the generating series of diagonal 3D rook paths

Let $a_n$ denote the number of ways in which a chess rook can move from a corner cell to the opposite corner cell of an $n \times n \times n$ three-dimensional chessboard, assuming that the piece moves closer to the goal cell at each step. We describe the computer-driven \emph{discovery and proof} of the fact that the generating series $G(x)= \sum_{n \geq 0} a_n x^n$ admits the following explicit expression in terms of a Gaussian hypergeometric function: G(x) = 1 + 6 \cdot \int_0^x \frac{\,\pFq21{1/3}{2/3}{2} {\frac{27 w(2-3w)}{(1-4w)^3}}}{(1-4w)(1-64w)} \, dw.Comment: To appear in "S\'eminaire Lotharingien de Combinatoire

Topics in chaotic secure communication

Results in nonlinear dynamics and chaos during this decade have been applied to problems in secure communications with limited success. Most of these applications have been based on the chaotic synchronization property discovered by Pecora and Carroll in 1989 [37]. Short [44, 45, 48] demonstrated the effectiveness of nonlinear dynamic (NLD) forecasting methods in breaking this class of communication schemes. In response, investigators have proposed enhancements to the basic synchronization technique in an attempt to improve the security properties. In this work two of these newer communication systems will be analyzed using NLD forecasting and other techniques to determine the level of security they provide. It will be shown that the transmitted waveform alone allows an eavesdropper to extract the message. During the course of this research, a new impulsively initialized, binary chaotic communication scheme has been developed, which eliminates the most significant weaknesses of its predecessors. This new approach is based on symbolic dynamics and chaotic control, and may be implemented using one-dimensional maps, which gives the designer more control over the statistics of the transmitted binary stream. Recent results in a certain class of one-dimensional chaotic maps will be discussed in this context. The potential for using NLD techniques in problems from standard digital communications will also be explored. The two problems which will be addressed are bit errors due to channel effects and co-channel interference. It will be shown that NLD reconstruction methods provide a way to exploit the short-term determinism that is present in these types of communication signals