Higher order generalization and its application in program verification

Generalization is a fundamental operation of inductive inference. While first order syntactic generalization (anti–unification) is well understood, its various extensions are often needed in applications. This paper discusses syntactic higher order generalization in a higher order language λ2 [1]. Based on the application ordering, we prove that least general generalization exists for any two terms and is unique up to renaming. An algorithm to compute the least general generalization is also presented. To illustrate its usefulness, we propose a program verification system based on higher order generalization that can reuse the proofs of similar programs

Generalization of Clauses under Implication

In the area of inductive learning, generalization is a main operation, and the usual definition of induction is based on logical implication. Recently there has been a rising interest in clausal representation of knowledge in machine learning. Almost all inductive learning systems that perform generalization of clauses use the relation theta-subsumption instead of implication. The main reason is that there is a well-known and simple technique to compute least general generalizations under theta-subsumption, but not under implication. However generalization under theta-subsumption is inappropriate for learning recursive clauses, which is a crucial problem since recursion is the basic program structure of logic programs. We note that implication between clauses is undecidable, and we therefore introduce a stronger form of implication, called T-implication, which is decidable between clauses. We show that for every finite set of clauses there exists a least general generalization under T-implication. We describe a technique to reduce generalizations under implication of a clause to generalizations under theta-subsumption of what we call an expansion of the original clause. Moreover we show that for every non-tautological clause there exists a T-complete expansion, which means that every generalization under T-implication of the clause is reduced to a generalization under theta-subsumption of the expansion.Comment: See http://www.jair.org/ for any accompanying file

Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term

Deep Neural Networks (DNNs) generalization is known to be closely related to the flatness of minima, leading to the development of Sharpness-Aware Minimization (SAM) for seeking flatter minima and better generalization. In this paper, we revisit the loss of SAM and propose a more general method, called WSAM, by incorporating sharpness as a regularization term. We prove its generalization bound through the combination of PAC and Bayes-PAC techniques, and evaluate its performance on various public datasets. The results demonstrate that WSAM achieves improved generalization, or is at least highly competitive, compared to the vanilla optimizer, SAM and its variants. The code is available at https://github.com/intelligent-machine-learning/dlrover/tree/master/atorch/atorch/optimizers.Comment: 10 pages. Accepted as a conference paper at KDD '2

The Moore-Penrose pseudo-inverse: theory, applications, and a generalization

The Moore-Penrose pseudo-inverse is the more widely known generalization of the inverse of a matrix, and has applications in many areas including least squares. We present its definition, some of its properties and its connection with left and right inverses. We also discuss two different methods for computing the pseudo-inverse. Finally, we show its applications to the standard least-squares problems and propose a generalization of the pseudo-inverse using a general dot product on ℝ^n.https://ecommons.udayton.edu/stander_posters/3209/thumbnail.jp

A Modular Order-sorted Equational Generalization Algorithm

Generalization, also called anti-unification, is the dual of unification. Given terms t and t , a generalizer is a term t of which t and t are substitution instances. The dual of a most general unifier (mgu) is that of least general generalizer (lgg). In this work, we extend the known untyped generalization algorithm to, first, an order-sorted typed setting with sorts, subsorts, and subtype polymorphism; second, we extend it to work modulo equational theories, where function symbols can obey any combination of associativity, commutativity, and identity axioms (including the empty set of such axioms); and third, to the combination of both, which results in a modular, order-sorted equational generalization algorithm. Unlike the untyped case, there is in general no single lgg in our framework, due to order-sortedness or to the equational axioms. Instead, there is a finite, minimal and complete set of lggs, so that any other generalizer has at least one of them as an instance. Our generalization algorithms are expressed by means of inference systems for which we give proofs of correctness. This opens up new applications to partial evaluation, program synthesis, and theorem proving for typed equational reasoning systems and typed rulebased languages such as ASF+SDF, Elan, OBJ, Cafe-OBJ, and Maude. © 2014 Elsevier Inc. All rights reserved. 1.M. Alpuente, S. Escobar, and J. Espert have been partially supported by the EU (FEDER) and the Spanish MEC/MICINN under grant TIN 2010-21062-C02-02, and by Generalitat Valenciana PROMETEO2011/052. J. Meseguer has been supported by NSF Grants CNS 09-04749, and CCF 09-05584.Alpuente Frasnedo, M.; Escobar Román, S.; Espert Real, J.; Meseguer, J. (2014). A Modular Order-sorted Equational Generalization Algorithm. Information and Computation. 235:98-136. https://doi.org/10.1016/j.ic.2014.01.006S9813623

Study of a homotopy continuation method for early orbit determination with the Tracking and Data Relay Satellite System (TDRSS)

A recent mathematical technique for solving systems of equations is applied in a very general way to the orbit determination problem. The study of this technique, the homotopy continuation method, was motivated by the possible need to perform early orbit determination with the Tracking and Data Relay Satellite System (TDRSS), using range and Doppler tracking alone. Basically, a set of six tracking observations is continuously transformed from a set with known solution to the given set of observations with unknown solutions, and the corresponding orbit state vector is followed from the a priori estimate to the solutions. A numerical algorithm for following the state vector is developed and described in detail. Numerical examples using both real and simulated TDRSS tracking are given. A prototype early orbit determination algorithm for possible use in TDRSS orbit operations was extensively tested, and the results are described. Preliminary studies of two extensions of the method are discussed: generalization to a least-squares formulation and generalization to an exhaustive global method

Asymptotic behaviour of the CUSUM of squares test under stochastic and deterministic time trends

We undertake a generalization of the cumulative sum of squares (CUSQ) test to the case of non-stationary autoregressive distributed lag models with quite general deterministic time trends. The test may be validly implemented with either ordinary least squares residuals or standardized forecast errors. Simulations suggest that there is little at stake in the choice between the two in the unit root case under Gaussian innovations, and that there is only very modest variation in the finite sample distribution across the parameter space.