Relational Representations in Reinforcement Learning: Review and Open Problems

This paper is about representation in RL.We discuss some of the concepts in representation and generalization in reinforcement learning and argue for higher-order representations, instead of the commonly used propositional representations. The paper contains a small review of current reinforcement learning systems using higher-order representations, followed by a brief discussion. The paper ends with research directions and open problems.\u

Orienting Ordered Scaffolds: Complexity and Algorithms

Despite the recent progress in genome sequencing and assembly, many of the currently available assembled genomes come in a draft form. Such draft genomes consist of a large number of genomic fragments (scaffolds), whose order and/or orientation (i.e., strand) in the genome are unknown. There exist various scaffold assembly methods, which attempt to determine the order and orientation of scaffolds along the genome chromosomes. Some of these methods (e.g., based on FISH physical mapping, chromatin conformation capture, etc.) can infer the order of scaffolds, but not necessarily their orientation. This leads to a special case of the scaffold orientation problem (i.e., deducing the orientation of each scaffold) with a known order of the scaffolds. We address the problem of orientating ordered scaffolds as an optimization problem based on given weighted orientations of scaffolds and their pairs (e.g., coming from pair-end sequencing reads, long reads, or homologous relations). We formalize this problem using notion of a scaffold graph (i.e., a graph, where vertices correspond to the assembled contigs or scaffolds and edges represent connections between them). We prove that this problem is NP-hard, and present a polynomial-time algorithm for solving its special case, where orientation of each scaffold is imposed relatively to at most two other scaffolds. We further develop an FPT algorithm for the general case of the OOS problem

Algorithmic and Statistical Perspectives on Large-Scale Data Analysis

In recent years, ideas from statistics and scientific computing have begun to interact in increasingly sophisticated and fruitful ways with ideas from computer science and the theory of algorithms to aid in the development of improved worst-case algorithms that are useful for large-scale scientific and Internet data analysis problems. In this chapter, I will describe two recent examples---one having to do with selecting good columns or features from a (DNA Single Nucleotide Polymorphism) data matrix, and the other having to do with selecting good clusters or communities from a data graph (representing a social or information network)---that drew on ideas from both areas and that may serve as a model for exploiting complementary algorithmic and statistical perspectives in order to solve applied large-scale data analysis problems.Comment: 33 pages. To appear in Uwe Naumann and Olaf Schenk, editors, "Combinatorial Scientific Computing," Chapman and Hall/CRC Press, 201

Networks, (K)nots, Nucleotides, and Nanostructures

Designing self-assembling DNA nanostructures often requires the identification of a route for a scaffolding strand of DNA through the target structure. When the target structure is modeled as a graph, these scaffolding routes correspond to Eulerian circuits subject to turning restrictions imposed by physical constraints on the strands of DNA. Existence of such Eulerian circuits is an NP-hard problem, which can be approached by adapting solutions to a version of the Traveling Salesperson Problem. However, the author and collaborators have demonstrated that even Eulerian circuits obeying these turning restrictions are not necessarily feasible as scaffolding routes by giving examples of nontrivially knotted circuits which cannot be traced by the unknotted scaffolding strand. Often, targets of DNA nanostructure self-assembly are modeled as graphs embedded on surfaces in space. In this case, Eulerian circuits obeying the turning restrictions correspond to A-trails, circuits which turn immediately left or right at each vertex. In any graph embedded on the sphere, all A-trails are unknotted regardless of the embedding of the sphere in space. We show that this does not hold in general for graphs on the torus. However, we show this property does hold for checkerboard-colorable graphs on the torus, that is, those graphs whose faces can be properly 2-colored, and provide a partial converse to this result. As a consequence, we characterize (with one exceptional family) regular triangulations of the torus containing unknotted A-trails. By developing a theory of sums of A-trails, we lift constructions from the torus to arbitrary n-tori, and by generalizing our work on A-trails to smooth circuit decompositions, we construct all torus links and certain sums of torus links from circuit decompositions of rectangular torus grids. Graphs embedded on surfaces are equivalent to ribbon graphs, which are particularly well-suited to modeling DNA nanostructures, as their boundary components correspond to strands of DNA and their twisted ribbons correspond to double-helices. Every ribbon graph has a corresponding delta-matroid, a combinatorial object encoding the structure of the ribbon-graph\u27s spanning quasi-trees (substructures having exactly one boundary component). We show that interlacement with respect to quasi-trees can be generalized to delta-matroids, and use the resulting structure on delta-matroids to provide feasible-set expansions for a family of delta-matroid polynomials, both recovering well-known expansions of this type (such as the spanning-tree expansion of the Tutte polynnomial) as well as providing several previously unknown expansions. Among these are expansions for the transition polynomial, a version of which has been used to study DNA nanostructure self-assembly, and the interlace polynomial, which solves a problem in DNA recombination

Fixed-parameter tractability and lower bounds for stabbing problems

We study the following general stabbing problem from a parameterized complexity point of view: Given a set $\mathcal S$ of $n$ translates of an object in \Rd, find a set of $k$ lines with the property that every object in $\mathcal S$ is ''stabbed'' (intersected) by at least one line. We show that when $S$ consists of axis-parallel unit squares in \Rtwo the (decision) problem of stabbing $S$ with axis-parallel lines is W[1]-hard with respect to $k$ (and thus, not fixed-parameter tractable unless FPT=W[1]) while it becomes fixed-parameter tractable when the squares are disjoint. We also show that the problem of stabbing a set of disjoint unit squares in \Rtwo with lines of arbitrary directions is W[1]--hard with respect to $k$. Several generalizations to other types of objects and lines with arbitrary directions are also presented. Finally, we show that deciding whether a set of unit balls in \Rd can be stabbed by one line is W[1]--hard with respect to the dimension $d$.Comment: Based on the MSc. Thesis of Daniel Werner, Free University Berlin, Berlin, German

Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”This position paper proposes that the study of embodied cognitive agents, such as humanoid robots, can advance our understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills. This in turn will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions. Four key areas of research challenges are discussed, specifically for the issues related to the understanding of: 1) how agents learn and represent compositional actions; 2) how agents learn and represent compositional lexica; 3) the dynamics of social interaction and learning; and 4) how compositional action and language representations are integrated to bootstrap the cognitive system. The review of specific issues and progress in these areas is then translated into a practical roadmap based on a series of milestones. These milestones provide a possible set of cognitive robotics goals and test scenarios, thus acting as a research roadmap for future work on cognitive developmental robotics.Peer reviewe

Enriching mathematics expectations in grade one with a Reggio-inspired emergent curriculum

"This thesis researches the mathematics learning of young children. In a school seriously invested in the Reggio Emilia experience for ten years, grade-one learning is project based. I pose the question is it possible to enrich numeracy expectations in the Ontario mathematics curriculum for grade one, while engaging in a Reggio-inspired emergent curriculum? As a teacher/researcher I observed and listened carefully to the students' interactions. I facilitated discussions, problem solving and directions for the project, while remaining open to the children's mathematical theories. I paid close attention to their hypotheses about proportion as well as their exploration into perimeters, and I supported their math manipulatives ""factory."" Pedagogical documentation in the form of graphic novels demonstrates how emergent curriculum weaves play-based inquiry with sophisticated mathematical thinking. I argue that emergent curriculum and provincial expectations can co-exist in a grade one classroom to infuse complex mathematical thinking with joy and beauty.