Streaming Tree Transducers

Theory of tree transducers provides a foundation for understanding expressiveness and complexity of analysis problems for specification languages for transforming hierarchically structured data such as XML documents. We introduce streaming tree transducers as an analyzable, executable, and expressive model for transforming unranked ordered trees in a single pass. Given a linear encoding of the input tree, the transducer makes a single left-to-right pass through the input, and computes the output in linear time using a finite-state control, a visibly pushdown stack, and a finite number of variables that store output chunks that can be combined using the operations of string-concatenation and tree-insertion. We prove that the expressiveness of the model coincides with transductions definable using monadic second-order logic (MSO). Existing models of tree transducers either cannot implement all MSO-definable transformations, or require regular look ahead that prohibits single-pass implementation. We show a variety of analysis problems such as type-checking and checking functional equivalence are solvable for our model.Comment: 40 page

Programming Using Automata and Transducers

Automata, the simplest model of computation, have proven to be an effective tool in reasoning about programs that operate over strings. Transducers augment automata to produce outputs and have been used to model string and tree transformations such as natural language translations. The success of these models is primarily due to their closure properties and decidable procedures, but good properties come at the price of limited expressiveness. Concretely, most models only support finite alphabets and can only represent small classes of languages and transformations. We focus on addressing these limitations and bridge the gap between the theory of automata and transducers and complex real-world applications: Can we extend automata and transducer models to operate over structured and infinite alphabets? Can we design languages that hide the complexity of these formalisms? Can we define executable models that can process the input efficiently? First, we introduce succinct models of transducers that can operate over large alphabets and design BEX, a language for analysing string coders. We use BEX to prove the correctness of UTF and BASE64 encoders and decoders. Next, we develop a theory of tree transducers over infinite alphabets and design FAST, a language for analysing tree-manipulating programs. We use FAST to detect vulnerabilities in HTML sanitizers, check whether augmented reality taggers conflict, and optimize and analyze functional programs that operate over lists and trees. Finally, we focus on laying the foundations of stream processing of hierarchical data such as XML files and program traces. We introduce two new efficient and executable models that can process the input in a left-to-right linear pass: symbolic visibly pushdown automata and streaming tree transducers. Symbolic visibly pushdown automata are closed under Boolean operations and can specify and efficiently monitor complex properties for hierarchical structures over infinite alphabets. Streaming tree transducers can express and efficiently process complex XML transformations while enjoying decidable procedures

XML stream transformer generation through program composition and dependency analysis

AbstractXML stream transformation, which sequentially processes the input XML data on the fly, makes it possible to process large sized data within a limited amount of memory. Though being efficient in memory-use, stream transformation requires stateful programming, which is error-prone and hard to manage.This paper proposes a scheme for generating XML stream transformers. Given an attribute grammar definition of transformation over an XML tree structure, we systematically derive a stream transformer in two steps. First, an attribute grammar definition of the XML stream transformation is inferred by applying a program composition method. Second, a finite state transition machine is constructed through a dependency analysis. Due to the closure property of the program composition method, our scheme also allows modular construction of XML stream transformers.We have implemented a prototype XML stream transformer generator, called altSAX. The experimental results show that the generated transformers are efficient in memory consumption as well as in execution time

XML stream transformer generation through program composition and dependency analysis

AbstractXML stream transformation, which sequentially processes the input XML data on the fly, makes it possible to process large sized data within a limited amount of memory. Though being efficient in memory-use, stream transformation requires stateful programming, which is error-prone and hard to manage.This paper proposes a scheme for generating XML stream transformers. Given an attribute grammar definition of transformation over an XML tree structure, we systematically derive a stream transformer in two steps. First, an attribute grammar definition of the XML stream transformation is inferred by applying a program composition method. Second, a finite state transition machine is constructed through a dependency analysis. Due to the closure property of the program composition method, our scheme also allows modular construction of XML stream transformers.We have implemented a prototype XML stream transformer generator, called altSAX. The experimental results show that the generated transformers are efficient in memory consumption as well as in execution time

Interactive Programming by Example

As of today, programming has never been so accessible. Yet, it remains a challenge for end-users: students, non-technical employees, experts in their domains outside of computer science, and so on. With its forecast potential for solving problems by only observing inputs and outputs, programming-by-example was supposed to alleviate complex tasks requiring programming for end-users. The initial ideas of macro-based editors paved the way to subsequent practical solutions, such as spreadsheet transformations from examples. Finding the right program is the core of the programming-by-example systems. However, users find it difficult to trust such generated programs. In this thesis, we contribute to proving that some forms of interaction alleviate, by having users provide examples, the problem of finding correct and reliable programs. We first report on two experiments that enable us to conjecture what kind of interaction brings benefits to programming-by-example. First, we present a new kind of game engine, Pong Designer. In this game engine, by using their finger, users program rules on the fly, by modifying the game state. We analyze its potential, and its eventual downsides that have probably prevented its wide adoption. Second, we present StriSynth, an interactive command-line tool that uses programming-by-example to transform string and collections. The resulting programs can also rename or otherwise manage files. We obtained the result that confirms that many users preferred StriSynth over usual programming languages, but would appreciate to have both. We then report on two new exciting experiments with verified results, using two forms of interaction truly benefiting programming-by-example. Third, on top of a programmingby- example-based engine for extracting structured data out of text files, in this thesis we study two interaction models implemented in a tool named FlashProg: a view of the program with notification about ambiguities, and the asking of clarification questions. In this thesis, we prove that these two interaction models enable users to perform tasks with less errors and to be more confident with the results. Last, for learning recursive tree-to-string functions (e.g., pretty-printers), in this thesis we prove that questioning breaks down the learning complexity from a cubic to a linear number of questions, in practice making programming-by-example even more accessible than regular programming. The implementation, named Prosy, could be easily added to integrated development environments

木を用いた構造化並列プログラミング

High-level abstractions for parallel programming are still immature. Computations on complicated data structures such as pointer structures are considered as irregular algorithms. General graph structures, which irregular algorithms generally deal with, are difficult to divide and conquer. Because the divide-and-conquer paradigm is essential for load balancing in parallel algorithms and a key to parallel programming, general graphs are reasonably difficult. However, trees lead to divide-and-conquer computations by definition and are sufficiently general and powerful as a tool of programming. We therefore deal with abstractions of tree-based computations. Our study has started from Matsuzaki’s work on tree skeletons. We have improved the usability of tree skeletons by enriching their implementation aspect. Specifically, we have dealt with two issues. We first have implemented the loose coupling between skeletons and data structures and developed a flexible tree skeleton library. We secondly have implemented a parallelizer that transforms sequential recursive functions in C into parallel programs that use tree skeletons implicitly. This parallelizer hides the complicated API of tree skeletons and makes programmers to use tree skeletons with no burden. Unfortunately, the practicality of tree skeletons, however, has not been improved. On the basis of the observations from the practice of tree skeletons, we deal with two application domains: program analysis and neighborhood computation. In the domain of program analysis, compilers treat input programs as control-flow graphs (CFGs) and perform analysis on CFGs. Program analysis is therefore difficult to divide and conquer. To resolve this problem, we have developed divide-and-conquer methods for program analysis in a syntax-directed manner on the basis of Rosen’s high-level approach. Specifically, we have dealt with data-flow analysis based on Tarjan’s formalization and value-graph construction based on a functional formalization. In the domain of neighborhood computations, a primary issue is locality. A naive parallel neighborhood computation without locality enhancement causes a lot of cache misses. The divide-and-conquer paradigm is known to be useful also for locality enhancement. We therefore have applied algebraic formalizations and a tree-segmenting technique derived from tree skeletons to the locality enhancement of neighborhood computations.電気通信大学201

Routing and Mobility on IPv6 over LoWPAN

The IoT means a world-wide network of interconnected objects based on standard communication protocols. An object in this context is a quotidian physical device augmented with sensing/actuating, processing, storing and communication capabilities. These objects must be able to interact with the surrounding environment where they are placed and to cooperate with neighbouring objects in order to accomplish a common objective. The IoT objects have also the capabilities of converting the sensed data into automated instructions and communicating them to other objects through the communication networks, avoiding the human intervention in several tasks. Most of IoT deployments are based on small devices with restricted computational resources and energy constraints. For this reason, initially the scientific community did not consider the use of IP protocol suite in this scenarios because there was the perception that it was too heavy to the available resources on such devices. Meanwhile, the scientific community and the industry started to rethink about the use of IP protocol suite in all IoT devices and now it is considered as the solution to provide connectivity between the IoT devices, independently of the Layer 2 protocol in use, and to connect them to the Internet. Despite the use of IP suite protocol in all devices and the amount of solutions proposed, many open issues remain unsolved in order to reach a seamless integration between the IoT and the Internet and to provide the conditions to IoT service widespread. This thesis addressed the challenges associated with the interconnectivity between the Internet and the IoT devices and with the security aspects of the IoT. In the interconnectivity between the IoT devices and the Internet the problem is how to provide valuable information to the Internet connected devices, independently of the supported IP protocol version, without being necessary accessed directly to the IoT nodes. In order to solve this problem, solutions based on Representational state transfer (REST) web services and IPv4 to IPv6 dual stack transition mechanism were proposed and evaluated. The REST web service and the transition mechanism runs only at the border router without penalizing the IoT constrained devices. The mitigation of the effects of internal and external security attacks minimizing the overhead imposed on the IoT devices is the security challenge addressed in this thesis. Three different solutions were proposed. The first is a mechanism to prevent remotely initiated transport level Denial of Service attacks that avoids the use of inefficient and hard to manage traditional firewalls. It is based on filtering at the border router the traffic received from the Internet and destined to the IoT network according to the conditions announced by each IoT device. The second is a network access security framework that can be used to control the nodes that have access to the network, based on administrative approval, and to enforce security compliance to the authorized nodes. The third is a network admission control framework that prevents IoT unauthorized nodes to communicate with IoT authorized nodes or with the Internet, which drastically reduces the number of possible security attacks. The network admission control was also exploited as a management mechanism as it can be used to manage the network size in terms of number of nodes, making the network more manageable, increasing its reliability and extending its lifetime.A IoT (Internet of Things) tem suscitado o interesse tanto da comunidade académica como da indústria, uma vez que os campos de aplicação são inúmeros assim como os potenciais ganhos que podem ser obtidos através do uso deste tipo de tecnologia. A IoT significa uma rede global de objetos ligados entre si através de uma rede de comunicações baseada em protocolos standard. Neste contexto, um objeto é um objeto físico do dia a dia ao qual foi adicionada a capacidade de medir e de atuar sobre variáveis físicas, de processar e armazenar dados e de comunicar. Estes objetos têm a capacidade de interagir com o meio ambiente envolvente e de cooperar com outros objetos vizinhos de forma a atingirem um objetivo comum. Estes objetos também têm a capacidade de converter os dados lidos em instruções e de as comunicar a outros objetos através da rede de comunicações, evitando desta forma a intervenção humana em diversas tarefas. A maior parte das concretizações de sistemas IoT são baseados em pequenos dispositivos autónomos com restrições ao nível dos recursos computacionais e de retenção de energia. Por esta razão, inicialmente a comunidade científica não considerou adequado o uso da pilha protocolar IP neste tipo de dispositivos, uma vez que havia a perceção de que era muito pesada para os recursos computacionais disponíveis. Entretanto, a comunidade científica e a indústria retomaram a discussão acerca dos benefícios do uso da pilha protocolar em todos os dispositivos da IoT e atualmente é considerada a solução para estabelecer a conetividade entre os dispositivos IoT independentemente do protocolo da camada dois em uso e para os ligar à Internet. Apesar do uso da pilha protocolar IP em todos os dispositivos e da quantidade de soluções propostas, são vários os problemas por resolver no que concerne à integração contínua e sem interrupções da IoT na Internet e de criar as condições para a adoção generalizada deste tipo de tecnologias. Esta tese versa sobre os desafios associados à integração da IoT na Internet e dos aspetos de segurança da IoT. Relativamente à integração da IoT na Internet o problema é como fornecer informação válida aos dispositivos ligados à Internet, independentemente da versão do protocolo IP em uso, evitando o acesso direto aos dispositivos IoT. Para a resolução deste problema foram propostas e avaliadas soluções baseadas em web services REST e em mecanismos de transição IPv4 para IPv6 do tipo pilha dupla (dual stack). O web service e o mecanismo de transição são suportados apenas no router de fronteira, sem penalizar os dispositivos IoT. No que concerne à segurança, o problema é mitigar os efeitos dos ataques de segurança internos e externos iniciados local e remotamente. Foram propostas três soluções diferentes, a primeira é um mecanismo que minimiza os efeitos dos ataques de negação de serviço com origem na Internet e que evita o uso de mecanismos de firewalls ineficientes e de gestão complexa. Este mecanismo filtra no router de fronteira o tráfego com origem na Internet é destinado à IoT de acordo com as condições anunciadas por cada um dos dispositivos IoT da rede. A segunda solução, é uma framework de network admission control que controla quais os dispositivos que podem aceder à rede com base na autorização administrativa e que aplica políticas de conformidade relativas à segurança aos dispositivos autorizados. A terceira é um mecanismo de network admission control para redes 6LoWPAN que evita que dispositivos não autorizados comuniquem com outros dispositivos legítimos e com a Internet o que reduz drasticamente o número de ataques à segurança. Este mecanismo também foi explorado como um mecanismo de gestão uma vez que pode ser utilizado a dimensão da rede quanto ao número de dispositivos, tornando-a mais fácil de gerir e aumentando a sua fiabilidade e o seu tempo de vida