Modelling Reactive Multimedia: Design and Authoring

Multimedia document authoring is a multifaceted activity, and authoring tools tend to concentrate on a restricted set of the activities involved in the creation of a multimedia artifact. In particular, a distinction may be drawn between the design and the implementation of a multimedia artifact. This paper presents a comparison of three different authoring paradigms, based on the common case study of a simple interactive animation. We present details of its implementation using the three different authoring tools, MCF, Fran and SMIL 2.0, and we discuss the conclusions that may be drawn from our comparison of the three approaches

Asynchronous Functional Reactive Programming for GUIs

Graphical user interfaces (GUIs) mediate many of our interactions with computers. Functional Reactive Programming (FRP) is a promising approach to GUI design, providing high-level, declarative, compositional abstractions to describe user interactions and time-dependent computations. We present Elm, a practical FRP language focused on easy creation of responsive GUIs. Elm has two major features: simple declarative support for Asynchronous FRP; and purely functional graphical layout. Asynchronous FRP allows the programmer to specify when the global ordering of event processing can be violated, and thus enables efficient concurrent execution of FRP programs; long-running computation can be executed asynchronously and not adversely affect the responsiveness of the user interface. Layout in Elm is achieved using a purely functional declarative framework that makes it simple to create and combine text, images, and video into rich multimedia displays. Together, Elm's two major features simplify the complicated task of creating responsive and usable GUIs.Engineering and Applied Science

Concurrency Controls in Event-Driven Programs

Functional reactive programming (FRP) is a programming paradigm that utilizes the concepts of functional programming and time-varying data types to create event-driven applications. In this paradigm, data types in which values can change over time are primitives and can be applied to functions. These values are composable and can be combined with functions to create values that react to changes in values from multiple sources. Events can be modeled as values that change in discrete time steps. Computation can be encoded as values that produce events, with combination operators, it enables us to write concurrent event-driven programs by combining the concurrent computation as events. Combined with the denotational approach of functional programming, we can write programs in a concise manner. The style of event-driven programming has been widely adopted for developing graphical user interface applications, since they need to process events concurrently to stay responsive. This makes FRP a fitting approach for managing complex state and handling of events concurrently. In recent years, real-time systems such as IoT (internet of things) applications have become an important field of computation. Applying FRP to real-time systems is still an active area of research.For IoT applications, they are commonly tasked to perform data capturing in real time and transmit them to other devices. They need to exchange data with other applications over the internet and respond in a timely manner. The data needs to be processed, for simple analysis or more computation intensive work such as machine learning. Designing applications that perform these tasks and remain efficient and responsive can be challenging. In this thesis, we demonstrate that FRP is a suitable approach for real-time applications. These applications require soft real-time requirements, where systems can tolerate tasks that fail to meet the deadline and the results of these tasks might still be useful.First, we design the concurrency abstractions needed for supporting asynchronous computation and use it as the basis for building the FRP abstraction. Our implementation is in Haskell, a functional programming language with a rich type system that allows us to model abstractions with ease. The concurrency abstraction is based on some of the ideas from the Haskell solution for asynchronous computation, which elegantly supports cancelation in a composable way. Based on the Haskell implementation, we extend our design with operators that are more suitable for building web applications. We translate our implementation to JavaScript as it is more commonly used for web application development, and implementing the RxJS interface. RxJS is a popular JavaScript library for reactive programming in web applications. By implementing the RxJS interface, we argue that our programming model implemented in Haskell is also applicable in mainstream languages such as JavaScript

Timeflies: Push-pull signal-function functional reactive programming

Functional Reactive Programming (FRP) is a promising class of abstractions for interactive programs. FRP systems provide values defined at all points in time (behaviors or signals) and values defined at countably many points in time (events) as abstractions. Signal-function FRP is a subclass of FRP which does not provide direct access to time-varying values to the programmer, but instead provides signal functions, which are reactive transformers of signals and events, as first-class objects in the program. All signal-function implementations of FRP to date have utilized demand-driven or pull-based evaluation for both events and signals, producing output from the FRP system whenever the consumer of the output is ready. This greatly simplifies the implementation of signal-function FRP systems, but leads to inefficient and wasteful evaluation of the FRP system when this strategy is employed to evaluate events, because the components of the signal function which process events must be computed whether or not there is an event occurrence. In contrast, an input-driven or push-based system evaluates the network whenever new input is available. This frees the system from evaluating the network when nothing has changed, and then only the components necessary to react to the input are re-evaluated. This form of evaluation has been applied to events in standard FRP systems but not in signal-function FRP systems. I describe the design and implementation of a signal-function FRP system which applies pull-based evaluation to signals and push-based evaluation to events (a push-pull system). The semantics of the system are discussed, and its performance and expressiveness for practical examples of interactive programs are compared to existing signal-function FRP systems through the implementation of a networking application

A Timed IO monad

Programming with explicit timing information is often tedious and error prone. This is especially visible in music programming where, when played, the specified durations of notes and rests must be shortened in order to compensate the actual duration of all surrounding processing. In this paper, we develop the notion of timed extension of a monad that aims at relieving programmers from such a burden. We show how, under simple conditions, such extensions can be built, and how useful features of monad programming such as asynchronous concurrency with promises or data-flow programming with monadic streams can be uniformly lifted to the resulting timed programming framework. Even though presented and developed in the abstract, the notion of timed extension of a monad is nevertheless illustrated by two concrete instances: a default timed IO monad where programmers specify durations in mi-croseconds, and a musically timed IO monad, where programmers specify durations in number of beats, the underlying tempo, that is, the speed of the music in beats per minute, possibly changed whenever needed

Asynchronous Monad for Reactive IoT Programming

The widespread adoption of Internet of Things (IoT) has given rise to systems which must reactively process signal data, while maintaining latency and throughput constraints. While the reactive programming paradigm partially addresses this problem, it does not offer a complete solution. We propose a design for a monad, AsyncM, and apply it to implement push-pull functional reactive programming (FRP) where side-effect bound events are performed in low-latency push streams, and time-series computations are performed in demand-driven pull streams. The design uses implicitly threaded cancellation tokens and continuation passing style to allow for implementations in languages which have single-threaded event-loops as well as languages which have multi-threading. We present our abstractions Stream, Signal, and Behavior and their functor, applicative, and (when applicable) monadic instances. We demonstrate how push streams (Stream) can be converted to pull streams (Signal) (and visa versa) to apply our model to practical use cases. We give a use case of a real-time IoT data analytics platform to demonstrate our design\u27s industrial significance where signal sample rates can change dynamically based on unknown factors such as network speed and processor resources. Application of the design to dynamic languages is shown throughout the paper in a reference JavaScript implementation

Keeping calm in the face of change: towards optimisation of FRP by reasoning about change

Functional Reactive Programming (FRP) is an approach to reactive programming where systems are structured as networks of functions operating on signals (time-varying values). FRP is based on the synchronous data-flow paradigm and supports both (an approximation to) continuous-time and discrete-time signals (hybrid systems).What sets FRP apart from most other languages for similar applications is its support for systems with dynamic structure and for higher-order reactive constructs. This paper contributes towards advancing the state of the art of FRP implementation by studying the notion of signal change and change propagation in a setting of structurally dynamic networks of n-ary signal functions operating on mixed continuous-time and discrete-time signals. We first define an ideal denotational semantics (time is truly continuous) for this kind of FRP, along with temporal properties, expressed in temporal logic, of signals and signal functions pertaining to change and change propagation. Using this framework, we then show how to reason about change; specifically, we identify and justify a number of possible optimisations, such as avoiding recomputation of unchanging values. Note that due to structural dynamism, and the fact that the output of a signal function may change because time is passing even if the input is unchanging, the problem is significantly more complex than standard change propagation in networks with static structure

Declarative Support for Prototyping Interactive Systems

The development of complex, multi-user, interactive systems is a difficult process that requires both a rapid iterative approach, and the ability to reason carefully about system designs. This thesis argues that a combination of declarative prototyping and formal specification provides a suitable way of satisfying these requirements. The focus of this thesis is on the development of software tools for prototyping interactive systems. In particular, it uses a declarative approach, based on the functional programming paradigm. This thesis makes two contributions. The most significant contribution is the presentation of FranTk, a new Graphical User Interface language, embedded in the functional language Haskell. It is suitable for prototyping complex, concurrent, multi-user systems. It allows systems to be built in a high level, structured manner. In particular, it provides good support for specifying real-time properties of such systems. The second contribution is a mechanism that allows a formal specification to be derived from a high level FranTk prototype. The approach allows this to be done automatically. This specification can then be checked, with tool support, to verify some safety properties about a system. To avoid the state space explosion problem that would be faced when verifying an entire system, we focus on partial verification. This concentrates on key areas of a design: in particular this means that we only derive a specification from parts of a prototype. To demonstrate the scalability of both the prototyping and verification approaches, this thesis uses a series of case studies including a multi-user design rationale editor and a prototype data-link Air Traffic Control system