LTLf/LDLf Non-Markovian Rewards

In Markov Decision Processes (MDPs), the reward obtained in a state is Markovian, i.e., depends on the last state and action. This dependency makes it difficult to reward more interesting long-term behaviors, such as always closing a door after it has been opened, or providing coffee only following a request. Extending MDPs to handle non-Markovian reward functions was the subject of two previous lines of work. Both use LTL variants to specify the reward function and then compile the new model back into a Markovian model. Building on recent progress in temporal logics over finite traces, we adopt LDLf for specifying non-Markovian rewards and provide an elegant automata construction for building a Markovian model, which extends that of previous work and offers strong minimality and compositionality guarantees

Adaptive Rich Media Presentations via Preference-Based Constrained Optimization

Personalization and adaptation of multi-media messages are well known and well studied problems. Ideally, each message should reflect its recipient\u27s interests, device capabilities, and network conditions. Such personalization is more difficult to carry out given a compound multi-media presentation containing multiple spatially and temporally related elements. This paper describes a novel formal, yet practical approach, and an implemented system prototype for authoring and adapting compound multi-media presentations. Our approach builds on recent advances in preference specification and preferences-based constrained optimization techniques