Portfolio management using partially observable Markov decision process

Portfolio theory is concerned with how an investor should divide his wealth among different securities. This problem was first formulated by Markowitz in 1952. Since then, other more sophisticated formulations have been introduced. However, practical issues like transactions costs and their effects on the portfolio choice in multiple stages have not been widely considered. In our work, we show that the portfolio management problem is appropriately formulated as a Partially Observable Markov Decision Process. We use a Monte Carlo method called "rollout" to approximate an optimal strategy for making decisions. To capture the behavior of stock prices over time, we use two well known models.2nd place, IS&T Graduate Group

The Trust-Based Interactive Partially Observable Markov Decision Process

Cooperative agent and robot systems are designed so that each is working toward the same common good. The problem is that the software systems are extremely complex and can be subverted by an adversary to either break the system or potentially worse, create sneaky agents who are willing to cooperate when the stakes are low and take selfish, greedy actions when the rewards rise. This research focuses on the ability of a group of agents to reason about the trustworthiness of each other and make decisions about whether to cooperate. A trust-based interactive partially observable Markov decision process (TI-POMDP) is developed to model the trust interactions between agents, enabling the agents to select the best course of action from the current state. The TI-POMDP is a novel approach to multiagent cooperation based on an interactive partially observable Markov decision process (I-POMDP) augmented with trust relationships. Experiments using the Defender simulation demonstrate the TI-POMDP\u27s ability to accurately track the trust levels of agents with hidden agendas The TI-POMDP provides agents with the information needed to make decisions based on their level of trust and model of the environment. Testing demonstrates that agents quickly identify the hidden trust levels and mitigate the impact of a deceitful agent in comparison with a trust vector model. Agents using the TI-POMDP model achieved 3.8 times the average reward of agents using a trust vector model

Toward Affective Dialogue Modeling using Partially Observable Markov Decision Processes

We propose a novel approach to developing a dialogue model which is able to take into account some aspects of the user’s emotional state and acts appropriately. The dialogue model uses a Partially Observable Markov Decision Process approach with observations composed of the observed user’s emotional state and action. A simple example of route navigation is explained to clarify our approach and preliminary results & future plans are briefly discussed

Technical report: Distribution Temporal Logic: combining correctness with quality of estimation

We present a new temporal logic called Distribution Temporal Logic (DTL) defined over predicates of belief states and hidden states of partially observable systems. DTL can express properties involving uncertainty and likelihood that cannot be described by existing logics. A co-safe formulation of DTL is defined and algorithmic procedures are given for monitoring executions of a partially observable Markov decision process with respect to such formulae. A simulation case study of a rescue robotics application outlines our approach