Optimizing Hypervideo Navigation Using a Markov Decision Process Approach

Interaction with hypermedia documents is a required feature for new sophisticated yet flexible multimedia applications. This paper presents an innovative adaptive technique to stream hypervideo that takes into account user behaviour. The objective is to optimize hypervideo prefetching in order to reduce the latency caused by the network. This technique is based on a model provided by a Markov Decision Process approach. The problem is solved using two methods: classical stochastic dynamic programming algorithms and reinforcement learning. Experimental results under stochastic network conditions are very promising

Adapting Content Delivery to Limited Resources and Inferred User Interest

This paper discusses adaptation policies for information systems that are subject to dynamic and stochastic contexts such as mobile access to multimedia web sites. In our approach, adaptation agents apply sequential decisional policies under uncertainty. We focus on the modeling of such decisional processes depending on whether the context is fully or partially observable. Our case study is a movie browsing service in a mobile environment that we model by using Markov decision processes (MDPs) and partially observable MDP (POMDP). We derive adaptation policies for this service, that take into account the limited resources such as the network bandwidth. We further refine these policies according to the partially observable users' interest level estimated from implicit feedback. Our theoretical models are validated through numerous simulations

Optimal Prefetching in Random Trees

International audienceWe propose and analyze a model for optimizing the prefetching of documents, in the situation where the connection between documents is discovered progressively. A random surfer moves along the edges of a random tree representing possible sequences of documents, which is known to a controller only up to depth d. A quantity k of documents can be prefetched between two movements. The question is to determine which nodes of the known tree should be prefetched so as to minimize the probability of the surfer moving to a node not prefetched. We analyzed the model with the tools of Markov decision process theory. We formally identified the optimal policy in several situations, and we identified it numerically in others

Stability Problems for Stochastic Models: Theory and Applications II

Most papers published in this Special Issue of Mathematics are written by the participants of the XXXVI International Seminar on Stability Problems for Stochastic Models, 21­25 June, 2021, Petrozavodsk, Russia. The scope of the seminar embraces the following topics: Limit theorems and stability problems; Asymptotic theory of stochastic processes; Stable distributions and processes; Asymptotic statistics; Discrete probability models; Characterization of probability distributions; Insurance and financial mathematics; Applied statistics; Queueing theory; and other fields. This Special Issue contains 12 papers by specialists who represent 6 countries: Belarus, France, Hungary, India, Italy, and Russia

A markov-model-based framework for supporting real-time generation of synthetic memory references effectively and efficiently

Driven by several real-life case studies and in-lab developments, synthetic memory reference generation has a long tradition in computer science research. The goal is that of reproducing the running of an arbitrary program, whose generated traces can later be used for simulations and experiments. In this paper we investigate this research context and provide principles and algorithms of a Markov-Model-based framework for supporting real-time generation of synthetic memory references effectively and efficiently. Specifically, our approach is based on a novel Machine Learning algorithm we called Hierarchical Hidden/ non Hidden Markov Model (HHnHMM). Experimental results conclude this paper

Prefetching and clustering techniques for network based storage.

The usage of network-based applications is increasing, as network speeds increase, and the use of streaming applications, e.g BBC iPlayer, YouTube etc., running over network infrastructure is becoming commonplace. These applications access data sequentially. However, as processor speeds and the amount of memory available increase, the rate at which streaming applications access data is now faster than the rate at which the blocks can be fetched consecutively from network storage. In addition to sequential access, the system also needs to promptly satisfy demand misses in order for applications to continue their execution. This thesis proposes a design to provide Quality-Of-Service (QoS) for streaming applications (sequential accesses) and demand misses, such that, streaming applications can run without jitter (once they are started) and demand misses can be satisfied in reasonable time using network storage. To implement the proposed design in real time, the thesis presents an analytical model to estimate the average time taken to service a demand miss. Further, it defines and explores the operational space where the proposed QoS could be provided. Using database techniques, this region is then encapsulated into an autonomous algorithm which is verified using simulation. Finally, a prototype Experimental File System (EFS) is designed and implemented to test the algorithm on a real test-bed

Center for Aeronautics and Space Information Sciences

This report summarizes the research done during 1991/92 under the Center for Aeronautics and Space Information Science (CASIS) program. The topics covered are computer architecture, networking, and neural nets