Scaling POMDPs For Selecting Sellers in E-markets-Extended Version

In multiagent e-marketplaces, buying agents need to select good sellers by querying other buyers (called advisors). Partially Observable Markov Decision Processes (POMDPs) have shown to be an effective framework for optimally selecting sellers by selectively querying advisors. However, current solution methods do not scale to hundreds or even tens of agents operating in the e-market. In this paper, we propose the Mixture of POMDP Experts (MOPE) technique, which exploits the inherent structure of trust-based domains, such as the seller selection problem in e-markets, by aggregating the solutions of smaller sub-POMDPs. We propose a number of variants of the MOPE approach that we analyze theoretically and empirically. Experiments show that MOPE can scale up to a hundred agents thereby leveraging the presence of more advisors to significantly improve buyer satisfaction

Maximizing Expected Impact in an Agent Reputation Network

Many multi-agent systems (MASs) are situated in stochastic environments. Some such systems that are based on the partially observ- able Markov decision process (POMDP) do not take the benevolence of other agents for granted. We propose a new POMDP-based framework which is general enough for the specification of a variety of stochastic MAS domains involving the impact of agents on each other’s reputa- tions. A unique feature of this framework is that actions are specified as either undirected (regular) or directed (towards a particular agent), and a new directed transition function is provided for modeling the effects of reputation in interactions. Assuming that an agent must maintain a good enough reputation to survive in the network, a planning algorithm is developed for an agent to select optimal actions in stochastic MASs. Preliminary evaluation is provided via an example specification and by determining the algorithm’s complexity

Provably robust decisions based on potentially malicious sources of information

Ministry of Education, Singapore under its Academic Research Funding Tier

Supply Side Optimisation in Online Display Advertising

On the Internet there are publishers (the supply side) who provide free contents (e.g., news) and services (e.g., email) to attract users. Publishers get paid by selling ad displaying opportunities (i.e., impressions) to advertisers. Advertisers then sell products to users who are converted by ads. Better supply side revenue allows more free content and services to be created, thus, benefiting the entire online advertising ecosystem. This thesis addresses several optimisation problems for the supply side. When a publisher creates an ad-supported website, he needs to decide the percentage of ads first. The thesis reports a large-scale empirical study of Internet ad density over past seven years, then presents a model that includes many factors, especially the competition among similar publishers, and gives an optimal dynamic ad density that generates the maximum revenue over time. This study also unveils the tragedy of the commons in online advertising where users' attention has been overgrazed which results in a global sub-optimum. After deciding the ad density, the publisher retrieves ads from various sources, including contracts, ad networks, and ad exchanges. This forms an exploration-exploitation problem when ad sources are typically unknown before trail. This problem is modelled using Partially Observable Markov Decision Process (POMDP), and the exploration efficiency is increased by utilising the correlation of ads. The proposed method reports 23.4% better than the best performing baseline in the real-world data based experiments. Since some ad networks allow (or expect) an input of keywords, the thesis also presents an adaptive keyword extraction system using BM25F algorithm and the multi-armed bandits model. This system has been tested by a domain service provider in crowdsourcing based experiments. If the publisher selects a Real-Time Bidding (RTB) ad source, he can use reserve price to manipulate auctions for better payoff. This thesis proposes a simplified game model that considers the competition between seller and buyer to be one-shot instead of repeated and gives heuristics that can be easily implemented. The model has been evaluated in a production environment and reported 12.3% average increase of revenue. The documentation of a prototype system for reserve price optimisation is also presented in the appendix of the thesis

Computational models of trust for cooperative evolution. Reputation based game theoretic models of trust for cooperative evolution in online business games.

Online services such as e-marketplaces, social networking sites, online gaming environments etc have grown in popularity in the recent years. These services represent situation where participants do not get to negotiate face to face before interaction and most of the time parties to transaction remain anonymous. It is thus necessary to have a system that rightly assesses trustworthiness of the other party in order to maintain quality assurance in such systems. Recent works on Trust and Reputation in online communities have focused on identifying probable defaulters, but less effort has been put to come up with system that make cooperation attractive over defection in order to achieve cooperation without enforcement. Our work in this regard concerns design and investigation of trust assessment systems that not only filter defaulters but also promote evolution of cooperativeness in player society. Based on the concept of game theory and prisoner¿s dilemma, we model business games and design incentive method, compensation method, acquaintance based assessment method and decision theoretic assessment method as mechanisms to assure trustworthiness in online business environments. Effectiveness of each of these methods in promoting the evolution of cooperation in player society has been investigated. Our results show that these methods contribute positively in promoting cooperative evolution. We have further extended our trust assessment model to suit the needs of a mobile ad-hoc network setting. The effectiveness of this model has been tested against its capability to reduce packet drop rate and energy conservation. In both of these the results show promise.EU Asia-link project TH/Asia Link/004(91712) East-West and its local host Kantipur Engineering College in Nepal, and Kathmandu University

A POMDP Based Approach to Optimally Select Sellers in Electronic Marketplaces

Selecting a seller in e-markets is a tedious task that we might want to delegate to an agent. Many approaches to constructing such agents have been proposed, building upon different foundations (decision theory, trust modeling) and making use of different information (direct experience with sellers, reputation of sellers, trustworthiness of other buyers called advisors, etc.). In this paper, we propose the SALE POMDP, a new approach based on the decision-theoretic framework of POMDPs. It enables optimal trade-offs of information gaining and exploiting actions, with the ultimate goal of maximizing buyer satisfaction. A unique feature of the model is that it allows querying advisors about the trustworthiness of other advisors. We represent the model as a factored POMDP, thereby enabling the use of computationally more efficient solution methods. Evaluation on the ART testbed demonstrates that SALE POMDP balances the cost of obtaining and benefit of more information more effectively, leading to more earnings, than traditional trust models. Experiments also show that it is more robust to deceptive advisors than a previous POMDP based approach, and that the factored formulation allows the solution of reasonably large instances of seller selection problems