99 research outputs found
Modelling and analysis of temporal preference drifts using a component-based factorised latent approach
In recommender systems, human preferences are identified by a number of individual components with complicated interactions and properties. Recently, the dynamicity of preferences has been the focus of several studies. The changes in user preferences can originate from substantial reasons, like personality shift, or transient and circumstantial ones, like seasonal changes in item popularities. Disregarding these temporal drifts in modelling user preferences can result in unhelpful recommendations. Moreover, different temporal patterns can be associated with various preference domains, and preference components and their combinations. These components comprise preferences over features, preferences over feature values, conditional dependencies between features, socially-influenced preferences, and bias. For example, in the movies domain, the user can change his rating behaviour (bias shift), her preference for genre over language (feature preference shift), or start favouring drama over comedy (feature value preference shift). In this paper, we first propose a novel latent factor model to capture the domain-dependent component-specific temporal patterns in preferences. The component-based approach followed in modelling the aspects of preferences and their temporal effects enables us to arbitrarily switch components on and off. We evaluate the proposed method on three popular recommendation datasets and show that it significantly outperforms the most accurate state-of-the-art static models. The experiments also demonstrate the greater robustness and stability of the proposed dynamic model in comparison with the most successful models to date. We also analyse the temporal behaviour of different preference components and their combinations and show that the dynamic behaviour of preference components is highly dependent on the preference dataset and domain. Therefore, the results also highlight the importance of modelling temporal effects but also underline the advantages of a component-based architecture that is better suited to capture domain-specific balances in the contributions of the aspects
Preference Learning in Automated Negotiation Using Gaussian Uncertainty Models
In this paper, we propose a general two-objective Markov Decision Process (MDP) modeling paradigm for automated negotiation with incomplete information, in which preference elicitation alternates with negotiation actions, with the objective to optimize negotiation outcomes. The key ingredient in our MDP framework is a stochastic utility model governed by a Gaussian law, formalizing the agent's belief (uncertainty) over the user's preferences. Our belief model is fairly general and can be updated in real time as new data becomes available, which makes it a fundamental modeling tool
Automated peer-to-peer negotiation for energy contract settlements in residential cooperatives
This paper presents an automated peer-to-peer negotiation
strategy for settling energy contracts among prosumers in a Residential
Energy Cooperative considering heterogeneity prosumer preferences. The
heterogeneity arises from prosumers' evaluation of energy contracts
through multiple societal and environmental criteria and the prosumers'
private preferences over those criteria. The prosumers engage in
bilateral negotiations with peers to mutually agree on periodical energy
contracts/loans consisting of the energy volume to be exchanged at that
period and the return time of the exchanged energy. The negotiating
prosumers navigate through a common negotiation domain consisting of
potential energy contracts and evaluate those contracts from their
valuations on the entailed criteria against a utility function that is
robust against generation and demand uncertainty. From the repeated
interactions, a prosumer gradually learns about the compatibility of its
peers in reaching energy contracts that are closer to Nash solutions.
Empirical evaluation on real demand, generation and storage profiles –
in multiple system scales – illustrates that the proposed negotiation
based strategy can increase the system efficiency (measured by
utilitarian social welfare) and fairness (measured by Nash social
welfare) over a baseline strategy and an individual flexibility control
strategy representing the status quo strategy. We thus elicit system
benefits from peer-to-peer flexibility exchange already without any
central coordination and market operator, providing a simple yet
flexible and effective paradigm that complements existing markets
Automated negotiation with Gaussian process-based utility models
Designing agents that can efficiently learn and integrate user's preferences into decision making processes is a key challenge in automated negotiation. While accurate knowledge of user preferences is highly desirable, eliciting the necessary information might be rather costly, since frequent user interactions may cause inconvenience. Therefore, efficient elicitation strategies (minimizing elicitation costs) for inferring relevant information are critical. We introduce a stochastic, inverse-ranking utility model compatible with the Gaussian Process preference learning framework and integrate it into a (belief) Markov Decision Process paradigm which formalizes automated negotiation processes with incomplete information. Our utility model, which naturally maps ordinal preferences (inferred from the user) into (random) utility values (with the randomness reflecting the underlying uncertainty), provides the basic quantitative modeling ingredient for automated (agent-based) negotiation
RLBOA: A modular reinforcement learning framework for autonomous negotiating agents
Negotiation is a complex problem, in which the variety of settings and opponents that may be encountered prohibits the use of a single predefined negotiation strategy. Hence the agent should be able to learn such a strategy autonomously. To this end we propose RLBOA, a modular framework that facilitates the creation of autonomous negotiation agents using reinforcement learning. The framework allows for the creation of agents that are capable of negotiating effectively in many different scenarios. To be able to cope with the large size of the state and action spaces and diversity of settings, we leverage the modular BOA-framework. This decouples the negotiation strategy into a Bidding strategy, an Opponent model and an Acceptance condition. Furthermore, we map the multidimensional contract space onto the utility axis which enables a compact and generic state and action description. We demonstrate the value of the RLBOA framework by implementing an agent that uses tabular Q-learning on the compressed state and action space to learn a bidding strategy.We show that the resulting agent is able to learn well-performing bidding strategies in a range of negotiation settings and is able to generalize across opponents and domains
Automated Negotiations under User Preference Uncertainty: A Linear Programming Approach
Autonomous agents negotiating on our behalf find applications in everyday life in many domains such as high frequency trading, cloud computing and the smart grid among others. The agents negotiate with one another to reach the best agreement for the users they represent. An obstacle in the future of automated negotiators is that the agent may not always have a priori information about the preferences of the user it represents. The purpose of this work is to develop an agent that will be able to negotiate given partial information about the user’s preferences. First, we present a new partial information model that is supplied to the agent, which is based on categorical data in the form of pairwise comparisons of outcomes instead of precise utility information. Using this partial information, we develop an estimation model that uses linear optimization and translates the information into utility estimates. We test our methods in a negotiation scenario based on a smart grid cooperative where agents participate in energy trade-offs. The results show that already with very limited information the model becomes accurate quickly and performs well in an actual negotiation setting. Our work provides valuable insight into how uncertainty affects an agent’s negotiation performance, how much information is needed to be able to formulate an accurate user model, and shows a capability of negotiating effectively with minimal user feedback
Autonomous Bidding Coordinated Acceptance in one-to-many negotiations
This work presents the Autonomous Bidding Coordinated Acceptance framework (ABCA): An agent-Team design that allows general bilateral agents to engage in oneto-many negotiations in a setting where (possibly overlapping) deals with multiple opponents are desirable. We propose also a coordinated acceptance strategy that uses the estimated outcomes of its bilateral negotiations while deciding to accept a deal
An optimal rewiring strategy for cooperative multiagent social learning
Multiagent coordination is a key problem in cooperative multiagent systems (MASs). It has been widely studied in both fixed-agent repeated interaction setting and static social learning framework. However, two aspects of dynamics in real-world MASs are currently neglected. First, the network topologies can change during the course of interaction dynamically. Second, the interaction utilities can be different among each pair of agents and usually unknown before interaction. Both issues mentioned above increase the difficulty of coordination. In this paper, we consider the multiagent social learning in a dynamic environment in which agents can alter their connections and interact with randomly chosen neighbors with unknown utilities beforehand. We propose an optimal rewiring strategy to select most beneficial peers to maximize the accumulated payoffs in long-run interactions. We empirically demonstrate the effects of our approach in a variety of large-scale MASs
- …