Rational bidding using reinforcement learning: an application in automated resource allocation

The application of autonomous agents by the provisioning and usage of computational resources is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic resource provisioning and usage of computational resources, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a framework for supporting consumers and providers in technical and economic preference elicitation and the generation of bids. Secondly, we introduce a consumer-side reinforcement learning bidding strategy which enables rational behavior by the generation and selection of bids. Thirdly, we evaluate and compare this bidding strategy against a truth-telling bidding strategy for two kinds of market mechanisms – one centralized and one decentralized

Q-Strategy: A Bidding Strategy for Market-Based Allocation of Grid Services

The application of autonomous agents by the provisioning and usage of computational services is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic service provisioning and usage of Grid services, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a bidding agent framework for implementing artificial bidding agents, supporting consumers and providers in technical and economic preference elicitation as well as automated bid generation by the requesting and provisioning of Grid services. Secondly, we introduce a novel consumer-side bidding strategy, which enables a goal-oriented and strategic behavior by the generation and submission of consumer service requests and selection of provider offers. Thirdly, we evaluate and compare the Q-strategy, implemented within the presented framework, against the Truth-Telling bidding strategy in three mechanisms – a centralized CDA, a decentralized on-line machine scheduling and a FIFO-scheduling mechanisms

Users’ time preference based stochastic resource allocation in cloud spot market: cloud provider’s perspective

Cloud Computing spot markets have enabled the users to make use of the spare computing capacities of the cloud providers at a relatively cheaper price which in turn has given the providers such as Amazon and Google an opportunity to earn extra money by auctioning-off the underutilized resources. However, resource availability is a problem in the spot market owing to spot-price fluctuations. Ignoring the customer’s preference is one of the potential reasons behind this. In this paper, we propose a time preference (value of service at different points of time) based stochastic integer linear programming model to allocate the cloud resources among the cloud users with a view to maximizing the revenue of cloud providers from the spot-market

Seamless Service Provisioning for Mobile Crowdsensing: Towards Integrating Forward and Spot Trading Markets

The challenge of exchanging and processing of big data over Mobile Crowdsensing (MCS) networks calls for the new design of responsive and seamless service provisioning as well as proper incentive mechanisms. Although conventional onsite spot trading of resources based on real-time network conditions and decisions can facilitate the data sharing over MCS networks, it often suffers from prohibitively long service provisioning delays and unavoidable trading failures due to its reliance on timely analysis of complex and dynamic MCS environments. These limitations motivate us to investigate an integrated forward and spot trading mechanism (iFAST), which entails a new hybrid service trading protocol over the MCS network architecture. In iFAST, the sellers (i.e., mobile users with sensing resources) can provide long-term or temporary sensing services to the buyers (i.e., sensing task owners). iFast enables signing long-term contracts in advance of future transactions through a forward trading mode, via analyzing historical statistics of the market, for which the notion of overbooking is introduced and promoted. iFAST further enables the buyers with unsatisfying service quality to recruit temporary sellers through a spot trading mode, upon considering the current market/network conditions. We analyze the fundamental blocks of iFAST, and provide a case study to demonstrate its superior performance as compared to existing methods. Finally, future research directions on reliable service provisioning for next-generation MCS networks are summarized

Automated Negotiation for Complex Multi-Agent Resource Allocation

The problem of constructing and analyzing systems of intelligent, autonomous agents is becoming more and more important. These agents may include people, physical robots, virtual humans, software programs acting on behalf of human beings, or sensors. In a large class of multi-agent scenarios, agents may have different capabilities, preferences, objectives, and constraints. Therefore, efficient allocation of resources among multiple agents is often difficult to achieve. Automated negotiation (bargaining) is the most widely used approach for multi-agent resource allocation and it has received increasing attention in the recent years. However, information uncertainty, existence of multiple contracting partners and competitors, agents\u27 incentive to maximize individual utilities, and market dynamics make it difficult to calculate agents\u27 rational equilibrium negotiation strategies and develop successful negotiation agents behaving well in practice. To this end, this thesis is concerned with analyzing agents\u27 rational behavior and developing negotiation strategies for a range of complex negotiation contexts. First, we consider the problem of finding agents\u27 rational strategies in bargaining with incomplete information. We focus on the principal alternating-offers finite horizon bargaining protocol with one-sided uncertainty regarding agents\u27 reserve prices. We provide an algorithm based on the combination of game theoretic analysis and search techniques which finds agents\u27 equilibrium in pure strategies when they exist. Our approach is sound, complete and, in principle, can be applied to other uncertainty settings. Simulation results show that there is at least one pure strategy sequential equilibrium in 99.7% of various scenarios. In addition, agents with equilibrium strategies achieved higher utilities than agents with heuristic strategies. Next, we extend the alternating-offers protocol to handle concurrent negotiations in which each agent has multiple trading opportunities and faces market competition. We provide an algorithm based on backward induction to compute the subgame perfect equilibrium of concurrent negotiation. We observe that agents\u27 bargaining power are affected by the proposing ordering and market competition and for a large subset of the space of the parameters, agents\u27 equilibrium strategies depend on the values of a small number of parameters. We also extend our algorithm to find a pure strategy sequential equilibrium in concurrent negotiations where there is one-sided uncertainty regarding the reserve price of one agent. Third, we present the design and implementation of agents that concurrently negotiate with other entities for acquiring multiple resources. Negotiation agents are designed to adjust 1) the number of tentative agreements and 2) the amount of concession they are willing to make in response to changing market conditions and negotiation situations. In our approach, agents utilize a time-dependent negotiation strategy in which the reserve price of each resource is dynamically determined by 1) the likelihood that negotiation will not be successfully completed, 2) the expected agreement price of the resource, and 3) the expected number of final agreements. The negotiation deadline of each resource is determined by its relative scarcity. Since agents are permitted to decommit from agreements, a buyer may make more than one tentative agreement for each resource and the maximum number of tentative agreements is constrained by the market situation. Experimental results show that our negotiation strategy achieved significantly higher utilities than simpler strategies. Finally, we consider the problem of allocating networked resources in dynamic environment, such as cloud computing platforms, where providers strategically price resources to maximize their utility. While numerous auction-based approaches have been proposed in the literature, our work explores an alternative approach where providers and consumers negotiate resource leasing contracts. We propose a distributed negotiation mechanism where agents negotiate over both a contract price and a decommitment penalty, which allows agents to decommit from contracts at a cost. We compare our approach experimentally, using representative scenarios and workloads, to both combinatorial auctions and the fixed-price model, and show that the negotiation model achieves a higher social welfare

Moving Toward A Consensus on Climate Policy: The Essential Role of Global Public Disclosure

Among climate scientists, there is no longer any serious debate about whether greenhouse gas emissions from human activity are altering the earth’s climate. There is also a broad consensus on two issues related to reducing emissions. First, developing countries must be full participants in global emissions control, because they will be most heavily impacted by global warming, and because they are rapidly approaching parity with developed countries in the scale of their emissions. Second, efficient emissions control will require carbon pricing via market-based instruments (charges or cap-and-trade). These points of consensus are sufficient to establish a clear way forward, despite continued disagreements over the choice of specific instrument and the appropriate carbon charge level. Since all market-based systems that regulate emissions sources require the same emissions information, the international community should immediately establish an institution mandated to collect, verify and publicly disclose information about emissions from all significant global carbon sources. Its mandate should extend to best-practice estimation and disclosure of emissions sources in countries that initially refuse to participate. This institution will serve four purposes. First, it will lay the necessary foundation for implementing any market-based system of emissions source regulation. Second, it will provide an excellent credibility test, since a country’s acceptance of full disclosure will signal its true willingness to participate in globally-efficient emissions reduction. Third, global public disclosure will itself reduce carbon emissions, by focusing stakeholder pressure on major emitters and providing reputational rewards for clean producers. Fourth, disclosure will make it very hard to cheat once market-based instruments are implemented. This will be essential for preserving the credibility of an international agreement to reduce emissions.climate change

Game theory for cooperation in multi-access edge computing

Cooperative strategies amongst network players can improve network performance and spectrum utilization in future networking environments. Game Theory is very suitable for these emerging scenarios, since it models high-complex interactions among distributed decision makers. It also finds the more convenient management policies for the diverse players (e.g., content providers, cloud providers, edge providers, brokers, network providers, or users). These management policies optimize the performance of the overall network infrastructure with a fair utilization of their resources. This chapter discusses relevant theoretical models that enable cooperation amongst the players in distinct ways through, namely, pricing or reputation. In addition, the authors highlight open problems, such as the lack of proper models for dynamic and incomplete information scenarios. These upcoming scenarios are associated to computing and storage at the network edge, as well as, the deployment of large-scale IoT systems. The chapter finalizes by discussing a business model for future networks.info:eu-repo/semantics/acceptedVersio

Automated Bidding in Computing Service Markets. Strategies, Architectures, Protocols

This dissertation contributes to the research on Computational Mechanism Design by providing novel theoretical and software models - a novel bidding strategy called Q-Strategy, which automates bidding processes in imperfect information markets, a software framework for realizing agents and bidding strategies called BidGenerator and a communication protocol called MX/CS, for expressing and exchanging economic and technical information in a market-based scheduling system