Learning Adaptive Display Exposure for Real-Time Advertising

In E-commerce advertising, where product recommendations and product ads are presented to users simultaneously, the traditional setting is to display ads at fixed positions. However, under such a setting, the advertising system loses the flexibility to control the number and positions of ads, resulting in sub-optimal platform revenue and user experience. Consequently, major e-commerce platforms (e.g., Taobao.com) have begun to consider more flexible ways to display ads. In this paper, we investigate the problem of advertising with adaptive exposure: can we dynamically determine the number and positions of ads for each user visit under certain business constraints so that the platform revenue can be increased? More specifically, we consider two types of constraints: request-level constraint ensures user experience for each user visit, and platform-level constraint controls the overall platform monetization rate. We model this problem as a Constrained Markov Decision Process with per-state constraint (psCMDP) and propose a constrained two-level reinforcement learning approach to decompose the original problem into two relatively independent sub-problems. To accelerate policy learning, we also devise a constrained hindsight experience replay mechanism. Experimental evaluations on industry-scale real-world datasets demonstrate the merits of our approach in both obtaining higher revenue under the constraints and the effectiveness of the constrained hindsight experience replay mechanism.Comment: accepted by CIKM201

Optimal Participation of Power Generating Companies in a Deregulated Electricity Market

The function of an electric utility is to make stable electric power available to consumers in an efficient manner. This would include power generation, transmission, distribution and retail sales. Since the early nineties however, many utilities have had to change from the vertically integrated structure to a deregulated system where the services were unbundled due to a rapid demand growth and need for better economic benefits. With the unbundling of services came competition which pushed innovation and led to the improvement of efficiency. In a deregulated power system, power generators submit offers to sell energy and operating reserve in the electricity market. The market can be described more as oligopolistic with a System Operator in-charge of the power grid, matching the offers to supply with the bid in demands to determine the market clearing price for each interval. This price is what is paid to all generators. Energy is sold in the day-ahead market where offers are submitted hours prior to when it is needed. The spot energy market caters to unforeseen rise in load demand and thus commands a higher price for electrical energy than the day-ahead market. A generating company can improve its profit by using an appropriate bidding strategy. This improvement is affected by the nature of bids from competitors and uncertainty in demand. In a sealed bid auction, bids are submitted simultaneously within a timeframe and are confidential, thus a generator has no information on rivals’ bids. There have been studies on methods used by generators to build optimal offers considering competition. However, many of these studies base estimations of rivals’ behaviour on analysis with sufficient bidding history data from the market. Historical data on bidding behaviour may not be readily available in practical systems. The work reported in this thesis explores ways a generator can make security-constrained offers in different markets considering incomplete market information. It also incorporates possible uncertainty in load forecasts. The research methodology used in this thesis is based on forecasting and optimization. Forecasts of market clearing price for each market interval are calculated and used in the objective function of profit maximization to get maximum benefit at the interval. Making these forecasts includes competition into the bid process. Results show that with information on historical data available, a generator can make adequate short-term analysis on market behaviour and thus optimize its benefits for the period. This thesis provides new insights into power generators’ approach in making optimal bids to maximize market benefits

Building and investigating generators' bidding strategies in an electricity market

In a deregulated electricity market environment, Generation Companies (GENCOs) compete with each other in the market through spot energy trading, bilateral contracts and other financial instruments. For a GENCO, risk management is among the most important tasks. At the same time, how to maximise its profit in the electricity market is the primary objective of its operations and strategic planning. Therefore, to achieve the best risk-return trade-off, a GENCO needs to determine how to allocate its assets. This problem is also called portfolio optimization. This dissertation presents advanced techniques for generator strategic bidding, portfolio optimization, risk assessment, and a framework for system adequacy optimisation and control in an electricity market environment. Most of the generator bidding related problems can be regarded as complex optimisation problems. In this dissertation, detailed discussions of optimisation methods are given and a number of approaches are proposed based on heuristic global optimisation algorithms for optimisation purposes. The increased level of uncertainty in an electricity market can result in higher risk for market participants, especially GENCOs, and contribute significantly to the drivers for appropriate bidding and risk management tasks for GENCOs in the market. Accordingly, how to build an optimal bidding strategy considering market uncertainty is a fundamental task for GENCOs. A framework of optimal bidding strategy is developed out of this research. To further enhance the effectiveness of the optimal bidding framework; a Support Vector Machine (SVM) based method is developed to handle the incomplete information of other generators in the market, and therefore form a reliable basis for a particular GENCO to build an optimal bidding strategy. A portfolio optimisation model is proposed to maximise the return and minimise the risk of a GENCO by optimally allocating the GENCO's assets among different markets, namely spot market and financial market. A new market pnce forecasting framework is given In this dissertation as an indispensable part of the overall research topic. It further enhances the bidding and portfolio selection methods by providing more reliable market price information and therefore concludes a rather comprehensive package for GENCO risk management in a market environment. A detailed risk assessment method is presented to further the price modelling work and cover the associated risk management practices in an electricity market. In addition to the issues stemmed from the individual GENCO, issues from an electricity market should also be considered in order to draw a whole picture of a GENCO's risk management. In summary, the contributions of this thesis include: 1) a framework of GENCO strategic bidding considering market uncertainty and incomplete information from rivals; 2) a portfolio optimisation model achieving best risk-return trade-off; 3) a FIA based MCP forecasting method; and 4) a risk assessment method and portfolio evaluation framework quantifying market risk exposure; through out the research, real market data and structure from the Australian NEM are used to validate the methods. This research has led to a number of publications in book chapters, journals and refereed conference proceedings

Utilization of Electric Prosumer Flexibility Incentivized by Spot and Balancing Markets

The use of energy flexibility to balance electricity demand and supply is becoming increasingly important due to the growing share of fluctuating energy sources. Electric flexibility regarding time or magnitude of consumption can be offered in the form of different products on electricity spot and balancing power markets. In the wake of the energy transition and because of new possibilities provided by digitalization, the decision intervals on these markets are becoming shorter and the controllability of electricity consumption and generation more small-scale. This evolution opens up new chances for formerly passive energy consumers. This thesis shows how electric flexibility can be monetized using the application example of commercial sites. These are often multimodal energy systems coupling electricity, heat, and gas, and thus deliver high flexibility potential. To leverage this potential, a comprehensive picture of demand-side flexibilization is provided and used to propose an energy management system and optimization for cost-optimized device schedules. The cost-optimization considers two simultaneous incentives: variable day-ahead spot market prices and revenues for offering possible schedule adjustments to the automatic Frequency Restoration Reserve (aFRR) balancing market. To solve the formulated optimization problem, a genetic algorithm is presented, tailored to the specific needs of consumers. In addition to addressing the trade-off between the two competing markets, the algorithm inherently considers the uncertain activation of aFRR bids and related catch-up effects. An analysis of the activation behavior of aFRR balancing market bids, based on a developed ex-post simulation, forms an important decision basis for the optimization. Finally, a simulation study concentrating on battery energy storage systems and combined heat and power plants on the consumer side enables the quantitative discussion of the optimization potential. The results show that consumers considering both markets simultaneously can achieve cost benefits that are up to multiples of those for pure day-ahead price optimization, despite the stochastic nature of aFRR balancing power activations. In conclusion, this thesis enables formerly passive electricity consumers to assume the role of alternative balancing service providers, hence contributing to the economic and reliable operation of power grids characterized by a high share of renewable energy sources

Energy Management

Forecasts point to a huge increase in energy demand over the next 25 years, with a direct and immediate impact on the exhaustion of fossil fuels, the increase in pollution levels and the global warming that will have significant consequences for all sectors of society. Irrespective of the likelihood of these predictions or what researchers in different scientific disciplines may believe or publicly say about how critical the energy situation may be on a world level, it is without doubt one of the great debates that has stirred up public interest in modern times. We should probably already be thinking about the design of a worldwide strategic plan for energy management across the planet. It would include measures to raise awareness, educate the different actors involved, develop policies, provide resources, prioritise actions and establish contingency plans. This process is complex and depends on political, social, economic and technological factors that are hard to take into account simultaneously. Then, before such a plan is formulated, studies such as those described in this book can serve to illustrate what Information and Communication Technologies have to offer in this sphere and, with luck, to create a reference to encourage investigators in the pursuit of new and better solutions

Agent-orientated auction mechanism and strategy design

Agent-based technology is playing an increasingly important role in today’s economy. Usually a multi-agent system is needed to model an economic system such as a market system, in which heterogeneous trading agents interact with each other autonomously. Two questions often need to be answered regarding such systems: 1) How to design an interacting mechanism that facilitates efficient resource allocation among usually self-interested trading agents? 2) How to design an effective strategy in some specific market mechanisms for an agent to maximise its economic returns? For automated market systems, auction is the most popular mechanism to solve resource allocation problems among their participants. However, auction comes in hundreds of different formats, in which some are better than others in terms of not only the allocative efficiency but also other properties e.g., whether it generates high revenue for the auctioneer, whether it induces stable behaviour of the bidders. In addition, different strategies result in very different performance under the same auction rules. With this background, we are inevitably intrigued to investigate auction mechanism and strategy designs for agent-based economics. The international Trading Agent Competition (TAC) Ad Auction (AA) competition provides a very useful platform to develop and test agent strategies in Generalised Second Price auction (GSP). AstonTAC, the runner-up of TAC AA 2009, is a successful advertiser agent designed for GSP-based keyword auction. In particular, AstonTAC generates adaptive bid prices according to the Market-based Value Per Click and selects a set of keyword queries with highest expected profit to bid on to maximise its expected profit under the limit of conversion capacity. Through evaluation experiments, we show that AstonTAC performs well and stably not only in the competition but also across a broad range of environments. The TAC CAT tournament provides an environment for investigating the optimal design of mechanisms for double auction markets. AstonCAT-Plus is the post-tournament version of the specialist developed for CAT 2010. In our experiments, AstonCAT-Plus not only outperforms most specialist agents designed by other institutions but also achieves high allocative efficiencies, transaction success rates and average trader profits. Moreover, we reveal some insights of the CAT: 1) successful markets should maintain a stable and high market share of intra-marginal traders; 2) a specialist’s performance is dependent on the distribution of trading strategies. However, typical double auction models assume trading agents have a fixed trading direction of either buy or sell. With this limitation they cannot directly reflect the fact that traders in financial markets (the most popular application of double auction) decide their trading directions dynamically. To address this issue, we introduce the Bi-directional Double Auction (BDA) market which is populated by two-way traders. Experiments are conducted under both dynamic and static settings of the continuous BDA market. We find that the allocative efficiency of a continuous BDA market mainly comes from rational selection of trading directions. Furthermore, we introduce a high-performance Kernel trading strategy in the BDA market which uses kernel probability density estimator built on historical transaction data to decide optimal order prices. Kernel trading strategy outperforms some popular intelligent double auction trading strategies including ZIP, GD and RE in the continuous BDA market by making the highest profit in static games and obtaining the best wealth in dynamic games

Pricing the Cloud: An Auction Approach

Cloud computing has changed the processing and service modes of information communication technology and has affected the transformation, upgrading and innovation of the IT-related industry systems. The rapid development of cloud computing in business practice has spawned a whole new field of interdisciplinary, providing opportunities and challenges for business management research. One of the critical factors impacting cloud computing is how to price cloud services. An appropriate pricing strategy has important practical means to stakeholders, especially to providers and customers. This study addressed and discussed research findings on cloud computing pricing strategies, such as fixed pricing, bidding pricing, and dynamic pricing. Another key factor for cloud computing is Quality of Service (QoS), such as availability, reliability, latency, security, throughput, capacity, scalability, elasticity, etc. Cloud providers seek to improve QoS to attract more potential customers; while, customers intend to find QoS matching services that do not exceed their budget constraints. Based on the existing study, a hybrid QoS-based pricing mechanism, which consists of subscription and dynamic auction design, is proposed and illustrated to cloud services. The results indicate that our hybrid pricing mechanism has potential to better allocate available cloud resources, aiming at increasing revenues for providers and reducing expenses for customers in practice