On the Temporal Effects of Mobile Blockers in Urban Millimeter-Wave Cellular Scenarios

Millimeter-wave (mmWave) propagation is known to be severely affected by the blockage of the line-of-sight (LoS) path. In contrast to microwave systems, at shorter mmWave wavelengths such blockage can be caused by human bodies, where their mobility within environment makes wireless channel alternate between the blocked and non-blocked LoS states. Following the recent 3GPP requirements on modeling the dynamic blockage as well as the temporal consistency of the channel at mmWave frequencies, in this paper a new model for predicting the state of a user in the presence of mobile blockers for representative 3GPP scenarios is developed: urban micro cell (UMi) street canyon and park/stadium/square. It is demonstrated that the blockage effects produce an alternating renewal process with exponentially distributed non-blocked intervals, and blocked durations that follow the general distribution. The following metrics are derived (i) the mean and the fraction of time spent in blocked/non-blocked state, (ii) the residual blocked/non-blocked time, and (iii) the time-dependent conditional probability of having blockage/no blockage at time t1 given that there was blockage/no blockage at time t0. The latter is a function of the arrival rate (intensity), width, and height of moving blockers, distance to the mmWave access point (AP), as well as the heights of the AP and the user device. The proposed model can be used for system-level characterization of mmWave cellular communication systems. For example, the optimal height and the maximum coverage radius of the mmWave APs are derived, while satisfying the required mean data rate constraint. The system-level simulations corroborate that the use of the proposed method considerably reduces the modeling complexity.Comment: Accepted, IEEE Transactions on Vehicular Technolog

Inefficiencies in Digital Advertising Markets

Digital advertising markets are growing and attracting increased scrutiny. This article explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and within advertising channel members, ad blocking, and ad fraud. Although these topics are not unique to digital advertising, each manifests in unique ways in markets for digital ads. The authors identify relevant findings in the academic literature, recent developments in practice, and promising topics for future research

Modeling Mutual Influence in Multi-Agent Reinforcement Learning

In multi-agent systems (MAS), agents rarely act in isolation but tend to achieve their goals through interactions with other agents. To be able to achieve their ultimate goals, individual agents should actively evaluate the impacts on themselves of other agents' behaviors before they decide which actions to take. The impacts are reciprocal, and it is of great interest to model the mutual influence of agent's impacts with one another when they are observing the environment or taking actions in the environment. In this thesis, assuming that the agents are aware of each other's existence and their potential impact on themselves, I develop novel multi-agent reinforcement learning (MARL) methods that can measure the mutual influence between agents to shape learning. The first part of this thesis outlines the framework of recursive reasoning in deep multi-agent reinforcement learning. I hypothesize that it is beneficial for each agent to consider how other agents react to their behavior. I start from Probabilistic Recursive Reasoning (PR2) using level-1 reasoning and adopt variational Bayes methods to approximate the opponents' conditional policies. Each agent shapes the individual Q-value by marginalizing the conditional policies in the joint Q-value and finding the best response to improving their policies. I further extend PR2 to Generalized Recursive Reasoning (GR2) with different hierarchical levels of rationality. GR2 enables agents to possess various levels of thinking ability, thereby allowing higher-level agents to best respond to less sophisticated learners. The first part of the thesis shows that eliminating the joint Q-value to an individual Q-value via explicitly recursive reasoning would benefit the learning. In the second part of the thesis, in reverse, I measure the mutual influence by approximating the joint Q-value based on the individual Q-values. I establish Q-DPP, an extension of the Determinantal Point Process (DPP) with partition constraints, and apply it to multi-agent learning as a function approximator for the centralized value function. An attractive property of using Q-DPP is that when it reaches the optimum value, it can offer a natural factorization of the centralized value function, representing both quality (maximizing reward) and diversity (different behaviors). In the third part of the thesis, I depart from the action-level mutual influence and build a policy-space meta-game to analyze agents' relationship between adaptive policies. I present a Multi-Agent Trust Region Learning (MATRL) algorithm that augments single-agent trust region policy optimization with a weak stable fixed point approximated by the policy-space meta-game. The algorithm aims to find a game-theoretic mechanism to adjust the policy optimization steps that force the learning of all agents toward the stable point