Decentralized Learning for Multi-player Multi-armed Bandits

We consider the problem of distributed online learning with multiple players in multi-armed bandits (MAB) models. Each player can pick among multiple arms. When a player picks an arm, it gets a reward. We consider both i.i.d. reward model and Markovian reward model. In the i.i.d. model each arm is modelled as an i.i.d. process with an unknown distribution with an unknown mean. In the Markovian model, each arm is modelled as a finite, irreducible, aperiodic and reversible Markov chain with an unknown probability transition matrix and stationary distribution. The arms give different rewards to different players. If two players pick the same arm, there is a "collision", and neither of them get any reward. There is no dedicated control channel for coordination or communication among the players. Any other communication between the users is costly and will add to the regret. We propose an online index-based distributed learning policy called ${\tt dUCB_4}$ algorithm that trades off \textit{exploration v. exploitation} in the right way, and achieves expected regret that grows at most as near-$O(\log^2 T)$. The motivation comes from opportunistic spectrum access by multiple secondary users in cognitive radio networks wherein they must pick among various wireless channels that look different to different users. This is the first distributed learning algorithm for multi-player MABs to the best of our knowledge.Comment: 33 pages, 3 figures. Submitted to IEEE Transactions on Information Theor

Mechanism Design for Demand Response Programs

Demand Response (DR) programs serve to reduce the consumption of electricity at times when the supply is scarce and expensive. The utility informs the aggregator of an anticipated DR event. The aggregator calls on a subset of its pool of recruited agents to reduce their electricity use. Agents are paid for reducing their energy consumption from contractually established baselines. Baselines are counter-factual consumption estimates of the energy an agent would have consumed if they were not participating in the DR program. Baselines are used to determine payments to agents. This creates an incentive for agents to inflate their baselines. We propose a novel self-reported baseline mechanism (SRBM) where each agent reports its baseline and marginal utility. These reports are strategic and need not be truthful. Based on the reported information, the aggregator selects or calls on agents to meet the load reduction target. Called agents are paid for observed reductions from their self-reported baselines. Agents who are not called face penalties for consumption shortfalls below their baselines. The mechanism is specified by the probability with which agents are called, reward prices for called agents, and penalty prices for agents who are not called. Under SRBM, we show that truthful reporting of baseline consumption and marginal utility is a dominant strategy. Thus, SRBM eliminates the incentive for agents to inflate baselines. SRBM is assured to meet the load reduction target. SRBM is also nearly efficient since it selects agents with the smallest marginal utilities, and each called agent contributes maximally to the load reduction target. Finally, we show that SRBM is almost optimal in the metric of average cost of DR provision faced by the aggregator

HEMODYNAMIC COMPLICATION IN ACUTE MYOCARDIAL INFARCTION

Universitatea de Stat de Medicină şi Farmacie „Nicolae Testemiţanu”, Chişinău, Republica MoldovaIntroduction. Acute myocardial infarction (AMI) is defined by the presence of myocardial necrosis. The pathogenesis that underlies AMI is complex and multifactorial. One of the most important components, however, is the role of thrombus formation leading to sudden coronary occlusion. AMI are one of the leading causes of death in the developed world. The incidence of MI was eight to nine times greater in people aged 55 to 64 years. Purpose of paper is to study the haemodynamic complication in AMI according to Killip classification. Materials and methods used. The retrospective cross-sectional study was based on 60 case histories analysis of patients admitted with AMI in the Cardiology Discipline, of The Holy Trinity Municipal Clinical Hospital. Results. All of the 60 patients with AMI were subjected to echo. Analysis of each patients showed that, 55.75% out of all patients had diastolic dysfunction, 65% of all patients had pulmonary hypertension, 60.9% of all patients had reduced ejection fraction and 93.2% of all patients had hypokinesia/akinesia of myocardial wall. All the patients were also subjected to coronary angiography that showed, that the majority of patients 71.6% had tricoronary artery lesions, 13.3% of all patients had bicoronary lesions and 8.3% of all patients had mono vascular lesions. In Killip 3, patients with tricoronary lesion were 78% and in Killip 4 with 81%. Our analysis showed that the tricoronary occlusion was found to be in 50% of patients in Killip 1, 68.75% of patients in Killip 2, 77.7% of patients in Killip 3 and 71.4% of patients in Killip 4. Conclusion. Severe hemodynamic complications like acute pulmonary edema and cardiogenic shock are seen in most of the patients after AMI. The contractility of the myocardium is reduced (hypokinesia), leading to decreased ejection fraction and insufficient blood supply to the organs causing cardiogenic shock. Hypertension causes a volume overload in the left ventricle and because of hypokinesia, the blood is pooled back into the lungs increasing the pulmonary artery pressure and fluid leakage from alveoli causing pulmonary edema. The formation of thrombus has led to tricoronary lesions and severe complications