Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions

Learning in MDPs with highly complex state representations is currently possible due to multiple advancements in reinforcement learning algorithm design. However, this incline in complexity, and furthermore the increase in the dimensions of the observation came at the cost of volatility that can be taken advantage of via adversarial attacks (i.e. moving along worst-case directions in the observation space). To solve this policy instability problem we propose a novel method to detect the presence of these non-robust directions via local quadratic approximation of the deep neural policy loss. Our method provides a theoretical basis for the fundamental cut-off between safe observations and adversarial observations. Furthermore, our technique is computationally efficient, and does not depend on the methods used to produce the worst-case directions. We conduct extensive experiments in the Arcade Learning Environment with several different adversarial attack techniques. Most significantly, we demonstrate the effectiveness of our approach even in the setting where non-robust directions are explicitly optimized to circumvent our proposed method.Comment: Published in ICML 202

Sûrnâmelerde 1582 şenliği

Cataloged from PDF version of article.A magnificent festival was held in İstanbul in 1582 for the circumcision of Şehzade Mehmed, the son of Sultan Murad III. This glorious festival did not only become the subject of many literary works but also generated the birth of a new literary genre. This genre, named “sûrnâme” (festival book), takes the imperial festivals as its subject. The first examples of these festival books are Câmi’u’l-Buhur Der Mecâlis-i Sûr (Gatherer of the Seas in the Gatherings of the Festival) of Gelibolulu Âlî and Sûrnâme-i Hümâyûn (Imperial Festival Book) of İntizâmî. The festival books serve as important sources for subjects such as the daily life of the Ottomans, court traditions, means of production and various forms of art. These texts have attracted many scholars with the value of their content. Nevertheless, these studies mainly focuse on the festival itself. How this festival is transformed into literary texts by the authors of festival books has not been thoroughly analyzed so far. This thesis introduces the genre of festival book, which flourishes related to the Ottoman tradition of imperial festival. In this study, the first examples of this new literary genre,which are Câmi’u’l-Buhur Der Mecâlis-i Sûr and Sûrnâme-i Hümâyûn, are examined in terms of their approach to the festival of 1582. The thesis also examines the way the authors of the festival books transformed the festival into literary texts, the discourse they adopt and the narrative forms they make use of. In this thesis, it has been determined that the festival books of 1582 reflect the language and literary conception of the period they were written in, both in terms of literary practices and the information they provide about the relationship between literature and the festival. It has been observed that, both of the authors aimed to form a literary language and searched for originality in transforming the festival into literary texts. Although İntizâmî and Gelibolulu Âlî narrate the same festival, their texts are very different from each other. It has been demonstrated that the authors of the festival books approach the festival differently. These different approaches are reflected by the narrative forms they made use of and also the discourse they adopted.Korkmaz, Gülsüm EzgiM.S

Deep Reinforcement Learning Policies Learn Shared Adversarial Features across MDPs

The use of deep neural networks as function approximators has led to striking progress for reinforcement learning algorithms and applications. Yet the knowledge we have on decision boundary geometry and the loss landscape of neural policies is still quite limited. In this paper, we propose a framework to investigate the decision boundary and loss landscape similarities across states and across MDPs. We conduct experiments in various games from Arcade Learning Environment, and discover that high sensitivity directions for neural policies are correlated across MDPs. We argue that these high sensitivity directions support the hypothesis that non-robust features are shared across training environments of reinforcement learning agents. We believe our results reveal fundamental properties of the environments used in deep reinforcement learning training, and represent a tangible step towards building robust and reliable deep reinforcement learning agents

Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness

Learning from raw high dimensional data via interaction with a given environment has been effectively achieved through the utilization of deep neural networks. Yet the observed degradation in policy performance caused by imperceptible worst-case policy dependent translations along high sensitivity directions (i.e. adversarial perturbations) raises concerns on the robustness of deep reinforcement learning policies. In our paper, we show that these high sensitivity directions do not lie only along particular worst-case directions, but rather are more abundant in the deep neural policy landscape and can be found via more natural means in a black-box setting. Furthermore, we show that vanilla training techniques intriguingly result in learning more robust policies compared to the policies learnt via the state-of-the-art adversarial training techniques. We believe our work lays out intriguing properties of the deep reinforcement learning policy manifold and our results can help to build robust and generalizable deep reinforcement learning policies

Appointment of limited commercial agent in joint stock companies

137 page

Klasik jeopolitik teorilerinin 2000-2016 yılları arasında Ortadoğu’ya yansımaları : Irak, Suriye, Libya ve Türkiye örnekleri

Jeopolitik, bir devletin coğrafi konumu, doğal kaynakları, demografik yapısı gibi özelliklerinin o devletin dış politikasına nasıl yansıdığıyla ilgilenir. Klasik jeopolitik teoriler ise kalpgâh, kalpgâhı çevreleyen ülkeler ve bunların özellikleri ile denizlere ve havaya hâkim olmanın dış politikaya nasıl yansıdığını açıklamaya çalışan teorilerdir. Aynı zamanda mekân kavramı klasik jeopolitikte önemlidir. Güçlü devletlerin güçlerini devam ettirebilmeleri için mekân arayışları vardır ve bu da politikalarını şekillendirirken zaman zaman şiddete de başvurdukları gerçeğini ortaya çıkarır. Bu çalışmada, klasik jeopolitik teorilerinin 2000-2016 yılları arasında Irak, Suriye, Libya ve Türkiye’de meydana gelen olaylar ya da ülkelerin mevcut sorunları üzerinden nasıl yansımalarının olduğu incelenmiştir. Irak, Suriye, Libya ve Türkiye’de meydana gelen olaylarda klasik jeopolitiğin etkisinin olduğu görülmüş, çalışmada bu etkilerin nedenlerine inilerek Ortadoğu’ya yansımasının nasıl olduğuna bakılmıştır. Irak örneğinde, 11 Eylül saldırganlarına destek verdiği ve kitle imha silahlarına sahip olduğu gerekçesiyle ve arka planda olan petrol hâkimiyetinden dolayı müdahale gerçekleştirilmiştir. Suriye örneğinde, rejim karşıtları ve rejimin çatışması dünya denkleminde Rusya ve ABD’nin karşı karşıya kalmasına neden olmuştur. İç savaş bölge ülkelerini de derinden etkilemiş, radikal dinci grupların doğuşuna neden olmuştur. Libya’da anti-demokratik rejimin yıkılması ve teröre desteğin önüne geçilmesi bağlamında müdahalede bulunulmuştur ancak yine de arka planda kaynak hâkimiyeti ve Akdeniz’e olan komşuluğu nedeniyle stratejik önemi bulunmaktadır. Türkiye açısındansa, Irak müdahalesinde Türkiye’nin desteğine kesin gözüyle bakan ABD’nin aradığı şekilde destek bulamaması Tezkere Krizini doğurmuştur. Yıllardır süregelen terör konusunda da ABD’nin Türkiye’ye yardımcı olmaması ilişkileri zora sokmaktadır. Klasik jeopolitik teoriler, adı her ne kadar klasik olarak geçse de günümüz 21. yüzyıl dünyasında etkilerini göstermektedir. Bunun örnekleri çalışmada sunulmuştur. -------------------- Geopolitics deals with how a state's characteristics such as its geographical location, its natural resources, its demographic structure are reflected in the foreign policy of that state. As for classical geopolitical theories, they try to explain how the heartland, the surrounding countries, and their characteristics, and the dominance of the seas and the air are reflected in the foreign policy. At the same time, the concept of “Raum” is important in classical geopolitics. Strong states have a search for Raum to sustain their power, and this reveals the fact that they sometimes resort to violence while shaping their policies. In this study, it is studied that between the years of 2000-2016, how the classical geopolitical theories have reflections in terms of the incidents and current problems in Iraq, Syria, Libya and Turkey. It has been seen that in the events which occurred in Iraq, Syria, Libya and Turkey, classical geopolitics has an effect and in this study the causes of these effects are analyzed on the grounds that how the Middle East is reflected. In the case of Iraq, an intervention was carried out on the grounds that it supported the September 11 attackers and possessed weapons of mass destruction and oil control in the background. In the case of Syria, the conflict of the opponents and the regime has caused Russia and the US confrontation in the world equation. The civil war deeply affected the countries of the region as well, causing the emergence of radical religious groups. Libya has been intervened in the context of the demolition of anti-democratic regime and support for terrorism, but it also has strategic importance due to its resource dominance and its proximity to the Mediterranean. With regard to Turkey, USA took it for granted that Turkey would support the US intervention in Iraq but as Turkey did not give the expected support, it led to the Tezkere crisis. As USA did not give support to Turkey for the on-going terrorism problem, it also jeopardizes the relations of the United States and Turkey Classical geopolitical theories, even though their names are classic, show their effects in today's 21st century world. Examples are presented in this study

Klasik jeopolitik teorilerinin 2000-2016 yılları arasında Ortadoğu’ya yansımaları : Irak, Suriye, Libya ve Türkiye örnekleri

Jeopolitik, bir devletin coğrafi konumu, doğal kaynakları, demografik yapısı gibi özelliklerinin o devletin dış politikasına nasıl yansıdığıyla ilgilenir. Klasik jeopolitik teoriler ise kalpgâh, kalpgâhı çevreleyen ülkeler ve bunların özellikleri ile denizlere ve havaya hâkim olmanın dış politikaya nasıl yansıdığını açıklamaya çalışan teorilerdir. Aynı zamanda mekân kavramı klasik jeopolitikte önemlidir. Güçlü devletlerin güçlerini devam ettirebilmeleri için mekân arayışları vardır ve bu da politikalarını şekillendirirken zaman zaman şiddete de başvurdukları gerçeğini ortaya çıkarır.Bu çalışmada, klasik jeopolitik teorilerinin 2000-2016 yılları arasında Irak, Suriye, Libya ve Türkiye’de meydana gelen olaylar ya da ülkelerin mevcut sorunları üzerinden nasıl yansımalarının olduğu incelenmiştir. Irak, Suriye, Libya ve Türkiye’de meydana gelen olaylarda klasik jeopolitiğin etkisinin olduğu görülmüş, çalışmada bu etkilerin nedenlerine inilerek Ortadoğu’ya yansımasının nasıl olduğuna bakılmıştır. Irak örneğinde, 11 Eylül saldırganlarına destek verdiği ve kitle imha silahlarına sahip olduğu gerekçesiyle ve arka planda olan petrol hâkimiyetinden dolayı müdahale gerçekleştirilmiştir. Suriye örneğinde, rejim karşıtları ve rejimin çatışması dünya denkleminde Rusya ve ABD’nin karşı karşıya kalmasına neden olmuştur. İç savaş bölge ülkelerini de derinden etkilemiş, radikal dinci grupların doğuşuna neden olmuştur. Libya’da anti-demokratik rejimin yıkılması ve teröre desteğin önüne geçilmesi bağlamında müdahalede bulunulmuştur ancak yine de arka planda kaynak hâkimiyeti ve Akdeniz’e olan komşuluğu nedeniyle stratejik önemi bulunmaktadır. Türkiye açısındansa, Irak müdahalesinde Türkiye’nin desteğine kesin gözüyle bakan ABD’nin aradığı şekilde destek bulamaması Tezkere Krizini doğurmuştur. Yıllardır süregelen terör konusunda da ABD’nin Türkiye’ye yardımcı olmaması ilişkileri zora sokmaktadır.Klasik jeopolitik teoriler, adı her ne kadar klasik olarak geçse de günümüz 21. yüzyıl dünyasında etkilerini göstermektedir. Bunun örnekleri çalışmada sunulmuştur.--------------------Geopolitics deals with how a state's characteristics such as its geographical location, its natural resources, its demographic structure are reflected in the foreign policy of that state. As for classical geopolitical theories, they try to explain how the heartland, the surrounding countries, and their characteristics, and the dominance of the seas and the air are reflected in the foreign policy. At the same time, the concept of “Raum” is important in classical geopolitics. Strong states have a search for Raum to sustain their power, and this reveals the fact that they sometimes resort to violence while shaping their policies.In this study, it is studied that between the years of 2000-2016, how the classical geopolitical theories have reflections in terms of the incidents and current problems in Iraq, Syria, Libya and Turkey. It has been seen that in the events which occurred in Iraq, Syria, Libya and Turkey, classical geopolitics has an effect and in this study the causes of these effects are analyzed on the grounds that how the Middle East is reflected. In the case of Iraq, an intervention was carried out on the grounds that it supported the September 11 attackers and possessed weapons of mass destruction and oil control in the background. In the case of Syria, the conflict of the opponents and the regime has caused Russia and the US confrontation in the world equation. The civil war deeply affected the countries of the region as well, causing the emergence of radical religious groups. Libya has been intervened in the context of the demolition of anti-democratic regime and support for terrorism, but it also has strategic importance due to its resource dominance and its proximity to the Mediterranean. With regard to Turkey, USA took it for granted that Turkey would support the US intervention in Iraq but as Turkey did not give the expected support, it led to the Tezkere crisis. As USA did not give support to Turkey for the on-going terrorism problem, it also jeopardizes the relations of the United States and TurkeyClassical geopolitical theories, even though their names are classic, show their effects in today's 21st century world. Examples are presented in this study

Üzüm Kullanılarak Üretilen Geleneksel Türk İçeceklerinin Antimikrobiyal, Aktioksidan Etkileri ve Toplam Fenolik İçerikleri

In the present study, antimicrobial effects of traditional beverages produced from grapes such as traditional grape pickles, grape juice, home-made hardaliye, and commercially produced hardaliye were investigated using microdilution method against Staphylococcus aureus ATCC 6538P, Escherichia coli ATCC 25922, Salmonella Typhimurium NRRL B4420, Bacillus cereus ATCC 10876, Saccharomyces cerevisiae NRRL Y- 12632, Lactobacillus acidophilus La-5 and L. rhamnosus LGG. in addition, total phenolic content was measured by the Folin-Ciocalteau method, and antioxidant activities of grape products were evaluated using DPPH assay. Results showed that grape containing beverages have antimicrobial effects on S. aureus, E. coli, S. typhimurium and B. cereus at various minimum inhibitory concentration (MIC) values in the range of 4.53- 150 mg/mL. the lowest MIC value of 4.53 mg/mL was obtained against E. coli for home-made hardaliye. MIC values of the traditional grape pickles were determined as 4.69 mg/mL and 9.38 mg/mL against S.aureus and E.coli, respectively. on the other hand, traditional grape pickles showed weak inhibitory effects against B.cereus with MIC value of 150 mg/mL. the bactericidal effect of these grape products was not detected for any of the test microorganisms however traditional foods produced by using grapes were showed inhibitory effects at different concentrations against tested microorganisms except for probiotics and the yeast. the total phenolic contents of the grape products were within the range of 865.27-2193.08 mg gallic acid equivalent (GAE)/L. Free radical scavenging activities of grape samples ranged from 46% to 90% and the grape juice was found to have the highest antioxidant activity. in conclusion, grape beverages have the potential to act as a antimicrobial and antioxidant agents for use as a natural antimicrobial and antioxidant product in the food industry.Bu çalışmada üzüm turşusu, üzüm suyu, ev yapımı ve ticari olarak üretilmiş hardaliye gibi üzüm ile üretilen geleneksel içeceklerin mikrodilüsyon yöntemi kullanılarak Staphylococcus aureus ATCC 6538P, Escherichia coli ATCC 25922, Salmonella Typhimurium NRRL B4420, Bacillus cereus ATCC 10876, Saccharomyces cerevisiae NRRL Y-12632, Lactobacillus acidophilus La-5 ve L. rhamnosus LGG’ye karşı antimikrobiyal etkisi araştırılmıştır. Ayrıca, üzüm ürünlerinin toplam fenolik içeriği Folin-Ciocalteau yöntemi ile ölçülmüş ve antioksidan aktiviteleri DPPH yöntemi ile değerlendirilmiştir. Sonuçlar, üzüm içeren bu içeceklerin S. aureus, E. coli, S. Typhimurium ve B. cereus üzerinde 4.53-150 mg/mL aralığında minimum inhibisyon konsantrasyonu (MİK) değerlerinde antimikrobiyal etkiye sahip olduğunu göstermiştir. Ev yapımı hardaliyenin en düşük MİK değerinin 4,53 mg/mL ile E. coli’ye karşı olduğu belirlenmiştir. Geleneksel üzüm turşunun S.aureus ve E.coli için MİK değerleri sırasıyla 4,69 mg/mL ve 9,38 mg/mL olarak belirlenmiştir. Diğer yandan, geleneksel üzüm turşusu, 150 mg/mL MİK değeri ile B.cereus'a karşı zayıf inhibisyon etkisi göstermiştir. Kullanılan bu üzüm ürünlerinin test edilen mikroorganizmaların hiçbirinde bakterisidal etkisi tespit edilememiştir, ancak üzüm kullanılarak üretilen geleneksel gıdaların, probiyotikler ve maya dışındaki test edilen mikroorganizmalara karşı farklı konsantrasyonlarda inhibe edici etki gösterdiği görülmüştür. Üzüm ürünlerinin toplam fenolik içerikleri 865,27-2193,08 mg gallik asit eşdeğeri (GAE)/L aralığındadır. Üzüm örneklerinin serbest radikal süpürme aktiviteleri %46 ile %90 arasında değişmiş olup, üzüm suyunun en yüksek antioksidan aktiviteye sahip olduğu bulunmuştur. Sonuç olarak, üzüm içeren geleneksel içeceklerin antimikrobiyal ve antioksidan ajan olarak kullanım potansiyelinin bulunduğu ve gıda sektöründe doğal bir antimikrobiyal ve antioksidan ürün olarak kullanılabileceği saptanmıştır