Emergence of Addictive Behaviors in Reinforcement Learning Agents

This paper presents a novel approach to the technical analysis of wireheading in intelligent agents. Inspired by the natural analogues of wireheading and their prevalent manifestations, we propose the modeling of such phenomenon in Reinforcement Learning (RL) agents as psychological disorders. In a preliminary step towards evaluating this proposal, we study the feasibility and dynamics of emergent addictive policies in Q-learning agents in the tractable environment of the game of Snake. We consider a slightly modified settings for this game, in which the environment provides a "drug" seed alongside the original "healthy" seed for the consumption of the snake. We adopt and extend an RL-based model of natural addiction to Q-learning agents in this settings, and derive sufficient parametric conditions for the emergence of addictive behaviors in such agents. Furthermore, we evaluate our theoretical analysis with three sets of simulation-based experiments. The results demonstrate the feasibility of addictive wireheading in RL agents, and provide promising venues of further research on the psychopathological modeling of complex AI safety problems

A psychopathological approach to safety engineering in AI and AGI

The complexity of dynamics in AI techniques is already approaching that of complex adaptive systems, thus curtailing the feasibility of formal controllability and reachability analysis in the context of AI safety. It follows that the envisioned instances of Artificial General Intelligence (AGI) will also suffer from challenges of complexity. To tackle such issues, we propose the modeling of deleterious behaviors in AI and AGI as psychological disorders, thereby enabling the employment of psychopathological approaches to analysis and control of misbehaviors. Accordingly, we present a discussion on the feasibility of the psychopathological approaches to AI safety, and propose general directions for research on modeling, diagnosis, and treatment of psychological disorders in AGI

Tunable Low Density Palladium Nanowire Foams

Nanostructured palladium foams offer exciting potential for applications in diverse fields such as catalyst, fuel cell, and particularly hydrogen storage technologies. We have fabricated palladium nanowire foams using a cross-linking and freeze-drying technique. These foams have a tunable density down to 0.1% of the bulk, and a surface area to volume ratio of up to 1,540,000:1. They exhibit highly attractive characteristics for hydrogen storage, in terms of loading capacity, rate of absorption and heat of absorption. The hydrogen absorption/desorption process is hysteretic in nature, accompanied by substantial lattice expansion/contraction as the foam converts between Pd and PdHx.Comment: 17 pages, 4 figure

OPA1-related auditory neuropathy: site of lesion and outcome of cochlear implantation.

Hearing impairment is the second most prevalent clinical feature after optic atrophy in Dominant Optic Atrophy associated with mutations in the OPA1 gene. In this study we characterized the hearing dysfunction in OPA1-linked disorders and provided effective rehabilitative options to improve speech perception. We studied two groups of OPA1 subjects, one comprising 11 patients (7 males; age range 13-79 years) carrying OPA1 mutations inducing haploinsufficiency, the other, 10 subjects (3 males; age range 5-58 years) carrying OPA1 missense mutations. Both groups underwent audiometric assessment with pure tone and speech perception evaluation, and otoacoustic emissions and auditory brainstem response recording. Cochlear potentials were recorded through transtympanic electrocochleography from the group of patients harboring OPA1 missense mutations and were compared to recordings obtained from 20 normally-hearing controls and from 19 subjects with cochlear hearing loss. Eight patients carrying OPA1 missense mutations underwent cochlear implantation. Speech perception measures and electrically-evoked auditory nerve and brainstem responses were obtained after one year of cochlear implant use. Nine out of 11 patients carrying OPA1 mutations inducing haploinsufficiency had normal hearing function. In contrast, all but one subject harboring OPA1 missense mutations displayed impaired speech perception, abnormal brainstem responses and presence of otoacoustic emissions consistent with auditory neuropathy. In electrocochleography recordings, cochlear microphonic had enhanced amplitudes while summating potential showed normal latency and peak amplitude consistent with preservation of both outer and inner hair cell activities. After cancelling the cochlear microphonic, the synchronized neural response seen in both normally-hearing controls and subjects with cochlear hearing loss was replaced by a prolonged, low-amplitude negative potential that decreased in both amplitude and duration during rapid stimulation consistent with neural generation. The use of cochlear implant improved speech perception in all but one patient. Brainstem potentials were recorded in response to electrical stimulation in five subjects out of six, whereas no compound action potential was evoked from the auditory nerve through the cochlear implant. These findings indicate that underlying the hearing impairment in patients carrying OPA1 missense mutations is a disordered synchrony in auditory nerve fiber activity resulting from neural degeneration affecting the terminal dendrites. Cochlear implantation improves speech perception and synchronous activation of auditory pathways by by-passing the site of lesion

Алгоритми та програмні рішення для тестування вразливості в інтерфейсі SQL у веб-програмах

Software security gains importance day by day and developers try to secure web applications as much as possible to protect confidentiality, integrity and availability that are described in the fundamental security model so-called CIA triad. SQL injection vulnerability which can violate the confidentiality and integrity principles of the CIA triad is reviewed, and SQL injection attack execution and protection techniques are explained. The common frameworks’ solutions against SQL injection vulnerability were compared, and this comparison shown the most used techniques in this domain. Error-based and time-based detection algorithms for SQL injection’s identification are developed to create a vulnerability scanner that can detect SQL attacks which cause vulnerability in web applications, and these algorithms are represented in form of UML-activity diagrams. In order to discover all possible links and forms to perform SQL injection vulnerability tests in the entire website, a web crawler is needed. Breadth-First Search (BFS) algorithm for developing the web crawler is proposed, and the appropriate pseudo code and activity diagram are provided. Besides, Common Vulnerability Scoring System (CVSS) that is used to measure severity score of attacks that can violate CIA triad principles is reviewed. Qualitative severity score rating scale of CVSS is explained. An example of CVSS calculation is represented. Necessary components of a vulnerability scanner are explained. A vulnerability scanner prototype is developed using explained algorithms. Process results of this vulnerability scanner’s usage for real web applications are represented. Conclusions are made, and goals of future work are defined.Безпека програмного забезпечення щоденно набуває все більшого значення, і розробники намагаються максимально захистити веб-програми, щоб забезпечити їх конфіденційність, цілісність та доступність, які описані в основній моделі безпеки так званої тріади CIA. Розглянута вразливість SQL-ін'єкцій, яка може порушувати принципи конфіденційності та цілісності тріади ЦРУ та пояснюються виконання SQL-атак та методи захисту від них. Було проведено порівняння загальних структурних рішень для усунення вразливості SQL-ін’єкцій, яке виявило найпоширеніші технології у цій галузі. Розроблені алгоритми виявлення на основі помилок та на основі вимірі часу для ідентифікації SQL-ін’єкцій для створення сканера вразливості, який може виявити SQL-атаки, що викликають уразливість в веб-додатках, і ці алгоритми представлені у формі UML-діаграм активності. Щоб виявити всі можливі посилання та форми для виконання тестів вразливості на всьому веб-сайті, потрібен пошуковий веб-робот. Запропоновано алгоритм Breadth-First Search (BFS) для розробки веб-сканеру, для нього наведено псевдокод та діаграма активності. Розглядається система загальної оцінки вразливості (CVSS), яка використовується для вимірювання ступеня тяжкості атак, що можуть порушувати принципи захисту тріади ЦРУ. Роз'яснено якісну оціночну шкалу CVSS. Представлений приклад розрахунку CVSS. Розроблено прототип сканера вразливості з використанням запропонованих алгоритмів. Результати застосування цього сканеру вразливості представлені прикладами оцінки реальних веб-застосувань. Зроблено висновки, визначені цілі майбутньої роботи

ETM-ANN Approach Application for Thiobenzamide and Quinolizidine Derivatives

The structure anti-influenza activity relationships of thiobenzamide and quinolizidine derivatives, being influenza fusion inhibitors, have been investigated using the electronic-topological method (ETM) and artificial neural network (ANN) method. Molecular fragments specific for active compounds and breaks of activity were calculated for influenza fusion inhibitors by applying the ETM. QSAR descriptors such as molecular weight, EHOMO, ELUMO, ΔE, chemical potential, softness, electrophilicity index, dipole moment, and so forth were calculated, and it was found to give good statistical qualities (classified correctly 92%, or 48 compounds from 52 in training set, and 69% or 9 compounds from 13 in the external test set). By using multiple linear regression, several QSAR models were performed with the help of calculated descriptors and the compounds activity data. Among the obtained QSAR models, statistically the most significant one is the one of skeleton 1 with R2 = 0.999

Sweeping with all graphical ingredients in a topological picturebook

Sweeping is a powerful method to generate 3-D shapes in geometric modeling. In this paper we formulate a general matrix to give a mathematical definition of twisted-profiled sweep objects as a discrete approximation. While conceptually simple, our result is, to our best knowledge, the first precise formulation of sweeping with all graphical ingredients, viz. twisting, scaling, rotation, and translation. Twisted-profiled sweeping surfaces defined by contour, profile, trajectory, and guide curves are thus represented in concatenated matrix formulation. In addition, we give interactive methods to generate sweep objects and present sample figures produced within the framework of our implementation Tb, a topological picturebook. © 1992

Betti numbers for numerical semigroup rings

We survey results related to the magnitude of the Betti numbers of numerical semigroup rings and of their tangent cones.Comment: 22 pages; v2: updated references. To appear in Multigraded Algebra and Applications (V. Ene, E. Miller Eds.