Exploring Neural Language Models via Analysis of Local and Global Self-Attention Spaces

Large pretrained language models using the transformer neural network architecture are becoming a dominant methodology for many natural language processing tasks, such as question answering, text classification, word sense disambiguation, text completion and machine translation. Commonly comprising hundreds of millions of parameters, these models offer state-of-the-art performance, but at the expense of interpretability. The attention mechanism is the main component of transformer networks. We present AttViz, a method for exploration of self-attention in transformer networks, which can help in explanation and debugging of the trained models by showing associations between text tokens in an input sequence. We show that existing deep learning pipelines can be explored with AttViz, which offers novel visualizations of the attention heads and their aggregations. We implemented the proposed methods in an online toolkit and an offline library. Using examples from news analysis, we demonstrate how AttViz can be used to inspect and potentially better understand what a model has learned

Traveling salesman problem on the map of Slovenia

The traveling salesman problem is a well known NP-hard problem. Given a list of cities and the distances between each pair of cities, the goal is to find the shortest possible route that visits each city and returns to the origin city. In the thesis we found the solution to the traveling salesman problem on 6007 settlements in Slovenia using the geographical distance between the settlements. To find the solution we used programs LKH and Concorde. With LKH we obtained the upper bound of traveling salesman tour. Than we used Concorde to obtain and improve the lower bound of the traveling salesman tour. We have found the tour of length 7733,125km and have shown that it is optimal

Vpliv nizkotemperaturne plazme na privzem vode, viabilnost in kaljivost paradižnika

Z raziskavo smo želeli ugotoviti ali obdelava semen paradižnika, (Solanum lycopersicum) z nizkotemperaturno kisikovo plazmo vpliva na viabilnost, kaljivost in privzem vode semen ter na dinamiko rasti korenin. V ta namen smo semena paradižnika obdelali z nizkotemperaturno plazmo (tlak 50 Pa in delovna moč cca. 1300 W). Parametri so bili ves čas obdelave enaki, le čas izpostavljenosti semen se je razlikoval (0, 5 s, 10 s, 30 s, 60 s). Nato smo z različnimi testi preverjali viabilnost in kaljivost semen, privzem vode v seme ter vpliv na dinamiko rasti korenin. Rezultati kažejo negativno povezavo med dolžino izpostavitve hladni plazmi ter viabilnostjo in kaljivostjo. Pri spremljanju dinamike privzema vode in rasti korenin v določenem časovnem obdobju je pri semenih, obdelanih 5 s s hladno plazmo, opazna nekoliko boljša dinamika rasti korenin in privzema vode semen glede na kontrolo

BoBER: web interface to the base of bioisosterically exchangeable replacements

Abstract We describe a novel freely available web server Base of Bioisosterically Exchangeable Replacements (BoBER), which implements an interface to a database of bioisosteric and scaffold hopping replacements. Bioisosterism and scaffold hopping are key concepts in drug design and optimization, and can be defined as replacements of biologically active compound’s fragments with other fragments to improve activity, reduce toxicity, change bioavailability or to diversify the scaffold space. Our web server enables fast and user-friendly searches for bioisosteric and scaffold replacements which were obtained by mining the whole Protein Data Bank. The working of the web server is presented on an existing MurF inhibitor as example. BoBER web server enables medicinal chemists to quickly search for and get new and unique ideas about possible bioisosteric or scaffold hopping replacements that could be used to improve hit or lead drug-like compounds

Patch testing with the European baseline series and 10 added allergens : single centre study of 748 patients

Background. The European baseline series (EBS) of contact allergens is subject to change. An allergen is considered for inclusion when routine patch testing of patients with suspected contact dermatitis results in ≥ 0.5% prevalence rate. Objectives. We aimed to determine the frequency of sensitizations to 30 EBS allergens and 10 locally added allergens. Additionally, we assessed the strength and evolution of reactions to all tested allergens and co-reactivity of additional allergens. Methods. Patch testing with our baseline series of 40 allergens was done in 748 consecutive adults. Tests were applied to the upper back and removed by patients after 48 hours. Readings were done on day 3 (D3) and D6 or D7 (D6/7). Positive reactions fulfilled the criteria of at least one plus (+) reaction. Retrospective analysis was done. Results. Eight allergens not listed in the EBS had ≥ 0.5% prevalence rate (i.e., cocamidopropyl betaine, thiomersal, disperse blue mix 106/124, 2-bromo-2-nitropropane-1,3-diol, diazolidinyl urea, propylene glycol, Compositae mix II, and dexamethasone-21-phosphate), and 16.6% of positive reactions would have been missed without D6/7 readings. Conclusion. We propose further studies to evaluate whether cocamidopropyl betaine, disperse blue mix 106/124, 2-bromo-2-nitropropane-1,3-diol, diazolidinyl urea, and Compositae mix II need to be added to the EBS