Distance Metric Learning Loss Functions in Few-Shot Scenarios of Supervised Language Models Fine-Tuning

This paper presents an analysis regarding an influence of the Distance Metric Learning (DML) loss functions on the supervised fine-tuning of the language models for classification tasks. We experimented with known datasets from SentEval Transfer Tasks. Our experiments show that applying the DML loss function can increase performance on downstream classification tasks of RoBERTa-large models in few-shot scenarios. Models fine-tuned with the use of SoftTriple loss can achieve better results than models with a standard categorical cross-entropy loss function by about 2.89 percentage points from 0.04 to 13.48 percentage points depending on the training dataset. Additionally, we accomplished a comprehensive analysis with explainability techniques to assess the models' reliability and explain their results

Revisiting Distance Metric Learning for Few-Shot Natural Language Classification

Distance Metric Learning (DML) has attracted much attention in image processing in recent years. This paper analyzes its impact on supervised fine-tuning language models for Natural Language Processing (NLP) classification tasks under few-shot learning settings. We investigated several DML loss functions in training RoBERTa language models on known SentEval Transfer Tasks datasets. We also analyzed the possibility of using proxy-based DML losses during model inference. Our systematic experiments have shown that under few-shot learning settings, particularly proxy-based DML losses can positively affect the fine-tuning and inference of a supervised language model. Models tuned with a combination of CCE (categorical cross-entropy loss) and ProxyAnchor Loss have, on average, the best performance and outperform models with only CCE by about 3.27 percentage points -- up to 10.38 percentage points depending on the training dataset

Knowledge centers as an innovative knowledge transfer mechanism : lesson learned from the program implemented in Lesser Poland

The aim of the paper is to present the lessons learnt from the "SPIN" regional public project. The project was implemented in the region of Lesser Poland. The objective of the project was to increase the intensity of knowledge transfer from universities to enterprises. The goal was achieved by establishing four Centres for Knowledge Transfer at major universities. Each of them was dedicated to a specific domain of knowledge - regional smart specialization - biotechnology, translational medicine, smart grids and energy-saving buildings. The paper discusses the implementation and effects of the project. The most important conclusions stemming from the project concern the fact that the context of the implementation needs to be taken into account during the project as well as the importance of leadership. More attention should also be devoted to the motivation and skills of those involved in the implementation

Entity Graph Extraction from Legal Acts -- a Prototype for a Use Case in Policy Design Analysis

This paper presents research on a prototype developed to serve the quantitative study of public policy design. This sub-discipline of political science focuses on identifying actors, relations between them, and tools at their disposal in health, environmental, economic, and other policies. Our system aims to automate the process of gathering legal documents, annotating them with Institutional Grammar, and using hypergraphs to analyse inter-relations between crucial entities. Our system is tested against the UNESCO Convention for the Safeguarding of the Intangible Cultural Heritage from 2003, a legal document regulating essential aspects of international relations securing cultural heritage.Comment: 17 pages, 10 figure

Kinetic and thermodynamic characterization of the interactions between the components of human plasma kinin-forming system and isolated and purified cell wall proteins of Candida albicans

Cell wall proteins of Candida albicans, besides their best known role in the adhesion of this fungal pathogen to host's tissues, also bind some soluble proteins, present in body fluids and involved in maintaining the biochemical homeostasis of the human organism. In particular, three plasma factors - high-molecular-mass kininogen (HK), factor XII (FXII) and prekallikrein (PPK) - have been shown to adhere to candidal cells. These proteins are involved in the surface-contact-catalyzed production of bradykinin-related peptides (kinins) that contribute to inflammatory states associated with microbial infections. We recently identified several proteins, associated with the candidal cell walls, and probably involved in the binding of HK. In our present study, a list of potential FXII- and PPK-binding proteins was proposed, using an affinity selection (on agarose-coupled FXII or PPK) from a whole mixture of β-1,3-glucanase-extrated cell wall-associated proteins and the mass-spectrometry protein identification. Five of these fungal proteins, including agglutinin-like sequence protein 3 (Als3), triosephosphate isomerase 1 (Tpi1), enolase 1 (Eno1), phosphoglycerate mutase 1 (Gpm1) and glucose-6-phosphate isomerase 1 (Gpi1), were purified and characterized in terms of affinities to the human contact factors, using the surface plasmon resonance measurements. Except Gpm1 that bound only PPK, and Als3 that exhibited an affinity to HK and FXII, the other isolated proteins interacted with all three contact factors. The determined dissociation constants for the identified protein complexes were of 10-7 M order, and the association rate constants were in a range of 104-105 M-1s-1. The identified fungal pathogen-host protein interactions are potential targets for novel anticandidal therapeutic approaches

Aspartic proteases and major cell wall components in Candida albicans trigger the release of neutrophil extracellular traps

Neutrophils use different mechanisms to cope with pathogens that invade the host organism. The most intriguing of these responses is a release of neutrophil extracellular traps (NETs) composed of decondensed chromatin and granular proteins with antimicrobial activity. An important potential target of NETs is Candida albicans—an opportunistic fungal pathogen that employs morphological and phenotype switches and biofilm formation during contact with neutrophils, accompanied by changes in epitope exposition that mask the pathogen from host recognition. These processes differ depending on infection conditions and are thus influenced by the surrounding environment. In the current study, we compared the NET release by neutrophils upon contact with purified main candidal cell surface components. We show here for the first time that in addition to the main cell wall-building polysaccharides (mannans and β-glucans), secreted aspartic proteases (Saps) trigger NETs with variable intensities. The most efficient NET-releasing response is with Sap4 and Sap6, which are known to be secreted by fungal hyphae. This involves mixed, ROS-dependent and ROS-independent signaling pathways, mainly through interactions with the CD11b receptor. In comparison, upon contact with the cell wall-bound Sap9 and Sap10, neutrophils responded via a ROS-dependent mechanism using CD16 and CD18 receptors for protease recognition. In addition to the Saps tested, the actuation of selected mediating kinases (Src, Syk, PI3K, and ERK) was also investigated. β-Glucans were found to trigger a ROS-dependent process of NET production with engagement of Dectin-1 as well as CD11b and CD18 receptors. Mannans were observed to be recognized by TLRs, CD14, and Dectin-1 receptors and triggered NET release mainly via a ROS-independent pathway. Our results thus strongly suggest that neutrophils activate NET production in response to different candidal components that are presented locally at low concentrations at the initial stages of infection. However, NET release seemed to be blocked by increasing numbers of fungal cells

Marketing

Praca recenzowana / peer-reviewed pape

Identyfication of Candida albicans surface proteins involved in the interactions with high molecular weight kininogen.

It is well known that some bacterial surface proteins can bind kininogen and other proteins of the contact system, what leads to the activation of the kallikrein / kinin system. Activation of the contact system with the participation of microorganisms can occur in different ways. Indirectly by the proteolytic activation of factor XII or prekallikrein, involving bacterial proteinases or direct by release of kinin from kininogen. This phenomenon can also occur on the cell surface of pathogenic fungi such as Candida spp, which may also contribute to the kinin production. Especially the possibility of binding of HK by C. albicans cells plays an important role in the activation of kalikrein/ kinin system and development of inflammation, sepsis and septic shock. The crucial role in such interaction is played by receptor proteins on the cell surface of the pathogen. The aim of this study was the purification of the cell wall proteins (CWP) of Candida albicans, and then evaluating their participation in interactions with high molecular weight kininogen. Prefractionation of CWP carried out in three ways: salting out, affinity chromatography on immobilized lectins (concanavalin A) and ion exchange chromatography using anion. It was shown that the strongest binding to HK exhibit proteins with molecular mass of 47 kDa and 60 kDa. Analysis with mass spectrometry revealed that the protein with a molecular mass of 47 kDa is an enolase. In the interactions with HK other proteins with molecular weight such as: 30 kDa, 50kDa, 40 kDa and 92 kDa may also be involved. The last one probably belong to the ALS family.Powszechnie wiadome jest, iż niektóre bakteryjne białka powierzchniowe mają zdolność wiązania kininogenów i pozostałych białek układu kontaktu, co w efekcie prowadzi do aktywacji układu kalikreina/kininy. Aktywacja układu kontaktu z udziałem mikroorganizmów może zachodzić na różne sposoby. Może to być proteolityczna aktywacja czynnika XII czy prekalikreiny, z udziałem proteinaz bakteryjnych oraz bezpośrednie wycinanie kinin z kininogenu. Zjawisko to może również występować na powierzchni komórek grzybów chorobotwórczych, takich jak Candida spp., które również mogą przyczyniać się do aktywacji produkcji kinin. Szczególnie możliwość wiązania cząsteczki HK przez komórki C. albicans odgrywa ważną rolę w aktywacja układu kalikreina/kininy i rozwoju stanu zapalnego, sepsy oraz wstrząsu septycznego. W oddziaływaniu tym główna rolę odgrywają białka receptorowe na powierzchni komórki patogenu. Celem niniejszej pracy było oczyszczenie białek ściany komórkowej Candida albicans, a następnie oszacowanie ich udziału w oddziaływaniu z kininogenem wysokocząsteczkowym. Podjęto trzy drogi wstępnego frakcjonowania CWP: wysalanie, chromatografię powinowactwa na złożu z unieruchomioną lektyn konkanawaliną A (ConA) oraz chromatografię jonowymienną z użyciem anionitu. Wykazano, że najsilniejsze wiązanie do HK wykazują białka o masach cząsteczkowych odpowiednio: 47 kDa oraz 60 kDa. Metodą spektrometrii masowej wykazano, iż białkiem o masie cząsteczkowej 47 kDa jest enolaza. W oddziaływaniu z HK mogą brać udział także białka o masach cząsteczkowych: 30 kDa, 50 kDa, 40 kDa oraz 92 kDa należące do rodziny białek Als

Polish Natural Language Inference and Factivity -- an Expert-based Dataset and Benchmarks

Despite recent breakthroughs in Machine Learning for Natural Language Processing, the Natural Language Inference (NLI) problems still constitute a challenge. To this purpose we contribute a new dataset that focuses exclusively on the factivity phenomenon; however, our task remains the same as other NLI tasks, i.e. prediction of entailment, contradiction or neutral (ECN). The dataset contains entirely natural language utterances in Polish and gathers 2,432 verb-complement pairs and 309 unique verbs. The dataset is based on the National Corpus of Polish (NKJP) and is a representative sample in regards to frequency of main verbs and other linguistic features (e.g. occurrence of internal negation). We found that transformer BERT-based models working on sentences obtained relatively good results ($\approx89\%$ F1 score). Even though better results were achieved using linguistic features ($\approx91\%$ F1 score), this model requires more human labour (humans in the loop) because features were prepared manually by expert linguists. BERT-based models consuming only the input sentences show that they capture most of the complexity of NLI/factivity. Complex cases in the phenomenon - e.g. cases with entitlement (E) and non-factive verbs - remain an open issue for further research