Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis

This paper explores the grounding issue regarding multimodal semantic representation from a computational cognitive-linguistic view. We annotate images from the Flickr30k dataset with five perceptual properties: Affordance, Perceptual Salience, Object Number, Gaze Cueing, and Ecological Niche Association (ENA), and examine their association with textual elements in the image captions. Our findings reveal that images with Gibsonian affordance show a higher frequency of captions containing 'holding-verbs' and 'container-nouns' compared to images displaying telic affordance. Perceptual Salience, Object Number, and ENA are also associated with the choice of linguistic expressions. Our study demonstrates that comprehensive understanding of objects or events requires cognitive attention, semantic nuances in language, and integration across multiple modalities. We highlight the vital importance of situated meaning and affordance grounding in natural language understanding, with the potential to advance human-like interpretation in various scenarios.Comment: 10 pages, 9 figure

Lexical Retrieval Hypothesis in Multimodal Context

Multimodal corpora have become an essential language resource for language science and grounded natural language processing (NLP) systems due to the growing need to understand and interpret human communication across various channels. In this paper, we first present our efforts in building the first Multimodal Corpus for Languages in Taiwan (MultiMoco). Based on the corpus, we conduct a case study investigating the Lexical Retrieval Hypothesis (LRH), specifically examining whether the hand gestures co-occurring with speech constants facilitate lexical retrieval or serve other discourse functions. With detailed annotations on eight parliamentary interpellations in Taiwan Mandarin, we explore the co-occurrence between speech constants and non-verbal features (i.e., head movement, face movement, hand gesture, and function of hand gesture). Our findings suggest that while hand gestures do serve as facilitators for lexical retrieval in some cases, they also serve the purpose of information emphasis. This study highlights the potential of the MultiMoco Corpus to provide an important resource for in-depth analysis and further research in multimodal communication studies

31st Annual Meeting and Associated Programs of the Society for Immunotherapy of Cancer (SITC 2016) : part two

Background The immunological escape of tumors represents one of the main ob- stacles to the treatment of malignancies. The blockade of PD-1 or CTLA-4 receptors represented a milestone in the history of immunotherapy. However, immune checkpoint inhibitors seem to be effective in specific cohorts of patients. It has been proposed that their efficacy relies on the presence of an immunological response. Thus, we hypothesized that disruption of the PD-L1/PD-1 axis would synergize with our oncolytic vaccine platform PeptiCRAd. Methods We used murine B16OVA in vivo tumor models and flow cytometry analysis to investigate the immunological background. Results First, we found that high-burden B16OVA tumors were refractory to combination immunotherapy. However, with a more aggressive schedule, tumors with a lower burden were more susceptible to the combination of PeptiCRAd and PD-L1 blockade. The therapy signifi- cantly increased the median survival of mice (Fig. 7). Interestingly, the reduced growth of contralaterally injected B16F10 cells sug- gested the presence of a long lasting immunological memory also against non-targeted antigens. Concerning the functional state of tumor infiltrating lymphocytes (TILs), we found that all the immune therapies would enhance the percentage of activated (PD-1pos TIM- 3neg) T lymphocytes and reduce the amount of exhausted (PD-1pos TIM-3pos) cells compared to placebo. As expected, we found that PeptiCRAd monotherapy could increase the number of antigen spe- cific CD8+ T cells compared to other treatments. However, only the combination with PD-L1 blockade could significantly increase the ra- tio between activated and exhausted pentamer positive cells (p= 0.0058), suggesting that by disrupting the PD-1/PD-L1 axis we could decrease the amount of dysfunctional antigen specific T cells. We ob- served that the anatomical location deeply influenced the state of CD4+ and CD8+ T lymphocytes. In fact, TIM-3 expression was in- creased by 2 fold on TILs compared to splenic and lymphoid T cells. In the CD8+ compartment, the expression of PD-1 on the surface seemed to be restricted to the tumor micro-environment, while CD4 + T cells had a high expression of PD-1 also in lymphoid organs. Interestingly, we found that the levels of PD-1 were significantly higher on CD8+ T cells than on CD4+ T cells into the tumor micro- environment (p < 0.0001). Conclusions In conclusion, we demonstrated that the efficacy of immune check- point inhibitors might be strongly enhanced by their combination with cancer vaccines. PeptiCRAd was able to increase the number of antigen-specific T cells and PD-L1 blockade prevented their exhaus- tion, resulting in long-lasting immunological memory and increased median survival

♯Irony or ♯Sarcasm - A Quantitative and Qualitative Study Based on Twitter

Current study is with the aim to identify similarities and distinctions between irony and sarcasm by adopting quantitative sentiment analysis as well as qualitative content analysis. The result of quantitative sentiment analysis shows that sarcastic tweets are used with more positive tweets than ironic tweets. The result of content analysis corresponds to the result of quantitative sentiment analysis in identifying the aggressiveness of sarcasm. On the other hand, from content analysis it shows that irony owns two senses. The first sense of irony is equal to aggressive sarcasm with speaker awareness. Thus, tweets of first sense of irony may attack a specific target, and the speaker may tag his/her tweet irony because the tweet itself is ironic. These tweets though tagged as irony are in fact sarcastic tweets. Different from this, the tweets of second sense of irony is tagged to classify an event to be ironic. However, from the distribution in sentiment analysis and examples in content analysis, irony seems to be more broadly used in its second sense.

Secrets of Lexical Conventionalization: A Quantitative and Qualitative Exploratory Analysis on Linguistic Factors

前人的在語詞上的研究有許多見解，主要可分為兩部分：語言理論上的分析和語言處理的應用。理論上的分析主要包含三個角度：研究語言現象歷史發展的歷史語言學，新詞共時表現的詞彙語義學，預測詞彙存留的計算語言學。他們都可以運用於字典學，設計語言教材，構建自然語言處理所需的資源。然而，在相關研究中少有同時採用量化和質性角度的探討。其次，前人研究中所選取的目標詞彙有其侷限性。同時，時間訊息以及各類語言學變相都應納入討論以及更深刻的了解詞彙穩定的肇因。詞彙以概念連結的組構模式以及隨著時間積累的心理詞庫都應在探討本議題時納入考量。因此，本文欲以量化和質性觀點切入研究，提出詞彙可能有的三種生命形態（擴散、穩定、失去活性），透過時間資訊以及六種語言學面向（聲韻、構詞、語意、句法、語用、社會語言學）來探討本議題，並期能將結果運用於詞彙預測以及資源建構。 量化分析的角度來看，線性回歸模型用以研究區分不同時間點詞彙的語言學特色。語用學顯著地解釋了1950年以前存在的詞彙期使用穩定度的高低，而1950年以後所造的詞是否在語言中穩定使用則有賴語法面向的因素來解釋。這樣的結果暗示詞彙活得越久越與經驗性和語用性知識相關，但對於近期新生的詞彙句法結構的結合性對於其是否會被穩定使用有著決定性的意義。新起的擴散詞以及存在數世紀的詞彙在使用穩定度上十分相似，但藉由邏輯回歸模型可以發現數音節、近義詞數、同義詞數目、在回文中使用的活躍度、是否為外來語成功區別擴散詞以及存在數世紀的詞彙。另方面，語言學特質的角度而言1950年後新生的詞彙與近來新起的擴散詞有相似的語言學特徵。所以將1950年以後新生的詞作為訓練資料建構預測模型來理解現下擴散的詞未來發展的趨勢。結果顯示目標詞前後共現的不同詞彙數有顯著的預測能力，達到0.6335的準確度。 質性分析的面向從同義詞間的競爭來探討，句法上的兼容性和該詞概念關係的豐富度應為是否能贏過其他同義詞而被大量使用的關鍵。此外，不同時間點生成的詞在貼文與回文中有不同的使用活性。不同於其他兩者擴散詞在回文中較為活躍，這暗示他們在類似回覆導向的口語風格中以及互動中較易擴散。根據這些研究發現，我們可以進一步應用於增補詞彙於語言資源中。語用上的穩定度、語法上的結合性，以及語意可作為增補詞彙的標準，較廣泛使用的異體詞，語意表達中較穩定使用的詞彙，以及來自同一概念經歷詞彙化的詞項皆收錄於增補後的詞，由此可知所提標準的涵蓋性。Previous studies have many insights in understanding lexical items. They can be generally captured into two parts: linguistic analysis and application. Linguistic analysis mainly includes three angles: studies on historical development of linguistic phenomenon from Historical Linguistics, probes on synchronic emergence of neologisms from Lexical Semantics, and prediction models built for understanding survival of words from Computational Linguistics. They can all be applied on including words for Lexicology, designing language teaching materials, and constructing resources for Natural Language Processing. However, there is rarely a single work include quantitative and qualitative methods simultaneously. Second, the generality of included target words in previous studies needs reconsideration. Meanwhile, temporal information of lexical items and various linguistic aspects should be invited to probe deeper for understanding factors contributing to conventionalization of a word. The conceptual associations of organization in mental lexicon and temporal accumulation for mental lexicon should all be considered when facing this issue. Thus, this thesis is aimed to conduct quantitative profiling and qualitative analysis as well as to apply them in constructing lexical resources with proposing three life stages of lexical items (diffusion, conventionalization, and inactivation), including target words from different temporal points, and adopting linguistic variables from six linguistic aspects (phonology, morphology, semantics, syntax, pragmatics, and sociolinguistics). In quantitative profiling, the linear regression model has built to distinguish words from different temporal points. The result shows that pragmatics can best account behavioral performance of words before 1950 and syntax can best capture words after 1950, which implies that words live longer may correlated with rich experiential and pragmatic using knowledge, but for those who are born recently their structurally syntactic compatibility plays important role in deciding their fluctuation in use. Diffused words are similar to words existing over centuries in their Revised Constant U. From logistic regression model it is found that number of syllable, number of near-synonym, number of synonym, activeness in used in comments, and borrowing from other language or not are statistically significant variables that distinguish diffused words and words existing over centuries. On the other hand, words born after 1950 and diffused words are quite similar in their linguistic characteristics. Prediction model based on training data from words after 1950 are built to foretell potential life of diffused words. It shows that number of types co-occurring before target words is statistically valued in prediction. With words before 1950 and recent diffused words as test data the accuracy of model reaches 0.6335. Qualitative analysis on competitions among words from the same synset indicates that structural compatibility and involved conceptual relations may be the key for one lexical item to winning over the other synonymous member. Besides, words coming from different temporal points show differences in their activeness in being used in comments and posts on PTT. Diffused words are more active in comments, which implies they are more correlated with feedback oriented oral style and diffused in interaction. With these findings we can further apply them on proposing suggestions for lexicology. Pragmatically stable in use, syntactic compatibility, and semantically number of senses are taken as standard to expanding inclusion of words. The updated inclusion of popularly used variants, more stable semantic representation, and words lexicalized from the same conceptual experiences indicates the inclusiveness of proposed standards

Guidelines for the use and interpretation of assays for monitoring autophagy (4th edition)

In 2008, we published the first set of guidelines for standardizing research in autophagy. Since then, this topic has received increasing attention, and many scientists have entered the field. Our knowledge base and relevant new technologies have also been expanding. Thus, it is important to formulate on a regular basis updated guidelines for monitoring autophagy in different organisms. Despite numerous reviews, there continues to be confusion regarding acceptable methods to evaluate autophagy, especially in multicellular eukaryotes. Here, we present a set of guidelines for investigators to select and interpret methods to examine autophagy and related processes, and for reviewers to provide realistic and reasonable critiques of reports that are focused on these processes. These guidelines are not meant to be a dogmatic set of rules, because the appropriateness of any assay largely depends on the question being asked and the system being used. Moreover, no individual assay is perfect for every situation, calling for the use of multiple techniques to properly monitor autophagy in each experimental setting. Finally, several core components of the autophagy machinery have been implicated in distinct autophagic processes (canonical and noncanonical autophagy), implying that genetic approaches to block autophagy should rely on targeting two or more autophagy-related genes that ideally participate in distinct steps of the pathway. Along similar lines, because multiple proteins involved in autophagy also regulate other cellular pathways including apoptosis, not all of them can be used as a specific marker for bona fide autophagic responses. Here, we critically discuss current methods of assessing autophagy and the information they can, or cannot, provide. Our ultimate goal is to encourage intellectual and technical innovation in the field