Towards a Unified Knowledge-Based Approach to Modality Choice

This paper advances a unified knowledge-based approach to the process of choosing the most appropriate modality or combination of modalities in multimodal output generation. We propose a Modality Ontology (MO) that models the knowledge needed to support the two most fundamental processes determining modality choice – modality allocation (choosing the modality or set of modalities that can best support a particular type of information) and modality combination (selecting an optimal final combination of modalities). In the proposed ontology we model the main levels which collectively determine the characteristics of each modality and the specific relationships between different modalities that are important for multi-modal meaning making. This ontology aims to support the automatic selection of modalities and combinations of modalities that are suitable to convey the meaning of the intended message

fMRI Evidence for Modality-Specific Processing of Conceptual Knowledge on Six Modalities

Traditional theories assume that amodal representations, such as feature lists and semantic networks, represent conceptual knowledge about the world. According to this view, the sensory, motor, and introspective states that arise during perception and action are irrelevant to representing knowledge. Instead the conceptual system lies outside modality-specific systems and operates according to different principles. Increasingly, however, researchers report that modality-specific systems become active during purely conceptual tasks, suggesting that these systems play central roles in representing knowledge (for a review, see Martin, 2001, Handbook of Functional Neuroimaging of Cognition). In particular, researchers report that the visual system becomes active while processing visual properties, and that the motor system becomes active while processing action properties. The present study corroborates and extends these findings. During fMRI, subjects verified whether or not properties could potentially be true of concepts (e.g., BLENDER-loud). Subjects received only linguistic stimuli, and nothing was said about using imagery. Highly related false properties were used on false trials to block word association strategies (e.g., BUFFALOwinged). To assess the full extent of the modality-specific hypothesis, properties were verified on each of six modalities. Examples include GEMSTONE-glittering (vision), BLENDER-loud (audition), FAUCET-turned (motor), MARBLE-cool (touch), CUCUMBER-bland (taste), and SOAP-perfumed (smell). Neural activity during property verification was compared to a lexical decision baseline. For all six sets of the modalityspecific properties, significant activation was observed in the respective neural system. Finding modality-specific processing across six modalities contributes to the growing conclusion that knowledge is grounded in modality-specific systems of the brain

A Pragmatic Analysis of the Pedagogical Implications of the Use of English Epistemic Modality Written Literary Discourse

This is a pragmatic study of the use of the items of epistemic modality in a literary discourse with the main aims to identify, analyze and describe the ways the items of epistemic modality are used. Their contextual meanings, functions, and implication to the pedagogical attempts are also unfolded. The results of the interpretative and descriptive analysis reveal that the items of epistemic modality are found to be very dominant which also suggests that the genre of narrative fiction is linguistically characterized by the utterances that are established on the basis of knowledge and reasoning. The items of epistemic modality are found to be polysemous and polyfunctional which are reflected pragmatically in the forms of politeness, negotiative and constructive functions. All these lead to the acknowledgement that the use of the items of linguistic modality in literary discourse and their usage for language teaching in the applied linguistic contexts is worth conducting

Conceivability and Modal Knowledge

Christopher Hill contends that the metaphysical modalities can be reductively explained in terms of the subjunctive conditional and that this reductive explanation yields two tests for determining the metaphysical modality of a proposition. He goes on to argue that his reductive account of the metaphysical modalities in conjunction with his account of modal knowledge underwrites the further conclusion that conceivability does not provide a reliable test for metaphysical possibility. I argue (1) that Hill’s reductive explanation of the metaphysical modalities in terms of the subjunctive conditional does not yield a reductive explanation of knowledge of metaphysical modality in terms of knowledge of subjunctive conditionals, and (2) that his account of modal knowledge is at odds with his contention that conceivability does not provide epistemic access to metaphysical possibility

Topic-Sensitive Epistemic 2D Truthmaker ZFC and Absolute Decidability

This paper aims to contribute to the analysis of the nature of mathematical modality, and to the applications of the latter to unrestricted quantification and absolute decidability. Rather than countenancing the interpretational type of mathematical modality as a primitive, I argue that the interpretational type of mathematical modality is a species of epistemic modality. I argue, then, that the framework of two-dimensional semantics ought to be applied to the mathematical setting. The framework permits of a formally precise account of the priority and relation between epistemic mathematical modality and metaphysical mathematical modality. The discrepancy between the modal systems governing the parameters in the two-dimensional intensional setting provides an explanation of the difference between the metaphysical possibility of absolute decidability and our knowledge thereof. I also advance an epistemic two-dimensional truthmaker semantics, if hyperintenisonal approaches are to be preferred to possible worlds semantics. I examine the relation between epistemic truthmakers and epistemic set theory

Quantitative Photo-acoustic Tomography with Partial Data

Photo-acoustic tomography is a newly developed hybrid imaging modality that combines a high-resolution modality with a high-contrast modality. We analyze the reconstruction of diffusion and absorption parameters in an elliptic equation and improve an earlier result of Bal and Uhlmann to the partial date case. We show that the reconstruction can be uniquely determined by the knowledge of 4 internal data based on well-chosen partial boundary conditions. Stability of this reconstruction is ensured if a convexity condition is satisfied. Similar stability result is obtained without this geometric constraint if 4n well-chosen partial boundary conditions are available, where $n$ is the spatial dimension. The set of well-chosen boundary measurements is characterized by some complex geometric optics (CGO) solutions vanishing on a part of the boundary.Comment: arXiv admin note: text overlap with arXiv:0910.250

Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation

The success of deep convolutional neural networks is partially attributed to the massive amount of annotated training data. However, in practice, medical data annotations are usually expensive and time-consuming to be obtained. Considering multi-modality data with the same anatomic structures are widely available in clinic routine, in this paper, we aim to exploit the prior knowledge (e.g., shape priors) learned from one modality (aka., assistant modality) to improve the segmentation performance on another modality (aka., target modality) to make up annotation scarcity. To alleviate the learning difficulties caused by modality-specific appearance discrepancy, we first present an Image Alignment Module (IAM) to narrow the appearance gap between assistant and target modality data.We then propose a novel Mutual Knowledge Distillation (MKD) scheme to thoroughly exploit the modality-shared knowledge to facilitate the target-modality segmentation. To be specific, we formulate our framework as an integration of two individual segmentors. Each segmentor not only explicitly extracts one modality knowledge from corresponding annotations, but also implicitly explores another modality knowledge from its counterpart in mutual-guided manner. The ensemble of two segmentors would further integrate the knowledge from both modalities and generate reliable segmentation results on target modality. Experimental results on the public multi-class cardiac segmentation data, i.e., MMWHS 2017, show that our method achieves large improvements on CT segmentation by utilizing additional MRI data and outperforms other state-of-the-art multi-modality learning methods.Comment: Accepted by AAAI 202