Semantic representation in the white matter pathway

<div><p>Object conceptual processing has been localized to distributed cortical regions that represent specific attributes. A challenging question is how object semantic space is formed. We tested a novel framework of representing semantic space in the pattern of white matter (WM) connections by extending the representational similarity analysis (RSA) to structural lesion pattern and behavioral data in 80 brain-damaged patients. For each WM connection, a neural representational dissimilarity matrix (RDM) was computed by first building machine-learning models with the voxel-wise WM lesion patterns as features to predict naming performance of a particular item and then computing the correlation between the predicted naming score and the actual naming score of another item in the testing patients. This correlation was used to build the neural RDM based on the assumption that if the connection pattern contains certain aspects of information shared by the naming processes of these two items, models trained with one item should also predict naming accuracy of the other. Correlating the neural RDM with various cognitive RDMs revealed that neural patterns in several WM connections that connect left occipital/middle temporal regions and anterior temporal regions associated with the object semantic space. Such associations were not attributable to modality-specific attributes (shape, manipulation, color, and motion), to peripheral picture-naming processes (picture visual similarity, phonological similarity), to broad semantic categories, or to the properties of the cortical regions that they connected, which tended to represent multiple modality-specific attributes. That is, the semantic space could be represented through WM connection patterns across cortical regions representing modality-specific attributes.</p></div

The RSA results of the WM connections showing significant effects of higher-order semantic space.

<p>For each connection, results (r(<i>p</i>)) are shown for the higher-order semantic space, raw semantic space, and broad object category, before or after controlling for various types of stimulus properties and in various subsets of patients. R values are the Spearman <i>r</i> between the neural RDM in the corresponding connection and the semantic RDM with various other properties controlled for.</p

WM connections representing higher-order semantic space.

<p>(A) Eight WM connections representing higher-order semantic space, with 11 GM regions being connected. The regions that fail to show successful within-item prediction or in right hemisphere are rendered gray. The four colored regions represent raw semantic effects or modality-specific attributes (red for manipulation, shape, and semantic; orange for manipulation and shape; and purple for shape and color). (B) The WM connections reconstructed using the HCP database. The blue streamlines are the WM connections between two GM regions (rendered in red and green). The masks of WM connections reconstructed with current data are shown in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2003993#pbio.2003993.s002" target="_blank">S2 Fig</a>. The RSA results of the eight WM connections, with bars showing the correlation strength (<i>r</i> value) between neural and semantic RDMs and error bars indicating ±1 standard error based on 1,000 times bootstrap resampling (see [<a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2003993#pbio.2003993.ref023" target="_blank">23</a>] for details) of the neural and behavioral RDM sets. The three WM connections did not survive all validation tests were shown in the dashed box. (C) The GM nodes representing semantic and modality-specific knowledge. The bar figure shows the RSA correlation strength (<i>r</i> value) of the semantic and modality-specific attributes in the colored GM regions in (A); the error bars indicate ±1 standard error; only positive values are shown. Note that for the superior ATL, in which RSA with semantic space was significant, its semantic effects diminished when controlling for modality-specific attribute RDMs. Asterisks indicate FDR <i>q</i> < 0.05. The object line drawings were done by first author Y.F.; the brain illustrations were generated using BrainNet Viewer [<a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2003993#pbio.2003993.ref028" target="_blank">28</a>] and DSI Studio (<a href="http://dsi-studio.labsolver.org/" target="_blank">http://dsi-studio.labsolver.org/</a>). The underlying data can be found at <a href="https://osf.io/h7upk/?view_only=52b8f86cffa14ed4844e4a1b9cd429cb" target="_blank">https://osf.io/h7upk/?view_only=52b8f86cffa14ed4844e4a1b9cd429cb</a>. ATL, anterior temporal lobe; CAL, calcarine sulcus; FDR, false discovery rate; GM, gray matter; HCP, Human Connectome Project; LING, lingual gyrus; midATL, middle anterior temporal lobe; MOG, middle occipital gyrus; MTG, middle temporal gyrus; PHG, parahippocampal gyrus; PoCG, postcentral gyrus; RDM, representational dissimilarity matrix; RSA, representational similarity analysis; STG, superior temporal gyrus; supATL, superior anterior temporal lobe; WM, white matter.</p

The construction and result of behavioral RDMs.

<p>(A) The multi-arrangement method. Twenty college students were asked to arrange object pictures according to their semantic (or modality-specific attribute) relatedness by dragging the items on a screen with a mouse. The distances between items on the screen would transform into an RDM. If two items, e.g., scissors and axe, showed a close distance, then they were assigned a low value in the RDM. (B) The results of the behavioral RDMs. Three broad types of distances were measured: semantic similarity, modality-specific attributes (shape, manipulation, color, and motion), and control models that are also potentially relevant to object naming (early visual, phonological, and object category matrix). The values of dissimilarity were transformed to percentile for display. Red indicates low dissimilarity (high similarity) and blue high dissimilarity (low similarity). (C) Visualization of the semantic RDM using multidimensional scaling. (D) The correlations among various behavioral RDMs. The object line drawings were done by the first author Y.F. The underlying data for this figure can be found at <a href="https://osf.io/h7upk/?view_only=52b8f86cffa14ed4844e4a1b9cd429cb" target="_blank">https://osf.io/h7upk/?view_only=52b8f86cffa14ed4844e4a1b9cd429cb</a>. F&V, fruit and vegetable; MDS, multidimensional scaling; RDM, representational dissimilarity matrix.</p

A flowchart for constructing a neural RDM in a WM connection.

<p>(A) The neuropsychological test. We asked patients to complete a picture-naming task containing 100 items. The response for each item was scored as 1 if correct or 0 if wrong. (B) The lesion mask (manually traced in T1 image) in a given patient was converted to MNI space and was then overlapped with the WM connection template constructed from a healthy population [<a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2003993#pbio.2003993.ref027" target="_blank">27</a>] to extract the voxel-wise lesion pattern on each WM connection. (C) The SVM classifier was trained on the naming accuracy of one item <i>i</i> (e.g., scissors) and lesion patterns on a WM connection in some patients (see <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2003993#sec011" target="_blank">Materials and methods</a>) and then used to generate the predicted naming score in the testing patients (1 or 0). The correspondence (simple matching coefficient) between the predicted score and the actual naming score of each of the other items (e.g., axe) across patients was calculated. This correlation reflects to what degree the lesion features that were useful to predict naming accuracy of item <i>i</i> could also be useful to predict item <i>j</i>, and thus was taken as the neural similarity between the naming process of the training item <i>i</i> and this other item <i>j</i> (scissors–axe similarity) on this connection. All cross-item and within-item similarity could be obtained this way, resulting in a 100 × 100 similarity matrix for this connection. (D) A sample neural RDM of a WM connection (between MTG and superior ATL). The values of dissimilarity were 1-similarity (obtained in C); red indicates low dissimilarity (high similarity) and blue high dissimilarity (low similarity). The object line drawings were done by the first author Y.F.; The brain figure was generated using BrainNet Viewer [<a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2003993#pbio.2003993.ref028" target="_blank">28</a>]. ATL, anterior temporal lobe; MTG, middle temporal gyrus; RDM, representational dissimilarity matrix; SVM, support vector machine; WM, white matter.</p