Using segmented objects in ostensive video shot retrieval

This paper presents a system for video shot retrieval in which shots are retrieved based on matching video objects using a combination of colour, shape and texture. Rather than matching on individual objects, our system supports sets of query objects which in total reflect the user’s object-based information need. Our work also adapts to a shifting user information need by initiating the partitioning of a user’s search into two or more distinct search threads, which can be followed by the user in sequence. This is an automatic process which maps neatly to the ostensive model for information retrieval in that it allows a user to place a virtual checkpoint on their search, explore one thread or aspect of their information need and then return to that checkpoint to then explore an alternative thread. Our system is fully functional and operational and in this paper we illustrate several design decisions we have made in building it

Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI

<p>Abstract</p> <p>Background</p> <p>Near-infrared spectroscopy (NIRS) is an optical imaging technology that has recently been investigated for use in a safe, non-invasive brain-computer interface (BCI) for individuals with severe motor impairments. To date, most NIRS-BCI studies have attempted to discriminate two mental states (e.g., a mental task and rest), which could potentially lead to a two-choice BCI system. In this study, we attempted to automatically differentiate three mental states - specifically, intentional activity due to 1) a mental arithmetic (MA) task and 2) a mental singing (MS) task, and 3) an unconstrained, "no-control (NC)" state - to investigate the feasibility of a three-choice system-paced NIRS-BCI.</p> <p>Results</p> <p>Deploying a dual-wavelength frequency domain near-infrared spectrometer, we interrogated nine sites around the frontopolar locations while 7 able-bodied adults performed mental arithmetic and mental singing to answer multiple-choice questions within a system-paced paradigm. With a linear classifier trained on a ten-dimensional feature set, an overall classification accuracy of 56.2% was achieved for the MA vs. MS vs. NC classification problem and all individual participant accuracies significantly exceeded chance (i.e., 33%). However, as anticipated based on results of previous work, the three-class discrimination was unsuccessful for three participants due to the ineffectiveness of the mental singing task. Excluding these three participants increases the accuracy rate to 62.5%. Even without training, three of the remaining four participants achieved accuracies approaching 70%, the value often cited as being necessary for effective BCI communication.</p> <p>Conclusions</p> <p>These results are encouraging and demonstrate the potential of a three-state system-paced NIRS-BCI with two intentional control states corresponding to mental arithmetic and mental singing.</p

Harmonizing two measures of adaptive functioning using computational approaches: prediction of vineland adaptive behavior scales II (VABS-II) from the adaptive behavior assessment system II (ABAS-II) scores

Background: Very large sample sizes are often needed to capture heterogeneity in autism, necessitating data sharing across multiple studies with diverse assessment instruments. In these cases, data harmonization can be a critical tool for deriving a single dataset for analysis. This can be done through computational approaches that enable the conversion of scores across various instruments. To this end, our study examined the use of analytical approaches for mapping scores on two measures of adaptive functioning, namely predicting the scores on the vineland adaptive behavior scales II (VABS) from the scores on the adaptive behavior assessment system II (ABAS). Methods: Data from the province of Ontario neurodevelopmental disorders network were used. The dataset included scores VABS and the ABAS for 720 participants (autism n = 547, 433 male, age: 11.31 ± 3.63 years; neurotypical n = 173, 95 male, age: 12.53 ± 4.05 years). Six regression approaches (ordinary least squares (OLS) linear regression, ridge regression, ElasticNet, LASSO, AdaBoost, random forest) were used to predict VABS total scores from the ABAS scores, demographic variables (age, sex), and phenotypic measures (diagnosis; core and co-occurring features; IQ; internalizing and externalizing symptoms). Results: The VABS scores were significantly higher than the ABAS scores in the autism group, but not the neurotypical group (median difference: 8, 95% CI = (7,9)). The difference was negatively associated with age (beta = -1.2 ± 0.12, t = -10.6, p < 0.0001). All estimators demonstrated similar performance, with no statistically significant differences in mean absolute error (MAE) values across estimators (MAE range: 4.96–6.91). The highest contributing features to the prediction model were ABAS composite score, diagnosis, and age. Limitations: This study has several strengths, including the large sample. We did not examine the conversion of domain scores across the two measures of adaptive functioning and suggest this as a future area of investigation. Conclusion: Overall, our results supported the feasibility of harmonization. Our results suggest that a linear regression model trained on the ABAS composite score, the ABAS raw domain scores, and age, sex, and diagnosis would provide an acceptable trade-off between accuracy, parsimony, and data collection and processing complexity

Dataset factors associated with age‐related changes in brain structure and function in neurodevelopmental conditions

With brain structure and function undergoing complex changes throughout childhood and adolescence, age is a critical consideration in neuroimaging studies, particularly for those of individuals with neurodevelopmental conditions. However, despite the increasing use of large, consortium‐based datasets to examine brain structure and function in neurotypical and neurodivergent populations, it is unclear whether age‐related changes are consistent between datasets and whether inconsistencies related to differences in sample characteristics, such as demographics and phenotypic features, exist. To address this, we built models of age‐related changes of brain structure (regional cortical thickness and regional surface area; N = 1218) and function (resting‐state functional connectivity strength; N = 1254) in two neurodiverse datasets: the Province of Ontario Neurodevelopmental Network and the Healthy Brain Network. We examined whether deviations from these models differed between the datasets, and explored whether these deviations were associated with demographic and clinical variables. We found significant differences between the two datasets for measures of cortical surface area and functional connectivity strength throughout the brain. For regional measures of cortical surface area, the patterns of differences were associated with race/ethnicity, while for functional connectivity strength, positive associations were observed with head motion. Our findings highlight that patterns of age‐related changes in the brain may be influenced by demographic and phenotypic characteristics, and thus future studies should consider these when examining or controlling for age effects in analyses

Using deep learning to predict internalizing problems from brain structure in youth

Internalizing problems (e.g., anxiety and depression) are associated with a wide range of adverse outcomes. While some predictors of internalizing problems are known (e.g., their frequent co-occurrence with neurodevelopmental (ND) conditions), the biological markers of internalizing problems are not well understood. Here, we used deep learning, a powerful tool for identifying complex and multi-dimensional brain-behaviour relationships, to predict cross-sectional and worsening longitudinal trajectories of internalizing problems. Data were extracted from four large-scale datasets: the Adolescent Brain Cognitive Development study, the Healthy Brain Network, the Human Connectome Project Development study, and the Province of Ontario Neurodevelopmental network. We developed deep learning models that used measures of brain structure (thickness, surface area, and volume) to (a) predict clinically significant internalizing problems cross-sectionally (N = 14,523); and (b) predict subsequent worsening trajectories (using the reliable change index) of internalizing problems (N = 10,540) longitudinally. A stratified cross-validation scheme was used to tune, train, and test the models, which were evaluated using the area under the receiving operating characteristic curve (AUC). The cross-sectional model performed well across the sample, reaching an AUC of 0.80 [95% CI: 0.71, 0.88]. For the longitudinal model, while performance was sub-optimal for predicting worsening trajectories in a sample of the general population (AUC = 0.66 [0.65, 0.67]), good performance was achieved in a small, external test set of primarily ND conditions (AUC = 0.80 [0.78, 0.81]), as well as across all ND conditions (AUC = 0.73 [0.70, 0.76]). Deep learning with features of brain structure is a promising avenue for biomarkers of internalizing problems, particularly for individuals who have a higher likelihood of experiencing difficulties

Characterizing replicability in the clustering structure of brain morphology in autism, attention-deficit/hyperactivity disorder, and obsessive compulsive disorder

In neurodevelopmental research, within-diagnosis heterogeneity and across-diagnosis overlap necessitate a shift from case-control designs to data-driven clustering approaches. However, our understanding of the replicability of these clustering structures across independent datasets remains limited. Our objective was to examine the replicability of clustering structure in measures of brain morphology in neurodiverse children across two independent datasets, namely the Province of Ontario Neurodevelopmental Disorder (POND) Network and the Healthy Brain Network (HBN). POND and HBN data were collected across various institutions in Ontario, Canada, and New York, United States, respectively. Participants were 5-19 years old and had diagnoses of autism, attention deficit/hyperactivity disorder (ADHD), obsessive compulsive disorder (OCD), or were neurotypical. We used measures of cortical volume, surface area, cortical thickness, and subgroup volume from structural MRI data. Principal component analysis (PCA) and clustering were used to examine the replicability of clustering structures across the datasets. Correlations among principle components, measures of clusterability, and alignment between the four brain measures as well as male/female subsets were examined. Brain-behaviour associations were examined using univariate and multivariate approaches. The POND dataset included 747 participants with (autism n = 312, ADHD n = 220, OCD n = 70, neurotypical n = 145). The HBN dataset included 582 participants (autism n = 60, ADHD n = 445, OCD n = 19, neurotypical n = 58). Our results showed significant between-dataset correlations in 82.1% of the principal components derived from brain measures. A two-cluster structure was replicated across datasets, brain measures, and the female/male subsets, however the participant composition of clusters were only aligned between cortical volume and surface area, and cortical thickness and subcortical volume. Regional effect sizes for between-cluster differences were highly correlated across datasets (beta = 0.92+/−0.01, p < 0.0001; adjusted R-squared=0.93). Data-driven clusters did not align with diagnostic labels across datasets. Brain-behaviour associations were only replicated for male subsets and subcortical volume using multivariate analysis. We found evidence of replicability of the clustering structure across two independent datasets; however, caution must be exercised in integrating multiple measures in clustering and interpretation of brain-behaviour associations

Large‐scale analysis of structural brain asymmetries during neurodevelopment: Associations with age and sex in 4265 children and adolescents

Only a small number of studies have assessed structural differences between the two hemispheres during childhood and adolescence. However, the existing findings lack consistency or are restricted to a particular brain region, a specific brain feature, or a relatively narrow age range. Here, we investigated associations between brain asymmetry and age as well as sex in one of the largest pediatric samples to date (n = 4265), aged 1–18 years, scanned at 69 sites participating in the ENIGMA (Enhancing NeuroImaging Genetics through Meta‐Analysis) consortium. Our study revealed that significant brain asymmetries already exist in childhood, but their magnitude and direction depend on the brain region examined and the morphometric measurement used (cortical volume or thickness, regional surface area, or subcortical volume). With respect to effects of age, some asymmetries became weaker over time while others became stronger; sometimes they even reversed direction. With respect to sex differences, the total number of regions exhibiting significant asymmetries was larger in females than in males, while the total number of measurements indicating significant asymmetries was larger in males (as we obtained more than one measurement per cortical region). The magnitude of the significant asymmetries was also greater in males. However, effect sizes for both age effects and sex differences were small. Taken together, these findings suggest that cerebral asymmetries are an inherent organizational pattern of the brain that manifests early in life. Overall, brain asymmetry appears to be relatively stable throughout childhood and adolescence, with some differential effects in males and females

Taxonomy of fundamental concepts of localization in cyber-physical and sensor networks

Localization is a fundamental task in Cyber-Physical Systems (CPS), where data is tightly coupled with the environment and the location where it is generated. The research literature on localization has reached a critical mass, and several surveys have also emerged. This review paper contributes on the state-of-the-art with the proposal of a new and holistic taxonomy of the fundamental concepts of localization in CPS, based on a comprehensive analysis of previous research works and surveys. The main objective is to pave the way towards a deep understanding of the main localization techniques, and unify their descriptions. Furthermore, this review paper provides a complete overview on the most relevant localization and geolocation techniques. Also, we present the most important metrics for measuring the accuracy of localization approaches, which is meant to be the gap between the real location and its estimate. Finally, we present open issues and research challenges pertaining to localization. We believe that this review paper will represent an important and complete reference of localization techniques in CPS for researchers and practitioners and will provide them with an added value as compared to previous surveys