6 research outputs found

    S3^3R: Self-supervised Spectral Regression for Hyperspectral Histopathology Image Classification

    Full text link
    Benefited from the rich and detailed spectral information in hyperspectral images (HSI), HSI offers great potential for a wide variety of medical applications such as computational pathology. But, the lack of adequate annotated data and the high spatiospectral dimensions of HSIs usually make classification networks prone to overfit. Thus, learning a general representation which can be transferred to the downstream tasks is imperative. To our knowledge, no appropriate self-supervised pre-training method has been designed for histopathology HSIs. In this paper, we introduce an efficient and effective Self-supervised Spectral Regression (S3^3R) method, which exploits the low rank characteristic in the spectral domain of HSI. More concretely, we propose to learn a set of linear coefficients that can be used to represent one band by the remaining bands via masking out these bands. Then, the band is restored by using the learned coefficients to reweight the remaining bands. Two pre-text tasks are designed: (1)S3^3R-CR, which regresses the linear coefficients, so that the pre-trained model understands the inherent structures of HSIs and the pathological characteristics of different morphologies; (2)S3^3R-BR, which regresses the missing band, making the model to learn the holistic semantics of HSIs. Compared to prior arts i.e., contrastive learning methods, which focuses on natural images, S3^3R converges at least 3 times faster, and achieves significant improvements up to 14% in accuracy when transferring to HSI classification tasks

    Gene-induced Multimodal Pre-training for Image-omic Classification

    Full text link
    Histology analysis of the tumor micro-environment integrated with genomic assays is the gold standard for most cancers in modern medicine. This paper proposes a Gene-induced Multimodal Pre-training (GiMP) framework, which jointly incorporates genomics and Whole Slide Images (WSIs) for classification tasks. Our work aims at dealing with the main challenges of multi-modality image-omic classification w.r.t. (1) the patient-level feature extraction difficulties from gigapixel WSIs and tens of thousands of genes, and (2) effective fusion considering high-order relevance modeling. Concretely, we first propose a group multi-head self-attention gene encoder to capture global structured features in gene expression cohorts. We design a masked patch modeling paradigm (MPM) to capture the latent pathological characteristics of different tissues. The mask strategy is randomly masking a fixed-length contiguous subsequence of patch embeddings of a WSI. Finally, we combine the classification tokens of paired modalities and propose a triplet learning module to learn high-order relevance and discriminative patient-level information.After pre-training, a simple fine-tuning can be adopted to obtain the classification results. Experimental results on the TCGA dataset show the superiority of our network architectures and our pre-training framework, achieving 99.47% in accuracy for image-omic classification. The code is publicly available at https://github.com/huangwudiduan/GIMP

    Effect of pulsed transcranial ultrasound stimulation at different number of tone-burst on cortico-muscular coupling

    No full text
    Abstract Background Pulsed transcranial ultrasound stimulation (pTUS) can modulate the neuronal activity of motor cortex and elicit muscle contractions. Cortico-muscular coupling (CMC) can serve as a tool to identify interaction between the oscillatory activity of the motor cortex and effector muscle. This research aims to explore the neuromodulatory effect of low-intensity, pTUS with different number of tone burst to neural circuit of motor-control system by analyzing the coupling relationship between motor cortex and tail muscle in mouse. The motor cortex of mice was stimulated by pulsed transcranial ultrasound with different number of tone bursts (NTB = 100 150 200 250 300). The local field potentials (LFPs) in tail motor cortex and electromyography (EMG) in tail muscles were recorded simultaneously during pTUS. The change of integral coupling strength between cortex and muscle was evaluated by mutual information (MI). The directional information interaction between them were analyzed by transfer entropy (TE). Results Almost all of the MI and TE values were significantly increased by pTUS. The results of MI showed that the CMC was significantly enhanced with the increase of NTB. The TE results showed the coupling strength of CMC in descending direction (from LFPs to EMG) was significantly higher than that in ascending direction (from EMG to LFPs) after stimulation. Furthermore, compared to NTB = 100, the CMC in ascending direction were significantly enhanced when NTB = 250, 300, and CMC in descending direction were significantly enhanced when NTB = 200, 250, 300. Conclusion These results confirm that the CMC between motor cortex and the tail muscles in mouse could be altered by pTUS. And by increasing the NTB (i.e. sonication duration), the coupling strength within the cortico-muscular circuit could be increased, which might further influence the motor function of mice. It demonstrates that, using MI and TE method, the CMC could be used for quantitatively evaluating the effect of pTUS with different NTBs, which might provide a new insight into the effect of pTUS neuromodulation in motor cortex

    Analysis of public opinion on food safety in Greater China with big data and machine learning

    No full text
    The Internet contains a wealth of public opinion on food safety, including views on food adulteration, food-borne diseases, agricultural pollution, irregular food distribution, and food production issues. To systematically collect and analyze public opinion on food safety in Greater China, we developed IFoodCloud, which automatically collects data from more than 3,100 public sources. Meanwhile, we constructed sentiment classification models using multiple lexicon-based and machine learning-based algorithms integrated with IFoodCloud that provide an unprecedented rapid means of understanding the public sentiment toward specific food safety incidents. Our best model's F1 score achieved 0.9737, demonstrating its great predictive ability and robustness. Using IFoodCloud, we analyzed public sentiment on food safety in Greater China and the changing trend of public opinion at the early stage of the 2019 Coronavirus Disease pandemic, demonstrating the potential of big data and machine learning for promoting risk communication and decision-making.ISSN:2665-927

    A deep learning model using hyperspectral image for EUS‐FNA cytology diagnosis in pancreatic ductal adenocarcinoma

    No full text
    Abstract Background and Aims Endoscopic ultrasonography‐guided fine‐needle aspiration/biopsy (EUS‐FNA/B) is considered to be a first‐line procedure for the pathological diagnosis of pancreatic cancer owing to its high accuracy and low complication rate. The number of new cases of pancreatic ductal adenocarcinoma (PDAC) is increasing, and its accurate pathological diagnosis poses a challenge for cytopathologists. Our aim was to develop a hyperspectral imaging (HSI)‐based convolution neural network (CNN) algorithm to aid in the diagnosis of pancreatic EUS‐FNA cytology specimens. Methods HSI images were captured of pancreatic EUS‐FNA cytological specimens from benign pancreatic tissues (n = 33) and PDAC (n = 39) prepared using a liquid‐based cytology method. A CNN was established to test the diagnostic performance, and Attribution Guided Factorization Visualization (AGF‐Visualization) was used to visualize the regions of important classification features identified by the model. Results A total of 1913 HSI images were obtained. Our ResNet18‐SimSiam model achieved an accuracy of 0.9204, sensitivity of 0.9310 and specificity of 0.9123 (area under the curve of 0.9625) when trained on HSI images for the differentiation of PDAC cytological specimens from benign pancreatic cells. AGF‐Visualization confirmed that the diagnoses were based on the features of tumor cell nuclei. Conclusions An HSI‐based model was developed to diagnose cytological PDAC specimens obtained using EUS‐guided sampling. Under the supervision of experienced cytopathologists, we performed multi‐staged consecutive in‐depth learning of the model. Its superior diagnostic performance could be of value for cytologists when diagnosing PDAC