Bayesian approaches of mixture copulas with applications

Copula theory has become one of the most important ideologies and methodologies for modeling the dependence among random variables. Rather than using point performance metrics such as Pearson linear correlation, copula functions enable us to construct the multivariate distributions among the concerned random variables by starting from the corresponding marginal distributions. Hence, it gives us a full description of the dependence mode. The most frequently used copula models are parametric copulas such as Gaussian, Clayton, and Gumbel copulas. However, in many practical scenarios, these copulas often fail to fully describe the dependence as real data often contain complex patterns with multi-modals. In addition, classic copulas are mostly studied in their bivariate form, leaving the application of copulas into higher dimensional data non-trivial. This thesis intends to approach the above-mentioned problems by utilizing Bayesian samplers into mixture copulas. In particular, we study the problems of estimating, selecting, and simulating mixture components of copulas by using Bayesian approaches. Families of multivariate elliptical and skew-elliptical copulas are given special attention as they can be naturally extended to higher dimensions. For applications, we apply our proposed approaches to study the dependence among financial markets. Meanwhile, we extend the application of our Bayesian mixture copulas to improve the oversampling methods for imbalance learning problems in the field of data science. The thesis mainly consists of four major parts. In the first part, we applied the Bayesian sparse finite mixture model to the copula mixture modeling, which enables us to estimate and select the correct finite mixture copulas simultaneously without having to repeatedly estimate various forms of models and compare their AICs or BICs. The second part focused on the construction of infinite mixture t copulas using the Dirichlet process prior. Although we are concentrated on the t copulas due to their usefulness in financial applications. This approach can be extended to more general copulas. The approaches further advance the previously proposed finite mixture Bayesian approaches despite being more complicated in terms of modeling. The third part further extends previous parts to construct the non-parametric Bayesian copula mixture models for serially correlated data. In particular, we discuss the modeling of the hidden Markov models (HMM) with multivariate emission distributions. We use copula theories to decompose the construction of multivariate emission distributions into univariate marginal distributions and a dependence structure. Meanwhile, many real-life applications of HMM have an unknown number of states, which need to be manually specified by analysts if the classic HMM method is used. Introducing the hierarchical Dirichlet process into the Copula-HMM model enables us to infer the number of unknown states from the dataset automatically. We thoroughly introduce the inference method of this non-parametric Bayesian copula-HMM model therein. The final part is about the introduction and study of the evaluation metrics of imbalance learning problems as well as applying the mixture copulas approach to solving the data imbalance. One major obstacle of applying the copulas approach to imbalanced datasets is the high dimensional features of many tasks. On the other hand, data science applications often include features that are discrete-valued, while most of the copulas literature only deals with continuous random vectors. Therefore, we develop the MCMC approaches for estimating the mixed valued copulas (i.e., the copula contains both continuous and discrete valued variables) and apply them to estimate the dataset and perform the oversampling. The Bayesian approach would be useful in these tasks as the real applications often involve high dimensional large dataset, whereas the classic MLE approaches struggle in this case due to the exponential complexity in evaluating the discrete dimensions. The approaches are applied to the simulated dataset to prove its validity in the paper. Meanwhile, the real oversampling task is performed using mixture copulas, and the results are compared with the classic random oversampling and the SMOTE approaches

CaSR Induces Osteoclast Differentiation and Promotes Bone Metastasis in Lung Adenocarcinoma

Objective: Explore the mechanism of CaSR's involvement in bone metastasis in lung adenocarcinoma. Methods: Immunohistochemistry (IHC) was used to detect the expression of calcium-sensing receptor (CaSR) in 120 cases of lung adenocarcinoma with bone metastasis. Stably transfected cell lines with CaSR overexpression and knockdown based on A549 cells were constructed. The expression of CaSR was verified by western blot and qPCR. The proliferation and migration abilities of A549 cells were tested using cholecystokinin-8 (CCK-8) and Transwell assays, respectively. Western blotting was used to detect the expression of matrix metalloproteinases MMP2, MMP9, CaSR, and NF-κB. The supernatant from each cell culture group was collected as a conditional co-culture solution to study the induction of osteoclast precursor cells and osteoblasts. Western blot and qPCR were used to validate the expression of bone matrix degradation-related enzymes cathepsin K and hormone calcitonin receptor (CTR) and osteoblast-induced osteoclast maturation and differentiation enzyme receptor activator of nuclear factor-κB ligand (RANKL), macrophage colony-stimulating factor (M-CSF), osteoprotegerin (OPG), and PTHrP. Immunofluorescent staining was used to detect F-actin ring formation and osteocalcin expression. Western blot results for NF-κB expression identified a regulatory relationship between NF-κB and CaSR. Results: CaSR expression in lung cancer tissues was significantly higher than that in adjacent and normal lung tissues. The expression of CaSR in lung cancer tissues with bone metastasis was higher than that in non-metastatic lung cancer tissues. The proliferation and migration ability of A549 cells increased significantly with overexpressed CaSR. The co-culture solution directly induced osteoclast precursor cells and the expression of bone matrix degradation-related enzymes significantly increased. Osteoblasts were significantly inhibited and osteoblast-induced osteoclast maturation and differentiation enzymes were significantly downregulated. It was found that the expression of NF-κB and PTHrP increased when CaSR was overexpressed. Osteoclast differentiation factor expression was also significantly increased, which directly induces osteoclast differentiation and maturation. These results were reversed when CaSR was knocked down. Conclusions: CaSR can positively regulate NF-κB and PTHrP expression in A549 cells with a high metastatic potential, thereby promoting osteoclast differentiation and maturation, and facilitating the occurrence and development of bone metastasis in lung adenocarcinoma

MPC-STANet: Alzheimer’s Disease Recognition Method based on Multiple Phantom Convolution and Spatial Transformation Attention Mechanism

Alzheimer\u27s disease (AD) is a progressive neurodegenerative disease with insidious and irreversible onset. The recognition of the disease stage of AD and the administration of effective interventional treatment are important to slow down and control the progression of the disease. However, due to the unbalanced distribution of the acquired data volume, the problem that the features change inconspicuously in different disease stages of AD, and the scattered and narrow areas of the feature areas (hippocampal region, medial temporal lobe, etc.), the effective recognition of AD remains a critical unmet need. Therefore, we first employ class-balancing operation using data expansion and Synthetic Minority Oversampling Technique (SMOTE) to avoid the AD MRI dataset being affected by classification imbalance in the training. Subsequently, a recognition network based on Multi-Phantom Convolution (MPC) and Space Conversion Attention Mechanism (MPC-STANet) with ResNet50 as the backbone network is proposed for the recognition of the disease stages of AD. In this study, we propose a Multi-Phantom Convolution in the way of convolution according to the channel direction and integrate it with the average pooling layer into two basic blocks of ResNet50: Conv Block and Identity Block to propose the Multi-Phantom Residual Block (MPRB) including Multi-Conv Block and Multi-Identity Block to better recognize the scattered and tiny disease features of Alzheimer\u27s disease. Meanwhile, the weight coefficients are extracted from both vertical and horizontal directions using the Space Conversion Attention Mechanism (SCAM) to better recognize subtle structural changes in the AD MRI images. The experimental results show that our proposed method achieves an average recognition accuracy of 96.25%, F1 score of 95%, and mAP of 93%, and the number of parameters is only 1.69 M more than ResNet50

Pre-trained transformer for adversarial purification

With more and more deep neural networks being deployed as various daily services, their reliability is essential. It's frightening that deep neural networks are vulnerable and sensitive to adversarial attacks, the most common one of which for the services is evasion-based. Recent works usually strengthen the robustness by adversarial training or leveraging the knowledge of an amount of clean data. However, in practical terms, retraining and redeploying the model need a large computational budget, leading to heavy losses to the online service. In addition, when adversarial examples of a certain attack are detected, only limited adversarial examples are available for the service provider, while much clean data may not be accessible. Given the mentioned problems, we propose a new scenario, RaPiD (Rapid Plug-in Defender), which is to rapidly defend against a certain attack for the frozen original service model with limitations of few clean and adversarial examples. Motivated by the generalization and the universal computation ability of pre-trained transformer models, we come up with a new defender method, CeTaD, which stands for Considering Pre-trained Transformers as Defenders. In particular, we evaluate the effectiveness and the transferability of CeTaD in the case of one-shot adversarial examples and explore the impact of different parts of CeTaD as well as training data conditions. CeTaD is flexible, able to be embedded into an arbitrary differentiable model, and suitable for various types of attacks

MPC-STANet: Alzheimer’s Disease Recognition Method based on Multiple Phantom Convolution and Spatial Transformation Attention Mechanism

Alzheimer\u27s disease (AD) is a progressive neurodegenerative disease with insidious and irreversible onset. The recognition of the disease stage of AD and the administration of effective interventional treatment are important to slow down and control the progression of the disease. However, due to the unbalanced distribution of the acquired data volume, the problem that the features change inconspicuously in different disease stages of AD, and the scattered and narrow areas of the feature areas (hippocampal region, medial temporal lobe, etc.), the effective recognition of AD remains a critical unmet need. Therefore, we first employ class-balancing operation using data expansion and Synthetic Minority Oversampling Technique (SMOTE) to avoid the AD MRI dataset being affected by classification imbalance in the training. Subsequently, a recognition network based on Multi-Phantom Convolution (MPC) and Space Conversion Attention Mechanism (MPC-STANet) with ResNet50 as the backbone network is proposed for the recognition of the disease stages of AD. In this study, we propose a Multi-Phantom Convolution in the way of convolution according to the channel direction and integrate it with the average pooling layer into two basic blocks of ResNet50: Conv Block and Identity Block to propose the Multi-Phantom Residual Block (MPRB) including Multi-Conv Block and Multi-Identity Block to better recognize the scattered and tiny disease features of Alzheimer\u27s disease. Meanwhile, the weight coefficients are extracted from both vertical and horizontal directions using the Space Conversion Attention Mechanism (SCAM) to better recognize subtle structural changes in the AD MRI images. The experimental results show that our proposed method achieves an average recognition accuracy of 96.25%, F1 score of 95%, and mAP of 93%, and the number of parameters is only 1.69 M more than ResNet50

A framework of lightweight deep cross-connected convolution kernel mapping support vector machines

Deep kernel mapping support vector machines have achieved good results in numerous tasks by mapping features from a low-dimensional space to a high-dimensional space and then using support vector machines for classification. However, the depth kernel mapping support vector machine does not take into account the connection of different dimensional spaces and increases the model parameters. To further improve the recognition capability of deep kernel mapping support vector machines while reducing the number of model parameters, this paper proposes a framework of Lightweight Deep Convolutional Cross-Connected Kernel Mapping Support Vector Machines (LC-CKMSVM). The framework consists of a feature extraction module and a classification module. The feature extraction module first maps the data from low-dimensional to high-dimensional space by fusing the representations of different dimensional spaces through cross-connections; then, it uses depthwise separable convolution to replace part of the original convolution to reduce the number of parameters in the module; The classification module uses a soft margin support vector machine for classification. The results on 6 different visual datasets show that LC-CKMSVM obtains better classification accuracies on most cases than the other five models

DCCAM-MRNet: Mixed Residual Connection Network with Dilated Convolution and Coordinate Attention Mechanism for Tomato Disease Identification

Tomato is an important and fragile crop. During the course of its development, it is frequently contaminated with bacteria or viruses. Tomato leaf diseases may be detected quickly and accurately, resulting in increased productivity and quality. Because of the intricate development environment of tomatoes and their inconspicuous disease spot features and small spot area, present machine vision approaches fail to reliably recognize tomato leaves. As a result, this research proposes a novel paradigm for detecting tomato leaf disease. The INLM (integration nonlocal means) filtering algorithm, for example, decreases the interference of surrounding noise on the features. Then, utilizing ResNeXt50 as the backbone, we create DCCAM-MRNet, a novel tomato image recognition network. Dilated Convolution (DC) was employed in STAGE 1 of the DCCAM-MRNet to extend the network\u27s perceptual area and locate the scattered disease spots on tomato leaves. The coordinate attention (CA) mechanism is then introduced to record cross-channel information and direction- and position-sensitive data, allowing the network to more accurately detect localized tomato disease spots. Finally, we offer a mixed residual connection (MRC) technique that combines residual block (RS-Block) and transformed residual block (TR-Block) (TRS-Block). This strategy can increase the network\u27s accuracy while also reducing its size. The DCCAM-classification MRNet\u27s accuracy is 94.3 percent, which is higher than the existing network, and the number of parameters is 0.11 M lesser than the backbone network ResNeXt50, according to the experimental results. As a result, combining INLM and DCCAM-MRNet to identify tomato diseases is a successful strategy