Auto-tune: PAC-Bayes Optimization over Prior and Posterior for Neural Networks

It is widely recognized that the generalization ability of neural networks can be greatly enhanced through carefully designing the training procedure. The current state-of-the-art training approach involves utilizing stochastic gradient descent (SGD) or Adam optimization algorithms along with a combination of additional regularization techniques such as weight decay, dropout, or noise injection. Optimal generalization can only be achieved by tuning a multitude of hyperparameters through grid search, which can be time-consuming and necessitates additional validation datasets. To address this issue, we introduce a practical PAC-Bayes training framework that is nearly tuning-free and requires no additional regularization while achieving comparable testing performance to that of SGD/Adam after a complete grid search and with extra regularizations. Our proposed algorithm demonstrates the remarkable potential of PAC training to achieve state-of-the-art performance on deep neural networks with enhanced robustness and interpretability.Comment: 30 pages, 15 figures, 7 table

Enhanced prediction accuracy with uncertainty quantification in monitoring CO2 sequestration using convolutional neural networks

Monitoring changes inside a reservoir in real time is crucial for the success of CO2 injection and long-term storage. Machine learning (ML) is well-suited for real-time CO2 monitoring because of its computational efficiency. However, most existing applications of ML yield only one prediction (i.e., the expectation) for a given input, which may not properly reflect the distribution of the testing data, if it has a shift with respect to that of the training data. The Simultaneous Quantile Regression (SQR) method can estimate the entire conditional distribution of the target variable of a neural network via pinball loss. Here, we incorporate this technique into seismic inversion for purposes of CO2 monitoring. The uncertainty map is then calculated pixel by pixel from a particular prediction interval around the median. We also propose a novel data-augmentation method by sampling the uncertainty to further improve prediction accuracy. The developed methodology is tested on synthetic Kimberlina data, which are created by the Department of Energy and based on a CO2 capture and sequestration (CCS) project in California. The results prove that the proposed network can estimate the subsurface velocity rapidly and with sufficient resolution. Furthermore, the computed uncertainty quantifies the prediction accuracy. The method remains robust even if the testing data are distorted due to problems in the field data acquisition. Another test demonstrates the effectiveness of the developed data-augmentation method in increasing the spatial resolution of the estimated velocity field and in reducing the prediction error.Comment: 42 pages (double-space), 14 figures, 1 tabl

Exploring achievement gamification on online medical quality based on machine learning and empirical analysis

How to improve online medical quality is an important challenge for practitioners of digital health platforms. Gamification creates new opportunities to deal with the problem persistent in online health services. To better understand the role of gamification in online health services context, this study intends to use the research method of machine learning and natural experiment to explore the impact of achievement gamification on online medical quality in online health services, as well as the moderating effects of doctors’ personality and image. Theoretically, this study will expand the application of game strategy in the field of healthcare, and make up for the deficiency of the effects of gamification on online medical quality. Practically, it provides guidance for promoting doctors\u27 online participation behavior, improves the quality of online health services, and suggests ways for optimizing the rational allocation of online health resources

The Influence of Buying vs. Receiving an IT-based Device on User Commitment

IT-based mobile devices (i.e., smart devices), especially those with health monitoring features, are popular gifts. However, little is known about a recipient’s commitment to using the smart device when it is obtained as a gift. To explore the influence of gift- giving on user perceptions and usage, three studies are reported. These studies build on the IT use literature, the gift-giving literature, and social exchange theory to investigate whether and how gift-giving leads to device commitment. Specifically, we found two contextual factor – receiving the smart device as a gift (versus buying for yourself) and providing emotional support when giving the gift – can increase recipients’ symbolic of the smart device. Additionally, recipients’ cognitive value of the smart device negatively moderates the effect of symbolic value on device commitment. The results provide novel insight into the relationship between IT use and gift-giving and provide implications for future research and the smart device industry

Anti-Inflammatory Effects of Cumin Essential Oil by Blocking JNK, ERK, and NF- κ

Cumin seeds (Cuminum cyminum L.) have been commonly used in food flavoring and perfumery. In this study, cumin essential oil (CuEO) extracted from seeds was employed to investigate the anti-inflammatory effects in lipopolysaccharide- (LPS-) stimulated RAW 264.7 cells and the underlying mechanisms. A total of 26 volatile constituents were identified in CuEO by GC-MS, and the most abundant constituent was cuminaldehyde (48.773%). Mitochondrial-respiration-dependent 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium (MTT) reduction assay demonstrated that CuEO did not exhibit any cytotoxic effect at the employed concentrations (0.0005–0.01%). Real-time PCR tests showed that CuEO significantly inhibited the mRNA expressions of inducible nitric oxide synthase (iNOS), cyclooxygenase (COX-2), interleukin- (IL-) 1, and IL-6. Moreover, western blotting analysis revealed that CuEO blocked LPS-induced transcriptional activation of nuclear factor-kappa B (NF-κB) and inhibited the phosphorylation of extracellular signal regulated kinase (ERK) and c-Jun N-terminal kinase (JNK). These results suggested that CuEO exerted anti-inflammatory effects in LPS-stimulated RAW 264.7 cells via inhibition of NF-κB and mitogen-activated protein kinases ERK and JNK signaling; the chemical could be used as a source of anti-inflammatory agents as well as dietary complement for health promotion

PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descent

Fine-tuning pretrained language models (PLMs) for downstream tasks is a large-scale optimization problem, in which the choice of the training algorithm critically determines how well the trained model can generalize to unseen test data, especially in the context of few-shot learning. To achieve good generalization performance and avoid overfitting, techniques such as data augmentation and pruning are often applied. However, adding these regularizations necessitates heavy tuning of the hyperparameters of optimization algorithms, such as the popular Adam optimizer. In this paper, we propose a two-stage fine-tuning method, PAC-tuning, to address this optimization challenge. First, based on PAC-Bayes training, PAC-tuning directly minimizes the PAC-Bayes generalization bound to learn proper parameter distribution. Second, PAC-tuning modifies the gradient by injecting noise with the variance learned in the first stage into the model parameters during training, resulting in a variant of perturbed gradient descent (PGD). In the past, the few-shot scenario posed difficulties for PAC-Bayes training because the PAC-Bayes bound, when applied to large models with limited training data, might not be stringent. Our experimental results across 5 GLUE benchmark tasks demonstrate that PAC-tuning successfully handles the challenges of fine-tuning tasks and outperforms strong baseline methods by a visible margin, further confirming the potential to apply PAC training for any other settings where the Adam optimizer is currently used for training.Comment: Accepted to EMNLP23 mai

Examining the Role of Technology Anxiety and Health Anxiety on Elderly Users’ Continuance Intention for Mobile Health Services Use

Mobile health (mHealth) is considered to be an important means of releasing the aging population problem. The efficiency of mHealth service can be increased by incorporating more elderly users and guaranteeing their continued use. However, limited attention has been directed toward investigating elderly users’ continuance intention for mHealth service use. Drawing upon the trust theory, we investigated elderly users’ characteristics, i.e. health anxiety and technology anxiety, to explain continuance intention. Survey data were collected comprising 261 valid responses to validate the research model and hypotheses. The results revealed that both cognitive and affective trust enhance continuance intention of mHealth services use. Health anxiety strengthens the effect of cognitive trust, but weakens the effect of affective trust, on the continuance intention. Furthermore, technology anxiety strengthens the effect of affective trust, but not that of cognitive trust, on the continuance intention. The limitations of our study and the theoretical and practical implications are discussed