Correlation between adherence rates measured by MEMS and self-reported questionnaires: a meta-analysis

<p>Abstract</p> <p>Purpose</p> <p>It is vital to understand the associations between the medication event monitoring systems (MEMS) and self-reported questionnaires (SRQs) because both are often used to measure medication adherence and can produce different results. In addition, the economic implication of using alternative measures is important as the cost of electronic monitoring devices is not covered by insurance, while self-reports are the most practical and cost-effective method in the clinical settings. This meta-analysis examined the correlations of two measurements of medication adherence: MEMS and SRQs.</p> <p>Methods</p> <p>The literature search (1980-2009) used PubMed, OVID MEDLINE, PsycINFO (EBSCO), CINAHL (EBSCO), OVID HealthStar, EMBASE (Elsevier), and Cochrane Databases. Studies were included if the correlation coefficients [Pearson (r_p) or Spearman (r_s)] between adherences measured by both MEMS and SRQs were available or could be calculated from other statistics in the articles. Data were independently abstracted in duplicate with standardized protocol and abstraction form including 1) first author's name; 2) year of publication; 3) disease status of participants; 4) sample size; 5) mean age (year); 6) duration of trials (month); 7) SRQ names if available; 8) adherence (%) measured by MEMS; 9) adherence (%) measured by SRQ; 10) correlation coefficient and relative information, including p-value, 95% confidence interval (CI). A meta-analysis was conducted to pool the correlation coefficients using random-effect model.</p> <p>Results</p> <p>Eleven studies (N = 1,684 patients) met the inclusion criteria. The mean of adherence measured by MEMS was 74.9% (range 53.4%-92.9%), versus 84.0% by SRQ (range 68.35%-95%). The correlation between adherence measured by MEMS and SRQs ranged from 0.24 to 0.87. The pooled correlation coefficient for 11 studies was 0.45 (p = 0.001, 95% confidence interval [95% CI]: 0.34-0.56). The subgroup meta-analysis on the seven studies reporting r_pand four studies reporting r_sreported the pooled correlation coefficient: 0.46 (p = 0.011, 95% CI: 0.33-0.59) and 0.43 (p = 0.0038, 95% CI: 0.23-0.64), respectively. No differences were found for other subgroup analyses.</p> <p>Conclusion</p> <p>Medication adherence measured by MEMS and SRQs tends to be at least moderately correlated, suggesting that SRQs give a good estimate of medication adherence.</p

Semantic Role Labeling Guided Out-of-distribution Detection

Identifying unexpected domain-shifted instances in natural language processing is crucial in real-world applications. Previous works identify the OOD instance by leveraging a single global feature embedding to represent the sentence, which cannot characterize subtle OOD patterns well. Another major challenge current OOD methods face is learning effective low-dimensional sentence representations to identify the hard OOD instances that are semantically similar to the ID data. In this paper, we propose a new unsupervised OOD detection method, namely Semantic Role Labeling Guided Out-of-distribution Detection (SRLOOD), that separates, extracts, and learns the semantic role labeling (SRL) guided fine-grained local feature representations from different arguments of a sentence and the global feature representations of the full sentence using a margin-based contrastive loss. A novel self-supervised approach is also introduced to enhance such global-local feature learning by predicting the SRL extracted role. The resulting model achieves SOTA performance on four OOD benchmarks, indicating the effectiveness of our approach. Codes will be available upon acceptance

Is ChatGPT a Good NLG Evaluator? A Preliminary Study

Recently, the emergence of ChatGPT has attracted wide attention from the computational linguistics community. Many prior studies have shown that ChatGPT achieves remarkable performance on various NLP tasks in terms of automatic evaluation metrics. However, the ability of ChatGPT to serve as an evaluation metric is still underexplored. Considering assessing the quality of natural language generation (NLG) models is an arduous task and NLG metrics notoriously show their poor correlation with human judgments, we wonder whether ChatGPT is a good NLG evaluation metric. In this report, we provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. In detail, we regard ChatGPT as a human evaluator and give task-specific (e.g., summarization) and aspect-specific (e.g., relevance) instruction to prompt ChatGPT to evaluate the generated results of NLG models. We conduct experiments on five NLG meta-evaluation datasets (including summarization, story generation and data-to-text tasks). Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments in most cases. In addition, we find that the effectiveness of the ChatGPT evaluator might be influenced by the creation method of the meta-evaluation datasets. For the meta-evaluation datasets which are created greatly depending on the reference and thus are biased, the ChatGPT evaluator might lose its effectiveness. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.Comment: Both first authors contributed equally. Technical Report, 11 pages. Accepted to the 4th New Frontiers in Summarization Workshop (NewSumm@EMNLP 2023

Stock Market Prediction via Deep Learning Techniques: A Survey

The stock market prediction has been a traditional yet complex problem researched within diverse research areas and application domains due to its non-linear, highly volatile and complex nature. Existing surveys on stock market prediction often focus on traditional machine learning methods instead of deep learning methods. Deep learning has dominated many domains, gained much success and popularity in recent years in stock market prediction. This motivates us to provide a structured and comprehensive overview of the research on stock market prediction focusing on deep learning techniques. We present four elaborated subtasks of stock market prediction and propose a novel taxonomy to summarize the state-of-the-art models based on deep neural networks from 2011 to 2022. In addition, we also provide detailed statistics on the datasets and evaluation metrics commonly used in the stock market. Finally, we highlight some open issues and point out several future directions by sharing some new perspectives on stock market prediction

Adjuvant therapy with Jianpi Huayu decoction improves overall and recurrence-free survival after hepatectomy for hepatocellular carcinoma: a retrospective propensity score-matching study

Hepatocellular carcinoma (HCC) patients experience high rates of recurrence following hepatectomy. Many herbal preparations used in traditional Chinese medicine have been shown to improve the postoperative condition of cancer patients. This retrospective study examined the efficacy and safety of Jianpi Huayu decoction (JPHYD) as adjuvant therapy for HCC following hepatectomy. HCC patients received postoperative management according to Chinese Society of Clinical Oncology recommendations, either alone (Control group) or in addition to daily JPHYD (1 week in hospital and 3 months after release). To reduce selection bias, we performed 1:1 propensity score matching between the Control and JPHYD groups. The main endpoint was recurrence-free survival (RFS), and secondary endpoints included overall survival (OS) and adverse event frequency. A total of 207 patients meeting inclusion criteria were enrolled, 127 in the Control group and 80 in the JPHYD group. Patients were then propensity score-matched, yielding each group of 80. Recurrence-free survival rate was significantly higher in the JPHYD group than in the Control group at 1 year (67.9% vs. 38.1%), 2 years (39.1% vs. 26.2%), and 3 years (31.3% vs. 26.2%) following hepatectomy (HR 0.5666 [95%CI, 0.3655 to 0.8784]; p = 0.0066). Additionally, OS was significantly higher in the JPHYD group than the Control group at 1 year (94.3% vs. 81.9%), 2 years (76.4% vs. 58.8%), and 3 years (66.3% vs. 51.4%) following hepatectomy (HR 0.5199 [95%CI, 0.2849 to 0.9490]; p = 0.027). Adverse events frequencies did not differ between the two groups. In conclusion, JPHYD can safely improve RFS and OS following hepatectomy for HCC