{\epsilon}-Neural Thompson Sampling of Deep Brain Stimulation for Parkinson Disease Treatment

Deep Brain Stimulation (DBS) stands as an effective intervention for alleviating the motor symptoms of Parkinson's disease (PD). Traditional commercial DBS devices are only able to deliver fixed-frequency periodic pulses to the basal ganglia (BG) regions of the brain, i.e., continuous DBS (cDBS). However, they in general suffer from energy inefficiency and side effects, such as speech impairment. Recent research has focused on adaptive DBS (aDBS) to resolve the limitations of cDBS. Specifically, reinforcement learning (RL) based approaches have been developed to adapt the frequencies of the stimuli in order to achieve both energy efficiency and treatment efficacy. However, RL approaches in general require significant amount of training data and computational resources, making it intractable to integrate RL policies into real-time embedded systems as needed in aDBS. In contrast, contextual multi-armed bandits (CMAB) in general lead to better sample efficiency compared to RL. In this study, we propose a CMAB solution for aDBS. Specifically, we define the context as the signals capturing irregular neuronal firing activities in the BG regions (i.e., beta-band power spectral density), while each arm signifies the (discretized) pulse frequency of the stimulation. Moreover, an {\epsilon}-exploring strategy is introduced on top of the classic Thompson sampling method, leading to an algorithm called {\epsilon}-Neural Thompson sampling ({\epsilon}-NeuralTS), such that the learned CMAB policy can better balance exploration and exploitation of the BG environment. The {\epsilon}-NeuralTS algorithm is evaluated using a computation BG model that captures the neuronal activities in PD patients' brains. The results show that our method outperforms both existing cDBS methods and CMAB baselines.Comment: 11 pages, 12 figures, 2 tables. To appear in the 15th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS'2024

Robust Reinforcement Learning through Efficient Adversarial Herding

Although reinforcement learning (RL) is considered the gold standard for policy design, it may not always provide a robust solution in various scenarios. This can result in severe performance degradation when the environment is exposed to potential disturbances. Adversarial training using a two-player max-min game has been proven effective in enhancing the robustness of RL agents. In this work, we extend the two-player game by introducing an adversarial herd, which involves a group of adversaries, in order to address ($\textit{i}$) the difficulty of the inner optimization problem, and ($\textit{ii}$) the potential over pessimism caused by the selection of a candidate adversary set that may include unlikely scenarios. We first prove that adversarial herds can efficiently approximate the inner optimization problem. Then we address the second issue by replacing the worst-case performance in the inner optimization with the average performance over the worst-$k$ adversaries. We evaluate the proposed method on multiple MuJoCo environments. Experimental results demonstrate that our approach consistently generates more robust policies

Knowledge-Enriched Visual Storytelling

Stories are diverse and highly personalized, resulting in a large possible output space for story generation. Existing end-to-end approaches produce monotonous stories because they are limited to the vocabulary and knowledge in a single training dataset. This paper introduces KG-Story, a three-stage framework that allows the story generation model to take advantage of external Knowledge Graphs to produce interesting stories. KG-Story distills a set of representative words from the input prompts, enriches the word set by using external knowledge graphs, and finally generates stories based on the enriched word set. This distill-enrich-generate framework allows the use of external resources not only for the enrichment phase, but also for the distillation and generation phases. In this paper, we show the superiority of KG-Story for visual storytelling, where the input prompt is a sequence of five photos and the output is a short story. Per the human ranking evaluation, stories generated by KG-Story are on average ranked better than that of the state-of-the-art systems. Our code and output stories are available at https://github.com/zychen423/KE-VIST.Comment: AAAI 202

Real-world Effectiveness and Tolerability of Interferon-free Direct-acting Antiviral for 15,849 Patients with Chronic Hepatitis C: A Multinational Cohort Study

BACKGROUND AND AIMS: As practice patterns and hepatitis C virus (HCV) genotypes (GT) vary geographically, a global real-world study from both East and West covering all GTs can help inform practice policy toward the 2030 HCV elimination goal. This study aimed to assess the effectiveness and tolerability of DAA treatment in routine clinical practice in a multinational cohort for patients infected with all HCV GTs, focusing on GT3 and GT6. METHODS: We analyzed the sustained virological response (SVR12) of 15,849 chronic hepatitis C patients from 39 Real-World Evidence from the Asia Liver Consortium for HCV clinical sites in Asia Pacific, North America, and Europe between 07/01/2014-07/01/2021. RESULTS: The mean age was 62±13 years, with 49.6% male. The demographic breakdown was 91.1% Asian (52.9% Japanese, 25.7% Chinese/Taiwanese, 5.4% Korean, 3.3% Malaysian, and 2.9% Vietnamese), 6.4% White, 1.3% Hispanic/Latino, and 1% Black/African-American. Additionally, 34.8% had cirrhosis, 8.6% had hepatocellular carcinoma (HCC), and 24.9% were treatment-experienced (20.7% with interferon, 4.3% with direct-acting antivirals). The largest group was GT1 (10,246 [64.6%]), followed by GT2 (3,686 [23.2%]), GT3 (1,151 [7.2%]), GT6 (457 [2.8%]), GT4 (47 [0.3%]), GT5 (1 [0.006%]), and untyped GTs (261 [1.6%]). The overall SVR12 was 96.9%, with rates over 95% for GT1/2/3/6 but 91.5% for GT4. SVR12 for GT3 was 95.1% overall, 98.2% for GT3a, and 94.0% for GT3b. SVR12 was 98.3% overall for GT6, lower for patients with cirrhosis and treatment-experienced (TE) (93.8%) but ≥97.5% for treatment-naive patients regardless of cirrhosis status. On multivariable analysis, advanced age, prior treatment failure, cirrhosis, active HCC, and GT3/4 were independent predictors of lower SVR12, while being Asian was a significant predictor of achieving SVR12. CONCLUSIONS: In this diverse multinational real-world cohort of patients with various GTs, the overall cure rate was 96.9%, despite large numbers of patients with cirrhosis, HCC, TE, and GT3/6. SVR12 for GT3/6 with cirrhosis and TE was lower but still excellent (\u3e91%)

[[alternative]]An Analytical Study of the Most Performed Repertoire for Ocarina Competitions and Concerts: An Example on Hao-Lun Hsu's Graduate Recital

[[abstract]]　　本詮釋報告將以筆者於2019年1月16日舉辦的畢業音樂會演出曲目中，挑選出適合當作陶笛比賽的曲目做為討論。　　陶笛在台灣發展了20年左右，這些年的發展雖然在硬體上突飛猛進，原本只是跟隨在日本的腳步發展，但是後來台灣的製笛技術因為許多前輩投入陶笛製作的這個領域後開始蓬勃發展。短短十年的時間，台灣製作的陶笛已經走向日本、韓國、大陸、香港甚至是陶笛的發源地意大利都有台灣陶笛的蹤影。然而在軟體的發展上顯得比較牛步些，台灣陶笛近20年來（1996-2019）的演奏歌曲，幾乎都是跟隨著日本陶笛演奏家們的腳步，尤其是日本陶笛演奏家野村宗次郎 先生的音樂一直是台灣陶笛玩家們模仿的對象，其音樂風格偏向心靈音樂，有舒服的旋律、簡單的技巧、單純的樂曲結構。由於不需要太艱深的吹奏技巧因而深受台灣陶笛吹奏者的喜愛，也影響了台灣陶笛音樂發展非常多年。隨者陶笛吹奏的人口越來越多，也有越來越多的其他樂器領域專業老師的投入，除了舉辦研習、講座之外，透過國際的音樂會交流慢慢的台灣陶笛的吹奏層次也提升了。當台灣音樂比賽的環境出現陶笛項目之後，隨即面臨的巨大問題是沒有足夠且正式的曲目足以應付音樂比賽。在筆者擔任南投、彰化、宜蘭以及嘉義等縣市的音樂比賽陶笛項目評審多年經驗中，不時可以出現陶笛吹奏流行歌曲參加比賽，在正式的音樂比賽中顯得格格不入。　　有鑑於此，筆者致力於陶笛吹奏曲目的開發，並引進國外的複數管陶笛擴展陶笛演奏的音域，使陶笛能夠演奏更多的樂曲，除此之外更創作了適合複管陶笛演奏的曲目，也為音樂比賽創作了可以發揮複管陶笛特色的演奏曲目。　　本詮釋報告挑選出經常在陶笛比賽與陶笛音樂中被吹奏改編自小提琴的獨奏曲目《情熱大陸》以及三首筆者特地為陶笛比賽設計的創作曲為討論對象。[[abstract]]　　This interpretive research mainly demonstrates works performed in my Graduate Recital held on January 16, 2019 and investigates selected works suitable for ocarina competitions.　　Although ocarina has been popular in Taiwan for almost twenty years, the development and establishment of repertoire for this instrument is still going after Japan, especially the compositions by Sojiro. However, the making technique of ocarina in Taiwan has been playing a leading role in the world.　　There are more and more people play ocarina-- people from kids to elders, from amateurs to professionals. Conferences and lectures about ocarina are also showing up internationally. Even so, according to my experiences on teaching ocarina and judging competitions for ocarina, I found that repertoire for this particular instrument is very limited and is with a huge gap between “classical” and “popular”. For example, one can often hear popular songs played in ocarina competitions along with some other performers' playing classical repertoire such as works by Bach and Mozart etc. 　　Therefore, I have been making effort to compost new works for ocarina and popularizing multi-chambered ocarinas. Meanwhile, I would like to analyze the most popular ocarina composition- Jounetsu Tairiku and my newly composed works for my own recital