Accelerating and Improving AlphaZero Using Population Based Training

AlphaZero has been very successful in many games. Unfortunately, it still consumes a huge amount of computing resources, the majority of which is spent in self-play. Hyperparameter tuning exacerbates the training cost since each hyperparameter configuration requires its own time to train one run, during which it will generate its own self-play records. As a result, multiple runs are usually needed for different hyperparameter configurations. This paper proposes using population based training (PBT) to help tune hyperparameters dynamically and improve strength during training time. Another significant advantage is that this method requires a single run only, while incurring a small additional time cost, since the time for generating self-play records remains unchanged though the time for optimization is increased following the AlphaZero training algorithm. In our experiments for 9x9 Go, the PBT method is able to achieve a higher win rate for 9x9 Go than the baselines, each with its own hyperparameter configuration and trained individually. For 19x19 Go, with PBT, we are able to obtain improvements in playing strength. Specifically, the PBT agent can obtain up to 74% win rate against ELF OpenGo, an open-source state-of-the-art AlphaZero program using a neural network of a comparable capacity. This is compared to a saturated non-PBT agent, which achieves a win rate of 47% against ELF OpenGo under the same circumstances.Comment: accepted by AAAI2020 as oral presentation. In this version, supplementary materials are adde

THE EFFECTS OF EXTERNAL LOAD ON LOWER EXTREMiTY ELECTROMYOGRAPHY AMPLITUDE DURING COUNTERMOVEMENT JUMP

The purpose of this study was to investigate the effects of different loads on the mean electromyography (EMG) amplitude of the gluteus maximus, biceps fernoris, vastus medialis, gastrocnemius, soleus, and tibialis anterior during the deceleration phase and the acceleration phase of the countermovement jumps (CMJ). Ten male physical education students performed different CMJs with and without an external load (0,2.5,5.0, 7.5, or 10.0 kg hold in arms). The results s h o w the amplitude of the gluteus maximus with load of 7.5 kg was higher than with load of 2.5 kg during the deceleration phase (p < .05), and the amplitude of the soleus with load of 10.0 kg was higher than with load of 2.5 kg during the acceleration phase (p < .05). It indicated that the activities of lower limb muscles were not influenced by the relative lower of external loading during CMJ

A Local-Pattern Related Look-Up Table

This paper describes a Relevance-Zone pattern table (RZT) that can be used to replace a traditional transposition table. An RZT stores exact game values for patterns that are discovered during a Relevance-Zone-Based Search (RZS), which is the current state-of-the-art in solving L&D problems in Go. Positions that share the same pattern can reuse the same exact game value in the RZT. The pattern matching scheme for RZTs is implemented using a radix tree, taking into consideration patterns with different shapes. To improve the efficiency of table lookups, we designed a heuristic that prevents redundant lookups. The heuristic can safely skip previously queried patterns for a given position, reducing the overhead to 10% of the original cost. We also analyze the time complexity of the RZT both theoretically and empirically. Experiments show the overhead of traversing the radix tree in practice during lookup remain flat logarithmically in relation to the number of entries stored in the table. Experiments also show that the use of an RZT instead of a traditional transposition table significantly reduces the number of searched nodes on two data sets of 7x7 and 19x19 L&D Go problems.Comment: Submitted to IEEE Transactions on Games (under review

Loading effects of anterior cervical spine fusion on adjacent segments

AbstractAdjacent segment degeneration typically follows anterior cervical spine fusion. However, the primary cause of adjacent segment degeneration remains unknown. Therefore, in order to identify the loading effects that cause adjacent segment degeneration, this study examined the loading effects to superior segments adjacent to fused bone following anterior cervical spine fusion. The C3–C6 cervical spine segments of 12 sheep were examined. Specimens were divided into the following groups: intact spine (group 1); and C5–C6 segments that were fused via cage-instrumented plate fixation (group 2). Specimens were cycled between 20° flexion and 15° extension with a displacement control of 1°/second. The tested parameters included the range of motion (ROM) of each segment, torque and strain on both the body and inferior articular process at the superior segments (C3–C4) adjacent to the fused bone, and the position of the neutral axis of stress at under 20° flexion and 15° extension. Under flexion and Group 2, torque, ROM, and strain on both the bodies and facets of superior segments adjacent to the fused bone were higher than those of Group 1. Under extension and Group 2, ROM for the fused segment was less than that of Group 1; torque, ROM, and stress on both the bodies and facets of superior segments adjacent to the fused bone were higher than those of Group 1. These analytical results indicate that the muscles and ligaments require greater force to achieve cervical motion than the intact spine following anterior cervical spine fusion. In addition, ROM and stress on the bodies and facets of the joint segments adjacent to the fused bone were significantly increased. Under flexion, the neutral axis of the stress on the adjacent segment moved backward, and the stress on the bodies of the segments adjacent to the fused bone increased. These comparative results indicate that increased stress on the adjacent segments is caused by stress-shielding effects. Furthermore, increased stress and ROM of the adjacent segments after long-term bone fusion may accelerate degeneration in adjacent segment

Evaluation of unilateral cage-instrumented fixation for lumbar spine

<p>Abstract</p> <p>Background</p> <p>To investigate how unilateral cage-instrumented posterior lumbar interbody fusion (PLIF) affects the three-dimensional flexibility in degenerative disc disease by comparing the biomechanical characteristics of unilateral and bilateral cage-instrumented PLIF.</p> <p>Methods</p> <p>Twelve motion segments in sheep lumbar spine specimens were tested for flexion, extension, axial rotation, and lateral bending by nondestructive flexibility test method using a nonconstrained testing apparatus. The specimens were divided into two equal groups. Group 1 received unilateral procedures while group 2 received bilateral procedures. Laminectomy, facectomy, discectomy, cage insertion and transpedicle screw insertion were performed sequentially after testing the intact status. Changes in range of motion (ROM) and neutral zone (NZ) were compared between unilateral and bilateral cage-instrumented PLIF.</p> <p>Results</p> <p>Both ROM and NZ, unilateral cage-instrumented PLIF and bilateral cage-instrumented PLIF, transpedicle screw insertion procedure did not revealed a significant difference between flexion-extension, lateral bending and axial rotation direction except the ROM in the axial rotation. The bilateral group's ROM (-1.7 ± 0. 8) of axial rotation was decreased significantly after transpedicle screw insertion procedure in comparison with the unilateral group (-0.2 ± 0.1). In the unilateral cage-instrumented PLIF group, the transpedicle screw insertion procedure did not demonstrate a significant difference between right and left side in the lateral bending and axial rotation direction.</p> <p>Conclusions</p> <p>Based on the results of this study, unilateral cage-instrumented PLIF and bilateral cage-instrumented PLIF have similar stability after transpedicle screw fixation in the sheep spine model. The unilateral approach can substantially reduce exposure requirements. It also offers the biomechanics advantage of construction using anterior column support combined with pedicle screws just as the bilateral cage-instrumented group. The unpleasant effect of couple motion resulting from inherent asymmetry was absent in the unilateral group.</p

Effects of R&D intensity on firm performance in Taiwan’s semiconductor industry

This study examined the impact of research and development (R&D) investment behaviour on the corporate performance of the Taiwanese semiconductor industry, which faced the economic downturn caused by the global financial crisis of 2008, for the period 2005–2016. A dynamic panel data model was used to empirically analyse the impact of R&D intensity on business performance. A generalised method of moments estimator was adopted to avoid endogeneity problems caused by adding dynamics to the model. Further, the model was used to explore the impact of the lag effect of R&D investments on business performance. It was found that significant R&D investments in a given period may reduce business performance in the same period and continue to influence it in the next few periods, thus indicating the presence of a positive and lagged effect of R&D investments in the high-tech industry. Firm size was also found to be positively correlated with business performance, that is, the larger the firm size, the greater is the use of resources for R&D, which, in turn, leads to more sophisticated technologies and profitable outcomes, forming a positive cycle. This indicates that R&D expenditures affect firms’sustainable management

Game Solving with Online Fine-Tuning

Game solving is a similar, yet more difficult task than mastering a game. Solving a game typically means to find the game-theoretic value (outcome given optimal play), and optionally a full strategy to follow in order to achieve that outcome. The AlphaZero algorithm has demonstrated super-human level play, and its powerful policy and value predictions have also served as heuristics in game solving. However, to solve a game and obtain a full strategy, a winning response must be found for all possible moves by the losing player. This includes very poor lines of play from the losing side, for which the AlphaZero self-play process will not encounter. AlphaZero-based heuristics can be highly inaccurate when evaluating these out-of-distribution positions, which occur throughout the entire search. To address this issue, this paper investigates applying online fine-tuning while searching and proposes two methods to learn tailor-designed heuristics for game solving. Our experiments show that using online fine-tuning can solve a series of challenging 7x7 Killall-Go problems, using only 23.54% of computation time compared to the baseline without online fine-tuning. Results suggest that the savings scale with problem size. Our method can further be extended to any tree search algorithm for problem solving. Our code is available at https://rlg.iis.sinica.edu.tw/papers/neurips2023-online-fine-tuning-solver.Comment: Accepted by the 37th Conference on Neural Information Processing Systems (NeurIPS 2023

Timeframe for return to driving for patients with minimally invasive knee arthroplasty is associated with knee performance on functional tests

BACKGROUND: This study hopes to establish the timeframe for a safe return to driving under different speed conditions for patients after minimally invasive total knee arthroplasty and further explores how well various kinds of functional tests on knee performance can predict the patients’ braking ability. METHODS: 14 patients with right knee osteoarthritis were included in the present study and instructed to perform three simulated driving tasks at preoperative, 2 weeks postoperative and 4 weeks postoperative. RESULTS: The results showed that the total braking time at 4 week postoperative has attained the preoperative level at the driving speed 50 and 70 km/hr but not at the driving speed 90 km/hr. It had significantly improving in knee reaction time and maximum isometric force at 4 weeks postoperative. Besides, there was a moderate to high correlation between the scores of the step counts and the total braking time. CONCLUSIONS: Summary, it is recommended that driving may be resumed 4 weeks after a right knee replacement but had to drive at low or moderate speed and the best predictor of safety driving is step counts

Activity-dependent neurorehabilitation beyond physical trainings: "mental exercise" through mirror neuron activation

The activity dependent brain repair mechanism has been widely adopted in many types of neurorehabilitation. The activity leads to target specific and non-specific beneficial effects in different brain regions, such as the releasing of neurotrophic factors, modulation of the cytokines and generation of new neurons in adult hood. However physical exercise program clinically are limited to some of the patients with preserved motor functions; while many patients suffered from paralysis cannot make such efforts. Here the authors proposed the employment of mirror neurons system in promoting brain rehabilitation by "observation based stimulation". Mirror neuron system has been considered as an important basis for action understanding and learning by mimicking others. During the action observation, mirror neuron system mediated the direct activation of the same group of motor neurons that are responsible for the observed action. The effect is clear, direct, specific and evolutionarily conserved. Moreover, recent evidences hinted for the beneficial effects on stroke patients after mirror neuron system activation therapy. Finally some music-relevant therapies were proposed to be related with mirror neuron system