XCS Classifier System with Experience Replay

XCS constitutes the most deeply investigated classifier system today. It bears strong potentials and comes with inherent capabilities for mastering a variety of different learning tasks. Besides outstanding successes in various classification and regression tasks, XCS also proved very effective in certain multi-step environments from the domain of reinforcement learning. Especially in the latter domain, recent advances have been mainly driven by algorithms which model their policies based on deep neural networks -- among which the Deep-Q-Network (DQN) is a prominent representative. Experience Replay (ER) constitutes one of the crucial factors for the DQN's successes, since it facilitates stabilized training of the neural network-based Q-function approximators. Surprisingly, XCS barely takes advantage of similar mechanisms that leverage stored raw experiences encountered so far. To bridge this gap, this paper investigates the benefits of extending XCS with ER. On the one hand, we demonstrate that for single-step tasks ER bears massive potential for improvements in terms of sample efficiency. On the shady side, however, we reveal that the use of ER might further aggravate well-studied issues not yet solved for XCS when applied to sequential decision problems demanding for long-action-chains

Deep Neural Networks and Data for Automated Driving

This open access book brings together the latest developments from industry and research on automated driving and artificial intelligence. Environment perception for highly automated driving heavily employs deep neural networks, facing many challenges. How much data do we need for training and testing? How to use synthetic data to save labeling costs for training? How do we increase robustness and decrease memory usage? For inevitably poor conditions: How do we know that the network is uncertain about its decisions? Can we understand a bit more about what actually happens inside neural networks? This leads to a very practical problem particularly for DNNs employed in automated driving: What are useful validation techniques and how about safety? This book unites the views from both academia and industry, where computer vision and machine learning meet environment perception for highly automated driving. Naturally, aspects of data, robustness, uncertainty quantification, and, last but not least, safety are at the core of it. This book is unique: In its first part, an extended survey of all the relevant aspects is provided. The second part contains the detailed technical elaboration of the various questions mentioned above

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp

Muscle activation patterns in shoulder impingement patients

Introduction: Shoulder impingement is one of the most common presentations of shoulder joint problems 1. It appears to be caused by a reduction in the sub-acromial space as the humerus abducts between 60o -120o – the 'painful arc'. Structures between the humeral head and the acromion are thus pinched causing pain and further pathology 2. Shoulder muscle activity can influence this joint space but it is unclear whether this is a cause or effect in impingement patients. This study aimed to observe muscle activation patterns in normal and impingement shoulder patients and determine if there were any significant differences. Method: 19 adult subjects were asked to perform shoulder abduction in their symptomatic arm and non-symptomatic. 10 of these subjects (age 47.9 ± 11.2) were screened for shoulder impingement, and 9 subjects (age 38.9 ± 14.3) had no history of shoulder pathology. Surface EMG was used to collect data for 6 shoulder muscles (Upper, middle and lower trapezius, serratus anterior, infraspinatus, middle deltoids) which was then filtered and fully rectified. Subjects performed 3 smooth unilateral abduction movements at a cadence of 16 beats of a metronome set at 60bpm, and the mean of their results was recorded. T-tests were used to indicate any statistical significance in the data sets. Significance was set at P<0.05. Results: There was a significant difference in muscle activation with serratus anterior in particular showing a very low level of activation throughout the range when compared to normal shoulder activation patterns (<30%). Middle deltoid recruitment was significantly reduced between 60-90o in the impingement group (30:58%).Trends were noted in other muscles with upper trapezius and infraspinatus activating more rapidly and erratically (63:25%; 60:27% respectively), and lower trapezius with less recruitment (13:30%) in the patient group, although these did not quite reach significance. Conclusion: There appears to be some interesting alterations in muscle recruitment patterns in impingement shoulder patients when compared against their own unaffected shoulders and the control group. In particular changes in scapula control (serratus anterior and trapezius) and lateral rotation (infraspinatus), which have direct influence on the sub-acromial space, should be noted. It is still not clear whether these alterations are causative or reactionary, but this finding gives a clear indication to the importance of addressing muscle reeducation as part of a rehabilitation programme in shoulder impingement patients