Replay as wavefronts and theta sequences as bump oscillations in a grid cell attractor network.

Grid cells fire in sequences that represent rapid trajectories in space. During locomotion, theta sequences encode sweeps in position starting slightly behind the animal and ending ahead of it. During quiescence and slow wave sleep, bouts of synchronized activity represent long trajectories called replays, which are well-established in place cells and have been recently reported in grid cells. Theta sequences and replay are hypothesized to facilitate many cognitive functions, but their underlying mechanisms are unknown. One mechanism proposed for grid cell formation is the continuous attractor network. We demonstrate that this established architecture naturally produces theta sequences and replay as distinct consequences of modulating external input. Driving inhibitory interneurons at the theta frequency causes attractor bumps to oscillate in speed and size, which gives rise to theta sequences and phase precession, respectively. Decreasing input drive to all neurons produces traveling wavefronts of activity that are decoded as replays

Reinforcement Learning for Racecar Control

This thesis investigates the use of reinforcement learning to learn to drive a racecar in the simulated environment of the Robot Automobile Racing Simulator. Real-life race driving is known to be difficult for humans, and expert human drivers use complex sequences of actions. There are a large number of variables, some of which change stochastically and all of which may affect the outcome. This makes driving a promising domain for testing and developing Machine Learning techniques that have the potential to be robust enough to work in the real world. Therefore the principles of the algorithms from this work may be applicable to a range of problems. The investigation starts by finding a suitable data structure to represent the information learnt. This is tested using supervised learning. Reinforcement learning is added and roughly tuned, and the supervised learning is then removed. A simple tabular representation is found satisfactory, and this avoids difficulties with more complex methods and allows the investigation to concentrate on the essentials of learning. Various reward sources are tested and a combination of three are found to produce the best performance. Exploration of the problem space is investigated. Results show exploration is essential but controlling how much is done is also important. It turns out the learning episodes need to be very long and because of this the task needs to be treated as continuous by using discounting to limit the size of the variables stored. Eligibility traces are used with success to make the learning more efficient. The tabular representation is made more compact by hashing and more accurate by using smaller buckets. This slows the learning but produces better driving. The improvement given by a rough form of generalisation indicates the replacement of the tabular method by a function approximator is warranted. These results show reinforcement learning can work within the Robot Automobile Racing Simulator, and lay the foundations for building a more efficient and competitive agent

Learning to Race through Coordinate Descent Bayesian Optimisation

In the automation of many kinds of processes, the observable outcome can often be described as the combined effect of an entire sequence of actions, or controls, applied throughout its execution. In these cases, strategies to optimise control policies for individual stages of the process might not be applicable, and instead the whole policy might have to be optimised at once. On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system. We consider the problem of optimising control policies to allow a robot to complete a given race track within a minimum amount of time. We assume that the robot has no prior information about the track or its own dynamical model, just an initial valid driving example. Localisation is only applied to monitor the robot and to provide an indication of its position along the track's centre axis. We propose a method for finding a policy that minimises the time per lap while keeping the vehicle on the track using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert space. We apply an algorithm to search more efficiently over high-dimensional policy-parameter spaces with BO, by iterating over each dimension individually, in a sequential coordinate descent-like scheme. Experiments demonstrate the performance of the algorithm against other methods in a simulated car racing environment.Comment: Accepted as conference paper for the 2018 IEEE International Conference on Robotics and Automation (ICRA

AlphaPilot: Autonomous Drone Racing

This paper presents a novel system for autonomous, vision-based drone racing combining learned data abstraction, nonlinear filtering, and time-optimal trajectory planning. The system has successfully been deployed at the first autonomous drone racing world championship: the 2019 AlphaPilot Challenge. Contrary to traditional drone racing systems, which only detect the next gate, our approach makes use of any visible gate and takes advantage of multiple, simultaneous gate detections to compensate for drift in the state estimate and build a global map of the gates. The global map and drift-compensated state estimate allow the drone to navigate through the race course even when the gates are not immediately visible and further enable to plan a near time-optimal path through the race course in real time based on approximate drone dynamics. The proposed system has been demonstrated to successfully guide the drone through tight race courses reaching speeds up to 8m/s and ranked second at the 2019 AlphaPilot Challenge.Comment: Accepted at Robotics: Science and Systems 2020, associated video at https://youtu.be/DGjwm5PZQT

A Sequential Two-Step Algorithm for Fast Generation of Vehicle Racing Trajectories

The problem of maneuvering a vehicle through a race course in minimum time requires computation of both longitudinal (brake and throttle) and lateral (steering wheel) control inputs. Unfortunately, solving the resulting nonlinear optimal control problem is typically computationally expensive and infeasible for real-time trajectory planning. This paper presents an iterative algorithm that divides the path generation task into two sequential subproblems that are significantly easier to solve. Given an initial path through the race track, the algorithm runs a forward-backward integration scheme to determine the minimum-time longitudinal speed profile, subject to tire friction constraints. With this fixed speed profile, the algorithm updates the vehicle's path by solving a convex optimization problem that minimizes the resulting path curvature while staying within track boundaries and obeying affine, time-varying vehicle dynamics constraints. This two-step process is repeated iteratively until the predicted lap time no longer improves. While providing no guarantees of convergence or a globally optimal solution, the approach performs very well when validated on the Thunderhill Raceway course in Willows, CA. The predicted lap time converges after four to five iterations, with each iteration over the full 4.5 km race course requiring only thirty seconds of computation time on a laptop computer. The resulting trajectory is experimentally driven at the race circuit with an autonomous Audi TTS test vehicle, and the resulting lap time and racing line is comparable to both a nonlinear gradient descent solution and a trajectory recorded from a professional racecar driver. The experimental results indicate that the proposed method is a viable option for online trajectory planning in the near future

Model predictive driving simulator motion cueing algorithm with actuator-based constraints

The simulator motion cueing problem has been considered extensively in the literature; approaches based on linear filtering and optimal control have been presented and shown to perform reasonably well. More recently, model predictive control (MPC) has been considered as a variant of the optimal control approach; MPC is perhaps an obvious candidate for motion cueing due to its ability to deal with constraints, in this case the platform workspace boundary. This paper presents an MPC-based cueing algorithm that, unlike other algorithms, uses the actuator positions and velocities as the constraints. The result is a cueing algorithm that can make better use of the platform workspace whilst ensuring that its bounds are never exceeded. The algorithm is shown to perform well against the classical cueing algorithm and an algorithm previously proposed by the authors, both in simulation and in tests with human drivers

RUNNER\u27S LOG AND PREDICTIVE PERFORMANCE ANALYTICS

This report, prepared for the Worcester Polytechnic Institute, describes the development of a running log application and the development and analysis of a data-centric approach to running performance prediction. The java application incorporated common UI principles as well as a community aspect to facilitate and encourage its use. The data-centric predictive model was developed by analyzing individuals who have performed similarly to the input. As tested with 1148 male track performances and 1265 female track performances, the data-centric approach provided predictions with an average error of 3.05 percent for men and 3.63 percent for women. These errors are approximately 9 percent and 20 percent lower, respectively, than the leading Purdy Points model

A comparison of completion times between a 1.5-mile run on an indoor track and treadmill in physically active individuals.

The purpose of this study was to compare the 1.5-mile run test performed on an indoor track to the 1.5-mile run test performed on a treadmill in a group of physically active adults 19 to 30 years of age. The study compared completion times on both tests, which were conducted within a two-week period at least seven days apart. A total of 30 subjects (n = 30) consisting of males (n = 15) and females (n = 15) participated in this study. On both the indoor track and treadmill runs, subjects completed 1.5 miles as fast as possible and completion times were recorded. Times were called out to subjects every quarter mile during each run. On the indoor track run, 18 laps on the inside lane was equivalent to 1.5 miles. On the treadmill run, subjects were responsible for adjusting their own speed using the treadmill's speed control. Findings and Conclusions: Data was analyzed using SAS (version 9.1). Differences between independent means were compared by two-tailed t-tests. Differences between dependent means were compared by two-tailed paired t-tests. Differences between proportions were compared by two-tailed Fisher's exact tests. Correlation between two variables was determined using Pearson's coefficient (one continuous variable and one categorical variable) or the phi coefficient or Fisher's exact test (two categorical variables). Multiple regression was used to determine the best-fitting model to predict treadmill times. P-values less than .05 indicated significant differences as well as collinearity. This study found no significant difference in completion times on a 1.5-mile run performed on a track versus a treadmill for both genders (p = .122), males only (p = .329), and females only (p = .223). These findings demonstrate that using a treadmill to conduct a 1.5-mile run test is an appropriate method that can be used to estimate cardiorespiratory endurance in young, physically active adults. Additionally, lifting weights was found to be a better predictor of treadmill completion time compared to both regular aerobic exercise and treadmill experience