Research on Standardization of Train Dispatcher Operations

Based on the actual operation of daily train reception and departure at stations, the author conducted research on the standardized operation evaluation of train dispatchers. Through the research of the project, a 1:1 station simulation training system is simulated, using technologies such as eye movement recognition, limb motion recognition, and speech recognition to record and analyze the “eye, finger, and mouth call” operation information of train dispatchers. Combined with the operation analysis of the CTC system, the standardization of train dispatchers’ work is achieved

Evolving developmental, recurrent and convolutional neural networks for deliberate motion planning in sparse reward tasks

Motion planning algorithms have seen a diverse set of approaches in a variety of disciplines. In the domain of artificial evolutionary systems, motion planning has been included in models to achieve sophisticated deliberate behaviours. These algorithms rely on fixed rules or little evolutionary influence which compels behaviours to conform within those specific policies, rather than allowing the model to establish its own specialised behaviour. In order to further these models, the constraints imposed by planning algorithms must be removed to grant greater evolutionary control over behaviours. That is the focus of this thesis. An examination of prevailing neuroevolution methods led to the use of two distinct approaches, NEAT and HyperNEAT. Both were used to gain an understanding of the components necessary to create neuroevolution planning. The findings accumulated in the formation of a novel convolutional neural network architecture with a recurrent convolution process. The architecture’s goal was to iteratively disperse local activations to greater regions of the feature space. Experimentation showed significantly improved robustness over contemporary neuroevolution techniques as well as an efficiency increase over a static rule set. Greater evolutionary responsibility is given to the model with multiple network combinations; all of which continually demonstrated the necessary behaviours. In comparison, these behaviours were shown to be difficult to achieve in a state-of-the-art deep convolutional network. Finally, the unique use of recurrent convolution is relocated to a larger convolutional architecture on an established benchmarking platform. Performance improvements are seen on a number of domains which illustrates that this recurrent mechanism can be exploited in alternative areas outside of planning. By presenting a viable neuroevolution method for motion planning a potential emerges for further systems to adopt and examine the capability of this work in prospective domains, as well as further avenues of experimentation in convolutional architectures

Solving the Order Batching and Sequencing Problem using Deep Reinforcement Learning

In e-commerce markets, on time delivery is of great importance to customer satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL) approach for deciding how and when orders should be batched and picked in a warehouse to minimize the number of tardy orders. In particular, the technique facilitates making decisions on whether an order should be picked individually (pick-by-order) or picked in a batch with other orders (pick-by-batch), and if so with which other orders. We approach the problem by formulating it as a semi-Markov decision process and develop a vector-based state representation that includes the characteristics of the warehouse system. This allows us to create a deep reinforcement learning solution that learns a strategy by interacting with the environment and solve the problem with a proximal policy optimization algorithm. We evaluate the performance of the proposed DRL approach by comparing it with several batching and sequencing heuristics in different problem settings. The results show that the DRL approach is able to develop a strategy that produces consistent, good solutions and performs better than the proposed heuristics.Comment: Preprin

A literature review of Artificial Intelligence applications in railway systems

Nowadays it is widely accepted that Artificial Intelligence (AI) is significantly influencing a large number of domains, including railways. In this paper, we present a systematic literature review of the current state-of-the-art of AI in railway transport. In particular, we analysed and discussed papers from a holistic railway perspective, covering sub-domains such as maintenance and inspection, planning and management, safety and security, autonomous driving and control, revenue management, transport policy, and passenger mobility. This review makes an initial step towards shaping the role of AI in future railways and provides a summary of the current focuses of AI research connected to rail transport. We reviewed about 139 scientific papers covering the period from 2010 to December 2020. We found that the major research efforts have been put in AI for rail maintenance and inspection, while very limited or no research has been found on AI for rail transport policy and revenue management. The remaining sub-domains received mild to moderate attention. AI applications are promising and tend to act as a game-changer in tackling multiple railway challenges. However, at the moment, AI research in railways is still mostly at its early stages. Future research can be expected towards developing advanced combined AI applications (e.g. with optimization), using AI in decision making, dealing with uncertainty and tackling newly rising cybersecurity challenges

Evolving Robust, Deliberate Motion Planning With a Shallow Convolutional Neural Network

Deep Convolutional Neural Networks (ConvNets) have seen great success on machine learning tasks in recent years but have shown difficulty with tasks that require long-term deliberative planning. Whereas, purpose-built hybrid network architectures have been able to solve increasingly challenging deliberate tasks in two-dimensional and three-dimensional artificial worlds. Starting from a purpose-built network and transitioning to a general architecture, like a deep ConvNet, may retain long-term deliberative planning while allowing greater flexibility in the task domain. This paper employs a standard genetic algorithm (GA) to train the weights of a ConvNet with a recurrent 3x3 filter to produce robust and deliberative motion planning. This technique resulted in an average of 98.97% completion over 10,000 runs in the most difficult deliberate task. This demonstrates that a shallow ConvNet with recurrent connections is capable of producing deliberate and robust motion planning. Further, the evolved ConvNet exhibits superior motion planning in the most challenging environments, when compared to the previous taskspecific motion-planning network

A Search For Principles of Basal Ganglia Function

The basal ganglia are a group of subcortical nuclei that contain about 100 million neurons in humans. Different modes of basal ganglia dysfunction lead to Parkinson's disease and Huntington's disease, which have debilitating motor and cognitive symptoms. However, despite intensive study, both the internal computational mechanisms of the basal ganglia, and their contribution to normal brain function, have been elusive. The goal of this thesis is to identify basic principles that underlie basal ganglia function, with a focus on signal representation, computation, dynamics, and plasticity. This process begins with a review of two current hypotheses of normal basal ganglia function, one being that they automatically select actions on the basis of past reinforcement, and the other that they compress cortical signals that tend to occur in conjunction with reinforcement. It is argued that a wide range of experimental data are consistent with these mechanisms operating in series, and that in this configuration, compression makes selection practical in natural environments. Although experimental work is outside the present scope, an experimental means of testing this proposal in the future is suggested. The remainder of the thesis builds on Eliasmith & Anderson's Neural Engineering Framework (NEF), which provides an integrated theoretical account of computation, representation, and dynamics in large neural circuits. The NEF provides considerable insight into basal ganglia function, but its explanatory power is potentially limited by two assumptions that the basal ganglia violate. First, like most large-network models, the NEF assumes that neurons integrate multiple synaptic inputs in a linear manner. However, synaptic integration in the basal ganglia is nonlinear in several respects. Three modes of nonlinearity are examined, including nonlinear interactions between dendritic branches, nonlinear integration within terminal branches, and nonlinear conductance-current relationships. The first mode is shown to affect neuron tuning. The other two modes are shown to enable alternative computational mechanisms that facilitate learning, and make computation more flexible, respectively. Secondly, while the NEF assumes that the feedforward dynamics of individual neurons are dominated by the dynamics of post-synaptic current, many basal ganglia neurons also exhibit prominent spike-generation dynamics, including adaptation, bursting, and hysterses. Of these, it is shown that the NEF theory of network dynamics applies fairly directly to certain cases of firing-rate adaptation. However, more complex dynamics, including nonlinear dynamics that are diverse across a population, can be described using the NEF equations for representation. In particular, a neuron's response can be characterized in terms of a more complex function that extends over both present and past inputs. It is therefore straightforward to apply NEF methods to interpret the effects of complex cell dynamics at the network level. The role of spike timing in basal ganglia function is also examined. Although the basal ganglia have been interpreted in the past to perform computations on the basis of mean firing rates (over windows of tens or hundreds of milliseconds) it has recently become clear that patterns of spikes on finer timescales are also functionally relevant. Past work has shown that precise spike times in sensory systems contain stimulus-related information, but there has been little study of how post-synaptic neurons might use this information. It is shown that essentially any neuron can use this information to perform flexible computations, and that these computations do not require spike timing that is very precise. As a consequence, irregular and highly-variable firing patterns can drive behaviour with which they have no detectable correlation. Most of the projection neurons in the basal ganglia are inhibitory, and the effect of one nucleus on another is classically interpreted as subtractive or divisive. Theoretically, very flexible computations can be performed within a projection if each presynaptic neuron can both excite and inhibit its targets, but this is hardly ever the case physiologically. However, it is shown here that equivalent computational flexibility is supported by inhibitory projections in the basal ganglia, as a simple consequence of inhibitory collaterals in the target nuclei. Finally, the relationship between population coding and synaptic plasticity is discussed. It is shown that Hebbian plasticity, in conjunction with lateral connections, determines both the dimension of the population code and the tuning of neuron responses within the coded space. These results permit a straightforward interpretation of the effects of synaptic plasticity on information processing at the network level. Together with the NEF, these new results provide a rich set of theoretical principles through which the dominant physiological factors that affect basal ganglia function can be more clearly understood