Optimizing the Performance of Robotic Mobile Fulfillment Systems

A robotic mobile fulfillment system is a novel type of automated part-to-picker material handling system. In this type of system, robots transport mobile shelves, called pods, containing items between the storage area and the workstations. It is well suited to e-commerce, due to its modularity and it's ability to adapt to changing orders patterns. Robots can nearly instantaneously switch between inbound and outbound tasks, pods can be continually repositioned to allow for automatic sorting of the inventory, pods can contain many different types of items, and unloaded robots can drive underneath pods, allowing them to use completely different routes than loaded robots. This thesis studies the performance of robotic mobile fulfillment systems by solving decision problems related to warehouse design, inventory and resource allocation, and real-time operations. For warehouse design, a new queueing network is developed that incorporates realistic robot movement, storage zones, and multi-line orders. For inventory allocation, we develop a new type of queueing network, the cross-class matching multi-class semi-open queueing network, which can be applied to other systems as well. Resource (re)allocation is modeled by combining queueing networks with Markov decision processes while including time-varying demand. This model compares benchmark policies from practice wit

Structural Properties of Optimal Fidelity Selection Policies for Human-in-the-loop Queues

We study optimal fidelity selection for a human operator servicing a queue of homogeneous tasks. The agent can service a task with a normal or high fidelity level, where fidelity refers to the degree of exactness and precision while servicing the task. Therefore, high-fidelity servicing results in higher-quality service but leads to larger service times and increased operator tiredness. We treat the cognitive state of the human operator as a lumped parameter that captures psychological factors such as workload and fatigue. The service time distribution of the human operator depends on her cognitive dynamics and the fidelity level selected for servicing the task. Her cognitive dynamics evolve as a Markov chain in which the cognitive state increases with high probability whenever she is busy and decreases while resting. The tasks arrive according to a Poisson process and the operator is penalized at a fixed rate for each task waiting in the queue. We address the trade-off between high-quality service of the task and consequent penalty due to subsequent increase in queue length using a discrete-time Semi-Markov Decision Process (SMDP) framework. We numerically determine an optimal policy and the corresponding optimal value function. Finally, we establish the structural properties of an optimal fidelity policy and provide conditions under which the optimal policy is a threshold-based policy

Variation in Lifting Behavior During a Highly Repetitive Manual Material Task

Presenters will provide

Decision-making and problem-solving methods in automation technology

The state of the art in the automation of decision making and problem solving is reviewed. The information upon which the report is based was derived from literature searches, visits to university and government laboratories performing basic research in the area, and a 1980 Langley Research Center sponsored conferences on the subject. It is the contention of the authors that the technology in this area is being generated by research primarily in the three disciplines of Artificial Intelligence, Control Theory, and Operations Research. Under the assumption that the state of the art in decision making and problem solving is reflected in the problems being solved, specific problems and methods of their solution are often discussed to elucidate particular aspects of the subject. Synopses of the following major topic areas comprise most of the report: (1) detection and recognition; (2) planning; and scheduling; (3) learning; (4) theorem proving; (5) distributed systems; (6) knowledge bases; (7) search; (8) heuristics; and (9) evolutionary programming

A Novel Approach to Analyze Inventory Allocation Decisions in Robotic Mobile Fulfillment Systems

The Robotic Mobile Fulfillment System is a newly developed automated, parts-to-picker material handling system. Storage shelves, also known as inventory pods, are moved by robots between the storage area and the workstations, which means that they can be continually repositioned during operations. This paper develops a queueing model for optimizing three key decision variables: (1) the number of pods per product (2) the ratio of the number of pick to the number of replenishment stations, and (3) the replenishment level per pod. We show that too few or too many pods per product leads to unnecessarily long order throughput times, that the ratio of the number of pick to the number of replenishment stations can be optimized for order throughput time, and that waiting to replenish until a pod is completely empty can severely decrease throughput performance

Tackling Unbounded State Spaces in Continuing Task Reinforcement Learning

While deep reinforcement learning (RL) algorithms have been successfully applied to many tasks, their inability to extrapolate and strong reliance on episodic resets inhibits their applicability to many real-world settings. For instance, in stochastic queueing problems, the state space can be unbounded and the agent may have to learn online without the system ever being reset to states the agent has seen before. In such settings, we show that deep RL agents can diverge into unseen states from which they can never recover due to the lack of resets, especially in highly stochastic environments. Towards overcoming this divergence, we introduce a Lyapunov-inspired reward shaping approach that encourages the agent to first learn to be stable (i.e. to achieve bounded cost) and then to learn to be optimal. We theoretically show that our reward shaping technique reduces the rate of divergence of the agent and empirically find that it prevents it. We further combine our reward shaping approach with a weight annealing scheme that gradually introduces optimality and log-transform of state inputs, and find that these techniques enable deep RL algorithms to learn high performing policies when learning online in unbounded state space domains

Inventory Allocation in Robotic Mobile Fulfillment Systems

A Robotic Mobile Fulfillment System is a recently developed automated, parts-to- picker material handling system. Robots can move storage shelves, also known as inventory pods, between the storage area and the workstations and can continually reposition them during operations. This paper shows how to optimize three key decision variables: (1) the number of pods per product (2) the ratio of the number of pick stations to replenishment stations, and (3) the replenishment level per pod. Our results show that throughput performance improves substantially when inventory is spread across multiple pods, when an optimum ratio between the number of pick stations to replenishment stations is achieved and when a pod is replenished before it is completely empty. This paper contributes methodologically by introducing a new type of Semi-Open Queueing Networks (SOQN): cross-class matching multi- class SOQN, by deriving necessary stability conditions, and by introducing a novel interpretation of the classes