Shampoo: Preconditioned Stochastic Tensor Optimization

Preconditioned gradient methods are among the most general and powerful tools in optimization. However, preconditioning requires storing and manipulating prohibitively large matrices. We describe and analyze a new structure-aware preconditioning algorithm, called Shampoo, for stochastic optimization over tensor spaces. Shampoo maintains a set of preconditioning matrices, each of which operates on a single dimension, contracting over the remaining dimensions. We establish convergence guarantees in the stochastic convex setting, the proof of which builds upon matrix trace inequalities. Our experiments with state-of-the-art deep learning models show that Shampoo is capable of converging considerably faster than commonly used optimizers. Although it involves a more complex update rule, Shampoo's runtime per step is comparable to that of simple gradient methods such as SGD, AdaGrad, and Adam

Memory-Efficient Adaptive Optimization

Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter, thus introducing significant memory overheads that restrict the size of the model being used as well as the number of examples in a mini-batch. We describe an effective and flexible adaptive optimization method with greatly reduced memory overhead. Our method retains the benefits of per-parameter adaptivity while allowing significantly larger models and batch sizes. We give convergence guarantees for our method, and demonstrate its effectiveness in training very large translation and language models with up to 2-fold speedups compared to the state-of-the-art

Manufacturing System Design for Resilience

AbstractUnexpected disruptive events in manufacturing systems always interrupt normal production conditions and cause production loss. A resilient system should be designed with the capability to suffer minimum production loss during disruptions, and settle itself to the steady state quickly after each disruption. In this paper, we define production loss (PL), throughput settling time (TST), and total underproduction time (TUT) as three metrics to measure system resilience, and use these measures to assist the design of multi-stage reconfigurable manufacturing systems. Numerical case studies are conducted to investigate how the system resilience is affected by different design factors, including system configuration, level of redundancy or flexibility, and buffer capacities

Capturing the design of mechanical components in VLSIs

This paper proposes an approach for implementing mechanical computer-aided design (CAD) systems with the aid of generic, multi-purpose VLSIs, as a partial substitute for software. Although software is not eliminated, a more equitable share of tasks is reached between software and hardware, providing for faster computer execution.Two approaches are presented; the first is based on "wired-in" design rules for every permissible solution, feasible but not practical; the second an approach which is based on a "solution-base" stored in ROMs and an imbedded search algorithm to retrieve the best solution. An appropriate search algorithm, called the virtual graphic method (VGM) has been developed. The VGM solves non-linear equations without performing arithmetic operations, and is therefore fast and easily implemented in hardware such as a VLSI. A flowchart and a conceptual diagram for VLSI implementation are presented.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/28110/1/0000559.pd

Real-time Map-building for Fast Mobile Robot Obstacle Avoidance

ABSTRACT This paper introduces HIMM (histogramic in-motion mapping), a new method for real-time map building with a mobile robot in motion. HIMM represents data in a two-dimensional array (called a histogram gjid) that is updated through rapid continuous sampling of the onboard range sensors during motion. Rapid in-motion sampling results in a statistical map representation that is well-suited to modeling inaccurate and noisy range-sensor data. HIMM is integral part of an obstacle avoidance algorithm and allows the robot to immediately use the mapped information in real-time obstacleavoidance. The benefits of this integrated approach are twofold: (1) quick, accurate mapping; and (2) safe navigation of the robot toward a given target. HIMM has been implemented and tested on a mobile robot. Its dual functionality was demonstrated through numerous tests in which maps of unknown obstacle courses were created, while the robot simultaneously performed real-time obstacle avoidance maneuvers at speeds of up to 0.78m/sec

Aerosol climatology using a tunable spectral variability cloud screening of AERONET data

Can cloud screening of an aerosol data set, affect the aerosol optical thickness (AOT) climatology? Aerosols, humidity and clouds are correlated. Therefore, rigorous cloud screening can systematically bias towards less cloudy conditions, underestimating the average AOT. Here, using AERONET data we show that systematic rejection of variable atmospheric optical conditions can generate such bias in the average AOT. Therefore we recommend (1) to introduce more powerful spectral variability cloud screening and (2) to change the philosophy behind present aerosol climatologies: Instead of systematically rejecting all cloud contaminations, we suggest to intentionally allow the presence of cloud contamination, estimate the statistical impact of the contamination and correct for it. The analysis, applied to 10 AERONET stations with approx. 4 years of data, shows almost no change for Rome (Italy), but up to a change in AOT of 0.12 in Beijing (PRC). Similar technique may be explored for satellite analysis, e.g. MODIS

Sustainable Living Factories for Next Generation Manufacturing

To be profitable and to generate sustainable value for all stakeholders, next generation manufacturers must develop capabilities to rapidly and economically respond to changing market needs while at the same time minimizing adverse impacts on the environment and benefiting society. 6R-based (Reduce, Reuse, Recycle, Recover, Redesign and Remanufacturing) sustainable manufacturing practices enable closed-loop and multi-life cycle material flow; they facilitate producing more sustainable products using manufacturing processes and systems that are more sustainable. Reconfigurable Manufacturing Systems (RMS) and its characteristics of scalability, convertibility, diagnosability, customization, modularity and integrability have emerged as a basis for living factories for next generation manufacturing that can significantly enhance the system sustainability by quickly adjusting system configuration and production processes to meet the market needs, and maintain the system values for generations of products. This paper examines the significance of developing such next generation manufacturing systems as the basis for futuristic sustainable living factories by adapting, integrating and implementing the RMS characteristics with the principles of sustainable manufacturing to achieve value creation for all stakeholders

Realtime curve interpolators

The amount of geometric information that must be transferred between a system and a computerized numerical control system creates a conflict between part precision on the one hand and feedrate fidelity and communications load on the other. This is the motivation for the development of new curve interpolation algorithms for CNC. The interpolation depends on the method of curve representation, i.e. the use of an implicit or a parametric from. Accordingly, the paper presents two realtime interpolation algorithms and compares them with existing interpolators. With the new interpolators, the amount of geometric information transferred from the system to the CNC system is reduced by orders of magnitude. Moreover, the contour errors caused by the new interpolators are much smaller than those caused by conventional interpolators.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/31229/1/0000132.pd