121 research outputs found
Shampoo: Preconditioned Stochastic Tensor Optimization
Preconditioned gradient methods are among the most general and powerful tools
in optimization. However, preconditioning requires storing and manipulating
prohibitively large matrices. We describe and analyze a new structure-aware
preconditioning algorithm, called Shampoo, for stochastic optimization over
tensor spaces. Shampoo maintains a set of preconditioning matrices, each of
which operates on a single dimension, contracting over the remaining
dimensions. We establish convergence guarantees in the stochastic convex
setting, the proof of which builds upon matrix trace inequalities. Our
experiments with state-of-the-art deep learning models show that Shampoo is
capable of converging considerably faster than commonly used optimizers.
Although it involves a more complex update rule, Shampoo's runtime per step is
comparable to that of simple gradient methods such as SGD, AdaGrad, and Adam
Memory-Efficient Adaptive Optimization
Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for
achieving state-of-the-art performance in machine translation and language
modeling. However, these methods maintain second-order statistics for each
parameter, thus introducing significant memory overheads that restrict the size
of the model being used as well as the number of examples in a mini-batch. We
describe an effective and flexible adaptive optimization method with greatly
reduced memory overhead. Our method retains the benefits of per-parameter
adaptivity while allowing significantly larger models and batch sizes. We give
convergence guarantees for our method, and demonstrate its effectiveness in
training very large translation and language models with up to 2-fold speedups
compared to the state-of-the-art
Manufacturing System Design for Resilience
AbstractUnexpected disruptive events in manufacturing systems always interrupt normal production conditions and cause production loss. A resilient system should be designed with the capability to suffer minimum production loss during disruptions, and settle itself to the steady state quickly after each disruption. In this paper, we define production loss (PL), throughput settling time (TST), and total underproduction time (TUT) as three metrics to measure system resilience, and use these measures to assist the design of multi-stage reconfigurable manufacturing systems. Numerical case studies are conducted to investigate how the system resilience is affected by different design factors, including system configuration, level of redundancy or flexibility, and buffer capacities
Capturing the design of mechanical components in VLSIs
This paper proposes an approach for implementing mechanical computer-aided design (CAD) systems with the aid of generic, multi-purpose VLSIs, as a partial substitute for software. Although software is not eliminated, a more equitable share of tasks is reached between software and hardware, providing for faster computer execution.Two approaches are presented; the first is based on "wired-in" design rules for every permissible solution, feasible but not practical; the second an approach which is based on a "solution-base" stored in ROMs and an imbedded search algorithm to retrieve the best solution. An appropriate search algorithm, called the virtual graphic method (VGM) has been developed. The VGM solves non-linear equations without performing arithmetic operations, and is therefore fast and easily implemented in hardware such as a VLSI. A flowchart and a conceptual diagram for VLSI implementation are presented.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/28110/1/0000559.pd
Real-time Map-building for Fast Mobile Robot Obstacle Avoidance
ABSTRACT This paper introduces HIMM (histogramic in-motion mapping), a new method for real-time map building with a mobile robot in motion. HIMM represents data in a two-dimensional array (called a histogram gjid) that is updated through rapid continuous sampling of the onboard range sensors during motion. Rapid in-motion sampling results in a statistical map representation that is well-suited to modeling inaccurate and noisy range-sensor data. HIMM is integral part of an obstacle avoidance algorithm and allows the robot to immediately use the mapped information in real-time obstacleavoidance. The benefits of this integrated approach are twofold: (1) quick, accurate mapping; and (2) safe navigation of the robot toward a given target. HIMM has been implemented and tested on a mobile robot. Its dual functionality was demonstrated through numerous tests in which maps of unknown obstacle courses were created, while the robot simultaneously performed real-time obstacle avoidance maneuvers at speeds of up to 0.78m/sec
Aerosol climatology using a tunable spectral variability cloud screening of AERONET data
Can cloud screening of an aerosol data set, affect the aerosol optical thickness (AOT) climatology? Aerosols, humidity and clouds are correlated. Therefore, rigorous cloud screening can systematically bias towards less cloudy conditions, underestimating the average AOT. Here, using AERONET data we show that systematic rejection of variable atmospheric optical conditions can generate such bias in the average AOT. Therefore we recommend (1) to introduce more powerful spectral variability cloud screening and (2) to change the philosophy behind present aerosol climatologies: Instead of systematically rejecting all cloud contaminations, we suggest to intentionally allow the presence of cloud contamination, estimate the statistical impact of the contamination and correct for it. The analysis, applied to 10 AERONET stations with approx. 4 years of data, shows almost no change for Rome (Italy), but up to a change in AOT of 0.12 in Beijing (PRC). Similar technique may be explored for satellite analysis, e.g. MODIS
Sustainable Living Factories for Next Generation Manufacturing
To be profitable and to generate sustainable value for all stakeholders, next generation manufacturers must develop capabilities to rapidly and economically respond to changing market needs while at the same time minimizing adverse impacts on the environment and benefiting society. 6R-based (Reduce, Reuse, Recycle, Recover, Redesign and Remanufacturing) sustainable manufacturing practices enable closed-loop and multi-life cycle material flow; they facilitate producing more sustainable products using manufacturing processes and systems that are more sustainable. Reconfigurable Manufacturing Systems (RMS) and its characteristics of scalability, convertibility, diagnosability, customization, modularity and integrability have emerged as a basis for living factories for next generation manufacturing that can significantly enhance the system sustainability by quickly adjusting system configuration and production processes to meet the market needs, and maintain the system values for generations of products. This paper examines the significance of developing such next generation manufacturing systems as the basis for futuristic sustainable living factories by adapting, integrating and implementing the RMS characteristics with the principles of sustainable manufacturing to achieve value creation for all stakeholders
Realtime curve interpolators
The amount of geometric information that must be transferred between a system and a computerized numerical control system creates a conflict between part precision on the one hand and feedrate fidelity and communications load on the other. This is the motivation for the development of new curve interpolation algorithms for CNC. The interpolation depends on the method of curve representation, i.e. the use of an implicit or a parametric from. Accordingly, the paper presents two realtime interpolation algorithms and compares them with existing interpolators. With the new interpolators, the amount of geometric information transferred from the system to the CNC system is reduced by orders of magnitude. Moreover, the contour errors caused by the new interpolators are much smaller than those caused by conventional interpolators.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/31229/1/0000132.pd
- …