6 research outputs found
Rover: An online Spark SQL tuning service via generalized transfer learning
Distributed data analytic engines like Spark are common choices to process
massive data in industry. However, the performance of Spark SQL highly depends
on the choice of configurations, where the optimal ones vary with the executed
workloads. Among various alternatives for Spark SQL tuning, Bayesian
optimization (BO) is a popular framework that finds near-optimal configurations
given sufficient budget, but it suffers from the re-optimization issue and is
not practical in real production. When applying transfer learning to accelerate
the tuning process, we notice two domain-specific challenges: 1) most previous
work focus on transferring tuning history, while expert knowledge from Spark
engineers is of great potential to improve the tuning performance but is not
well studied so far; 2) history tasks should be carefully utilized, where using
dissimilar ones lead to a deteriorated performance in production. In this
paper, we present Rover, a deployed online Spark SQL tuning service for
efficient and safe search on industrial workloads. To address the challenges,
we propose generalized transfer learning to boost the tuning performance based
on external knowledge, including expert-assisted Bayesian optimization and
controlled history transfer. Experiments on public benchmarks and real-world
tasks show the superiority of Rover over competitive baselines. Notably, Rover
saves an average of 50.1% of the memory cost on 12k real-world Spark SQL tasks
in 20 iterations, among which 76.2% of the tasks achieve a significant memory
reduction of over 60%.Comment: Accepted by KDD 202
Study on Thermal Coupling Characteristics of Constrained Blades Based on Spin Softening
The effects of spin softening and thermal shock had important influence upon the structural stability of blades in rotary machinery. Based on the theories of rotor dynamics and thermodynamics, a dynamic model of rotating blade was built. Considering the effect of spin softening, the research on vibration characteristics of high speed rotating blades was carried out under the centrifugal field and thermal coupling. The results had demonstrated that frequency of blade vibration increases with rising rotating velocity due to the effect of centrifugal force, whilst the frequency of all orders declined with the influence of thermal coupling. The conclusion derived from this paper had both theoretical and empirical value on retrofitting, optimal-designing, as well as engineering application for high speed rotating blades
Thermal Error Measurement and Analysis of Vertical Machining Center
In this paper, the thermal calculation and thermal-structural coupling analysis of the vertical machining center are carried out to obtain the theoretical values of thermal generation and thermal displacement. Combined with the thermal structure analysis of vertical machining center, sensors are placed at the key positions with large thermal displacement to measure and analyze the thermal error data, and the law of temperature and thermal deformation of machining center is found out. This research provides theoretical basis for thermal error compensation
Thermal Error Measurement and Analysis of Vertical Machining Center
In this paper, the thermal calculation and thermal-structural coupling analysis of the vertical machining center are carried out to obtain the theoretical values of thermal generation and thermal displacement. Combined with the thermal structure analysis of vertical machining center, sensors are placed at the key positions with large thermal displacement to measure and analyze the thermal error data, and the law of temperature and thermal deformation of machining center is found out. This research provides theoretical basis for thermal error compensation
An Integrated Condition Monitoring Method for Rotating Machinery Based on Optimum Healthy State
The degradation of a machine is nonlinear, which brings challenges to its performance assessment during condition monitoring, especially when there is a run-in period. Technically, the quantification of mechanical degradation is to define a distance metric from a health baseline. This paper develops an integrated condition monitoring scheme, where the degradation evaluation and fault diagnosis are combined by using one technical framework. Specifically, an optimum healthy state (OHS) is determined based on the clustering center of the self-organizing map (SOM) neural network instead of the commonly used initial working state. Then, the distance metric deviating from the OHS is defined as a health index, where the perceptual vibration hashing is improved to make it more sensitive to degradation. Visualized fault diagnosis is carried out by the SOM when the health index exceeds the preset threshold. Two cases with experiments are conducted to demonstrate the accuracy and robustness of the proposed method