Three-state disk model for high quality and energy efficient streaming media servers

Energy conservation and emission reduction is an increasingly prominent and global issue in green computing. Among the various components of a streaming media server, the storage system is the biggest power consumer. In this paper, a Three-State Disk Model (3SDM) is proposed to conserve energy for streaming media servers without losing quality. According to the load threshold, the disks are dynamically divided into three states: overload, normal and standby. With the requests arriving and departing, the disk state transition among these three states. The purpose of 3SDM is to skew the load among the disks to achieve high quality and energy efficiency for streaming media applications. The load of disks in overload state will move to disks in normal state to improve the quality of service (QoS) level. The load of disks in normal state will be packed together to switch some disks into standby state to save energy. The key problem here is to identify the blocks that need migrating among disks. A sliding window replacement (SWR) algorithm is developed for this purpose, which calculates the block weight based on the request frequency falling within the window of a block. Employing a validated simulator, this paper evaluates the SWR algorithm for conventional disks based on the proposed 3SDM model. The results show that this scheme is able to yield energy efficient streaming media servers

Priori information and sliding window based prediction algorithm for energy-efficient storage systems in cloud

One of the major challenges in cloud computing and data centers is the energy conservation and emission reduction. Accurate prediction algorithms are essential for building energy efficient storage systems in cloud computing. In this paper, we first propose a Three-State Disk Model (3SDM), which can describe the service quality and energy consumption states of a storage system accurately. Based on this model, we develop a method for achieving energy conservation without losing quality by skewing the workload among the disks to transmit the disk states of a storage system. The efficiency of this method is highly dependent on the accuracy of the information predicting the blocks to be accessed and the blocks not be accessed in the near future. We develop a priori information and sliding window based prediction (PISWP) algorithm by taking advantage of the priori information about human behavior and selecting suitable size of sliding window. The PISWP method targets at streaming media applications, but we also check its efficiency on other two applications, news in webpage and new tool released. Disksim, an established storage system simulator, is applied in our experiments to verify the effect of our method for various users’ traces. The results show that this prediction method can bring a high degree energy saving for storage systems in cloud computing environment

Searching Transferable Mixed-Precision Quantization Policy through Large Margin Regularization

Mixed-precision quantization (MPQ) suffers from time-consuming policy search process (i.e., the bit-width assignment for each layer) on large-scale datasets (e.g., ISLVRC-2012), which heavily limits its practicability in real-world deployment scenarios. In this paper, we propose to search the effective MPQ policy by using a small proxy dataset for the model trained on a large-scale one. It breaks the routine that requires a consistent dataset at model training and MPQ policy search time, which can improve the MPQ searching efficiency significantly. However, the discrepant data distributions bring difficulties in searching for such a transferable MPQ policy. Motivated by the observation that quantization narrows the class margin and blurs the decision boundary, we search the policy that guarantees a general and dataset-independent property: discriminability of feature representations. Namely, we seek the policy that can robustly keep the intra-class compactness and inter-class separation. Our method offers several advantages, i.e., high proxy data utilization, no extra hyper-parameter tuning for approximating the relationship between full-precision and quantized model and high searching efficiency. We search high-quality MPQ policies with the proxy dataset that has only 4% of the data scale compared to the large-scale target dataset, achieving the same accuracy as searching directly on the latter, and improving the MPQ searching efficiency by up to 300 times

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization

Exemplar-based colorization approaches rely on reference image to provide plausible colors for target gray-scale image. The key and difficulty of exemplar-based colorization is to establish an accurate correspondence between these two images. Previous approaches have attempted to construct such a correspondence but are faced with two obstacles. First, using luminance channels for the calculation of correspondence is inaccurate. Second, the dense correspondence they built introduces wrong matching results and increases the computation burden. To address these two problems, we propose Semantic-Sparse Colorization Network (SSCN) to transfer both the global image style and detailed semantic-related colors to the gray-scale image in a coarse-to-fine manner. Our network can perfectly balance the global and local colors while alleviating the ambiguous matching problem. Experiments show that our method outperforms existing methods in both quantitative and qualitative evaluation and achieves state-of-the-art performance.Comment: Accepted by ECCV2022; 14 pages, 10 figure

SALI: A Scalable Adaptive Learned Index Framework based on Probability Models

The growth in data storage capacity and the increasing demands for high performance have created several challenges for concurrent indexing structures. One promising solution is learned indexes, which use a learning-based approach to fit the distribution of stored data and predictively locate target keys, significantly improving lookup performance. Despite their advantages, prevailing learned indexes exhibit constraints and encounter issues of scalability on multi-core data storage. This paper introduces SALI, the Scalable Adaptive Learned Index framework, which incorporates two strategies aimed at achieving high scalability, improving efficiency, and enhancing the robustness of the learned index. Firstly, a set of node-evolving strategies is defined to enable the learned index to adapt to various workload skews and enhance its concurrency performance in such scenarios. Secondly, a lightweight strategy is proposed to maintain statistical information within the learned index, with the goal of further improving the scalability of the index. Furthermore, to validate their effectiveness, SALI applied the two strategies mentioned above to the learned index structure that utilizes fine-grained write locks, known as LIPP. The experimental results have demonstrated that SALI significantly enhances the insertion throughput with 64 threads by an average of 2.04x compared to the second-best learned index. Furthermore, SALI accomplishes a lookup throughput similar to that of LIPP+.Comment: Accepted by Conference SIGMOD 24, June 09-15, 2024, Santiago, Chil

Origin and Characteristics of the Crude Oils and Condensates in the Callovian-Oxfordian Carbonate Reservoirs of the Amu Darya Right Bank Block, Turkmenistan

AbstractThe Amu Darya Right Bank Block is located northeast of the Amu Darya basin, a large petroliferous sedimentary basin, with abundant natural gas resources in carbonate rocks under the ultra-thick gypsum-salt layer. Oil fields producing crude oils have recently been found around large gas fields. Unraveling the origins of the crude oils is crucial for effective petroleum exploration and exploitation. The origin of gas condensates and crude oils was unraveled through the use of comprehensively analytical and interpretative geochemical approaches. Based on oil-source correlation, the reservoir forming process has been restored. The bulk geochemical parameters of the local source rocks of the ADRBB indicated that the local sources have hydrocarbon generation and accumulation potential. The middle-lower Jurassic coal-bearing mudstone is gas prone, while the mudstone of the Callovian-Oxfordian gap layer is oil prone, and the organic matter type of Callovian-Oxfordian carbonate rocks is the mixed type between the two previous source rocks. The interpretation schemes for compositions of n-alkanes, pristane and phytane, C27–C28–C29 sterane distributions, C19+C20–C21–C23 tricyclic terpane distributions, extended tricyclic terpane ratio, and δ13C indicated that crude oil is likely from marine organic matter, while condensates mainly originate from terrestrial organic matter. However, from the perspective of the 18α-trisnorneohopane/17α-trisnorhopane and isomerization ratio of C29 sterane, condensates are too mature to have originated in the local source rocks of the ADRBB, whose maturity is well comparable with that of crude oils. The geochemical, geologic, and tectonic evolutions collectively indicate that the crude oils were most likely generated and migrated from the relatively shallow, lowly mature gap layer and Callovian-Oxfordian carbonate rocks of the ADRBB, while the condensates mostly originated from the relatively deep and highly mature middle-lower coal-bearing mudstone and Callovian-Oxfordian carbonate rocks in the Murgab depression in the southeast of the basin. Basement faults are the key factors affecting the types of oil and gas reservoirs. During the periods of oil and gas migration, traps with basement faults mainly captured natural gas and condensates and traps without basement faults were enriched with crude oils generated from local source rocks

ALOR: Adaptive Layout Optimization of Raft Groups for Heterogeneous Distributed Key-Value Stores

International audienceMany distributed key-value storage systems employ the simple and effective Raft protocol to ensure data consistency. They usually assume a homogeneous node hardware configuration for the underlying cluster and thus adopt even data distribution schemes. However, today’s distributed systems tend to be heterogeneous in nodes’ I/O devices due to the regular worn I/O device replacement and the emergence of expensive new storage media (e.g., non-volatile memory). In this paper, we propose a new data layout scheme called Adaptive Layout Optimization of Raft groups (ALOR), considering the hardware heterogeneity of the cluster. ALOR aims to optimize the data layout of Raft groups to achieve a better practical load balance, which leads to higher performance. ALOR consists of two components: leader migration in Raft groups and skewed data layout based on cold data migration. We conducted experiments on a practical heterogeneous cluster, and the results indicate that, on average, ALOR improves throughput by 36.89%, reduces latency and 99th percentile tail latency by 24.54% and 21.32%, respectively

Efficient subgraph matching on large RDF graphs using MapReduce

Abstract With the popularity of knowledge graphs growing rapidly, large amounts of RDF graphs have been released, which raises the need for addressing the challenge of distributed subgraph matching queries. In this paper, we propose an efficient distributed method to answer subgraph matching queries on big RDF graphs using MapReduce. In our method, query graphs are decomposed into a set of stars that utilize the semantic and structural information embedded RDF graphs as heuristics. Two optimization techniques are proposed to further improve the efficiency of our algorithms. One algorithm, called RDF property filtering, filters out invalid input data to reduce intermediate results; the other is to improve the query performance by postponing the Cartesian product operations. The extensive experiments on both synthetic and real-world datasets show that our method outperforms the close competitors S2X and SHARD by an order of magnitude on average