Outlier detection from ETL Execution trace

Extract, Transform, Load (ETL) is an integral part of Data Warehousing (DW) implementation. The commercial tools that are used for this purpose captures lot of execution trace in form of various log files with plethora of information. However there has been hardly any initiative where any proactive analyses have been done on the ETL logs to improve their efficiency. In this paper we utilize outlier detection technique to find the processes varying most from the group in terms of execution trace. As our experiment was carried on actual production processes, any outlier we would consider as a signal rather than a noise. To identify the input parameters for the outlier detection algorithm we employ a survey among developer community with varied mix of experience and expertise. We use simple text parsing to extract these features from the logs, as shortlisted from the survey. Subsequently we applied outlier detection technique (Clustering based) on the logs. By this process we reduced our domain of detailed analysis from 500 logs to 44 logs (8 Percentage). Among the 5 outlier cluster, 2 of them are genuine concern, while the other 3 figure out because of the huge number of rows involved.Comment: 2011 3rd International Conference on Electronics Computer Technology (ICECT 2011

Development of High-Performance Graphene-HgCdTe Detector Technology for Mid-Wave Infrared Applications

A high-performance graphene-based HgCdTe detector technology is being developed for sensing over the mid-wave infrared (MWIR) band for NASA Earth Science, defense, and commercial applications. This technology involves the integration of graphene into HgCdTe photodetectors that combines the best of both materials and allows for higher MWIR(2-5 m) detection performance compared to photodetectors using only HgCdTe material. The interfacial barrier between the HgCdTe-based absorber and the graphene layer reduces recombination of photogenerated carriers in the detector. The graphene layer also acts as high mobility channel that whisks away carriers before they recombine, further enhancing the detector performance. Likewise, HgCdTe has shown promise for the development of MWIR detectors with improvements in carrier mobility and lifetime. The room temperature operational capability of HgCdTe-based detectors and arrays can help minimize size, weight, power and cost for MWIR sensing applications such as remote sensing and earth observation, e.g., in smaller satellite platforms. The objective of this work is to demonstrate graphene-based HgCdTe room temperature MWIR detectors and arrays through modeling, material development, and device optimization. The primary driver for this technology development is the enablement of a scalable, low cost, low power, and small footprint infrared technology component that offers high performance, while opening doors for new earth observation measurement capabilities

An imputation-based approach for parameter estimation in the presence of ambiguous censoring with application in industrial supply chain

This paper describes a novel approach based on "proportional imputation" when identical units produced in a batch have random but independent installation and failure times. The current problem is motivated by a real life industrial production-delivery supply chain where identical units are shipped after production to a third party warehouse and then sold at a future date for possible installation. Due to practical limitations, at any given time point, the exact installation as well as the failure times are known for only those units which have failed within that time frame after the installation. Hence, in-house reliability engineers are presented with a very limited, as well as partial, data to estimate different model parameters related to installation and failure distributions. In reality, other units in the batch are generally not utilized due to lack of proper statistical methodology, leading to gross misspecification. In this paper we have introduced a likelihood based parametric and computationally efficient solution to overcome this problem.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS348 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org