Random Correlation Matrix and De-Noising

In Finance, the modeling of a correlation matrix is one of the important problems. In particular, the correlation matrix obtained from market data has the noise. Here we apply the de-noising processing based on the wavelet analysis to the noisy correlation matrix, which is generated by a parametric function with random parameters. First of all, we show that two properties, i.e. symmetry and ones of all diagonal elements, of the correlation matrix preserve via the de-noising processing and the efficiency of the de-nosing processing by numerical experiments. We propose that the de-noising processing is one of the effective methods in order to reduce the noise in the noisy correlation matrix.correlation matrix, calibration, rank reduction, de-noising, wavelet analysis

Bayesian wavelet de-noising with the caravan prior

According to both domain expert knowledge and empirical evidence, wavelet coefficients of real signals tend to exhibit clustering patterns, in that they contain connected regions of coefficients of similar magnitude (large or small). A wavelet de-noising approach that takes into account such a feature of the signal may in practice outperform other, more vanilla methods, both in terms of the estimation error and visual appearance of the estimates. Motivated by this observation, we present a Bayesian approach to wavelet de-noising, where dependencies between neighbouring wavelet coefficients are a priori modelled via a Markov chain-based prior, that we term the caravan prior. Posterior computations in our method are performed via the Gibbs sampler. Using representative synthetic and real data examples, we conduct a detailed comparison of our approach with a benchmark empirical Bayes de-noising method (due to Johnstone and Silverman). We show that the caravan prior fares well and is therefore a useful addition to the wavelet de-noising toolbox.Comment: 32 pages, 15 figures, 4 table

A Study on Clustering for Clustering Based Image De-Noising

In this paper, the problem of de-noising of an image contaminated with Additive White Gaussian Noise (AWGN) is studied. This subject is an open problem in signal processing for more than 50 years. Local methods suggested in recent years, have obtained better results than global methods. However by more intelligent training in such a way that first, important data is more effective for training, second, clustering in such way that training blocks lie in low-rank subspaces, we can design a dictionary applicable for image de-noising and obtain results near the state of the art local methods. In the present paper, we suggest a method based on global clustering of image constructing blocks. As the type of clustering plays an important role in clustering-based de-noising methods, we address two questions about the clustering. The first, which parts of the data should be considered for clustering? and the second, what data clustering method is suitable for de-noising.? Then clustering is exploited to learn an over complete dictionary. By obtaining sparse decomposition of the noisy image blocks in terms of the dictionary atoms, the de-noised version is achieved. In addition to our framework, 7 popular dictionary learning methods are simulated and compared. The results are compared based on two major factors: (1) de-noising performance and (2) execution time. Experimental results show that our dictionary learning framework outperforms its competitors in terms of both factors.Comment: 9 pages, 8 figures, Journal of Information Systems and Telecommunications (JIST

Contractive De-noising Auto-encoder

Auto-encoder is a special kind of neural network based on reconstruction. De-noising auto-encoder (DAE) is an improved auto-encoder which is robust to the input by corrupting the original data first and then reconstructing the original input by minimizing the reconstruction error function. And contractive auto-encoder (CAE) is another kind of improved auto-encoder to learn robust feature by introducing the Frobenius norm of the Jacobean matrix of the learned feature with respect to the original input. In this paper, we combine de-noising auto-encoder and contractive auto- encoder, and propose another improved auto-encoder, contractive de-noising auto- encoder (CDAE), which is robust to both the original input and the learned feature. We stack CDAE to extract more abstract features and apply SVM for classification. The experiment result on benchmark dataset MNIST shows that our proposed CDAE performed better than both DAE and CAE, proving the effective of our method.Comment: Figures edite

Vibration signal de-noising based on empirical wavelet transform autocorrelation analysis

In diesel engine fault diagnosis, non-stationary vibration signal is easily disturbed by strong noise. In view of the shortcomings of empirical mode decomposition (EMD) and wavelet transform in de-noising, a de-noising method is proposed, which is Empirical Wavelet Transform (EWT) autocorrelation analysis. Taking advantages of EMD and wavelet transform, the Fourier spectrum is adaptively divided by EWT, and the intrinsic mode components of different frequency are extracted through constructed wavelet filter, and the method can effectively eliminate modal aliasing and solve adaptive problems in wavelet de-noising. At the same time, autocorrelation analysis can make the random noise decay to zero, and the modal components with high frequency random noise are dealt by autocorrelation analysis. The method is used to de-noising and compared the de-noising effect of EWT and EMD. Results show that the method can effectively decompose the intrinsic mode component with less number, and there is no false mode, and the de-noising effect is better than EMD de-noising. The method is feasible and effective in de-noising by vibration signals of diesel engine

Fault diagnosis of rotor using EMD thresholding-based de-noising combined with probabilistic neural network

De-noising of signal processing is crucial for fault diagnosis in order to successfully conduct feature extraction and is an efficient method for accurate determination of cause. In this paper, the empirical mode decomposition (EMD) thresholding-based de-noising method and probabilistic neural network (PNN) are respectively used in the de-noising of the vibration signal and rotor fault diagnosis and compared with wavelet thresholding-based de-noising technology and back propagation neural network (BPNN). The results show that the clear iterative EMD interval thresholding performs better than wavelet thresholding in the de-noising of the vibration signal, and avoids the determination of wavelet basis and decomposition level. In addition, the PNN created by feature samples does not require training and has a higher accuracy than BPNN