Clustering‐based real‐time anomaly detection—A breakthrough in big data technologies

Off late, the ever increasing usage of a connected Internet-of-Things devices has consequently augmented the volume of real-time network data with high velocity. At the same time, threats on networks become inevitable; hence, identifying anomalies in real time network data has become crucial. To date, most of the existing anomaly detection approaches focus mainly on machine learning techniques for batch processing. Meanwhile, detection approaches which focus on the real-time analytics somehow deficient in its detection accuracy while consuming higher memory and longer execution time. As such, this paper proposes a novel framework which focuses on real-time anomaly detection based on big data technologies. In addition, this paper has also developed streaming sliding window local outlier factor coreset clustering algorithms (SSWLOFCC), which was then implemented into the framework. The proposed framework that comprises BroIDS, Flume, Kafka, Spark streaming, SparkMLlib, Matplot and HBase was evaluated to substantiate its efficacy, particularly in terms of accuracy, memory consumption, and execution time. The evaluation is done by performing critical comparative analysis using existing approaches, such as K-means, hierarchical density-based spatial clustering of applications with noise (HDBSCAN), isolation forest, spectral clustering and agglomerative clustering. Moreover, Adjusted Rand Index and memory profiler package were used for the evaluation of the proposed framework against the existing approaches. The outcome of the evaluation has substantially proven the efficacy of the proposed framework with a much higher accuracy rate of 96.51% when compared to other algorithms. Besides, the proposed framework also outperformed the existing algorithms in terms of lesser memory consumption and execution time. Ultimately the proposed solution enable analysts to precisely track and detect anomalies in real time. © 2019 John Wiley & Sons, Ltd

Menopause is associated with accelerated lung function decline

RATIONALE: Menopause is associated with changes in sex hormones, which affect immunity, inflammation, and osteoporosis and may impair lung function. Lung function decline has not previously been investigated in relation to menopause. OBJECTIVES: To study whether lung function decline, assessed by forced vital capacity and forced expiratory volume in one second, is accelerated in women who undergo menopause. METHODS: The population-based longitudinal European Community Respiratory Health Survey provided serum samples, spirometry and questionnaire data about respiratory and reproductive health from three study waves (N=1438). We measured follicle stimulating hormone and luteinizing hormone and added information on menstrual patterns, to determine menopausal status using latent class analysis. Associations with lung function decline were investigated using linear mixed effects models, adjusting for age, height, weight, packyears, current smoking, age at completed full-time education, spirometer and including study center as random effect. MEASUREMENTS AND MAIN RESULTS: Menopausal status was associated with accelerated lung function decline. The adjusted mean forced vital capacity decline was increased by -10.2 ml/yr (95% Confidence interval -13.1 to -7.2) in transitional women and -12.5 ml/yr (-16.2 to -8.9) in postmenopausal women, compared to women menstruating regularly. The adjusted mean forced expiratory volume in one second decline increased by -3.8 ml/yr (-6.3 to -2.9) in transitional women and -5.2 ml/yr (-8.3 to -2.0) in postmenopausal women. CONCLUSIONS: Lung function declined more rapidly among transitional and postmenopausal women, in particular for forced vital capacity, beyond the expected age change. Clinicians should be aware that respiratory health often deteriorates during reproductive aging