A Big Data Smart Agricultural System: Recommending Optimum Fertilisers For Crops

Nutrients are important to promote plant growth and nutrient deficiency is the primary factor limiting crop production. However, excess fertilisers can also have a negative impact on crop quality and yield, cause an increase in pollution and decrease producer profit. Hence, determining the suitable quantities of fertiliser for every crop is very useful. Currently, the agricultural systems with internet of things make very large data volumes. Exploiting agricultural Big Data will help to extract valuable information. However, designing and implementing a large scale agricultural data warehouse are very challenging. The data warehouse is a key module to build a smart crop system to make proficient agronomy recommendations. In our paper, an electronic agricultural record (EAR) is proposed to integrate many separate datasets into a unified dataset. Then, to store and manage the agricultural Big Data, we built an agricultural data warehouse based on Hive and Elasticsearch. Finally, we applied some statistical methods based on our data warehouse to extract fertiliser information such as a case study. These statistical methods propose the recommended quantities of fertiliser components across a wide range of environmental and crop management conditions, such as nitrogen (N), phosphorus (P) and potassium (K) for the top ten most popular crops in EU

User Selection Approaches to Mitigate the Straggler Effect for Federated Learning on Cell-Free Massive MIMO Networks

This work proposes UE selection approaches to mitigate the straggler effect for federated learning (FL) on cell-free massive multiple-input multiple-output networks. To show how these approaches work, we consider a general FL framework with UE sampling, and aim to minimize the FL training time in this framework. Here, training updates are (S1) broadcast to all the selected UEs from a central server, (S2) computed at the UEs sampled from the selected UE set, and (S3) sent back to the central server. The first approach mitigates the straggler effect in both Steps (S1) and (S3), while the second approach only Step (S3). Two optimization problems are then formulated to jointly optimize UE selection, transmit power and data rate. These mixed-integer mixed-timescale stochastic nonconvex problems capture the complex interactions among the training time, the straggler effect, and UE selection. By employing the online successive convex approximation approach, we develop a novel algorithm to solve the formulated problems with proven convergence to the neighbourhood of their stationary points. Numerical results confirm that our UE selection designs significantly reduce the training time over baseline approaches, especially in the networks that experience serious straggler effects due to the moderately low density of access points.Comment: submitted for peer review

Deep Nested Clustering Auto-Encoder for Anomaly-Based Network Intrusion Detection

Anomaly-based intrusion detection system(AIDS) plays an increasingly important role in detecting complex,multi-stage network attacks, especially zero-day attacks. Although there have been improvements both in practical applications and the research environment, there are still many unresolved accuracy-related concerns. The two fundamental limitations that contribute to these concerns are: i) the succinct, concise, latent representation learning of the normal network data, and ii) the optimization volume of normal regions in latent space. Recent studies have suggested many ways to learn the latent representation of normal network data in a semi-supervised manner to construct AIDS. However, these approaches are still affected by the above limitations,mainly due to the inability to process high data dimensionality or ineffectively explore the underlying architecture of the data. In this paper, we propose a novel Deep Nested Clustering Auto Encoder (DNCAE ) model to thoroughly overcome the aforementioned difficulties and improve the performance o fnetwork attack detection. The proposed model consists of two nested Deep Auto-Encoders(DAE) to learn the informative and tighter data representation space. In addition, the DNCAE model integrates the clustering technique into the latent layer of the outer DAE to learn the optimal arrangement of datapoints in the latent space. This harmonious combination allows us to effectively deal with the limitations outlined. The performance of the proposed model is evaluated using standard datasets including NSL-KDD,UNSW-NB15, and six scenarios of CIC-IDS2017(Tuesday, Wednesday, Thursday-Morning, Friday-Morning, Friday-Afternoon Port Scan,Friday-Afternoon DDoS).The experimental results strongly confirm that the proposed model clearly out performs th baselines and the existing methods for network anomaly detection. IndexTerms—Latent Representation, DeepAuto-Encoder, Clustering, AnomalyDetection, Intrusion Detection Syste

Energy-Efficient Design for Downlink Cloud Radio Access Networks

This work aims to maximize the energy efficiency of a downlink cloud radio access network (C-RAN), where data is transferred from a baseband unit in the core network to several remote radio heads via a set of edge routers over capacity-limited fronthaul links. The remote radio heads then send the received signals to their users via radio access links. We formulate a new mixed-integer nonlinear problem in which the ratio of network throughput and total power consumption is maximized. This challenging problem formulation includes practical constraints on routing, predefined minimum data rates, fronthaul capacity and maximum RRH transmit power. By employing the successive convex quadratic programming framework, an iterative algorithm is proposed with guaranteed convergence to a Fritz John solution of the formulated problem. Significantly, each iteration of the proposed algorithm solves only one simple convex program. Numerical examples with practical parameters confirm that the proposed joint optimization design markedly improves the C-RAN's energy efficiency compared to benchmark schemes.This work is supported in part by an ECR-HDR scholarship from The University of Newcastle, in part by the Australian Research Council Discovery Project grants DP170100939 and DP160101537, in part by Vietnam National Foundation for Science and Technology Development under grant number 101.02-2016.11 and in part by a startup fund from San Diego State University