Anomaly detection in agri warehouse construction

As with many sectors, strategists and decision makers in the agricultural sector have a requirement to predict key measures such as product and feed pricing in order to maintain their position and, in some cases, to survive in their industry. Predictive algorithms in the area of Agri Analytics have shown to be very difficult due to the wide range of parameters and often unpredictable nature of some of these variables. Improving the predictive capability of Agri planners requires access to up-to-date external information in addition to the analyses provided by their own in-house databases. This motivates the need for an Agri Data Warehouse together with appropriate cleaning and transformation processes. However, with the availability of rich and wide ranging sources of Agri data now available online, there is a strong motivation to process as much current, online information as possible. In this work, we introduce the Agri Data Warehouse built for the DATAS project which not only harvests from a large number of online sources but also adopts an anomaly detection and labelling process to assist transformation and loading into the warehouse

An architecture and services for constructing data marts from online data sources

The Agri sector has shown an exponential growth in both the requirement for and the production and availability of data. In parallel with this growth, Agri organisations often have a need to integrate their in-house data with international, web-based datasets. Generally, data is freely available from official government sources but there is very little unity between sources, often leading to significant manual overhead in the development of data integration systems and the preparation of reports. While this has led to an increased use of data warehousing technology in the Agri sector, the issues of cost in terms of both time to access data and the financial costs of generating the Extract-Transform-Load layers remain high. In this work, we examine more lightweight data marts in an infrastructure which can support on-demand queries. We focus on the construction of data marts which combine both enterprise and web data, and present an evaluation which verifies the transformation process from source to data mart

Multi-resolution forecast aggregation for time series in agri datasets

A wide variety of phenomena are characterized by time dependent dynamics that can be analyzed using time series methods. Various time series analysis techniques have been presented, each addressing certain aspects of the data. In time series analysis, forecasting is a challenging problem when attempting to estimate extended time horizons which effectively encapsulate multi-step-ahead (MSA) predictions. Two original solutions to MSA are the direct and the recursive approaches. Recent studies have mainly focused on combining previous methods as an attempt to overcome the problem of discarding sequential correlation in the direct strategy or accumulation of error in the recursive strategy. This paper introduces a technique known as Multi-Resolution Forecast Aggregation (MRFA) which incorporates an additional concept known as Resolutions of Impact. MRFA is shown to have favourable prediction capabilities in comparison to a number of state of the art methods

On-Premise AIOps Infrastructure for a Software Editor SME: An Experience Report

Information Technology has become a critical component in various industries, leading to an increased focus on software maintenance and monitoring. With the complexities of modern software systems, traditional maintenance approaches have become insufficient. The concept of AIOps has emerged to enhance predictive maintenance using Big Data and Machine Learning capabilities. However, exploiting AIOps requires addressing several challenges related to the complexity of data and incident management. Commercial solutions exist, but they may not be suitable for certain companies due to high costs, data governance issues, and limitations in covering private software. This paper investigates the feasibility of implementing on-premise AIOps solutions by leveraging open-source tools. We introduce a comprehensive AIOps infrastructure that we have successfully deployed in our company, and we provide the rationale behind different choices that we made to build its various components. Particularly, we provide insights into our approach and criteria for selecting a data management system and we explain its integration. Our experience can be beneficial for companies seeking to internally manage their software maintenance processes with a modern AIOps approach

Automating data mart construction from semi-structured data sources

The global food and agricultural industry has a total market value of USD 8 trillion in 2016, and decision makers in the Agri sector require appropriate tools and up-to-date information to make predictions across a range of products and areas. Traditionally, these requirements are met with information processed into a data warehouse and data marts constructed for analyses. Increasingly however, data is coming from outside the enterprise and often in unprocessed forms. As these sources are outside the control of companies, they are prone to change and new sources may appear. In these cases, the process of accommodating these sources can be costly and very time consuming. To automate this process, what is required is a sufficiently robust Extract-Transform-Load (ETL) process; external sources are mapped to some form of ontology, and an integration process to merge the specific data sources. In this paper, we present an approach to automating the integration of data sources in an Agri environment, where new sources are examined before an attempt to merge them with existing data marts. Our validation uses three separate case studies of real world data to demonstrate the robustness of our approach and the efficiency of materialising data mart

Using artificial intelligence to automate meat cut Identification from the semimembranosus muscle on beef boning lines

The identification of different meat cuts for labelling and quality control on production lines is still largely a manual process. As a result, it is a labor-intensive exercise with the potential for error but also bacterial cross-contamination. Artificial intelligence is used in many disciplines to identify objects within images but these approaches usually require a considerable volume of images for training and validation. The objective of this study was to identify five different meat cuts from images and weights collected by a trained operator within the working environment of a commercial Irish beef plant. Individual cut images and weights from 7987 meats cuts extracted from Semimembranosus muscles (i.e., Topside muscle), post-editing, were available. A variety of classical neural networks and a novel Ensemble machine learning approaches were then tasked with identifying each individual meat cut; performance of the approaches was dictated by accuracy (the percentage of correct predictions); precision (the ratio of correctly predicted objects relative to the number of objects identified as positive), and recall (also known as true positive rate or sensitivity). A novel Ensemble approach outperformed a selection of the classical neural networks including convolutional neural network (CNN) and residual network (ResNET). The accuracy, precision, and recall for the novel Ensemble method were 99.13%, 99.00%, and 98.00%, respectively, while that of the next best method were 98.00%, 98.00%, and 95.00%, respectively. The Ensemble approach, which requires relatively few gold-standard measures, can readily be deployed under normal abattoir conditions; the strategy could also be evaluated in the cuts from other primals or indeed other species

A methodology for automating graph construction and evaluation

Graphs and graph analytics facilitate new approaches to machine learning. They also provide the ability to extract new insights from the same datasets as used in traditional machine learning experiments. For this reason, many researchers are seeking to exploit graph databases in pursuit of better performance for their predictive models. However, the construction of a graph from relational or flat models such as CSV files is not a straightforward transformation. A careful selection of nodes and relationships is required to ensure an optimal construction of the target graph. Overly large graphs can cause performance issues for a number of graph algorithms and thus, graph compression is an important part of the construction process. This research has 2 components: the usage of graphs to integrate multiple data sources and a graph transformation methodology to create the integrated schema and populate the graph. Our approach to validation uses link prediction and community detection graph analytics to evaluate the graphs built using our methodology

Decentralized Accessibility of e-commerce Products through Blockchain Technology

A distributed and transparent ledger system is considered for various \textit{e}-commerce products including health medicines, electronics, security appliances, food products and many more to ensure technological and e-commerce sustainability. This solution, named as ‘PRODCHAIN’, is a generic blockchain framework with lattice-based cryptographic processes for reducing the complexity for tracing the e-commerce products. Moreover, we have introduced a rating based consensus process called Proof of Accomplishment (PoA). The solution has been analyzed and experimental studies are performed on Ethereum network. The results are discussed in terms of latency and throughput which prove the efficiency of PRODCHAIN in \textit{e}-commerce products and services.The presented solution is beneficial for improving the traceability of the products ensuring the social and financial sustainability. This work will help the researchers to gain knowledge about the blockchain implications for supply chain possibilities in future developments for society