Not AvailableData are playing an important role making good planning and policies for agricultural growth and
development. Population growth and climate change are worldwide trends that are increasing the
importance of using big data science to improve agriculture. Add to that land degradation increasing
marginal land and loss of biodiversity are better deals with study of big data science. Crop data
can be break down into bits and bytes it will give better study about the crop development by using
advance data analytics tools for betterment of agriculture. Here, talk about some important tools
and techniques to handle and study the big data

Anil Kumar

Kanchan Sinha

Mrinmoy Ray

Prakash Kumar

Sanjeev Panwar

Sukanta Dash

Vipin Kumar Chaudhary

KRISHI Publications and Data Repository

Kumar et al.210Role of big data in agriculture- A statistical prospectivePrakash Kumar1, Anil Kumar1, Sanjeev Panwar2, Sukanta Dash1, Kanchan Sinha ,Vipin Kumar Chaudhary and Mrinmoy Ray1ICAR-Indian Agricultural Statistics Research Institutee-mail: prakash289111@gmail.comReceived : August 2017 ; Revised Accepted: January 2018ABSTRACTData are playing an important role making good planning and policies for agricultural growth anddevelopment. Population growth and climate change are worldwide trends that are increasing theimportance of using big data science to improve agriculture. Add to that land degradation increasingmarginal land and   loss of biodiversity are better deals with study of big data science. Crop datacan be break down into bits and bytes it will give better study about the crop development by usingadvance data analytics tools for betterment of agriculture. Here, talk about some important toolsand techniques to handle and study the big data.Key words: Cloud Computing Analytics, Internet of Things, Big Data, Environmental Conditions,Machine Learning, Data Mining, HARVIST, WEKA System, R-software.Ann. Agric. Res. New Series Vol. 39 (2) : 210-215 (2018)Big data is a collection of large and complexdata sets which becomes difficult to process usingon-hand database management tools ortraditional data processing techniques. Big Datacharacterize by four components: (a) volume(how big the data is), (b) velocity (how fast thedata is being collected), (c) variety (how varieddata being collected is) and (d) Veracity (big datasolutions must validate the accuracy of the bulkyamount of rapidly incoming data). Big datarequires a shift in computing architecture so thatusers can handle both the data storage andanalyzing large volumes of data economically.The total amount of farmland in the worldis limited and according to UN estimates theglobal population will grow 47% to 8.9 billion in2050. Major food grain crop yields like wheat, riceand maize are all affected in the run up to 2050due to global warming and climate change.Research activities involving to advancement  ofagricultural research mainly concerning on thegenomics, agricultural bioinformatics, andcomputational biology of plants and animalshave been enabling scientists and researchorganizations to better feed the world popula-tions  and improve the quality of agriculturalproduce. These disciplines are involved in verylarge data sets and computationally expensiveanalyses.The major problem of agricultural growthis traditional agriculture practiced which isbarrier to use scientifically developed agronomicpractices, such as planting, irrigating and harvest-ing using at suitable point in time. Weatherforecasting and predictive analytics can be usedby collecting real-time data on weather, soil, cropstage to make smarter decisions for cropcultivation. Advanced statistical research is ableto build improved models and simulations thatcan predict future conditions and help farmersmake proactive decisions.Farmers need to understand how to useoptimized plant density in limited cultivable*Correspondence address : prakash289111@gmail.com1 ICAR-Indian Agricultural Statistics Research InstituteRole of big data in agriculture- A statistical prospective 211(2005) were explained Heterogeneous Agri-cultural Research via Interactive, ScalableTechnology (HARVIST) project. Their goal is tointegrate multiple Earth Science data sources intoa single graphical user interface that allows forthe investigation of connections between differentvariables and focus on relationships betweenweather and crop yield, but the system we arecreating will be capable of integrating data forother studies as well. Oswaldo et al. (2011) focuseson Computational solutions to large-scale datamanagement and analysis which presents cloudand heterogeneous computing as solutions fortackling large-scale and high-dimensional datasets. Tsuchiya et al. (2012) introduced twofundamental technologies to analyze massiveamount of data: one distributed data store andcomplex event processing, and other workflowdescription for distributed data processing. Wagaand Rabah (2014) studied big data managementand cloud computing analytics for sustainableagriculture. They focuses on environmentalconditions' data like rainfall, winds, temperatureetc. and the use of particular cloud computinganalytical tool to get some meaningful informa-tion from it which can be utilized by farmers forstrategic and successful Agriculture.SOFTWARE TOOLS AND TECHNIQUES USED FORHANDLE AND ANALYSIS OF "BIG DATA"Cloud Computing Analytics :Cloud computing promises enough capacityin terms of storage and processing power toelastically handle data and involves deliveringhosted services over the Internet.  It can deals onenvironmental conditions data like rainfall,humidity condition, winds speed, environmentaltemperature etc. and the use of particular cloudcomputing analytical tool to get some meaningfulinformation from it which can be utilized byfarmers for strategic and successful Agriculture.The data collected ranges from soil moisture tonitrogen and other nutrients levels present in soil.The use of these data will allow farmers to gain aclearer picture of farming, receiving updates ofthe land in real time. For example, the data canmonitor pests, which then allows farmers totarget problem areas rather than over using pestarea. Seeds sowing, fertilizing application andmaintaining the crops are time-sensitive andheavily influenced by the weather. Forecastedwhether determined it's going to rain or not canalso influence when to irrigate fields. To reducethe food grains loss due to undetermined factorscan be controlled by using suitable grainharvesting time and their transportations anddistribution in appropriate time using big dataanalysis. Mobile app can be implemented underdigital India programmed use to figure-out theprecise date, time and expected amount of rainfallto avoid the seed loss due to dry or high moisturecontent in soil. The world population will crossover 9 billion by the year 2050, the hunger gap isexpected to widen. Scientist's mission is toimprove productivity through using big dataanalysis like weather forecasting, real-timeoptimization of farming machinery, monitoringof market price, automated irrigationrecommendations system, mobile-basedinformation system etc. In these fieldsorganizations in the United States by now operatecloud-based farming information systems thatapply soil observations and weathermeasurements to forecast weather for the nextseven days. For instance, many organizations inIndia take part to develop a system that monitorsabove conditions to maintain the quality andconsistency of agricultural produce.Piatetsky-Shapiro and Frawley (1991) weredescribed knowledge discovery in databases anddefined data mining is the process of discoveringpreviously unknown and potentially interestingpatterns in large datasets. Breiman (1992) workedon meta-classifiers is a Recent developments incomputational learning theory have led tomethods that enhance the performance or extendthe capabilities of these basic learning schemes.Boosting and bagging classifier methods used forclassifications. Garner et al. (1995) was developeda process model for applying data mining tech-niques to data, with the goal of incorporating theinduced domain information into a softwaremodule. Hall and Smith (1998) and the wrappermethod (John and Kohavi, 1997) worked on filteralgorithms provide facilities for general mani-pulation of attributes for example, to insert anddelete attributes from the dataset. Wagstaff et al.Kumar et al.212controls.  To reduce water pollution cause byinadequate use of fertilizer in soil through soilanalysis using cloud computing.Illustration: Using hyper-spectral camera/aerial photography the images collected fromfarmer's field → Data analyzed by cloudcomputing service provider like Ceres Imaging(They provide farmers spectral data to optimizewater and plants nutrients) → Results tell farmershow healthy their plants were, whether there wasan adequate amount of fertilizer and water usedor not, whether there were diseases or pestscontrolled or not.Pros: Lower operational cost, less timetaken, enable heavy duty data crunching to betterprocess and better explore internet information,pay for uses to recovery fixed expenses on hard-ware, software development, their maintenanceand support.Cons: Security concerns, reliable internetconnections, data ownership, processingresponsibility, data and processing are mercy ofservice provider etc.RR is an open source language which is usedfor linear and non-linear modeling, forecasting,data mining, time series analysis, supportsprocedural programming with functions, matrixcalculation and visualization of data. It is aninterpreted language which gives results througha command-line interpreter. During the lastdecade R programming language has become asingle most important tool for computationalstatistics for both the students as well asagricultural and industrial researchers. To handlethe big data R have some libraries/packages likeRhdfs, Rodbc, ffbase and rmr2 etc.  R supportsmany languages such as C, C++ and Python etc.to directly manipulate the R objects and it willalso take less memory up to 2-4 GB only, dependson hardware configuration of system. R providesbetter connection with its different packages toovercome the problem of storage. To handle bigdata in R five strategies should be considered:(1) Sampling : A good sampling strategies aresometimes very useful to reduce the size of abig data.(2) Requirement of bigger hardware: By increasingthe size of the machines memory. Today Rcan address 8 TB of RAM if it runs on 64-bitmachines.(3) Store objects on hard disc and analyze it chunkwise: There are different specific packagesavailable in R which facilitates to store datain hard disc and analyzed it chunk wise.(4) Integration of high performing programminglanguages such as C++ or Java etc. and(5) Use of different interpreters along with R todeals big data such as pqR, TERR and Rinterpreter in Java etc.Heterogeneous Agricultural Research ViaInteractive, Scalable Technology (HARVIST)The HARVIST system will provide multiplemachine learning and data analysis algorithmsthat can be applied to the agricultural data. Hereit is mainly utilized machine learning techniqueswhich are a group of supervised/unsupervisedlearning methods that can be applied forclassification data, clustering and dimensionreduction with adding multivariate spatialmodeling methods. HARVIST operates onremote sensing data which are collected throughMulti-angle Imaging Spectro-Radiometer (MISR)instrument. These data are required advancedclassification technique using HARVIST throughsupport vector machine (SVM). This classifier canbe used by scientists or policy makers toconstructing land cover/land use (LCLU) databases. It seeks to address both shortcomings bydemonstrating the technology required toperform large-scale studies of the interactionsbetween agriculture and climate.As shown in Figure 1, the HARVIST systemwill incorporate data from weather stations,remote sensing instruments and historical cropyield databases to generate highly accuratepredictions. Quickly and interactively resultobtains for a large area HARVIST techniques alsoemploy to analyze soil properties with micro-environment around crop which gives morepredictive accuracy of the system.HARVIST Analysis Methods mainly involvestwo data analysis methods one support vectormachines (SVMs) and others clustering techni-Role of big data in agriculture- A statistical prospective 213ques. These techniques are used for multispectraldata from remote-sensing images. To solve pixelClassification and clustering problem: Supportvector machines (SVMs) are useful for classifi-cation when the user has several specific classesof interest and the objective is to build a suitableclassifier that automatically classifies big data set.In contrast, clustering methods are useful toidentify overall trends present in the data setwhen the classes of interest are not known.For prediction problem multivariate spatialmodels can be used. If samples are independent,this assumption for the given statistical modelswhich incorporate spatial dependencies canprovide more accurate predictions. The linear/non linear model can be used to predict valuesfor multiple response variables simultaneouslyand provided a straight forward method forestimating the uncertainty associated with eachpredictionThe Waikato Environment for KnowledgeAnalysis (WEKA)The WEKA system applying data miningtechniques to analyze large data sets Holmeset al. (1994). It can incorporate the followingtools:For data mining it is required to transformthe raw data into an appropriate form by a datapre-processing routines, supporting the mani-pulation. Appling feature selection tools whichcan useful for identifying irrelevant attributes.Classifiers and some other data miningalgorithms can be use to handle numerical as wellas categorical learning tasks. Metaclassifiers canbe used for enhancing the performance ofclassification. Appling bench marking tools thatcan be use for compare the relative performanceof different learning algorithms over severaldatasets. To understand the WEKA system itis required to know about process of data mining.Fig. 1. The HARVIST System Architecture.Kumar et al.214Data Mining Process ModelApplying process model for data miningtechniques to agricultural data, with the softwaremodule incorporated with induced domaininformation. The feature of this model can beillustrated as a two-way interaction between thedata mining experts and agricultural dataprovider. To good fit between a representationof the data and a data mining algorithm machinelearning technique has been takes several cyclesthrough the process model. WEKA used themachine learning techniques and for understandthe process we should first know the classifiersthat is defined as set of rules or the form of adecision tree that can be used to predict theclassification of a new data instance. ZeroR is themost primitive learning scheme which is inWEKA. For classification of new data item, it ispredicts the most frequent category value in thetraining data for given problems with a nominalclass value, or predicts the average class valuefor numeric prediction problems. To combiningmultiple models meta-classifiers can be usedwhich is recently developing computationallearning theory to enhance the performance ofthese basic learning schemes. Meta classifiersmeans to make predictions, instead of using asingle classifier, it can used committee ofclassifiers. Two of the most prominent methodsfor constructing ensemble classifiers are baggingand boosting. Bagging works by buildingseparate models of the training data using asampling technique that deletes some instancesand replicates others. But, boosting is iterativeprocess that can use instead of sampling freshtraining data, each new model is influenced bythe performance of those built previously.Promotion and relegation of instances that areAlgorithm : for bagging• In  training data: n be the number of instances• Perform t-time iterations for each• Using deletion and replication for sample n instancesrandomly• Model building by  apply a learning technique• Store the model• EndAlgorithm : for boosting• Assign equal weight to each instances• Do t iterations for each instances• Model building by  apply a learning technique fromthe weighted instances• Store the resultant model• Down-weight all instance which correctly classifiedby the model• Endincorrectly and correctly classified in previousiterations correspondingly.Clustering is technique to find the naturalgroupings in the dataset. Clustering is followedby a second learning stage, in which a classifieris used to induce a rule set that allocates eachinstance in the dataset using clustering algorithm.WEKA used the EM clustering algorithm thatmakes the assumption, common to otherclustering algorithms, that the attributes in thedataset represent independent random variables.Association rules in WEKA: It is a type oflearning scheme commonly used in marketbasket analysis (MBA).This algorithms can be useto analyzing consumer purchasing patternsparticularly, products detection that arefrequently purchased together. These algorithmswere developed in response to the largetransaction data formed by barcode-basedpurchasing/selling systems.CONCLUSIONBig Data bring new opportunities to modernsociety and plays an important role in agriculturalscience; producers make more informed decisionsregarding crop production practices. Therefore,higher yields with lower costs can be full filled byapplying the suggestion provided by crop advisorsand service providers. To setup new agro-basedbusiness opportunities like industrial farming,contact farming and agro-based industries etc. canbe grown by using big data analysis. It is necessaryto develop the algorithms for big data analyseswhich can solve the problem arise in laboratory toimplementing it in field. Cloud Computinganalytics, R-software, HARVIST analysis methodsand WEKA system are some significant tools thatRole of big data in agriculture- A statistical prospective 215REFERENCESBreiman, L. 1992. Bagging predictors. MachineLearning, 24: 123-140.Fan, J., Han, F. and Liu, H. 2014.Challenges ofbig data analysis. National Science Review, 1(2):293-314.Garner, S.R., Cunningham, S.J., Holmes, G.,Nevill-Manning, C.G. and Witten, I.H. 1995.Applying a Machine Learning Workbench:Experience with Agricultural Databases.Proceedings of the Machine Learning inPractice Workshop, 12th International MachineLearning Conference (Tahoe City, CA, USA).Hall, M.A. and Smith, L.A. 1998. Practical featuresubset selection for machine learning.Proceedings of the Australian Computer ScienceConference, Perth, Australia, 181-191.Holmes, G., Donkin, A. and  Witten, I.H. 1994.Weka: A machine learning workbench.  ProcSecond Australia and New Zealand Con-ference on Intelligent Information Systems,Brisbane, Australia.John, G.H. and Kohavi, R. 1997. Wrappers forfeature subset selection. Artificial Intelli-gence, 97(1-2): 273-324.Oswaldo T., Prins, P., Marc S. and Ritsert C.J.2011. Big data, but are we ready? NatureReviews Genetics, 12: 224.Tsuchiya , S., Sakamoto, Y., Tsuchimoto, Y. andLee, V. 2012. Big data processing in cloudenvironments. Fujitsu Science TechnologyJournal, 48(2): 159-168.Waga, D. and Rabah K. 2014. EnvironmentalConditions' Big Data Management and CloudComputing Analytics for SustainableAgriculture. World Journal of ComputerApplication and Technology, 2(3): 73-81.Wagstaff, K. L., Mazzoni, D. and Sain, S. 2005.HARVIST: A system for agricultural andweather studies using advanced statisticalmodels. Proceedings of the Earth-Sun SystemTechnology Conference. University ofMaryland Inn (USA).are used to understand how to big data use foragricultural development. The challenges to datascientists in big data analytics including storingmodels, information sharing, maintaining datasecurity, data accessing, patterns finding in data,new computing to deals agricultural data,advanced analytics tools and redesigning miningalgorithms etc. It is required to adopt emergingtechnologies that deals with big data challenges inagriculture.

Role of big data in Agriculture-A Statistical Prospective

http://krishi.icar.gov.in/jspui/bitstream/123456789/74395/2/Annals-39%282%29%2c%202018-96-101.pdf

Role of big data in Agriculture-A Statistical Prospective

Abstract

Similar works

Full text

Available Versions

KRISHI Publications and Data Repository