Alliance for a Data Revolution: CGIAR Platform for Big Data in Agriculture 2017 Convention Report

On September 19-22, 2017 the Consultative Group for International Agricultural Research1 (CGIAR) gathered over 300 local and international researchers, non-profits, public and private sector actors for the first CGIAR Platform for Big Data in Agriculture Convention, hosted by the International Center for Tropical Agriculture (CIAT) in Palmira, Colombia. The Convention marked the programmatic launch of the Platform, which aims to enable the development sector to embrace data and other digital technology approaches to solve agricultural development problems faster, better and at greater scale. The Platform works across the CGIAR network and CGIAR Research Programs (CRPs) and with the gamut of stakeholders in the agriculture sector as they grapple with creation, curation, and sharing data to enable new approaches to complex development challenges. The Platform is designed around three strategic pillars: Organize, Convene, and Inspire. The first aims to organize data so datasets are findable, accessible, and interoperable so they can be used increasingly in big data analytics. In addition, this pillar will develop open digital infrastructures for the sector that support the CGIAR’s work and enable new partnerships and innovations. The aim to convene analysts, researchers and public, private and non-profit actors in the agriculture sector will build new partnerships that both shape and fully leverage digital technologies in support of global agricultural development. The final pillar is to inspire these actors to push the limits of research and innovation to generate new data-driven approaches that solve real world development problems faster, cheaper, and more efficiently

Big Data Coordination Platform: Full Proposal 2017-2022

This proposal for a Big Data and ICT Platform therefore focuses on enhancing CGIAR and partner capacity to deliver big data management, analytics and ICT-focused solutions to CGIAR target geographies and communities. The ultimate goal of the platform is to harness the capabilities of Big Data to accelerate and enhance the impact of international agricultural research. It will support CGIAR’s mission by creating an enabling environment where data are expertly managed and used effectively to strengthen delivery on CGIAR SRF’s System Level Outcome (SLO) targets. Critical gaps were identified during the extensive scoping consultations with CGIAR researchers and partners (provided in Annex 8). The Platform will achieve this through ambitious partnerships with initiatives and organizations outside CGIAR, both upstream and downstream, public and private. It will focus on promoting CGIAR-wide collaboration across CRPs and Centers, in addition to developing new partnership models with big data leaders at the global level. As a result, CGIAR and partner capacity will be enhanced, external partnerships will be leveraged, and an institutional culture of collaborative data management and analytics will be established. Important international public goods such as new global and regional datasets will be developed, alongside new methods that support CGIAR to use the data revolution as an additional means of delivering on SLOs

CGIAR Platform for Big Data in Agriculture - Plan of Work and Budget 2021

The CGIAR Platform for Big Data in Agriculture is a cross-cutting program of the global CGIAR consortium of non-profit research institutes looking into virtually every aspect of food security spanning: genomics, breeding, agroecology, climate science, and the socioeconomic drivers and context of food systems change. The Platform tends to data standards and data sharing, digital innovation strategy and technology transfer, and research into the intersection of digital technologies and agricultural development in emerging regions

Big Data Refinement

"Big data" has become a major area of research and associated funding, as well as a focus of utopian thinking. In the still growing research community, one of the favourite optimistic analogies for data processing is that of the oil refinery, extracting the essence out of the raw data. Pessimists look for their imagery to the other end of the petrol cycle, and talk about the "data exhausts" of our society. Obviously, the refinement community knows how to do "refining". This paper explores the extent to which notions of refinement and data in the formal methods community relate to the core concepts in "big data". In particular, can the data refinement paradigm can be used to explain aspects of big data processing

BDGS: A Scalable Big Data Generator Suite in Big Data Benchmarking

Data generation is a key issue in big data benchmarking that aims to generate application-specific data sets to meet the 4V requirements of big data. Specifically, big data generators need to generate scalable data (Volume) of different types (Variety) under controllable generation rates (Velocity) while keeping the important characteristics of raw data (Veracity). This gives rise to various new challenges about how we design generators efficiently and successfully. To date, most existing techniques can only generate limited types of data and support specific big data systems such as Hadoop. Hence we develop a tool, called Big Data Generator Suite (BDGS), to efficiently generate scalable big data while employing data models derived from real data to preserve data veracity. The effectiveness of BDGS is demonstrated by developing six data generators covering three representative data types (structured, semi-structured and unstructured) and three data sources (text, graph, and table data)