Towards batch-processing on cold storage devices

Large amounts of data in storage systems is cold, i.e., Written Once and Read Occasionally (WORO). The rapid growth of massive-scale archival and historical data increases the demand for petabyte-scale cheap storage for such cold data. A Cold Storage Device (CSD) is a disk-based storage system which is designed to trade off performance for cost and power efficiency. Inevitably, the design restrictions used in CSD's results in performance limitations. These limitations are not a concern for WORO workloads, however, the very low price/performance characteristics of CSDs makes them interesting for other applications, e.g., batch processes, too. Applications, however, can be very slow on CSD's if they do not take their characteristics into account. In this paper we design two strategies for data partitioning in CSDs -- a crucial operation in many batch analytics tasks like hash-join, near-duplicate detection, and data localization. We show that our strategies can efficiently use CSDs for batch processing of terabyte-scale data by accelerating data partitioning by 3.5x in our experiments

Cheap Data Analytics using Cold Storage Devices

Enterprise databases use storage tiering to lower capital and operational expenses. In such a setting, data waterfalls from an SSD-based high-performance tier when it is "hot" (frequently accessed) to a disk-based capacity tier and finally to a tape-based archival tier when "cold" (rarely accessed). To address the unprecedented growth in the amount of cold data, hardware vendors introduced new devices named Cold Storage Devices (CSD) explicitly targeted at cold data workloads. With access latencies in tens of seconds and cost/GB as low as $0.01/GB/month, CSD provide a middle ground between the low-latency (ms), high-cost, HDD-based capacity tier, and high-latency (min to h), low-cost, tape-based, archival tier. Driven by the price/performance aspect of CSD, this paper makes a case for using CSD as a replacement for both capacity and archival tiers of enterprise databases. Although CSD offer major cost savings, we show that current database systems can suffer from severe performance drop when CSD are used as a replacement for HDD due to the mismatch between design assumptions made by the query execution engine and actual storage characteristics of the CSD. We then build a CSD-driven query execution framework, called Skipper, that modifies both the database execution engine and CSD scheduling algorithms to be aware of each other. Using results from our implementation of the architecture based on PostgreSQL and OpenStack Swift, we show that Skipper is capable of completely masking the high latency overhead of CSD, thereby opening up CSD for wider adoption as a storage tier for cheap data analytics over cold data

A Roadmap to Reduce U.S. Food Waste by 20 Percent

The magnitude of the food waste problem is difficult to comprehend. The U.S. spends $218 billion a year -- 1.3% of GDP -- growing, processing, transporting, and disposing of food that is never eaten. The causes of food waste are diverse, ranging from crops that never get harvested, to food left on overfilled plates, to near-expired milk and stale bread. ReFED is a coalition of over 30 business, nonprofit, foundation, and government leaders committed to building a different future, where food waste prevention, recovery, and recycling are recognized as an untapped opportunity to create jobs, alleviate hunger, and protect the environment -- all while stimulating a new multi-billion dollar market opportunity. ReFED developed A Roadmap to Reduce U.S. Food Waste as a data-driven guide to collectively take action to reduce food waste at scale nationwide.This Roadmap report is a guide and a call to action for us to work together to solve this problem. Businesses can save money for themselves and their customers. Policymakers can unleash a new wave of local job creation. Foundations can take a major step in addressing environmental issues and hunger. And innovators across all sectors can launch new products, services, and business models. There will be no losers, only winners, as food finds its way to its highest and best use

Using big data for customer centric marketing

This chapter deliberates on “big data” and provides a short overview of business intelligence and emerging analytics. It underlines the importance of data for customer-centricity in marketing. This contribution contends that businesses ought to engage in marketing automation tools and apply them to create relevant, targeted customer experiences. Today’s business increasingly rely on digital media and mobile technologies as on-demand, real-time marketing has become more personalised than ever. Therefore, companies and brands are striving to nurture fruitful and long lasting relationships with customers. In a nutshell, this chapter explains why companies should recognise the value of data analysis and mobile applications as tools that drive consumer insights and engagement. It suggests that a strategic approach to big data could drive consumer preferences and may also help to improve the organisational performance.peer-reviewe

Actionable Supply Chain Management Insights for 2016 and Beyond

The summit World Class Supply Chain 2016: Critical to Prosperity , contributed to addressing a need that the Supply Chain Management (SCM) field’s current discourse has deemed as critical: that need is for more academia-­‐industry collaboration to develop the field’s body of actionable knowledge. Held on May 4th, 2016 in Milton, Ontario, the summit addressed that need in a way that proved to be both effective and distinctive in the Canadian SCM environment. The summit, convened in partnership between Wilfrid Laurier University’s Lazaridis School of Business & Economics and CN Rail, focused on building actionable SCM knowledge to address three core questions: What are the most significant SCM issues to be confronted now and beyond 2016? What SCM practices are imperative now and beyond 2016? What are optimal ways of ensuring that (a) issues of interest to SCM practitioners inform the scholarly activities of research and teaching and (b) the knowledge generated from those scholarly activities reciprocally guide SCM practice? These are important questions for supply chain professionals in their efforts to make sense of today’s business environment that is appropriately viewed as volatile, uncertain, complex, and ambiguous. The structure of the deliberations to address these questions comprised two keynote presentations and three panel discussions, all of which were designed to leverage the collective wisdom that comes from genuine peer-­‐to-­‐peer dialogue between the SCM practitioners and SCM scholars. Specifically, the structure aimed for a balanced blend of industry and academic input and for coverage of the SCM issues of greatest interest to attendees (as determined through a pre-­‐summit survey of attendees). The structure produced impressively wide-­‐ranging deliberations on the aforementioned questions. The essence of the resulting findings from the summit can be distilled into three messages: Given today’s globally significant trends such as changes in population demographics, four highly impactful levers that SCM executives must expertly handle to attain excellence are: collaboration; information; technology; and talent Government policy, especially for infrastructure, is a significant determinant of SCM excellence There is tremendous potential for mutually beneficial industry-academia knowledge co-creation/sharing aimed at research and student training This white paper reports on those findings as well as on the summit’s success in realizing its vision of fostering mutually beneficial industry-academia dialogue. The paper also documents what emerged as matters that are inadequately understood and should therefore be targeted in the ongoing quest for deeper understanding of actionable SCM insights. Deliberations throughout the day on May 4th, 2016 and the encouraging results from the pre-­‐summit and post-­‐summit surveys have provided much inspiration to enthusiastically undertake that quest. The undertaking will be through initiatives that include future research projects as well as next year’s summit–World Class Supply Chain 2017