MODISTools - downloading and processing MODIS remotely sensed data in R

Remotely sensed data – available at medium to high resolution across global spatial and temporal scales – are a valuable resource for ecologists. In particular, products from NASA's MODerate-resolution Imaging Spectroradiometer (MODIS), providing twice-daily global coverage, have been widely used for ecological applications. We present MODISTools, an R package designed to improve the accessing, downloading, and processing of remotely sensed MODIS data. MODISTools automates the process of data downloading and processing from any number of locations, time periods, and MODIS products. This automation reduces the risk of human error, and the researcher effort required compared to manual per-location downloads. The package will be particularly useful for ecological studies that include multiple sites, such as meta-analyses, observation networks, and globally distributed experiments. We give examples of the simple, reproducible workflow that MODISTools provides and of the checks that are carried out in the process. The end product is in a format that is amenable to statistical modeling. We analyzed the relationship between species richness across multiple higher taxa observed at 526 sites in temperate forests and vegetation indices, measures of aboveground net primary productivity. We downloaded MODIS derived vegetation index time series for each location where the species richness had been sampled, and summarized the data into three measures: maximum time-series value, temporal mean, and temporal variability. On average, species richness covaried positively with our vegetation index measures. Different higher taxa show different positive relationships with vegetation indices. Models had high R2 values, suggesting higher taxon identity and a gradient of vegetation index together explain most of the variation in species richness in our data. MODISTools can be used on Windows, Mac, and Linux platforms, and is available from CRAN and GitHub (https://github.com/seantuck12/MODISTools)

An Alternate Construction of an Access-Optimal Regenerating Code with Optimal Sub-Packetization Level

Given the scale of today's distributed storage systems, the failure of an individual node is a common phenomenon. Various metrics have been proposed to measure the efficacy of the repair of a failed node, such as the amount of data download needed to repair (also known as the repair bandwidth), the amount of data accessed at the helper nodes, and the number of helper nodes contacted. Clearly, the amount of data accessed can never be smaller than the repair bandwidth. In the case of a help-by-transfer code, the amount of data accessed is equal to the repair bandwidth. It follows that a help-by-transfer code possessing optimal repair bandwidth is access optimal. The focus of the present paper is on help-by-transfer codes that employ minimum possible bandwidth to repair the systematic nodes and are thus access optimal for the repair of a systematic node. The zigzag construction by Tamo et al. in which both systematic and parity nodes are repaired is access optimal. But the sub-packetization level required is $r^k$ where $r$ is the number of parities and $k$ is the number of systematic nodes. To date, the best known achievable sub-packetization level for access-optimal codes is $r^{k/r}$ in a MISER-code-based construction by Cadambe et al. in which only the systematic nodes are repaired and where the location of symbols transmitted by a helper node depends only on the failed node and is the same for all helper nodes. Under this set-up, it turns out that this sub-packetization level cannot be improved upon. In the present paper, we present an alternate construction under the same setup, of an access-optimal code repairing systematic nodes, that is inspired by the zigzag code construction and that also achieves a sub-packetization level of $r^{k/r}$.Comment: To appear in National Conference on Communications 201

A scheme for supporting automatic data migration on multicomputers

A data migration mechanism is proposed that allows an explicit and controlled mapping of data to memory. While read or write copies of each data element can be assigned to any processor's memory, longer term storage of each data element is assigned to a specific location in the memory of a particular processor. Data is presented that suggests that the scheme may be a practical method for efficiently supporting data migration

Convertible Codes: New Class of Codes for Efficient Conversion of Coded Data in Distributed Storage

Erasure codes are typically used in large-scale distributed storage systems to provide durability of data in the face of failures. In this setting, a set of k blocks to be stored is encoded using an [n, k] code to generate n blocks that are then stored on different storage nodes. A recent work by Kadekodi et al. [Kadekodi et al., 2019] shows that the failure rate of storage devices vary significantly over time, and that changing the rate of the code (via a change in the parameters n and k) in response to such variations provides significant reduction in storage space requirement. However, the resource overhead of realizing such a change in the code rate on already encoded data in traditional codes is prohibitively high. Motivated by this application, in this work we first present a new framework to formalize the notion of code conversion - the process of converting data encoded with an [n^I, k^I] code into data encoded with an [n^F, k^F] code while maintaining desired decodability properties, such as the maximum-distance-separable (MDS) property. We then introduce convertible codes, a new class of code pairs that allow for code conversions in a resource-efficient manner. For an important parameter regime (which we call the merge regime) along with the widely used linearity and MDS decodability constraint, we prove tight bounds on the number of nodes accessed during code conversion. In particular, our achievability result is an explicit construction of MDS convertible codes that are optimal for all parameter values in the merge regime albeit with a high field size. We then present explicit low-field-size constructions of optimal MDS convertible codes for a broad range of parameters in the merge regime. Our results thus show that it is indeed possible to achieve code conversions with significantly lesser resources as compared to the default approach of re-encoding