Assessing Apache Spark Streaming with Scientific Data

Processing real-world data requires the ability to analyze data in real-time. Data processing engines like Hadoop come short when results are needed on the fly. Apache Spark\u27s streaming library is increasingly becoming a popular choice as it can stream and analyze a significant amount of data. To showcase and assess the ability of Spark various metrics were designed and operated using data collected from the USGODAE data catalog. The latency of streaming in Apache Spark was measured and analyzed against many nodes in the cluster. Scalability was monitored by adding and removing nodes in the middle of a streaming job. Fault tolerance was verified by stopping nodes in the middle of a job and making sure that the job was rescheduled and completed on other node/s. A full stack application was designed that would automate data collection, data processing and visualizing the results. Google Maps API was used to visualize results by color coding the world map with values from various analytics

Assessing Apache Spark Streaming with Scientific Data

Processing real-world data requires the ability to analyze data in real-time. Data processing engines like Hadoop come short when results are needed on the fly. Apache Spark\u27s streaming library is increasingly becoming a popular choice as it can stream and analyze a significant amount of data. To showcase and assess the ability of Spark various metrics were designed and operated using data collected from the USGODAE data catalog. The latency of streaming in Apache Spark was measured and analyzed against many nodes in the cluster. Scalability was monitored by adding and removing nodes in the middle of a streaming job. Fault tolerance was verified by stopping nodes in the middle of a job and making sure that the job was rescheduled and completed on other node/s. A full stack application was designed that would automate data collection, data processing and visualizing the results. Google Maps API was used to visualize results by color coding the world map with values from various analytics

A Shell-Neutral Modeling Approach Yields Sustainable Oyster Harvest Estimates: A Retrospective Analysis of the Louisiana State Primary Seed Grounds

A numerical model is presented that defines a sustainability criterion as no net loss of shell, and calculates a sustainable harvest of seed (\u3c75 mm) and sack or market oysters (\u3e= 75 mm). Stock assessments of the Primary State Seed Grounds conducted east of the Mississippi from 2009 to 2011 show a general trend toward decreasing abundance of sack and seed oysters. Retrospective simulations provide estimates of annual sustainable harvests. Comparisons of simulated sustainable harvests with actual harvests show a trend toward unsustainable harvests toward the end of the time series. Stock assessments combined with shell-neutral models can be used to estimate sustainable harvest and manage cultch through shell planting when actual harvest exceeds sustainable harvest. For exclusive restoration efforts (no fishing allowed), the model provides a metric for restoration success namely, shell accretion. Oyster fisheries that remove shell versus reef restorations that promote shell accretion, although divergent in their goals, are convergent in their management; both require vigilant attention to shell budgets