System Support For Stream Processing In Collaborative Cloud-Edge Environment

Stream processing is a critical technique to process huge amount of data in real-time manner. Cloud computing has been used for stream processing due to its unlimited computation resources. At the same time, we are entering the era of Internet of Everything (IoE). The emerging edge computing benefits low-latency applications by leveraging computation resources at the proximity of data sources. Billions of sensors and actuators are being deployed worldwide and huge amount of data generated by things are immersed in our daily life. It has become essential for organizations to be able to stream and analyze data, and provide low-latency analytics on streaming data. However, cloud computing is inefficient to process all data in a centralized environment in terms of the network bandwidth cost and response latency. Although edge computing offloads computation from the cloud to the edge of the Internet, there is not a data sharing and processing framework that efficiently utilizes computation resources in the cloud and the edge. Furthermore, the heterogeneity of edge devices brings more difficulty to the development of collaborative cloud-edge applications. To explore and attack the challenges of stream processing system in collaborative cloudedge environment, in this dissertation we design and develop a series of systems to support stream processing applications in hybrid cloud-edge analytics. Specifically, we develop an hierarchical and hybrid outlier detection model for multivariate time series streams that automatically selects the best model for different time series. We optimize one of the stream processing system (i.e., Spark Streaming) to reduce the end-to-end latency. To facilitate the development of collaborative cloud-edge applications, we propose and implement a new computing framework, Firework that allows stakeholders to share and process data by leveraging both the cloud and the edge. A vision-based cloud-edge application is implemented to demonstrate the capabilities of Firework. By combining all these studies, we provide comprehensive system support for stream processing in collaborative cloud-edge environment

Introducing distributed dynamic data-intensive (D3) science: Understanding applications and infrastructure

A common feature across many science and engineering applications is the amount and diversity of data and computation that must be integrated to yield insights. Data sets are growing larger and becoming distributed; and their location, availability and properties are often time-dependent. Collectively, these characteristics give rise to dynamic distributed data-intensive applications. While "static" data applications have received significant attention, the characteristics, requirements, and software systems for the analysis of large volumes of dynamic, distributed data, and data-intensive applications have received relatively less attention. This paper surveys several representative dynamic distributed data-intensive application scenarios, provides a common conceptual framework to understand them, and examines the infrastructure used in support of applications.Comment: 38 pages, 2 figure

Translate Data Into Meaning: integration of meteorology and geomatics to generate meaningful information for decision makers

A variety of actors at all scales and acting in different domains such as emergency management, agriculture, sports and leisure and commercial activities, are becoming more aware of the challenges and opportunities that meteorological data analysis poses for their operational goals. The increasing availability of meteorological data coupled with a rapid improvement in technology led to the widespread dissemination of the weather information to a variety of users on a regular basis. Particularly through the internet and mobile application all users, despite their varied background, can access to big amount of data with a high potential to gather essential input that can significantly help their decisions. At the same time, simply creating and disseminating information without context does not necessarily offer an added value to sèecific users. One of the main issues is related to the scientific approach of weather analysis and to the representation of results, which are hardly understandable for non-technical users and therefore not easily usable to make decisions. As a result, there are several researches aiming at finding new ways of supporting decision making by supplying easy to use information. The main objective of this thesis is therefore to provide guidance on how to identify and characterize the needs for meaningful and usable information among various users of meteorology, including members of the public, emergency managers, other government decision makers, and private-sector entities, both direct users and intermediaries. In particular a methodology for the integration of meteorological data and GIS capabilities is investigated and applied to three different end users having similarities and differences. Scientific analysis, results and cartographic products are adapted to specific requirements, experience and perceptions of the three different users

Small UAS Detect and Avoid Requirements Necessary for Limited Beyond Visual Line of Sight (BVLOS) Operations

Potential small Unmanned Aircraft Systems (sUAS) beyond visual line of sight (BVLOS) operational scenarios/use cases and Detect And Avoid (DAA) approaches were collected through a number of industry wide data calls. Every 333 Exemption holder was solicited for this same information. Summary information from more than 5,000 exemption holders is documented, and the information received had varied level of detail but has given relevant experiential information to generalize use cases. A plan was developed and testing completed to assess Radio Line Of Sight (RLOS), a potential key limiting factors for safe BVLOS ops. Details of the equipment used, flight test area, test payload, and fixtures for testing at different altitudes is presented and the resulting comparison of a simplified mathematical model, an online modeling tool, and flight data are provided. An Operational Framework that defines the environment, conditions, constraints, and limitations under which the recommended requirements will enable sUAS operations BVLOS is presented. The framework includes strategies that can build upon Federal Aviation Administration (FAA) and industry actions that should result in an increase in BVLOS flights in the near term. Evaluating approaches to sUAS DAA was accomplished through five subtasks: literature review of pilot and ground observer see and avoid performance, survey of DAA criteria and recommended baseline performance, survey of existing/developing DAA technologies and performance, assessment of risks of selected DAA approaches, and flight testing. Pilot and ground observer see and avoid performance were evaluated through a literature review. Development of DAA criteria—the emphasis here being well clear— was accomplished through working with the Science And Research Panel (SARP) and through simulations of manned and unmanned aircraft interactions. Information regarding sUAS DAA approaches was collected through a literature review, requests for information, and direct interactions. These were analyzed through delineation of system type and definition of metrics and metric values. Risks associated with sUAS DAA systems were assessed by focusing on the Safety Risk Management (SRM) pillar of the SMS (Safety Management System) process. This effort (1) identified hazards related to the operation of sUAS in BVLOS, (2) offered a preliminary risk assessment considering existing controls, and (3) recommended additional controls and mitigations to further reduce risk to the lowest practical level. Finally, flight tests were conducted to collect preliminary data regarding well clear and DAA system hazards

The Application of Data Mining Techniques to Learning Analytics and Its Implications for Interventions with Small Class Sizes

There has been significant progress in the development of techniques to deliver effective technology enhanced learning systems in education, with substantial progress in the field of learning analytics. These analyses are able to support academics in the identification of students at risk of failure or withdrawal. The early identification of students at risk is critical to giving academic staff and institutions the opportunity to make timely interventions. This thesis considers established machine learning techniques, as well as a novel method, for the prediction of student outcomes and the support of interventions, including the presentation of a variety of predictive analyses and of a live experiment. It reviews the status of technology enhanced learning systems and the associated institutional obstacles to their implementation and deployment. Many courses are comprised of relatively small student cohorts, with institutional privacy protocols limiting the data readily available for analysis. It appears that very little research attention has been devoted to this area of analysis and prediction. I present an experiment conducted on a final year university module, with a student cohort of 23, where the data available for prediction is limited to lecture/tutorial attendance, virtual learning environment accesses and intermediate assessments. I apply and compare a variety of machine learning analyses to assess and predict student performance, applied at appropriate points during module delivery. Despite some mixed results, I found potential for predicting student performance in small student cohorts with very limited student attributes, with accuracies comparing favourably with published results using large cohorts and significantly more attributes. I propose that the analyses will be useful to support module leaders in identifying opportunities to make timely academic interventions. Student data may include a combination of nominal and numeric data. A large variety of techniques are available to analyse numeric data, however there are fewer techniques applicable to nominal data. I summarise the results of what I believe to be a novel technique to analyse nominal data by making a systematic comparison of data pairs. In this thesis I have surveyed existing intelligent learning/training systems and explored the contemporary AI techniques which appear to offer the most promising contributions to the prediction of student attainment. I have researched and catalogued the organisational and non-technological challenges to be addressed for successful system development and implementation and proposed a set of critical success criteria to apply. This dissertation is supported by published work

The Sensor Network Workbench: Towards Functional Specification, Verification and Deployment of Constrained Distributed Systems

As the commoditization of sensing, actuation and communication hardware increases, so does the potential for dynamically tasked sense and respond networked systems (i.e., Sensor Networks or SNs) to replace existing disjoint and inflexible special-purpose deployments (closed-circuit security video, anti-theft sensors, etc.). While various solutions have emerged to many individual SN-centric challenges (e.g., power management, communication protocols, role assignment), perhaps the largest remaining obstacle to widespread SN deployment is that those who wish to deploy, utilize, and maintain a programmable Sensor Network lack the programming and systems expertise to do so. The contributions of this thesis centers on the design, development and deployment of the SN Workbench (snBench). snBench embodies an accessible, modular programming platform coupled with a flexible and extensible run-time system that, together, support the entire life-cycle of distributed sensory services. As it is impossible to find a one-size-fits-all programming interface, this work advocates the use of tiered layers of abstraction that enable a variety of high-level, domain specific languages to be compiled to a common (thin-waist) tasking language; this common tasking language is statically verified and can be subsequently re-translated, if needed, for execution on a wide variety of hardware platforms. snBench provides: (1) a common sensory tasking language (Instruction Set Architecture) powerful enough to express complex SN services, yet simple enough to be executed by highly constrained resources with soft, real-time constraints, (2) a prototype high-level language (and corresponding compiler) to illustrate the utility of the common tasking language and the tiered programming approach in this domain, (3) an execution environment and a run-time support infrastructure that abstract a collection of heterogeneous resources into a single virtual Sensor Network, tasked via this common tasking language, and (4) novel formal methods (i.e., static analysis techniques) that verify safety properties and infer implicit resource constraints to facilitate resource allocation for new services. This thesis presents these components in detail, as well as two specific case-studies: the use of snBench to integrate physical and wireless network security, and the use of snBench as the foundation for semester-long student projects in a graduate-level Software Engineering course

ACUTA Journal of Telecommunications in Higher Education

In This Issue Mobil VoIP for the Converged Campus Managing Remote IP Telephony Systems Cellular Service at University of Toledo Working with Wireless lmpact of Handheld Devices at Washington state university Point: Presence-Aware Communication Tools Counterpoint: Accessibility: Are You Too Connected? IP Telephony Payoff at Adelphi Institutional Excellence Award SUNY Upstate Medical University GIS Technology Brings the Layered Look to Campus Interview President\u27s Message From the Executive Director Here\u27s My Advic