Коллективные потоковые вычисления: реляционные модели и алгоритмы

Recently, microtask crowdsourcing has become a popular approach for addressing various data mining problems. Crowdsourcing workflows for approaching such problems are composed of several data processing stages which require consistent representation for making the work reproducible. This paper is devoted to the problem of reproducibility and formalization of the microtask crowdsourcing process. A computational model for microtask crowdsourcing based on an extended relational model and a dataflow computational model has been proposed. The proposed collaborative dataflow computational model is designed for processing the input data sources by executing annotation stages and automatic synchronization stages simultaneously. Data processing stages and connections between them are expressed by using collaborative computation workflows represented as loosely connected directed acyclic graphs. A synchronous algorithm for executing such workflows has been described. The computational model has been evaluated by applying it to two tasks from the computational linguistics field: concept lexicalization refining in electronic thesauri and establishing hierarchical relations between such concepts. The “Add–Remove–Confirm” procedure is designed for adding the missing lexemes to the concepts while removing the odd ones. The “Genus–Species–Match” procedure is designed for establishing “is-a” relations between the concepts provided with the corresponding word pairs. The experiments involving both volunteers from popular online social networks and paid workers from crowdsourcing marketplaces confirm applicability of these procedures for enhancing lexical resources. В последнее время краудсорсинг на основе выполения микрозадач получил широкое применение в области анализа неструктурированных данных. Разрабатываются специализированные методики, состоящие из множества этапов обработки исходных данных, требующих согласованности их представления для обеспечения воспроизводимости работы. Данная статья посвящена решению проблемы воспроизводимости и формализации процесса краудсорсинга микрозадачами. Предложена модель коллективных потоковых вычислений на основе расширенной реляционной модели и потоковой модели вычислений. Модель предназначена для обработки исходных данных в виде реляционных отношений путем параллельного выполнения этапов разметки микрозадачами и этапов автоматической синхронизации. Этапы обработки данных и связи между ними записываются с использованием схемы коллективных вычислений, представляющей собой слабо связный ориентированный ациклический граф. Описан синхронный алгоритм выполнения схем коллективных вычислений. Продемонстрированы приложения модели в области компьютерной лингвистики для уточнения лексикализации понятий в электронных тезаурусах и построения родо-видовых отношений между понятиями при помощи краудсорсинга. Процедура «добавить–удалить–подтвердить» позволяет внести в лексикализацию понятий недостающие лексемы и исключить посторонние. Процедура «род–вид–сопоставить» позволяет сформировать гипо-гиперонимические отношения между понятиями на основе соответствующих родо-видовых пар слов. Результаты экспериментов на материалах открытого электронного тезауруса русского языка подтверждают применимость разработанных процедур для развития лексических ресурсов. В экспериментах приняли участие как волонтеры из популярных социальных сетей, так и пользователи бирж краудсорсинга (за вознаграждение в форме микроплатежей).

A Framework for Time-Controlled and Portable WSN Applications

Abstract. Body sensor network applications require a large amount of data to be communicated over radio frequency. The radio transceiver is typically the largest source of power dissipation; improvements on energy consumption can thus be achieved by enabling on-node processing to reduce the number of packets to be transmitted. On-node processing is facilitated by a timely control over process execution to sequence operations on data; yet, the latter must be enabled while keeping highlevel software abstracted from both underlying software and hardware intricacies to accommodate portability to the wide range of hardware and software platforms. We investigated the challenges of implementing software services for on-node processing and devised constructs and system abstractions that integrate applications, drivers, time synchronization and MAC functionality into a system software which presents limited dependency between components and enables timely control of processes. We support our claims with a performance evaluation of the software tools implemented within the FreeRTOS micro-kernel

An Architecture for On-Demand Wireless Sensor Networks

abstract: Majority of the Sensor networks consist of low-cost autonomously powered devices, and are used to collect data in physical world. Today's sensor network deployments are mostly application specific & owned by a particular entity. Because of this application specific nature & the ownership boundaries, this modus operandi hinders large scale sensing & overall network operational capacity. The main goal of this research work is to create a mechanism to dynamically form personal area networks based on mote class devices spanning ownership boundaries. When coupled with an overlay based control system, this architecture can be conveniently used by a remote client to dynamically create sensor networks (personal area network based) even when the client does not own a network. The nodes here are "borrowed" from existing host networks & the application related to the newly formed network will co-exist with the native applications thanks to concurrency. The result allows users to embed a single collection tree onto spatially distant networks as if they were within communication range. This implementation consists of core operating system & various other external components that support injection maintenance & dissolution sensor network applications at client's request. A large object data dissemination protocol was designed for reliable application injection. The ability of this system to remotely reconfigure a network is useful given the high failure rate of real-world sensor network deployments. Collaborative sensing, various physical phenomenon monitoring also be considered as applications of this architecture.Dissertation/ThesisM.S. Computer Science 201

Run-time fault diagnosis in wireless sensor systems

Software failures in wireless sensor systems are notoriously difficult to debug. Resource constraints in wireless deployments substantially restrict visibility into the root causes of system and application level faults. At the same time, the high deployment cost of wireless sensor systems often far exceeds the cumulative cost of all other sensor hardware, such that software failures that completely disable a node are prohibitively expensive to repair in real-world applications, e.g. by on-site visits to replace or reset nodes. This thesis describes NodeMD, a fault management system designed to improve node debugging capabilities prior to deployment, and enable remote debugging on in-situ sensor nodes that fail. This system successfully implements lightweight run-time detection, logging, and notification of software faults on wireless mote-class devices. NodeMD introduces a debug mode that catches a failure before it completely disables a node and drops the node into a state that enables further diagnosis and correction, thus avoiding on-site redeployment. We present a detailed analysis of NodeMD on real world applications of wireless sensor systems

Simplifying Embedded System Development Through Whole-Program Compilers

As embedded systems embrace ever more complicated microcontrollers, they present both new capability and new complexity. To simplify their development, some lessons of computer application development will translate with additional work. This thesis offers one such translation. It shows how whole-program compilers - those that broadly analyze a program\u27s entire source code - can achieve performance gains and remove faults in embedded system applications. In so doing, this yields a novel stackless threading system named UnStacked C. UnStacked C enables cooperative multithreading without the risk of stack overflows in embedded system applications. We also propose a novel preemption system called Lazy Preemption. Unstacked C with Lazy Preemption enables stackless preemptive multithreading in embedded systems. These remove the possibility of thread stack overflows, but also significantly reduces the memory required for multithreading in embedded system

Abstractions for safe concurrent programming in networked embedded systems

Over the last several years, large-scale wireless mote networks have made possible the exploration of a new class of highly-concurrent and highly-distributed applications. As the horizon of what kinds of applications can be built on these networked embedded systems keeps expanding, there is a need to keep the activity of programming such systems easy, efficient, and scalable. We make three major contributions in this paper. First, we present a library for TinyOS and nesC that enables true multi-threading on a mote. This library includes support for all mote platforms in use currently (AVR, MSP). Second, we present a tool that can effectively and accurately compute stack requirements for multithreaded programs. Such analysis ensures that the stacks allocated to individual threads are correctly sized. Finally, we present a collection of programming abstractions that simplifies the construction of concurrent systems for the mote platform. We also present experimental results obtained from several example systems built using our concurrent programming abstractions and the underlying thread library. Categories and Subject Descriptors C.3 [Special purpose and Application-based systems]: Real-time and embedded systems; D.1.3 [Concurrent Programming]: Parallel programming; D.3.3 [Language Constructs and Features]: Concurrent programming structure