Any Data, Any Time, Anywhere: Global Data Access for Science

Data access is key to science driven by distributed high-throughput computing (DHTC), an essential technology for many major research projects such as High Energy Physics (HEP) experiments. However, achieving efficient data access becomes quite difficult when many independent storage sites are involved because users are burdened with learning the intricacies of accessing each system and keeping careful track of data location. We present an alternate approach: the Any Data, Any Time, Anywhere infrastructure. Combining several existing software products, AAA presents a global, unified view of storage systems - a "data federation," a global filesystem for software delivery, and a workflow management system. We present how one HEP experiment, the Compact Muon Solenoid (CMS), is utilizing the AAA infrastructure and some simple performance metrics.Comment: 9 pages, 6 figures, submitted to 2nd IEEE/ACM International Symposium on Big Data Computing (BDC) 201

Applying computer vision for detection of diseases in plants

Early detection and quantification of diseases in food plants are critical to agriculture industry and national food security. However, limitation in technology and cost has limited the success of applying Computer Vision in Plant Science. This research builds on the recent advance of Machine Learning, GPU and smartphones to tackle the problem of fast and low cost diagnosis of plant disease. In particular, we choose soybean as the subject for applying automatic disease detection. The reason is because soybean is an important crop for the state of Iowa and an important source of food for America. The plant is however, highly vulnerable to several type of diseases. This thesis consists of two sub-analyses of soybean diseases, which are: First, detection of a single disease in soybean, particularly Sudden Death Syndrome (SDS) with high detail (including location and severity). Second, detection of multiple diseases in soybean, using mobile phones which are resource- constraine

Distributed virtual environment scalability and security

Distributed virtual environments (DVEs) have been an active area of research and engineering for more than 20 years. The most widely deployed DVEs are network games such as Quake, Halo, and World of Warcraft (WoW), with millions of users and billions of dollars in annual revenue. Deployed DVEs remain expensive centralized implementations despite significant research outlining ways to distribute DVE workloads. This dissertation shows previous DVE research evaluations are inconsistent with deployed DVE needs. Assumptions about avatar movement and proximity - fundamental scale factors - do not match WoW’s workload, and likely the workload of other deployed DVEs. Alternate workload models are explored and preliminary conclusions presented. Using realistic workloads it is shown that a fully decentralized DVE cannot be deployed to today’s consumers, regardless of its overhead. Residential broadband speeds are improving, and this limitation will eventually disappear. When it does, appropriate security mechanisms will be a fundamental requirement for technology adoption. A trusted auditing system (“Carbon”) is presented which has good security, scalability, and resource characteristics for decentralized DVEs. When performing exhaustive auditing, Carbon adds 27% network overhead to a decentralized DVE with a WoW-like workload. This resource consumption can be reduced significantly, depending upon the DVE’s risk tolerance. Finally, the Pairwise Random Protocol (PRP) is described. PRP enables adversaries to fairly resolve probabilistic activities, an ability missing from most decentralized DVE security proposals. Thus, this dissertations contribution is to address two of the obstacles for deploying research on decentralized DVE architectures. First, lack of evidence that research results apply to existing DVEs. Second, the lack of security systems combining appropriate security guarantees with acceptable overhead

On the Impact of Practical P2P Incentive Mechanisms on User Behavior

In this paper we report on the results of a large-scale measurement study of two popular peer-topeer systems, namely BitTorrent and eMule, that use practical and lightweight incentive mechanisms to encourage cooperation between users. We focus on identifying the strategic behavior of users in response to those incentive mechanisms. Our results illustrate a gap between what system designers and researchers expect from users in reaction to an incentive mechanism, and how users react to those incentives. In particular, we observe that the majority of BitTorrent users appear to cooperate well, despite the existence of known ways to tamper with the incentive mechanism, users engaging in behavior that could be regarded as cheating comprised only around 10% of BitTorrent’s population. That is, although we know that users can easily cheat, they actually do not currently appear to cheat at a large enough scale. In the eMule system, we identify several distinct classes of users based on their behavior. A large fraction of users appears to perceive cooperation as a good strategy, and openly share all the files they obtained. Other users engage in more subtle strategic choices, by actively optimizing the number and types of files they share in order to improve their standing in eMule’s waiting queues; they tend to remove files for which downloading is complete and keep a limited total volume of files shared

Self-adapting parallel metric-space search engine for variable query loads

This research focuses on automatically adapting a search engine size in response to fluctuations in query workload. Deploying a search engine in an Infrastructure as a Service (IaaS) cloud facilitates allocating or deallocating computer resources to or from the engine. Our solution is to contribute an adaptive search engine that will repeatedly re-evaluate its load and, when appropriate, switch over to a dierent number of active processors. We focus on three aspects and break them out into three sub-problems as follows: Continually determining the Number of Processors (CNP), New Grouping Problem (NGP) and Regrouping Order Problem (ROP). CNP means that (in the light of the changes in the query workload in the search engine) there is a problem of determining the ideal number of processors p active at any given time to use in the search engine and we call this problem CNP. NGP happens when changes in the number of processors are determined and it must also be determined which groups of search data will be distributed across the processors. ROP is how to redistribute this data onto processors while keeping the engine responsive and while also minimising the switchover time and the incurred network load. We propose solutions for these sub-problems. For NGP we propose an algorithm for incrementally adjusting the index to t the varying number of virtual machines. For ROP we present an ecient method for redistributing data among processors while keeping the search engine responsive. Regarding the solution for CNP, we propose an algorithm determining the new size of the search engine by re-evaluating its load. We tested the solution performance using a custom-build prototype search engine deployed in the Amazon EC2 cloud. Our experiments show that when we compare our NGP solution with computing the index from scratch, the incremental algorithm speeds up the index computation 2{10 times while maintaining a similar search performance. The chosen redistribution method is 25% to 50% faster than other methods and reduces the network load around by 30%. For CNP we present a deterministic algorithm that shows a good ability to determine a new size of search engine. When combined, these algorithms give an adapting algorithm that is able to adjust the search engine size with a variable workload

An Architecture for Global Distributed SIP Network Using IPv4 Anycast

Tato diplomová práce se zabývá metodami pro výběr nejbližší RTP proxy k VoIP klientům s použitím IP anycastu. RTP proxy servery jsou umístěny v síti Internetu a přeposílají RTP data pro VoIP klienty za síťovými překladači adres(NAT). Bez zeměpisně rozmístěných RTP proxy serverů a metod pro nalezení nejbližšího RTP proxy serveru by došlo ke zbytečnému poklesu kvality přenosu médialních dat a velkému zpoždení. Tento dokument navrhuje 4 metody a jejich porovnání s podrobnějšími rozbory metod s využitím DNS resolvování a přímo SIP protokolu. Tento dokument také obsahuje měření chování IP anycastu v porovnání mezi metrikami směrování a metrikami časovými. Nakonec dokumentu je také uvedena implemetace na SIP Express Router platformě.This thesis is about using IP anycast-based methods for locating RTP proxy servers close to VoIP clients. The RTP proxy servers are hosts on the public Internet that relay RTP media between VoIP clients in a way that accomplishes traversal over Network Address Translators (NATs). Without geographically-dispersed RTP proxy servers and methods to find one in client's proximity, voice latency may be unbearably long and dramatically reduce perceived voice quality. This document proposes four methods their comparison with further design of DNS-based and SIP-based methods. It includes IP anycast measurements that provides an overview of IP anycast behaviour in terms of routing metrics and latency metrics. It also includes implementation on SIP Express Router platform.

Security features using a distributed file system

Tese de mestrado em Segurnaça Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2011Informação sensível como por exemplo dados provenientes the firewalls ou sistemas de detecção de intrusões, é preciso que seja armazenada durante longos períodos de tempo por razões legais ou para fins de análise forense. Com o crescimento das fontes geradores deste tipo de dados dentro de uma empresa, torna-se imperioso encontrar uma solução que cumpra os requisitos de escalabilidade, segurança, disponibilidade, performance e baixa manutenção com custos controlados. Na sequência desta necessidade, este projecto visa fazer uma análise sobre vários sistemas de ficheiros distribuídos por forma a encontrar uma solução que responda aos requisitos de performance e segurança de uma aplicação interna da Portugal Telecom. Para validar a solução, o projecto inclui a concepção de um protótipo que pretende simular as condições de execução dessa aplicação.Sensitive information such as firewall logs or data from intrusion detection systems, has to be stored for long periods of time for legal reasons or for later forensic analysis. With the growth of the sources generating this type of data within a company, it is imperative to find a solution that meets the requirements of scalability, security, availability, performance and low maintenance while keeping the costs under control. Following this need, this project aims to make an analysis of several distributed file systems in order to find a solution that meets both the performance and security requirements of an internal application of Portugal Telecom. To validate the solution, the project includes the design of a prototype that aims to simulate the execution environment of that application