The Ultralight project: the network as an integrated and managed resource for data-intensive science

Looks at the UltraLight project which treats the network interconnecting globally distributed data sets as a dynamic, configurable, and closely monitored resource to construct a next-generation system that can meet the high-energy physics community's data-processing, distribution, access, and analysis needs

AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

The Motivation, Architecture and Demonstration of Ultralight Network Testbed

In this paper we describe progress in the NSF-funded Ultralight project and a recent demonstration of Ultralight technologies at SuperComputing 2005 (SC|05). The goal of the Ultralight project is to help meet the data-intensive computing challenges of the next generation of particle physics experiments with a comprehensive, network-focused approach. Ultralight adopts a new approach to networking: instead of treating it traditionally, as a static, unchanging and unmanaged set of inter-computer links, we are developing and using it as a dynamic, configurable, and closely monitored resource that is managed from end-to-end. Thus we are constructing a next-generation global system that is able to meet the data processing, distribution, access and analysis needs of the particle physics community. In this paper we present the motivation for, and an overview of, the Ultralight project. We then cover early results in the various working areas of the project. The remainder of the paper describes our experiences of the Ultralight network architecture, kernel setup, application tuning and configuration used during the bandwidth challenge event at SC|05. During this Challenge, we achieved a record-breaking aggregate data rate in excess of 150 Gbps while moving physics datasets between many sites interconnected by the Ultralight backbone network. The exercise highlighted the benefits of Ultralight's research and development efforts that are enabling new and advanced methods of distributed scientific data analysis

The Design and Demonstration of the Ultralight Testbed

In this paper we present the motivation, the design, and a recent demonstration of the UltraLight testbed at SC|05. The goal of the Ultralight testbed is to help meet the data-intensive computing challenges of the next generation of particle physics experiments with a comprehensive, network- focused approach. UltraLight adopts a new approach to networking: instead of treating it traditionally, as a static, unchanging and unmanaged set of inter-computer links, we are developing and using it as a dynamic, configurable, and closely monitored resource that is managed from end-to-end. To achieve its goal we are constructing a next-generation global system that is able to meet the data processing, distribution, access and analysis needs of the particle physics community. In this paper we will first present early results in the various working areas of the project. We then describe our experiences of the network architecture, kernel setup, application tuning and configuration used during the bandwidth challenge event at SC|05. During this Challenge, we achieved a record-breaking aggregate data rate in excess of 150 Gbps while moving physics datasets between many Grid computing sites

The Design and Implementation of the Transatlantic Mission-Oriented Production and Experimental Networks

In this paper we present the design and implementation of the mission-oriented USLHCNet for HEP research community and the UltraLight network testbed. The design philosophy for these networks is to help meet the data-intensive computing challenges of the next generation of particle physics experiments with a comprehensive, network-focused approach. Instead of treating the network as a static, unchanging and unmanaged set of intercomputer links, we are developing and using it as a dynamic, configurable, and closely monitored resource that is managed from end-to-end. In this paper we will present our work in the various areas of the project including infrastructure construction, protocol research and application development. Our goal is to construct a next-generation global system that is able to meet the data processing, distribution, access and analysis needs of the particle physics community

Architectures and protocols for sub-wavelength optical networks: contributions to connectionless and connection-oriented data transport

La ràpida evolució d’Internet i l’àmplia gamma de noves aplicacions (per exemple, multimèdia, videoconferència, jocs en línia, etc.) ha fomentat canvis revolucionaris en la manera com ens comuniquem. A més, algunes d’aquestes aplicacions demanden grans quantitats de recursos d’ample de banda amb diversos requeriments de qualitat de servei (QoS). El desenvolupament de la multiplexació per divisió de longitud d’ona (WDM) en els anys noranta va fer molt rendible la disponibilitat d’ample de banda. Avui dia, les tecnologies de commutació òptica de circuits són predominants en el nucli de la xarxa, les quals permeten la configuració de canals (lightpaths) a través de la xarxa. No obstant això, la granularitat d’aquests canals ocupa tota la longitud d’ona, el que fa que siguin ineficients per a proveir canals de menor ample de banda (sub-longitud d’ona). Segons la comunitat científica, és necessari augmentar la transparència dels protocols, així com millorar l’aprovisionament d’ample de banda de forma dinàmica. Per tal de fer això realitat, és necessari desenvolupar noves arquitectures. La commutació òptica de ràfegues i de paquets (OBS/OPS), són dues de les tecnologies proposades. Aquesta tesi contribueix amb tres arquitectures de xarxa destinades a millorar el transport de dades sub-longitud d’ona. En primer lloc, aprofundim en la naturalesa sense connexió en OBS. En aquest cas, la xarxa incrementa el seu dinamisme a causa de les transmissions a ràfega. A més, les col·lisions entre ràfegues degraden el rendiment de la xarxa fins i tot a càrregues molt baixes. Per fer front a aquestes col·lisions, es proposa un esquema de resolució de col·lisions pro actiu basat en un algorisme d’encaminament i assignació de longitud d’ona (RWA) que balanceja de forma automàtica i distribuïda la càrrega en la xarxa. En aquest protocol, el RWA i la transmissió de ràfegues es basen en l’explotació i exploració de regles de commutació que incorporen informació sobre contencions i encaminament. Per donar suport a aquesta arquitectura, s’utilitzen dos tipus de paquets de control per a l’encaminament de les ràfegues i l’actualització de les regles de commutació, respectivament. Per analitzar els beneficis del nou algorisme, s’utilitzen quatre topologies de xarxa diferents. Els resultats indiquen que el mètode proposat millora en diferents marges la resta d’algorismes RWA en funció de la topologia i sense penalitzar altres paràmetres com el retard extrem a extrem. La segona contribució proposa una arquitectura híbrida sense i orientada a connexió sobre la base d’un protocol de control d’accés al medi (MAC) per a xarxes OBS (DAOBS). El MAC ofereix dos mètodes d’accés: arbitratge de cua (QA) per a la transmissió de ràfegues sense connexió, i pre-arbitratge (PA) per serveis TDM orientats a connexió. Aquesta arquitectura permet una àmplia gamma d’aplicacions sensibles al retard i al bloqueig. Els resultats avaluats a través de simulacions mostren que en l’accés QA, les ràfegues de més alta prioritat tenen garantides zero pèrdues i latències d’accés molt baixes. Pel que fa a l’accés PA, es reporta que la duplicació de la càrrega TDM augmenta en més d’un ordre la probabilitat de bloqueig, però sense afectar en la mateixa mesura les ràfegues sense connexió. En aquest capítol també es tracten dos dels problemes relacionats amb l’arquitectura DAOBS i el seu funcionament. En primer lloc, es proposa un model matemàtic per aproximar el retard d’accés inferior i superior com a conseqüència de l’accés QA. En segon lloc, es formula matemàticament la generació i optimització de les topologies virtuals que suporten el protocol per a l’escenari amb tràfic estàtic. Finalment, l’última contribució explora els beneficis d’una arquitectura de xarxa òptica per temps compartit (TSON) basada en elements de càlcul de camins (PCE) centralitzats per tal d’evitar col·lisions en la xarxa. Aquesta arquitectura permet garantir l’aprovisionament orientat a connexió de canals sub-longitud d’ona. En aquest capítol proposem i simulem tres arquitectures GMPLS/PCE/TSON. A causa del enfocament centralitzat, el rendiment de la xarxa depèn en gran mesura de l’assignació i aprovisionament de les connexions. Amb aquesta finalitat, es proposen diferents algorismes d’assignació de ranures temporals i es comparen amb les corresponents formulacions de programació lineal (ILP) per al cas estàtic. Per al cas de tràfic dinàmic, proposem i avaluem mitjançant simulació diferents heurístiques. Els resultats mostren els beneficis de proporcionar flexibilitat en els dominis temporal i freqüencial a l’hora d’assignar les ranures temporals.The rapid evolving Internet and the broad range of new data applications (e.g., multimedia, video-conference, online gaming, etc.) is fostering revolutionary changes in the way we communicate. In addition, some of these applications demand for unprecedented amounts of bandwidth resources with diverse quality of service (QoS). The development of wavelength division multiplexing (WDM) in the 90's made very cost-effective the availability of bandwidth. Nowadays, optical circuit switching technologies are predominant in the core enabling the set up of lightpaths across the network. However, full-wavelength lightpath granularity is too coarse, which results to be inefficient for provisioning sub-wavelength channels. As remarked by the research community, an open issue in optical networking is increasing the protocol transparency as well as provisioning true dynamic bandwidth allocation at the network level. To this end, new architectures are required. Optical burst/packet switching (OBS/OPS) are two such proposed technologies under investigation. This thesis contributes with three network architectures which aim at improving the sub-wavelength data transport from different perspectives. First, we gain insight into the connectionless nature of OBS. Here, the network dynamics are increased due to the short-lived burst transmissions. Moreover, burst contentions degrade the performance even at very low loads. To cope with them, we propose a proactive resolution scheme by means of a distributed auto load-balancing routing and wavelength assignment (RWA) algorithm for wavelength-continuity constraint networks. In this protocol, the RWA and burst forwarding is based on the exploitation and exploration of switching rule concentration values that incorporate contention and forwarding desirability information. To support such architecture, forward and backward control packets are used in the burst forwarding and updating rules, respectively. In order to analyze the benefits of the new algorithm, four different network topologies are used. Results indicate that the proposed method outperforms the rest of tested RWA algorithms at various margins depending on the topology without penalizing other parameters such as end-to-end delay. The second contribution proposes a hybrid connectionless and connection-oriented architecture based on a medium access control (MAC) protocol for OBS networks (DAOBS). The MAC provides two main access mechanisms: queue arbitrated (QA) for connectionless bursts and pre-arbitrated (PA) for TDM connection-oriented services. Such an architecture allows for a broad range of delay-sensitive applications or guaranteed services. Results evaluated through simulations show that in the QA access mode highest priority bursts are guaranteed zero losses and very low access latencies. Regarding the PA mode, we report that doubling the offered TDM traffic load increases in more than one order their connection blocking, slightly affecting the blocking of other connectionless bursts. In this chapter, we also tackle two of the issues related with the DAOBS architecture and its operation. Firstly, we model mathematically the lower and upper approximations of the access delay as a consequence of the connectionless queue arbitrated access. Secondly, we formulate the generation of the virtual light-tree overlay topology for the static traffic case.Postprint (published version

Distributed and Load-Adaptive Self Configuration in Sensor Networks

Proactive self-configuration is crucial for MANETs such as sensor networks, as these are often deployed in hostile environments and are ad hoc in nature. The dynamic architecture of the network is monitored by exchanging so-called Network State Beacons (NSBs) between key network nodes. The Beacon Exchange rate and the network state define both the time and nature of a proactive action to combat network performance degradation at a time of crisis. It is thus essential to optimize these parameters for the dynamic load profile of the network. This paper presents a novel distributed adaptive optimization Beacon Exchange selection model which considers distributed network load for energy efficient monitoring and proactive reconfiguration of the network. The results show an improvement of 70% in throughput, while maintaining a guaranteed quality-of- service for a small control-traffic overhead

Next-generation optical access seamless Evolution: concluding results of the European FP7 project OASE

Increasing bandwidth demand drives the need for next-generation optical access (NGOA) networks that can meet future end-user service requirements. This paper gives an overview of NGOA solutions, the enabling optical access network technologies, architecture principles, and related economics and business models. NGOA requirements (including peak and sustainable data rate, reach, cost, node consolidation, and open access) are proposed, and the different solutions are compared against such requirements in different scenarios (in terms of population density and system migration). Unsurprisingly, it is found that different solutions are best suited for different scenarios. The conclusions drawn from such findings allow us to formulate recommendations in terms of technology, strategy, and policy. The paper is based on the main results of the European FP7 OASE Integrated Project that ran between January 1, 2010 and February 28, 2013