On Correlated Availability in Internet Distributed Systems

International audienceAs computer networks rapidly increase in size and speed, Internet-distributed systems such as P2P, volunteer computing, and Grid systems are increasingly common. A precise and accurate characterization of Internet resources is important for the design and evaluation of such Internet-distributed systems, yet our picture of the Internet landscape is not perfectly clear. To improve this picture, we measure and characterize the time dynamics of availability in a large-scale Internet-distributed system with over 110,000 hosts. Our characterization focuses on identifying patterns of correlated availability. We determine scalable and accurate clustering techniques and distance metrics for automatically detecting significant availability patterns. By means of clustering, we identify groups of resources with correlated availability that exhibit similar time effects. Then we show how these correlated clusters of resources can be used to improve resource management for parallel applications in the context of volunteer computing

Investigating grid computing technologies for use with commercial simulation packages

As simulation experimentation in industry become more computationally demanding, grid computing can be seen as a promising technology that has the potential to bind together the computational resources needed to quickly execute such simulations. To investigate how this might be possible, this paper reviews the grid technologies that can be used together with commercial-off-the-shelf simulation packages (CSPs) used in industry. The paper identifies two specific forms of grid computing (Public Resource Computing and Enterprise-wide Desktop Grid Computing) and the middleware associated with them (BOINC and Condor) as being suitable for grid-enabling existing CSPs. It further proposes three different CSP-grid integration approaches and identifies one of them to be the most appropriate. It is hoped that this research will encourage simulation practitioners to consider grid computing as a technologically viable means of executing CSP-based experiments faster

Learning by volunteer computing, thinking and gaming: What and how are volunteers learning by participating in Virtual Citizen Science?

Citizen Science (CS) refers to a form of research collaboration that engages volunteers without formal scientific training in contributing to empirical scientific projects. Virtual Citizen Science (VCS) projects engage participants in online tasks. VCS has demonstrated its usefulness for research, however little is known about its learning potential for volunteers. This paper reports on research exploring the learning outcomes and processes in VCS. In order to identify different kinds of learning, 32 exploratory interviews of volunteers were conducted in three different VCS projects. We found six main learning outcomes related to different participants' activities in the project. Volunteers learn on four dimensions that are directly related to the scope of the VCS project: they learn at the task/game level, acquire pattern recognition skills, on-topic content knowledge, and improve their scientific literacy. Thanks to indirect opportunities of VCS projects, volunteers learn on two additional dimensions: off topic knowledge and skills, and personal development. Activities through which volunteers learn can be categorized in two levels: at a micro (task/game) level that is direct participation to the task, and at a macro level, i.e. use of project documentation, personal research on the Internet, and practicing specific roles in project communities. Both types are influenced by interactions with others in chat or forums. Most learning happens to be informal, unstructured and social. Volunteers do not only learn from others by interacting with scientists and their peers, but also by working for others: they gain knowledge, new status and skills by acting as active participants, moderators, editors, translators, community managers, etc. in a project community. This research highlights these informal and social aspects in adult learning and science education and also stresses the importance for learning through the indirect opportunities provided by the project: the main one being the opportunity to participate and progress in a project community, according to one's tastes and skills

Correlated Resource Models of Internet End Hosts

Understanding and modelling resources of Internet end hosts is essential for the design of desktop software and Internet-distributed applications. In this paper we develop a correlated resource model of Internet end hosts based on real trace data taken from the SETI@home project. This data covers a 5-year period with statistics for 2.7 million hosts. The resource model is based on statistical analysis of host computational power, memory, and storage as well as how these resources change over time and the correlations between them. We find that resources with few discrete values (core count, memory) are well modeled by exponential laws governing the change of relative resource quantities over time. Resources with a continuous range of values are well modeled with either correlated normal distributions (processor speed for integer operations and floating point operations) or log-normal distributions (available disk space). We validate and show the utility of the models by applying them to a resource allocation problem for Internet-distributed applications, and demonstrate their value over other models. We also make our trace data and tool for automatically generating realistic Internet end hosts publicly available