A Brief History of Web Crawlers

Web crawlers visit internet applications, collect data, and learn about new web pages from visited pages. Web crawlers have a long and interesting history. Early web crawlers collected statistics about the web. In addition to collecting statistics about the web and indexing the applications for search engines, modern crawlers can be used to perform accessibility and vulnerability checks on the application. Quick expansion of the web, and the complexity added to web applications have made the process of crawling a very challenging one. Throughout the history of web crawling many researchers and industrial groups addressed different issues and challenges that web crawlers face. Different solutions have been proposed to reduce the time and cost of crawling. Performing an exhaustive crawl is a challenging question. Additionally capturing the model of a modern web application and extracting data from it automatically is another open question. What follows is a brief history of different technique and algorithms used from the early days of crawling up to the recent days. We introduce criteria to evaluate the relative performance of web crawlers. Based on these criteria we plot the evolution of web crawlers and compare their performanc

Building Robust E-learning Software Systems Using Web Technologies

Building a robust e-learning software platform represents a major challenge for both the project manager and the development team. Since functionalities of these software systems improves and grows by the day, several aspects must be taken into consideration – e.g. workflows, use-casesor alternative scenarios – in order to create a well standardized and fully functional integrated learning management system. The paper will focus on a model of implementation for an e-learning software system, analyzing its features, its functional mechanisms as well as exemplifying an implementation algorithm. A list of some of the mostly used web technologies (both server-side and client-side) will be analyzed and a discussion over major security leaks of web applicationswill also be put in discussion.E-learning, E-testing, Web Technology, Software System, Web Platform

Browser-based Analysis of Web Framework Applications

Although web applications evolved to mature solutions providing sophisticated user experience, they also became complex for the same reason. Complexity primarily affects the server-side generation of dynamic pages as they are aggregated from multiple sources and as there are lots of possible processing paths depending on parameters. Browser-based tests are an adequate instrument to detect errors within generated web pages considering the server-side process and path complexity a black box. However, these tests do not detect the cause of an error which has to be located manually instead. This paper proposes to generate metadata on the paths and parts involved during server-side processing to facilitate backtracking origins of detected errors at development time. While there are several possible points of interest to observe for backtracking, this paper focuses user interface components of web frameworks.Comment: In Proceedings TAV-WEB 2010, arXiv:1009.330

VirusPKT: A Search Tool For Assimilating Assorted Acquaintance For Viruses

Viruses utilize various means to circumvent the immune detection in the biological systems. Several mathematical models have been investigated for the description of viral dynamics in the biological system of human and various other species. One common strategy for evasion and recognition of viruses is, through acquaintance in the systems by means of search engines. In this perspective a search tool have been developed to provide a wider comprehension about the structure and other details on viruses which have been narrated in this paper. This provides an adequate knowledge in evolution and building of viruses, its functions through information extraction from various websites. Apart from this, tool aim to automate the activities associated with it in a self-maintainable, self-sustainable, proactive one which has been evaluated through analysis made and have been discussed in this paper

HTTP Mailbox - Asynchronous RESTful Communication

We describe HTTP Mailbox, a mechanism to enable RESTful HTTP communication in an asynchronous mode with a full range of HTTP methods otherwise unavailable to standard clients and servers. HTTP Mailbox allows for broadcast and multicast semantics via HTTP. We evaluate a reference implementation using ApacheBench (a server stress testing tool) demonstrating high throughput (on 1,000 concurrent requests) and a systemic error rate of 0.01%. Finally, we demonstrate our HTTP Mailbox implementation in a human assisted web preservation application called "Preserve Me".Comment: 13 pages, 6 figures, 8 code blocks, 3 equations, and 3 table

Project Management Methodology for the Development of M-Learning Web Based Applications

M-learning web based applications are a particular case of web applications designed to be operated from mobile devices. Also, their purpose is to implement learning aspects. Project management of such applications takes into account the identified peculiarities. M-learning web based application characteristics are identified. M-learning functionality covers the needs of an educational process. Development is described taking into account the mobile web and its influences over the analysis, design, construction and testing phases. Activities building up a work breakdown structure for development of m-learning web based applications are presented. Project monitoring and control techniques are proposed. Resources required for projects are discussed.Software Project Management, M-Learning Applications, Web Applications, Methodology

Type-based Dependency Analysis for JavaScript

Dependency analysis is a program analysis that determines potential data flow between program points. While it is not a security analysis per se, it is a viable basis for investigating data integrity, for ensuring confidentiality, and for guaranteeing sanitization. A noninterference property can be stated and proved for the dependency analysis. We have designed and implemented a dependency analysis for JavaScript. We formalize this analysis as an abstraction of a tainting semantics. We prove the correctness of the tainting semantics, the soundness of the abstraction, a noninterference property, and the termination of the analysis.Comment: Technical Repor

Reverse Engineering and Testing of Rich Internet Applications

The World Wide Web experiences a continuous and constant evolution, where new initiatives, standards, approaches and technologies are continuously proposed for developing more effective and higher quality Web applications. To satisfy the growing request of the market for Web applications, new technologies, frameworks, tools and environments that allow to develop Web and mobile applications with the least effort and in very short time have been introduced in the last years. These new technologies have made possible the dawn of a new generation of Web applications, named Rich Internet Applications (RIAs), that offer greater usability and interactivity than traditional ones. This evolution has been accompanied by some drawbacks that are mostly due to the lack of applying well-known software engineering practices and approaches. As a consequence, new research questions and challenges have emerged in the field of web and mobile applications maintenance and testing. The research activity described in this thesis has addressed some of these topics with the specific aim of proposing new and effective solutions to the problems of modelling, reverse engineering, comprehending, re-documenting and testing existing RIAs. Due to the growing relevance of mobile applications in the renewed Web scenarios, the problem of testing mobile applications developed for the Android operating system has been addressed too, in an attempt of exploring and proposing new techniques of testing automation for these type of applications

Towards a virtual research environment for paediatric endocrinology across Europe

Paediatric endocrinology is a medical specialty dealing with variations of physical growth and sexual development in childhood. Genetic anomalies that can cause disorders of sexual development in children are rare. Given this, sharing and collaboration on the small number of cases that occur is needed by clinical experts in the field. The EU-funded EuroDSD project (www.eurodsd.eu) is one such collaboration involving clinical centres and clinical and genetic experts across Europe. Through the establishment of a virtual research environment (VRE) supporting sharing of data and a variety of clinical and bioinformatics analysis tools, EuroDSD aims to provide a research infrastructure for research into disorders of sex development. Security, ethics and information governance are at the heart of this infrastructure. This paper describes the infrastructure that is being built and the inherent challenges in security, availability and dependability that must be overcome for the enterprise to succeed