Agents, Bookmarks and Clicks: A topical model of Web traffic

Analysis of aggregate and individual Web traffic has shown that PageRank is a poor model of how people navigate the Web. Using the empirical traffic patterns generated by a thousand users, we characterize several properties of Web traffic that cannot be reproduced by Markovian models. We examine both aggregate statistics capturing collective behavior, such as page and link traffic, and individual statistics, such as entropy and session size. No model currently explains all of these empirical observations simultaneously. We show that all of these traffic patterns can be explained by an agent-based model that takes into account several realistic browsing behaviors. First, agents maintain individual lists of bookmarks (a non-Markovian memory mechanism) that are used as teleportation targets. Second, agents can retreat along visited links, a branching mechanism that also allows us to reproduce behaviors such as the use of a back button and tabbed browsing. Finally, agents are sustained by visiting novel pages of topical interest, with adjacent pages being more topically related to each other than distant ones. This modulates the probability that an agent continues to browse or starts a new session, allowing us to recreate heterogeneous session lengths. The resulting model is capable of reproducing the collective and individual behaviors we observe in the empirical data, reconciling the narrowly focused browsing patterns of individual users with the extreme heterogeneity of aggregate traffic measurements. This result allows us to identify a few salient features that are necessary and sufficient to interpret the browsing patterns observed in our data. In addition to the descriptive and explanatory power of such a model, our results may lead the way to more sophisticated, realistic, and effective ranking and crawling algorithms.Comment: 10 pages, 16 figures, 1 table - Long version of paper to appear in Proceedings of the 21th ACM conference on Hypertext and Hypermedi

A Multi-perspective Analysis of Carrier-Grade NAT Deployment

As ISPs face IPv4 address scarcity they increasingly turn to network address translation (NAT) to accommodate the address needs of their customers. Recently, ISPs have moved beyond employing NATs only directly at individual customers and instead begun deploying Carrier-Grade NATs (CGNs) to apply address translation to many independent and disparate endpoints spanning physical locations, a phenomenon that so far has received little in the way of empirical assessment. In this work we present a broad and systematic study of the deployment and behavior of these middleboxes. We develop a methodology to detect the existence of hosts behind CGNs by extracting non-routable IP addresses from peer lists we obtain by crawling the BitTorrent DHT. We complement this approach with improvements to our Netalyzr troubleshooting service, enabling us to determine a range of indicators of CGN presence as well as detailed insights into key properties of CGNs. Combining the two data sources we illustrate the scope of CGN deployment on today's Internet, and report on characteristics of commonly deployed CGNs and their effect on end users

On the Interaction between TCP and the Wireless Channel in CDMA2000 Networks

In this work, we conducted extensive active measurements on a large nationwide CDMA2000 1xRTT network in order to characterize the impact of both the Radio Link Protocol and more importantly, the wireless scheduler, on TCP. Our measurements include standard TCP/UDP logs, as well as detailed RF layer statistics that allow observability into RF dynamics. With the help of a robust correlation measure, normalized mutual information, we were able to quantify the impact of these two RF factors on TCP performance metrics such as the round trip time, packet loss rate, instantaneous throughput etc. We show that the variable channel rate has the larger impact on TCP behavior when compared to the Radio Link Protocol. Furthermore, we expose and rank the factors that influence the assigned channel rate itself and in particular, demonstrate the sensitivity of the wireless scheduler to the data sending rate. Thus, TCP is adapting its rate to match the available network capacity, while the rate allocated by the wireless scheduler is influenced by the sender's behavior. Such a system is best described as a closed loop system with two feedback controllers, the TCP controller and the wireless scheduler, each one affecting the other's decisions. In this work, we take the first steps in characterizing such a system in a realistic environment

TCP over CDMA2000 Networks: A Cross-Layer Measurement Study

Modern cellular channels in 3G networks incorporate sophisticated power control and dynamic rate adaptation which can have significant impact on adaptive transport layer protocols, such as TCP. Though there exists studies that have evaluated the performance of TCP over such networks, they are based solely on observations at the transport layer and hence have no visibility into the impact of lower layer dynamics, which are a key characteristic of these networks. In this work, we present a detailed characterization of TCP behavior based on cross-layer measurement of transport layer, as well as RF and MAC layer parameters. In particular, through a series of active TCP/UDP experiments and measurement of the relevant variables at all three layers, we characterize both, the wireless scheduler and the radio link protocol in a commercial CDMA2000 network and assess their impact on TCP dynamics. Somewhat surprisingly, our findings indicate that the wireless scheduler is mostly insensitive to channel quality and sector load over short timescales and is mainly affected by the transport layer data rate. Furthermore, with the help of a robust correlation measure, Normalized Mutual Information, we were able to quantify the impact of the wireless scheduler and the radio link protocol on various TCP parameters such as the round trip time, throughput and packet loss rate

The Effects of Parent-Child Interaction Therapy on Symptoms and Impairment in Young Children with Attention-Deficit/Hyperactivity Disorder

Attention-Deficit/Hyperactivity Disorder (ADHD) is an externalizing disorder beginning in childhood with symptoms and impairment persisting into adulthood for many individuals. Although ADHD is identifiable and diagnostically valid during the preschool years, the majority of the research has focused on treating school-aged children. Some research suggests that behavioral parent training (BPT) programs are efficacious in treating young children. Yet, results are inconsistent and previous research has methodological limitations that restrict interpretations of the data. Parent-Child Interaction Therapy (PCIT), a widely-disseminated evidence-based BPT program for young children with disruptive behavior problems, incorporates components (e.g., in vivo coaching) that may be particularly effective for this population. Therefore, the current study investigated the effects of PCIT on ADHD symptoms and impairment utilizing a non-concurrent multiple baseline design across 4 children (ages 5 and 6). Results demonstrated a visible increase in child on-task behavior in 50% of the children, a decrease in maternal report of child ADHD symptoms, impairment, and ODD symptoms, and maternal report of treatment satisfaction. These findings suggest that PCIT may reduce attention and hyperactivity/impulsivity problems in addition to oppositional behaviors. Despite these promising results, the inconsistent findings pertaining to on-task behavior, persistence of school difficulties and parenting stress demonstrate the need for further intervention. Additionally, limitations pertaining to the design, sample, and measurement are discussed

Using current uptime to improve failure detection in peer-to-peer networks

Peer-to-Peer (P2P) networks share computer resources or services through the exchange of information between participating nodes. These nodes form a virtual network overlay by creating a number of connections with one another. Due to the transient nature of nodes within these systems any connection formed should be monitored and maintained to ensure the routing table is kept up-to-date. Typically P2P networks predefine a fixed keep-alive period, a maximum interval in which connected nodes must exchange messages. If no other message has been sent within this interval then keep-alive messages are exchanged to ensure the corresponding node has not left the system. A fixed periodic interval can be viewed as a centralised, static and deterministic mechanism; maintaining overlays in an predictable, reliable and non-adaptive fashion. Several studies have shown that older peers are more likely to remain in the network longer than their short-lived counterparts. Therefore using the distribution of peer session times and the current age of peers as key attributes, we propose three algorithms which allow connections to extend the interval between successive keep-alive messages based upon the likelihood that a corresponding node will remain in the system. By prioritising keep-alive messages to nodes that are more likely to fail, our algorithms reduce the expected delay between failures occurring and their subsequent detection. Using extensively empirical analysis, we analyse the properties of these algorithms and compare them to the standard periodic approach in unstructured and structured network topologies, using tracedriven simulations based upon measured network data. Furthermore we also investigate the effect of nodes that misreport their age upon our adaptive algorithms and detail an efficient keep-alive algorithm that can adapt to the limitations network address translation devices

Web Mining-Based Objective Metrics for Measuring Website Navigatability

Web site design is critical to the success of electronic commerce and digital government. Effective design requires appropriate evaluation methods and measurement metrics. The current research examines Web site navigability, a fundamental structural aspect of Web site design. We define Web site navigability as the extent to which a visitor can use a Web site’s hyperlink structure to locate target contents successfully in an easy and efficient manner. We propose a systematic Web site navigability evaluation method built on Web mining techniques. To complement the subjective self-reported metrics commonly used by previous research, we develop three objective metrics for measuring Web site navigability on the basis of the Law of Surfing. We illustrate the use of the proposed methods and measurement metrics with two large Web sites

Landmark-based approaches for goal recognition as planning

This article is a revised and extended version of two papers published at AAAI 2017 (Pereira et al., 2017b) and ECAI 2016 (Pereira and Meneguzzi, 2016). We thank the anonymous reviewers that helped improve the research in this article. The authors thank Shirin Sohrabi for discussing the way in which the algorithms of Sohrabi et al. (2016) should be configured, and Yolanda Escudero-Martın for providing code for the approach of E-Martın et al. (2015) and engaging with us. We also thank Miquel Ramırez and Mor Vered for various discussions, and Andre Grahl Pereira for a discussion of properties of our algorithm. Felipe thanks CNPq for partial financial support under its PQ fellowship, grant number 305969/2016-1.Peer reviewedPostprin

Development and Testing of a New Transport Protocol Optimized for Multimedia Internet Transactions.

The TCP/IP protocol, which carries over 95% of data across the Internet, was first published in 1974 at a time when packet-switching was a new technology and computer communications were dominated by the virtual-circuit paradigm. Computer networking has changed dramatically in the past quarter-century, but the underpinnings of TCP have remained virtually unchanged. Many of TCP's most significant design assumptions are no longer valid in the modern Internet. As a result, TCP typically exhibits extremely poor performance including congestion, underutilization of bandwidth, and server overload. Despite these facts, and increasing evidence that TCP/IP is not suited to many of the application protocols it supports, only incremental improvements have been widely researched and no viable alternatives have come to prominence. This dissertation proposes a new transport protocol, the Multimedia Transaction Protocol (MTP), which has been created to meet the needs of modern applications operating in a modern network environment. This new protocol has been designed to handle transaction style client-server interactions across an unreliable, highly congested, packet-switched network. Experimental and simulation results show that MTP provides an order of magnitude improvement in throughput while contributing to network stability and greatly reducing latency. This work characterizes the modern transport environment, describes the design and implementation of MTP, and presents initial test results

The Dynamics of Internet Traffic: Self-Similarity, Self-Organization, and Complex Phenomena

The Internet is the most complex system ever created in human history. Therefore, its dynamics and traffic unsurprisingly take on a rich variety of complex dynamics, self-organization, and other phenomena that have been researched for years. This paper is a review of the complex dynamics of Internet traffic. Departing from normal treatises, we will take a view from both the network engineering and physics perspectives showing the strengths and weaknesses as well as insights of both. In addition, many less covered phenomena such as traffic oscillations, large-scale effects of worm traffic, and comparisons of the Internet and biological models will be covered.Comment: 63 pages, 7 figures, 7 tables, submitted to Advances in Complex System