Unsolicited commercial e-mail (spam): integrated policy and practice

The internet offers a cost-effective medium to build better relationships with customers than has been possible with traditional marketing media. Internet technologies, such as electronic mail, web sites and digital media, offer companies the ability to expand their customer reach, to target specific communities, and to communicate and interact with customers in a highly customised manner. In the last few years, electronic mail has emerged as an important marketing tool to build and maintain closer relationships both with customers and with prospects. E-mail marketing has become a popular choice for companies as it greatly reduces the costs associated with previously conventional methods such as direct mailing, cataloguing (i.e. sending product catalogues to potential customers) and telecommunication marketing. As small consumers obtain e-mail addresses, the efficiency of using e-mail as a marketing tool will grow. While e-mail may be a boon for advertisers, it is a problem for consumers, corporations and internet service providers since it is used for sending 'spam' (junk-mail). Unsolicited commercial e-mail (UCE), which is commonly called spam, impinges on the privacy of individual internet users. It can also cost users in terms of the time spent reading and deleting the messages, as well as in a direct financial sense where users pay time-based connection fees. Spam, which most frequently takes the form of mass mailing advertisements, is a violation of internet etiquette (EEMA, 2002). This thesis shows that spam is an increasing problem for information society citizens. For the senders of spam, getting the message to millions of people is easy and cost-effective, but for the receivers the cost of receiving spam is financial, time-consuming, resource-consuming, possibly offensive or even illegal, and also dangerous for information systems. The problem is recognised by governments who have attempted legislative measures, but these have had little impact because of the combined difficulties of crossing territorial boundaries and of continuously evasive originating addresses. Software developers are attempting to use technology to tackle the problem, but spammers keep one step ahead, for example by adapting subject headings to avoid filters. Filters have difficulty differentiating between legitimate e-mail and unwanted e-mail, so that while we may reduce our junk we may also reduce our wanted messages. Putting filter control into the hands of individual users results in an unfair burden, in that there is a cost of time and expertise from the user. Where filter control is outsourced to expert third parties, solving the time and expertise problems, the cost becomes financial. Given the inadequacy of legislation, and the unreliability of technical applications to resolve the problem, there is an unfair burden on information society citizens. This research has resulted in the conclusion that cooperation between legislation and technology is the most effective way to handle and manage spam, and that therefore a defence in depth should be based on a combination of those two strategies. The thesis reviews and critiques attempts at legislation, self-regulation and technical solutions. It presents a case for an integrated and user-oriented approach, and provides recommendations

A system analysis of the spam problem

Thesis (S.M.)--Massachusetts Institute of Technology, Engineering Systems Division, Technology and Policy Program, 2005.Includes bibliographical references (leaves 90-94).This thesis considers the problem of the large amount of unwanted email that is being sent and received, which lowers the aggregate value of email as a communication medium from what it would otherwise be. This problem is commonly known as the "spam problem." Solutions to the spam problem involve curbing the adverse affects of existing technology as well as steering technology development in a socially beneficial direction. Unlike some other technology and policy problems, the reasons for the existence of this problem are well known and the desired effects of ideal solutions can be readily articulated. However, attempted solutions to date have not made much progress at solving the problem. I posit that this failure stems from the fact the spam problem is really a complex system, and that solutions to date have not been designed to interact with this system in a useful manner. I show that the spam problem is a complex system, and should be dealt with by developing strategies to holistically interact with it. Such strategies must embrace both technical and legal realities simultaneously in order to be successful. They must also avoid causing negative side effects that negate their purpose. First, I build a model of the system surrounding the spam problem in the form of a Causal Loop Diagram. This diagram shows the causal interactions between the various technical, legal, social, and economic forces that are present in the spam system. Using this diagram, I then identify a number of places that solutions could interact with this system. These places comprise a set of possible levers that could be pulled to alleviate the spam problem. This set of levers is then used to make sense of the attempted and suggested solutions to date.(cont.) Various solutions are grouped by how they interact with the system. These solution categories are then presented in detail by showing, diagrammatically, how they positively and negatively affect the spam system through their interactions with it. In so doing, I attempt to argue persuasively that much of the current energy expended toward the spam problem is largely unnecessary, and in some cases, counterproductive. I additionally argue that because of the current reality of the spam problem, i.e. particular facts, we are already in a decent position to largely solve this problem by just redirecting current efforts toward more appropriate activity. Such appropriate activity is suggested, which includes steps to increase the identifiability of email in order to enable more successful litigation. Finally, an optimistic conclusion is reached that there are no fundamental reasons why the spam problem can not be dealt with in such a manner to ensure the continued usefulness of email as a communication medium.by Gabriel R. Weinberg.S.M

Computing with Granular Words

Computational linguistics is a sub-field of artificial intelligence; it is an interdisciplinary field dealing with statistical and/or rule-based modeling of natural language from a computational perspective. Traditionally, fuzzy logic is used to deal with fuzziness among single linguistic terms in documents. However, linguistic terms may be related to other types of uncertainty. For instance, different users search ‘cheap hotel’ in a search engine, they may need distinct pieces of relevant hidden information such as shopping, transportation, weather, etc. Therefore, this research work focuses on studying granular words and developing new algorithms to process them to deal with uncertainty globally. To precisely describe the granular words, a new structure called Granular Information Hyper Tree (GIHT) is constructed. Furthermore, several technologies are developed to cooperate with computing with granular words in spam filtering and query recommendation. Based on simulation results, the GIHT-Bayesian algorithm can get more accurate spam filtering rate than conventional method Naive Bayesian and SVM; computing with granular word also generates better recommendation results based on users’ assessment when applied it to search engine

POISED: Spotting Twitter Spam Off the Beaten Paths

Cybercriminals have found in online social networks a propitious medium to spread spam and malicious content. Existing techniques for detecting spam include predicting the trustworthiness of accounts and analyzing the content of these messages. However, advanced attackers can still successfully evade these defenses. Online social networks bring people who have personal connections or share common interests to form communities. In this paper, we first show that users within a networked community share some topics of interest. Moreover, content shared on these social network tend to propagate according to the interests of people. Dissemination paths may emerge where some communities post similar messages, based on the interests of those communities. Spam and other malicious content, on the other hand, follow different spreading patterns. In this paper, we follow this insight and present POISED, a system that leverages the differences in propagation between benign and malicious messages on social networks to identify spam and other unwanted content. We test our system on a dataset of 1.3M tweets collected from 64K users, and we show that our approach is effective in detecting malicious messages, reaching 91% precision and 93% recall. We also show that POISED's detection is more comprehensive than previous systems, by comparing it to three state-of-the-art spam detection systems that have been proposed by the research community in the past. POISED significantly outperforms each of these systems. Moreover, through simulations, we show how POISED is effective in the early detection of spam messages and how it is resilient against two well-known adversarial machine learning attacks

Detecting Abnormal Behavior in Web Applications

The rapid advance of web technologies has made the Web an essential part of our daily lives. However, network attacks have exploited vulnerabilities of web applications, and caused substantial damages to Internet users. Detecting network attacks is the first and important step in network security. A major branch in this area is anomaly detection. This dissertation concentrates on detecting abnormal behaviors in web applications by employing the following methodology. For a web application, we conduct a set of measurements to reveal the existence of abnormal behaviors in it. We observe the differences between normal and abnormal behaviors. By applying a variety of methods in information extraction, such as heuristics algorithms, machine learning, and information theory, we extract features useful for building a classification system to detect abnormal behaviors.;In particular, we have studied four detection problems in web security. The first is detecting unauthorized hotlinking behavior that plagues hosting servers on the Internet. We analyze a group of common hotlinking attacks and web resources targeted by them. Then we present an anti-hotlinking framework for protecting materials on hosting servers. The second problem is detecting aggressive behavior of automation on Twitter. Our work determines whether a Twitter user is human, bot or cyborg based on the degree of automation. We observe the differences among the three categories in terms of tweeting behavior, tweet content, and account properties. We propose a classification system that uses the combination of features extracted from an unknown user to determine the likelihood of being a human, bot or cyborg. Furthermore, we shift the detection perspective from automation to spam, and introduce the third problem, namely detecting social spam campaigns on Twitter. Evolved from individual spammers, spam campaigns manipulate and coordinate multiple accounts to spread spam on Twitter, and display some collective characteristics. We design an automatic classification system based on machine learning, and apply multiple features to classifying spam campaigns. Complementary to conventional spam detection methods, our work brings efficiency and robustness. Finally, we extend our detection research into the blogosphere to capture blog bots. In this problem, detecting the human presence is an effective defense against the automatic posting ability of blog bots. We introduce behavioral biometrics, mainly mouse and keyboard dynamics, to distinguish between human and bot. By passively monitoring user browsing activities, this detection method does not require any direct user participation, and improves the user experience

XRay: Enhancing the Web's Transparency with Differential Correlation

Today's Web services - such as Google, Amazon, and Facebook - leverage user data for varied purposes, including personalizing recommendations, targeting advertisements, and adjusting prices. At present, users have little insight into how their data is being used. Hence, they cannot make informed choices about the services they choose. To increase transparency, we developed XRay, the first fine-grained, robust, and scalable personal data tracking system for the Web. XRay predicts which data in an arbitrary Web account (such as emails, searches, or viewed products) is being used to target which outputs (such as ads, recommended products, or prices). XRay's core functions are service agnostic and easy to instantiate for new services, and they can track data within and across services. To make predictions independent of the audited service, XRay relies on the following insight: by comparing outputs from different accounts with similar, but not identical, subsets of data, one can pinpoint targeting through correlation. We show both theoretically, and through experiments on Gmail, Amazon, and YouTube, that XRay achieves high precision and recall by correlating data from a surprisingly small number of extra accounts.Comment: Extended version of a paper presented at the 23rd USENIX Security Symposium (USENIX Security 14