Efficient Subgraph Matching on Billion Node Graphs

The ability to handle large scale graph data is crucial to an increasing number of applications. Much work has been dedicated to supporting basic graph operations such as subgraph matching, reachability, regular expression matching, etc. In many cases, graph indices are employed to speed up query processing. Typically, most indices require either super-linear indexing time or super-linear indexing space. Unfortunately, for very large graphs, super-linear approaches are almost always infeasible. In this paper, we study the problem of subgraph matching on billion-node graphs. We present a novel algorithm that supports efficient subgraph matching for graphs deployed on a distributed memory store. Instead of relying on super-linear indices, we use efficient graph exploration and massive parallel computing for query processing. Our experimental results demonstrate the feasibility of performing subgraph matching on web-scale graph data.Comment: VLDB201

Transfomer Models: From Model Inspection to Applications in Patents

L'elaborazione del linguaggio naturale viene utilizzata per affrontare diversi compiti, sia di tipo linguistico, come ad esempio l'etichettatura della parte del discorso, il parsing delle dipendenze, sia più specifiche, come ad esempio la traduzione automatica e l'analisi del sentimento. Per affrontare questi compiti, nel tempo sono stati sviluppati approcci dedicati.Una metodologia che aumenta le prestazioni in tutti questi casi in modo unificato è la modellazione linguistica, che consiste nel preaddestrare un modello per sostituire i token mascherati in grandi quantità di testo, in modo casuale all'interno di pezzi di testo o in modo sequenziale uno dopo l'altro, per sviluppare rappresentazioni di uso generale che possono essere utilizzate per migliorare le prestazioni in molti compiti contemporaneamente.L'architettura di rete neurale che attualmente svolge al meglio questo compito è il transformer, inoltre, le dimensioni del modello e la quantità dei dati sono essenziali per lo sviluppo di rappresentazioni ricche di informazioni. La disponibilità di insiemi di dati su larga scala e l'uso di modelli con miliardi di parametri sono attualmente il percorso più efficace verso una migliore rappresentazione del testo.Tuttavia, i modelli di grandi dimensioni comportano una maggiore difficoltà nell'interpretazione dell'output che forniscono. Per questo motivo, sono stati condotti diversi studi per indagare le rappresentazioni fornite da modelli di transformers.In questa tesi indago questi modelli da diversi punti di vista, studiando le proprietà linguistiche delle rappresentazioni fornite da BERT, per capire se le informazioni che codifica sono localizzate all'interno di specifiche elementi della rappresentazione vettoriale. A tal fine, identifico pesi speciali che mostrano un'elevata rilevanza per diversi compiti di sondaggio linguistico. In seguito, analizzo la causa di questi particolari pesi e li collego alla distribuzione dei token e ai token speciali.Per completare questa analisi generale ed estenderla a casi d'uso più specifici, studio l'efficacia di questi modelli sui brevetti. Utilizzo modelli dedicati, per identificare entità specifiche del dominio, come le tecnologie o per segmentare il testo dei brevetti. Studio sempre l'analisi delle prestazioni integrandola con accurate misurazioni dei dati e delle proprietà del modello per capire se le conclusioni tratte per i modelli generici valgono anche in questo contesto.Natural Language Processing is used to address several tasks, linguistic related ones, e.g. part of speech tagging, dependency parsing, and downstream tasks, e.g. machine translation, sentiment analysis. To tackle these tasks, dedicated approaches have been developed over time.A methodology that increases performance on all tasks in a unified manner is language modeling, this is done by pre-training a model to replace masked tokens in large amounts of text, either randomly within chunks of text or sequentially one after the other, to develop general purpose representations that can be used to improve performance in many downstream tasks at once.The neural network architecture currently best performing this task is the transformer, moreover, model size and data scale are essential to the development of information-rich representations. The availability of large scale datasets and the use of models with billions of parameters is currently the most effective path towards better representations of text.However, with large models, comes the difficulty in interpreting the output they provide. Therefore, several studies have been carried out to investigate the representations provided by transformers models trained on large scale datasets.In this thesis I investigate these models from several perspectives, I study the linguistic properties of the representations provided by BERT, a language model mostly trained on the English Wikipedia, to understand if the information it codifies is localized within specific entries of the vector representation. Doing this I identify special weights that show high relevance to several distinct linguistic probing tasks. Subsequently, I investigate the cause of these special weights, and link them to token distribution and special tokens.To complement this general purpose analysis and extend it to more specific use cases, given the wide range of applications for language models, I study their effectiveness on technical documentation, specifically, patents. I use both general purpose and dedicated models, to identify domain-specific entities such as users of the inventions and technologies or to segment patents text. I always study performance analysis complementing it with careful measurements of data and model properties to understand if the conclusions drawn for general purpose models hold in this context as well

Damned If You Do, Doomed If You Don\u27t: Patenting Legal Methods and its Effect on Lawyers\u27 Professional Responsibilites

Imagine, before advising each client, having to confer with the U.S. Patent and Trademark Office (USPTO) to determine whether another lawyer already owns a patent to the legal strategy you wish to propose. Imagine having to pay someone so your client can follow legal advice you wish to impart. Worse yet, imagine having to forego the most favorable legal course of action for your client simply because your client cannot afford it! While these possibilities may seem outlandish, this is precisely what courts may soon decide. Judicial affirmation of the patentability of legal strategies could become a stark reality sooner than lawyers think. In light of our patent system’s history, this prospect should come as no surprise. Both federal courts and the USPTO have continually pushed the envelope of patentable subject matter. With increasing vigor, the USPTO issues patents in previously unpatentable areas. Take Robert Slane, a financial adviser in Florida, for instance. In 2003, Slane received the first tax strategy patent, which covers the use of unqualified stock options in grantor-retained annuity trusts, or GRATs. Just three years later, Slane succeeded in filing the first suit alleging infringement of a patented tax strategy. Slane sued former chairman and CEO of Aetna Insurance, John Rowe, alleging Rowe, as part of an estate plan, funded several GRATs covered by Slane’s patent. While the parties settled in March 2007, courts will likely render legal rulings on the patentability of other legal methods in the future. This possibility seems particularly likely as professionals like Slane pursue similar measures to protect their patent rights, and as the number of patents issued for various methods increases. Since the USPTO began issuing tax strategy patents in 2003, many insist patentability for other legal methods is inevitable. To be sure, this notion will be contested. History reveals that with virtually every technological development, professionals grapple with patentability. Despite concerns about extending patent protection, history foretells that the case for patentability will likely prevail. Whatever qualms critics express about patentability, professionals learn to operate within the parameters of patent protection. Because industries in the past have adapted to patent protection, many patent experts discount concerns within the legal profession. Since other professions have adapted to patent protection, the argument goes, lawyers should adapt to patent protection as well. Indeed, not long after courts affirmed software patentability, the industry learned to adjust —albeit not without a fight. Similar reactions occurred in 1998 when the Federal Circuit extended patent protection to business methods. Tax strategies, today’s latest rave in the patent arena, have also entered the realm of patentability with ease, with the USPTO establishing a patent classification almost exclusively for tax strategies and issuing a number of patents as a result. Since tax strategies are not significantly distinct from other kinds of legal methods, the USPTO could theoretically grant patents for legal methods in other areas. In fact, the USPTO already has. With patentability efforts penetrating most professions, one might question whether patent protection will tamper with one of the underlying traits that propels our legal system—creativity. Creativity lies at the crux of any successful career. Successful lawyering is often rooted in the ability to innovatively combat complex societal problems while simultaneously serving a client’s interests. This dual responsibility coupled with the unique nature of the lawyer-client relationship sets the legal profession apart from many others. Whether relying on personal creativity or “following the paths others have blazed,” creativity, used as a tool in the legal profession, can prove life-altering—both for the client and the lawyer. As this Note will demonstrate, imposing legal restrictions on creativity has the potential not only to frustrate good lawyering efforts but also to impinge on overriding societal interests. With rumors buzzing throughout the legal community on reported patent filings for legal strategies, many wonder: Will courts extend patent protection to innovative legal methods? Would such court findings undermine societal interests? More specifically, given an attorney’s professional obligations toward clients, would patent protection contravene societal interests and unnecessarily hamper a client’s best interests? It is beyond the scope of this Note to enumerate all the problems associated with patenting legal methods and strategies. Rather, this Note aims to engage in a normative analysis to determine whether patenting legal strategies should occur in light of a lawyer’s professional duties toward a client. The primary objectives of this discussion are to raise awareness within the legal community and cast light on a number of concerns that all lawyers should consider. With so much at stake, it is incumbent that all lawyers examine the repercussions stemming from patenting legal strategies. This Note argues patent protection should not extend to legal methods because of the professional responsibilities lawyers owe to the profession and to clients. Part II explains why legal methods merit discussion today. To appreciate the effects legal method patents could have on the legal profession, Part II then presents an overview of pertinent patent law and explains how legal methods are likely eligible for patent protection. Part III raises the question whether legal methods should receive patent protection in light of social and economic considerations. Part IV addresses implications for a lawyer’s professional obligations if courts extend patent protection over legal methods. And Part V considers who bears the ultimate costs from legal method patents

What if DRM Fails?: Seeking Patronage in the iWasteland and the Virtual O

Section 1201 of the 1998 Digital Millennium Copyright Act provided sweeping protection for technological measures or virtual locks on digital content to protect the entertainment industries-including music, films, games, and consumer electronics. Manufacturers use digital rights management (DRM) authorized under the law to lock down all software embedded in products, movies on DVDs, and audio files sold on iTunes and other Internet sites. DRM unfairly extends copyright and that legal protection is unnecessary to the robust development of new creative works Critics of the DMCA have charged that the law has extended well past its anti-piracy role to undermine fair use, threaten free speech, and thwart product interoperability. Embedded in the debate are philosophical assumptions about the role of copyright to incentivize authors, musicians, artists, and other creators, as well as the role of law, technology, and social norms to enforce the customs of copyright protection. The article explains the arenas where the DMCA has benefited and hindered the broader cause of copyright and reviews the implications of copyright in the absence of technological measures for enforcement to see if there is any incentive left in the copyright regime

Decentralised computer systems

The architecture of the Web was designed to enable decentralised exchange of information. Early architects envisioned an egalitarian yet organic society thriving in cyberspace. The reality of the Web today, unfortunately, does not bear out these visions: information networks have repeatedly shown a tendency towards consolidation and centralisation with the current Web split between a handful of large corporations. The advent of Bitcoin and successor blockchain networks re-ignited interest in developing alternatives to the centralised Web and paving a way back to the earlier architectural visions for the Web. This has led to immense hype around these technologies with the cryptocurrency market valued at several hundred billions of dollars at the time of writing. With great hype, apparently, come great scams. I start off by analysing the use of Bitcoin as an enabler for crime and then present both technical solutions as well as policy recommendations to mitigate the harm these crimes cause. These policy recommendations then lead us on to look more closely at cryptocurrency's tamer cousin: permissioned blockchains. These systems, while less revolutionary in their premise, nevertheless aim to provide sweeping improvements in the efficiency and transparency of existing enterprise systems. To see whether they work in practice, I present the results of my work in delivering a production permissioned blockchain system to real users. This involves comparing several permissioned blockchain systems, exploring their deficiencies and developing solutions for the most egregious of those. Lastly, I do a deep dive into one of the most persistent technical issues with permissioned blockchains, and decentralised networks in general: the lack of scalability in their consensus mechanisms. I present two novel consensus algorithms that aim to improve upon the state of the art in several ways. The first is designed to enable existing permissioned blockchain networks to scale to thousands of nodes. The second presents an entirely new way of building decentralised consensus systems utilising a trie-based data structure at its core as opposed to the usual linear ledgers used in current systems