Siva : The IPFS Search Engine

広島大学工学研究科 2019年度 修士論

Ditto: Towards Decentralised Similarity Search for Web3 Services

The Web has become an integral part of life, and over the past decade, it has become increasingly centralised, leading to a number of challenges such as censorship and control, particularly in search engines. Recently, the paradigm of the decentralised Web (DWeb), or Web3, has emerged, which aims to provide decentralised alternatives to current systems with decentralised control, transparency, and openness. In this paper we introduce Ditto, a decentralised search mechanism for DWeb content, based on similarity search. Ditto uses locality sensitive hashing (LSH) to extract similarity signatures and records from content, which are stored on a decentralised index on top of a distributed hash table (DHT). Ditto uniquely supports numerous underlying content networks and types, and supports various use-cases, including keyword-search. Our evaluation shows that our system is feasible and that our search quality, delay, and overhead are comparable to those currently accepted by users of DWeb and search systems

Improving the Security and Performance of Web Applications Running on the Distributed IPFS

While cloud computing is gaining widespread adoption these days, some challenges are emerging around security, performance, and reliability of centralized cloud resources. Decentralized services are introduced as an effective way to overcome the limitations of cloud services. Blockchain technology with its associated decentralization is used to develop decentralized application platforms. The interplanetary file system (IPFS) is built on top of a distributed system consisting of a group of nodes that shares the data and also takes advantage of blockchain to permanently store the data. The IPFS is very useful in transferring data between people. This project focuses on blockchain technology, distributed file sharing system IPFS, and their applications in software development. It talks about different types of blockchain, its advantages and challenges, and what we can do to overcome the challenges of decentralized technology

Complex queries over decentralised systems for geodata retrieval

none4sìDecentralised systems have been proved to be quite effective to allow for trusted and accountable data sharing, without the need to resort to a centralised party that collects all the information. While complete decentralisation provides important advantages in terms of data sovereignty, absence of bottlenecks and reliability, it also adds some issues concerned with efficient data lookup and the possibility to implement complex queries without reintroducing centralised components. In this paper, we describe a system that copes with these issues, thanks to a multi-layer lookup scheme based on Distributed Hash Tables that allows for multiple keyword-based searches. The service of peer nodes participating in this discovery service is controlled and rewarded for their contribution. Moreover, the governance of this process is completely automated through the use of smart contracts, thus building a Decentralised Autonomous Organization (DAO). Finally, we present a use case where road hazards are collected in order to test the goodness of our system for geodata retrieval. Then, we show results from a performance evaluation that confirm the viability of the proposal. © 2022 The Authors. IET Networks published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology. openZichichi, Mirko; Serena, Luca; Ferretti, Stefano; D'Angelo, GabrieleZichichi, Mirko; Serena, Luca; Ferretti, Stefano; D'Angelo, Gabriel

Ditto: Towards Decentralised Similarity Search for Web3 Services

The Web has become an integral part of life, and over the past decade, it has become increasingly centralised, leading to a number of challenges such as censorship and control, particularly in search engines. Recently, the paradigm of the decentralised Web (DWeb), or Web3, has emerged, which aims to provide decentralised alternatives to current systems with decentralised control, transparency, and openness. In this paper we introduce Ditto, a decentralised search mechanism for DWeb content, based on similarity search. Ditto uses locality sensitive hashing (LSH) to extract similarity signatures and records from content, which are stored on a decentralised index on top of a distributed hash table (DHT). Ditto uniquely supports numerous underlying content networks and types, and supports various use-cases, including keyword-search. Our evaluation shows that our system is feasible and that our search quality, delay, and overhead are comparable to those currently accepted by users of DWeb and search systems

Decentralized Personal Data Marketplaces: How Participation in a DAO Can Support the Production of Citizen-Generated Data

Big Tech companies operating in a data-driven economy offer services that rely on their users’ personal data and usually store this personal information in “data silos” that prevent transparency about their use and opportunities for data sharing for public interest. In this paper, we present a solution that promotes the development of decentralized personal data marketplaces, exploiting the use of Distributed Ledger Technologies (DLTs), Decentralized File Storages (DFS) and smart contracts for storing personal data and managing access control in a decentralized way. Moreover, we focus on the issue of a lack of efficient decentralized mechanisms in DLTs and DFSs for querying a certain type of data. For this reason, we propose the use of a hypercube-structured Distributed Hash Table (DHT) on top of DLTs, organized for efficient processing of multiple keyword-based queries on the ledger data. We test our approach with the implementation of a use case regarding the creation of citizen-generated data based on direct participation and the involvement of a Decentralized Autonomous Organization (DAO). The performance evaluation demonstrates the viability of our approach for decentralized data searches, distributed authorization mechanisms and smart contract exploitation

Keyword-based multimedia data lookup in decentralized systems

I sistemi decentralizzati hanno permesso agli utenti di condividere informazioni senza la presenza di un intermediario centralizzato che possiede la sovranità sui dati scambiati, rischi di sicurezza e la possibilità di colli di bottiglia. Tuttavia, sono rari i sistemi pratici per il recupero delle informazioni salvate su di essi che non includano una componente centralizzata. In questo lavoro di tesi viene presentato lo sviluppo di un'applicazione il cui scopo è quello di consentire agli utenti di caricare immagini in un'architettura totalmente decentralizzata, grazie ai Decentralized File Storage e alla successiva ricerca e recupero di tali oggetti attraverso una Distributed Hash Table (DHT) in cui sono memorizzati i necessari Content IDentifiers (CID).\\ L'obiettivo principale è stato quello di trovare una migliore allocazione delle immagini all'interno del DHT attraverso l'uso dell'International Standard Content Code (ISCC), ovvero uno standard ISO che, attraverso funzioni hash content-driven, locality-sensitive e similarity-preserving, assegna i CID IPFS delle immagini ai nodi del DHT in modo efficiente, per ridurre il più possibile i salti tra i nodi e recuperare immagini coerenti con la query eseguita. Verranno, poi, analizzati i risultati ottenuti dall'allocazione dei CID delle immagini nei nodi mettendo a confronto ISCC e hash crittografico SHA-256, per verificare se ISCC rappresenti meglio la somiglianza tra le immagini allocando le immagini simili in nodi vicini tra loro

A Survey on Content Retrieval on the Decentralised Web

The control, governance, and management of the web have become increasingly centralised, resulting in security, privacy, and censorship concerns. Decentralised initiatives have emerged to address these issues, beginning with decentralised file systems. These systems have gained popularity, with major platforms serving millions of content requests daily. Complementing the file systems are decentralised search engines and name registry infrastructures, together forming the basis of a decentralised web. This survey paper analyses research trends and emerging technologies for content retrieval on the decentralised web, encompassing both academic literature and industrial projects. Several challenges hinder the realisation of a fully decentralised web. Achieving comparable performance to centralised systems without compromising decentralisation is a key challenge. Hybrid infrastructures, blending centralised components with verifiability mechanisms, show promise to improve decentralised initiatives. While decentralised file systems have seen more mature deployments, they still face challenges such as usability, performance, privacy, and content moderation. Integrating these systems with decentralised name-registries offers a potential for improved usability with human-readable and persistent names for content. Further research is needed to address security concerns in decentralised name-registries and enhance governance and crypto-economic incentive mechanisms

A Survey on Content Retrieval on the Decentralised Web

The control, governance, and management of the web have become increasingly centralised, resulting in security, privacy, and censorship concerns. Decentralised initiatives have emerged to address these issues, beginning with decentralised file systems. These systems have gained popularity, with major platforms serving millions of content requests daily. Complementing the file systems are decentralised search engines and name registry infrastructures, together forming the basis of a decentralised web . This survey paper analyses research trends and emerging technologies for content retrieval on the decentralised web, encompassing both academic literature and industrial projects. Several challenges hinder the realisation of a fully decentralised web. Achieving comparable performance to centralised systems without compromising decentralisation is a key challenge. Hybrid infrastructures, blending centralised components with verifiability mechanisms, show promise to improve decentralised initiatives. While decentralised file systems have seen more mature deployments, they still face challenges such as usability, performance, privacy, and content moderation. Integrating these systems with decentralised name-registries offers a potential for improved usability with human-readable and persistent names for content. Further research is needed to address security concerns in decentralised name-registries and enhance governance and crypto-economic incentive mechanisms