2 research outputs found
Neural Locality Sensitive Hashing for Entity Blocking
Locality-sensitive hashing (LSH) is a fundamental algorithmic technique
widely employed in large-scale data processing applications, such as
nearest-neighbor search, entity resolution, and clustering. However, its
applicability in some real-world scenarios is limited due to the need for
careful design of hashing functions that align with specific metrics. Existing
LSH-based Entity Blocking solutions primarily rely on generic similarity
metrics such as Jaccard similarity, whereas practical use cases often demand
complex and customized similarity rules surpassing the capabilities of generic
similarity metrics. Consequently, designing LSH functions for these customized
similarity rules presents considerable challenges. In this research, we propose
a neuralization approach to enhance locality-sensitive hashing by training deep
neural networks to serve as hashing functions for complex metrics. We assess
the effectiveness of this approach within the context of the entity resolution
problem, which frequently involves the use of task-specific metrics in
real-world applications. Specifically, we introduce NLSHBlock (Neural-LSH
Block), a novel blocking methodology that leverages pre-trained language
models, fine-tuned with a novel LSH-based loss function. Through extensive
evaluations conducted on a diverse range of real-world datasets, we demonstrate
the superiority of NLSHBlock over existing methods, exhibiting significant
performance improvements. Furthermore, we showcase the efficacy of NLSHBlock in
enhancing the performance of the entity matching phase, particularly within the
semi-supervised setting
A Comprehensive Framework for Testing Database-Centric Software Applications
The database is a critical component of many modern software applications. Recent reports indicate that the vast majority of database use occurs from within an application program. Indeed, database-centric applications have been implemented to create digital libraries, scientific data repositories, and electronic commerce applications. However, a database-centric application is very different from a traditional software system because it interacts with a database that has a complex state and structure. This dissertation formulates a comprehensive framework to address the challenges that are associated with the efficient and effective testing of database-centric applications. The database-aware approach to testing includes: (i) a fault model, (ii) several unified representations of a program's database interactions, (iii) a family of test adequacycriteria, (iv) a test coverage monitoring component, and (v) tools for reducing and re-ordering a test suite during regression testing.This dissertation analyzes the worst-case time complexity of every important testing algorithm. This analysis is complemented by experiments that measure the efficiency and effectiveness of thedatabase-aware testing techniques. Each tool is evaluated by using it to test six database-centric applications. The experiments show thatthe database-aware representations can be constructed with moderate time and space overhead. The adequacy criteria call for test suitesto cover 20% more requirements than traditional criteria and this ensures the accurate assessment of test suite quality. It is possibleto enumerate data flow-based test requirements in less than one minute and coverage tree path requirements are normally identified in no morethan ten seconds. The experimental results also indicate that the coverage monitor can insert instrumentation probes into all six of theapplications in fewer than ten seconds. Although instrumentation may moderately increase the static space overhead of an application, the coverage monitoring techniques only increase testing time by 55% on average. A coverage tree often can be stored in less than five seconds even though the coverage report may consume up to twenty-fivemegabytes of storage. The regression tester usually reduces or prioritizes a test suite in under five seconds. The experiments also demonstrate that the modified test suite is frequently more streamlined than the initial tests