CORE
🇺🇦
make metadata, not war
Services
Services overview
Explore all CORE services
Access to raw data
API
Dataset
FastSync
Content discovery
Recommender
Discovery
OAI identifiers
OAI Resolver
Managing content
Dashboard
Bespoke contracts
Consultancy services
Support us
Support us
Membership
Sponsorship
Community governance
Advisory Board
Board of supporters
Research network
About
About us
Our mission
Team
Blog
FAQs
Contact us
research
A parallel framework for in-memory construction of term-partitioned inverted indexes
Authors
C. Aykanat
T. Kucukyilmaz
A. Turk
Publication date
1 January 2012
Publisher
'Oxford University Press (OUP)'
Doi
Cite
Abstract
Cataloged from PDF version of article.With the advances in cloud computing and huge RAMs provided by 64-bit architectures, it is possible to tackle large problems using memory-based solutions. Construction of term-based, partitioned, parallel inverted indexes is a communication intensive task and suitable for memory-based modeling. In this paper, we provide an efficient parallel framework for in-memory construction of term-based partitioned, inverted indexes. We show that, by utilizing an efficient bucketing scheme, we can eliminate the need for the generation of a global vocabulary. We propose and investigate assignment schemes that can reduce the communication overheads while minimizing the storage and final query processing imbalance. We also present a study on how communication among processors should be carried out with limited communication memory in order to reduce the total inversion time. We present several different communication-memory organizations and discuss their advantages and shortcomings. The conducted experiments indicate promising results. © 2012 The Author. Published by Oxford University Press on behalf of The British Computer Society
Similar works
Full text
Open in the Core reader
Download PDF
Available Versions
Bilkent University Institutional Repository
See this paper in CORE
Go to the repository landing page
Download from data provider
oai:repository.bilkent.edu.tr:...
Last time updated on 12/11/2016
Bilkent University Institutional Repository
See this paper in CORE
Go to the repository landing page
Download from data provider
oai:repository.bilkent.edu.tr:...
Last time updated on 12/11/2016