International audienceExperience with the parallelization of an index generator for desktop search is presented. Several configurations of the index generator are compared on three different Intel platforms with 4, 8, and 32 cores. The optimal configurations for these platforms are not intuitive and are markedly different for the three platforms. For finding the optimal configuration, detailed measurements and experimentation were necessary. Several recommendations for parallel software design are derived from this study

Meder, David J.

Tichy, Walter F.

KITopen

  Karlsruhe Reports in Informatics 2010,9 Edited by Karlsruhe Institute of Technology,  Faculty of Informatics    ISSN 2190-4782        Parallelizing an Index Generator for Desktop Search         David J. Meder ; Walter F. Tichy                 2010   KIT –  University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association                   Please note: This Report has been published on the Internet under the following Creative Commons License: http://creativecommons.org/licenses/by-nc-nd/3.0/de. Parallelizing an Index Generator forDesktop SearchDavid J. Meder and Walter F. TichyInstitute for Program Structures and Data Organization (IPD)University of Karlsruhe76131 Karlsruhe, Germany{meder, tichy}@ipd.uni-karlsruhe.deAbstract. Experience with the parallelization of an index generator fordesktop search is presented. Several configurations of the index generatorwere compared on Intel platforms with 4, 8 and 32 cores. The optimalconfigurations were not intuitive and markedly different for the threeplatforms. For finding the optimum, detailed measurements and experi-mentation were necessary. Several recommendations for parallel softwaredesign follow from this case study.1 IntroductionDeveloping multi-threaded applications for multicore computers is challenging.Since little knowledge about parallelizing non-numeric applications is available,it is appropriate at this time to perform case studies about parallelizing a widerange of applications. By documenting these case studies, such as the BZip2 casestudy [4] or the examples covered in [7], a rational process for parallel softwaredesign might emerge. In this vain, we conducted and documented an in-depthstudy of parallelizing desktop search. Desktop search is performed on a PC, lap-top, smart phone, or similar. In its simplest form, it returns a list of files thatcontain a given combination of search terms. The search uses an inverted indexthat lists for every term the files in which the term occurs. We chose desktopsearch for several reasons: It is a wide-spread, non-numeric application availableon virtually every computing device with a file system, and it is worth paral-lelizing. The application is simple enough to permit experimenting with manyalternatives, yet challenging enough in that the optimal solution is not obvious.It is also an I/O intensive application. In an earlier study, Pankratius et al.[5] conducted a competition among student teams to parallelize desktop search.Surprisingly, the team with the best performance used software transactionalmemory, while the team in second place used locks. However, the two solutionswere not comparable, because the teams stored different amounts of data in theTechnical Report 2010-9, Institute for Program Structures and Data Or-ganization (IPD), University of Karlsruhe, Germany, May 11, 2010.index. Furthermore, the competition was performed under time pressure. Quitenaturally the question arose what the best performance would be, given enoughtime to try out several alternatives. We present our approach on how to par-allelize the index generation of desktop search using locks. We present resultsfor three different platforms: a 4-core and a 8-core Intel platform in our lab anda 32-core Intel platform made available through Intel’s public manycore testinglab [2].2 What to parallelize and how?Before writing any concurrent code, one has to identify the components thatcan and should be parallelized. In case of the index generator, there are at leastthree parts which could be parallelized independently or in combination.Filename generation: Traverse the directory hierarchy to generate the namesof the files to be indexed (Stage 1).Term extraction: Scan files and extract terms (Stage 2).Index update: Add the extracted terms to an index structure (Stage 3).At the outset of the project, we faced a number of questions: Which of thosestages are the dominant ones and worth parallelizing? Is it traversing the filesystem from some root, opening and scanning individual files, or building theindex? Or is the disk the slowest part, in which case there is no hope for signif-icant speed up? None of these questions was answerable without measurement.Furthermore, it was unclear how to parallelize stages 1 and 3.2.1 Stage 1: Filename GenerationTraversing the directory hierarchy from a given root is an I/O intensive pro-cess, whose performance depends on variables such as the number of directories,the number of files contained in directories, the number, transfer rate and seektimes of the installed drives, and the buffering of the operating system. Paral-lelizing directory traversal is difficult, because directory trees are unbalanced.Another problem is how to distribute the filenames to multiple term extractorsin a balanced way, since the file lengths are uneven. Work queues, round-robindistribution, assignment based on file lengths, or work stealing are the main op-tions considered. Concurrent access to the filename data structure or the workqueues was likely to slow everything down. We didn’t even know whether it wasworth parallelizing filename generation.2.2 Stage 2: Term ExtractionThe most I/O intensive job of the index generator is reading the files. It wasunclear how many threads could be employed in stage 2 before the file systembottlenecked; furthermore, the best configuration was likely to be dependenton platform characteristics, such as clock rate of cores, size of caches, and I/Operformance. A single configuration was not going to be optimal for all platforms.Another question was how to handle duplicates of terms: Terms typically appearmultiple times in a given document. Should a term be entered into the indexevery time it is found, or should the term extractor construct a condensed wordlist without duplicates from each file and then insert the list of terms all atonce? The former technique might overwhelm the index with locking requests,while the latter approach might simply duplicate work that the index was wellprepared to handle anyway.2.3 Stage 3: Index UpdateThe main question concerns the relative speeds of index update and term ex-traction. Would it be enough to let the extractor threads update the index witha synchronized update method, or would it pay to have a separate process forindex update that received sets of terms via a buffer? Is synchronization thebottleneck? If so, there is a way to avoid synchronization entirely, by applying apattern we call ”Join Forces”. The idea behind this pattern is to let each termextractor build its own index and join the indices at the end. This approachwould eliminate all synchronization, except for a barrier before the join opera-tion. Would it be enough to join the indices with a single thread, or should aparallel reduction setup with multiple joining processes be used?3 ParallelizationTo answer some of the questions posed in the previous section, we needed toget some facts about the performance of the various stages. The first step wasto set up a benchmark. It consists of about 51.000 ASCII text files, containingmany small files and five large text files. On the whole the file set containsabout 869 MB of data, created by extracting plain text versions from wordprocessor files. Handling complex word processor formats directly in the termextractor would have been too distracting at the time, even though it would bean interesting extension now. Plain text made scanning faster, but it also madethe parallelization problem harder: the faster the term extractor runs, the lessopportunity for speedup exists.Next, we implemented a sequential version of the index generator and timedthe individual parts. The execution times are shown in Table 1. Generatingfilenames only takes 5 seconds, or between 2 to 5 percent of total runtime. Withthis information, it was clear that parallelizing the file system traversal wasunnecessary. To avoid synchronization operations, we decided to use a singlethread for stage 1, which would generate the complete set of filenames in mainmemory before starting term extraction.The next question was whether scanning the files was worth parallelizing, orwhether the whole program was I/O-bound. To decide this, we built an emptyscanner, i.e., a loop that simply reads each file byte by byte, but without anyterm extraction. Reading the benchmark from start to finish takes between 77-80Table 1. Execution times for sequential index generationExecution time (s)filenamegenerationread filesread files andextract termsindexupdate4-core platform 5.0 77.0 88.0 22.08-core platform 4.0 47.0 61.0 29.032-core platform 5.0 73.0 80.0 28.0seconds on the three platforms. Extracting the terms adds another 7-14 seconds.(For more complex formats, this part would take longer.) Now it was obviousthat the sequential version was not I/O bound. For safety, a back-of-the envelopecomparison with disk transfer and seek times confirmed that there was enoughI/O bandwidth for reading multiple files in parallel. However, we still neededa balanced work distribution. After trying a distribution that took file sizesinto account, we found that simply assigning files round-robin was the fastestapproach. Given k term extractors, the filename generator fills k vectors withfilenames in round-robin fashion. Each term extractor then processes its privatevector of filenames without any interference or synchronization. Running thefilename generator concurrently with the term extractors proved to be highlyinefficient, because of a pair of lock operations for every filename generated andconsumed.The most difficult part was the interaction with the index. With a few tests,it became clear that having a single index for all threads was not always a goodchoice. But we didn’t know what the right balance was. Only experimentationwould answer this question. The next section provides some of the data points.In some of the experiments, we used the auto-tuner by Scha¨fer et al. [7], butcouldn’t use it throughout, because this auto-tuner was built for C#, while ourimplementation was written in C++ for extra speed.The problem of how to handle term duplicates was not answered by mea-surement, but by analysis. The question was whether each term extractor shouldimplement a private index for eliminating duplicates, or whether term extractorsshould insert terms immediately (and potentially repeatedly) into a shared index.The latter solution would be similar to the distributed map-reduce implementa-tion in [1]. But we thought that the former solution had a higher performancepotential. The lookup time would be about the same for both methods. However,the shared index must also store the filename associated with the term. This inturn means that once a term has been looked up in the index, a search mustcheck whether the pair (term, filename) had been added previously (duplicate).This linear search for duplicates is eliminated entirely if the term extractor entersthe list of terms per file en bloc, without duplicates: Since each file is scannedexactly once, we need not check whether the filename already exists in this case.We chose to implement this approach in all configurations. This choice also hasthe benefit of passing large chunks of data from term extractor to index, whichreduces the number of buffering and locking operations. Perhaps the distributedmap-reduce implementation of index generation would also benefit from thistechnique.We implemented the index with a hash map provided by the Boost C++Library. The duplicate elimination in the term extractors uses a hash set. Bothdata structures use the FNV1 hash function [3] to calculate the hash values.4 Performance ResultsThe following three alternative implementations of the index generator have beencompared:Implementation 1: Use a single shared index and lock it on update.Implementation 2: Replicate the shared index and join the replicates at theend.Implementation 3: Same as Implementation 2, but don’t join indices (becausethe search can work with multiple indices in parallel).We ran those implementations on three systems: A 4-core Intel machine (IntelCore2Quad Q6600, 2.4 GHz, 4 GB RAM, Windows 7 64 bit), a 8-core Intelmachine (Intel Xeon E5320, 1.86 GHz, 8 GB RAM, Ubuntu 8.10 64 bit) and a32-core Intel machine (Intel Xeon X7560, 2.27 GHz, 8 GB RAM, RHEL 4 64bit). Each of the implementations was run using different numbers of threads forterm extraction, index update, and index joining, as discussed in section 2. Anycombination of thread counts – for example Implementation 2 running with 3threads for term extraction, 3 threads for index update and 1 thread for joiningindices – was run 5 times on each system. We report the averages per platform.The sequential implementation on the 4-core machine takes about 220 sec-onds. All three parallel implementations achieve nearly the same speed-up ofabout 4.7 (see Table 2).Table 2. Execution time and speed-up for the best configurations on the 4-core Intelmachine. Each configuration tuple (x, y, z) describes the number of threads used interm extraction, index update, and index join.4-core Intel machinebest config. exec. time (s) speed-up varianceSequential - 220.0 - -Implementation 1 (3, 1, 0) 46.7 4.71 0.0%Implementation 2 (3, 5, 1) 46.9 4.70 -0.21%Implementation 3 (3, 2, 0) 46.4 4.74 +0.85%The 8-core machine executes the sequential implementation in about 105seconds which is almost twice as fast as on the 4-core machine. The different im-plementations achieve different speed-ups as shown in Table 3. Implementation1 takes the most time to execute whereas Implementation 3 achieves the bestspeed-up of about 2.12 on this machine.Table 3. Execution time and speed-up for the best configurations on the 8-core Intelmachine. Each configuration tuple (x, y, z) describes the number of threads, used interm extraction, index update, and index join.8-core Intel machinebest config. exec. time (s) speed-up varianceSequential - 105.0 - -Implementation 1 (3, 2, 0) 59.5 1.76 0.0%Implementation 2 (6, 2, 1) 57.7 1.82 +3.4%Implementation 3 (6, 2, 0) 49.5 2.12 +16.5%On the 32-core machine, the sequential implementation takes about 90 sec-onds, which is significantly faster than on the 4-core machine. In contrast to the4-core system, the different implementations achieve different speed-ups as shownin Table 4. The performance results for this system show that Implementation1 takes longest to execute with a speed-up of about 1.96 while Implementation3 achieves a total speed-up of 3.5.Table 4. Execution time and speed-up for the best configurations on the 32-core Intelmachine. Each configuration tuple (x, y, z) describes the number of threads, used interm extraction, index update, and index join.32-core Intel machinebest config. exec. time (s) speed-up varianceSequential - 90.0 - -Implementation 1 (8, 4, 0) 45.9 1.96 0.0%Implementation 2 (8, 4, 1) 36.4 2.47 +26.0%Implementation 3 (9, 4, 0) 25.7 3.50 +78.6%5 Lessons Learned and ConclusionThere are typically numerous ways to parallelize an application, and index gen-eration is no exception. To arrive at a fast parallel implementation, one shouldproceed as follows:1. Use benchmarks and measurements to identify the components with thehighest parallelization potential.2. Beware of bottlenecks, such as I/O operations and shared data structureswith locks.3. Develop alternative parallel designs.4. Use back-of-the-envelope analysis with data from 1. to explore alternatives.5. Experiment with alternatives, where necessary. In particular, test differentthread allocations.6. Use an auto-tuner to speed up exploring the design space.We presented a rational development process; however this is not how itreally happened. We went through a number of dead ends caused by some ofthe reasons pointed out by Parnas et al. [6]: A lot of design details emergedwhile implementing the application, and we were influenced by design ideas fromprevious experience. But presenting a rational process is beneficial nevertheless,as Parnas pointed out [6]:”Those who read the software documentation want to understand theprograms, not to relive their discovery. By presenting rationalized docu-mentation we provide what they need.”By contributing this case study we hope to help make parallel software design amore rational and goal-oriented process.In the future we will analyze how to integrate the search query functionalityand parallelize it as well, for instance by using multiple indices. Better workdistribution strategies, more file formats, larger benchmarks, and more platformsare additional work items.Acknowledgments. We thank Dr. Victor Pankratius for many fruitful discus-sions and Christoph Scha¨fer for experiments with his auto-tuner. We also thankIntel for making the Manycore Testing Lab available.References1. Dean J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters,Sixth Symposium on Operating Systems Design and Implementation (2004)2. Intel Manycore Testing Lab, http://software.intel.com/en-us/articles/intel-many-core-testing-lab/ March 2010.3. Noll, L.C.: FNV hash http://isthe.com/chongo/tech/comp/fnv/ March 20104. Pankratius, V., Jannesari, A., Tichy, W.F.: Parallelizing BZip2: A Case Study inMulticore Software Engineering, IEEE Software, 70-77, November 20095. Pankratius, V., Adl-Tabatabai, A., Otto, F.: Does Transactional Memory Keep ItsPromises? Results from an Empirical Study. Technical Report 2009-12, Universityof Karlsruhe (2009)6. Parnas, D.L., Clements, P.C.: A rational design process: How and why to fake it.IEEE Trans. Softw. Eng. 12, 251–257 (1986)7. Scha¨fer, C.A., Pankratius, V., Tichy, W.F.: Engineering Parallel Applications withTunable Architectures. International Conference on Software Engineering (2010)

Parallelizing an Index Generator for Desktop Search

https://publikationen.bibliothek.kit.edu/1000018097/1978801

Parallelizing an Index Generator for Desktop Search

Abstract

Similar works

Full text

Available Versions

KITopen