Getafix: Workload-aware distributed interactive analytics

Distributed interactive analytics engines (Druid, Redshift, Pinot) need to achieve low query latency while using the least storage space. This paper presents a solution to the problem of replication of data blocks and routing of queries. Our techniques decide the replication level of individual data blocks (based on popularity, access counts), as well as output optimal placement patterns for such data blocks. For the static version of the problem (given set of queries accessing some segments), our techniques are provably optimal in both storage and query latency. For the dynamic version of the problem, we build a system called Getafix that dynamically tracks data block popularity, adjusts replication levels, dynamically routes queries, and garbage collects less useful data blocks. We implemented Getafix into Druid, the most popular open-source interactive analytics engine. Our experiments use both synthetic traces and production traces from Yahoo! Inc.’s production Druid cluster. Compared to existing techniques Getafix either improves storage space used by up to 3.5x while achieving comparable query latency, or improves query latency by up to 60% while using comparable storage.Ope

Screen real estate ownership based mechanism for negotiating advertisement display

As popularity of online video grows, a number of models of advertising are emerging. It is typically the brokers – usually the operators of websites – who maintain the balance between content and advertising. Existing approaches focus primarily on personalizing advertisements for viewer segments, with minimal decision-making capacity for individual viewers. We take a resource ownership view on this problem. We view consumers’ attention space, which can be abstracted as a display screen for an engaged viewer, as precious resource owned by the viewer. Viewers pay for the content they wish to view in dollars, as well as in terms of their attention. Specifically, advertisers may make partial payment for a viewer’s content, in return for receiving the viewer’s attention to their advertising. Our approach, named “FlexAdSense”, is based on CyberOrgs model, which encapsulates distributed owned resources for multi-agent computations. We build a market of viewers’ attention space in which advertisers can trade, just as viewers can trade in a market of content. We have developed key mechanisms to give viewers flexible control over the display of advertisements in real time. Specific policies needed for automated negotiations can be plugged-in. This approach relaxes the exclusivity of the relationship between advertisers and brokers, and empowers viewers, enhancing their viewing experience. This thesis presents the rationale, design, implementation, and evaluation of FlexAdSense. Feature comparison with existing advertising mechanisms shows how FlexAdSense enables viewers to control with fine-grained flexibility. Experimental results demonstrate the scalability of the approach, as the number of viewers increases. A preliminary analysis of user overhead illustrates minimal attention overhead for viewers as they customize their policies