Modern applications in this digital age collect a staggering amount of time series data from economic growth rates to electrical household consumption habits. To make sense of it, domain analysts interactively sift through these time series collections in search of critical relationships between and recurring patterns within these time series. The ONEX (Online Exploration of Time Series) system supports effective exploratory analysis of time series collections composed of heterogeneous, variable-length and misaligned time series using robust alignment dynamic time warping (DTW) methods. To assure real-time responsiveness even for these complex and compute-intensive analytics, ONEX precomputes and then encodes time series relationships based on the inexpensive-to-compute Euclidean distance into the ONEX base. Thereafter, based on a solid formal foundation, ONEX uses DTW-enhanced analytics to correctly extract relevant time series matches on this Euclidean-prepared ONEX base. Our live interactive demonstration shows how our ONEX exploratory tool, supported by a rich array of visual interactions and expressive visualizations, enables efficient mining and interpretation of the MATTERS real data collection composed of economic, social, and education data trends across the fifty American states. © 2017 ACM

Ahsan, R.

Lovering, C.

Neamtu, R.

Nguyen, C.

Rundensteiner, Elke

Sárközy, Gábor

Rodica Neamtu

Ramoza Ahsan

Charles Lovering

Cuong Nguyen

Elke Rundensteiner

Gabor Sarkozy

Crossref

Interactive Time Series Analytics Powered by ONEX

Repository of the Academy's Library

Interactive Time Series Analytics Powered by ONEXRodica Neamtu, Ramoza Ahsan, Charles Lovering, Cuong Nguyen,Elke A. Rundensteiner, and Gabor SarkozyWorcester Polytechnic Institute, Worcester, MA 01609rneamtu | rahsan | cjlovering | ctnguyendinh | rundenst | gsarkozy @wpi.eduABSTRACTModern applications in this digital age collect a stagger-ing amount of time series data from economic growth ratesto electrical household consumption habits. To make senseof it, domain analysts interactively sift through these timeseries collections in search of critical relationships betweenand recurring patterns within these time series. The ONEX(Online Exploration of Time Series) system supports effec-tive exploratory analysis of time series collections composedof heterogeneous, variable-length and misaligned time se-ries using robust alignment dynamic time warping (DTW)methods. To assure real-time responsiveness even for thesecomplex and compute-intensive analytics, ONEX precom-putes and then encodes time series relationships based on theinexpensive-to-compute Euclidean distance into the ONEXbase. Thereafter, based on a solid formal foundation, ONEXuses DTW-enhanced analytics to correctly extract relevanttime series matches on this Euclidean-prepared ONEX base.Our live interactive demonstration shows how our ONEX ex-ploratory tool, supported by a rich array of visual interac-tions and expressive visualizations, enables efficient miningand interpretation of the MATTERS real data collectioncomposed of economic, social, and education data trendsacross the fifty American states.1. INTRODUCTIONMotivation and Background. In the era when timeseries are prevalent in most application domains, we extractimportant insights from time series datasets. Let’s considerthe use case of finding and leveraging time series similaritiesfor decision making. In 2013 in Massachusetts, organizationsset out to repeal the Sales and Use Tax on computer andsoftware services, perceived as potentially having a negativeimpact on the economic health of the state. Vast amountsof data were analyzed to show similarities between tax ratesand fluctuations of social and economic factors, all mod-eled as time series, obtained from a large spectrum of pub-Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.SIGMOD’17, May 14-19, 2017, Chicago, IL, USAc© 2017 ACM. ISBN 978-1-4503-4197-4/17/05. . . $15.00DOI: http://dx.doi.org/10.1145/3035918.3058729lic governmental websites such as the Tax Policy Center1,the Census Bureau2, and the Bureau of Economic Analysis3. Finding and interpreting the similarities between diverseeconomic indicators represented as time series were centralto this process.(1) Analysts had to answer complex questions that were notalways based on traditional similarity searches. For exam-ple, they had to look for recurring similarity patterns in thegrowth or unemployment rate of a state over a few years. (2)The presence of data from different domains reported overspecific intervals required comparisons of time series of dif-ferent lengths and alignments, as the impact of a tax changemight play out with different time durations.(3) Data from diverse domains required using different pa-rameter settings such as similarity thresholds, leading torepeated and redundant computations for each parametervalue. For example, the most suitable thresholds for study-ing similarity of demographic data are different than thoseused for growth rates.Challenges. To enable interactive time series analytics,many challenges must be tackled:1. High data cardinality leading to decreased responsiveness.Time series datasets such as census, financial and tax datatend to be huge. To answer questions involving varying tem-poral granularities one needs to consider the whole time se-ries as well as all shorter subsequences. Given the huge num-ber of such subsequences, performing similarity comparisonsamong them is impractical.2. Sequences of different lengths and alignments. To per-form meaningful comparisons between sequences of differentlengths and alignments, complex elastic distances such asDynamic Time Warping (DTW) [2] must be used. Its pop-ularity among robust alignment tools is only overshadowedby its high computational cost [6]. The exploration of thehuge number of pairwise similarity comparisons between se-quences using DTW leads inherently to long response times.Many systems settle either on increased latency or on de-creased accuracy using cheaper-to-compute distances.3. Offering rich classes of exploratory queries. As discussedabove, to gain insights into data more operations are needed,including finding repeating patterns and best matches todiscover similarity patterns, all performed using varying pa-rameters such as similarity thresholds.State of the Art. Most systems face the trade-off be-tween accuracy and time response especially when dealing1http://www.taxpolicycenter.org/2www.census.gov3www.bea.gov/1595with high volumes of data. Some provide an exact or ahighly accurate solution [7] at the expense of responsiveness.Others [1] use preprocessing steps to improve the timely re-sponsiveness, but their requirement for setting many differ-ent parameters limits their efficiency [6]. Clearly, a dilemmaexists between choosing complex similarity distances andtime responsiveness, namely, while shorter time responsesare guaranteed by the use of fast-to-compute distances likethe Euclidean Distance [4], such distances cannot handle se-quences with different time alignments. Meaningful compar-isons of such sequences require the use of time-warping dis-tances like DTW, whose computational complexity [3] leadsto slow responsiveness and poor scaling as data grows.ONEX: Our Approach for Time Series Analytics.ONEX offers a viable compromise between real time respon-siveness and the high computational complexity needs of theelastic distances. We counter the prohibitively expensive useof DTW by employing a “marriage” of two distances: we usethe computationally inexpensive Euclidean Distance to con-struct compact similarity groups for specific lengths, andthen we support the exploration of these groups using therobust time-warping method DTW [2]. This unique com-bination yields very accurate results at much reduced re-sponse time rates, as DTW is successfully applied over thecompact ONEX base instead of the raw data [5]. Our theo-retical foundation rests on the proof of a triangle inequalitybetween ED and DTW that builds a conceptual bridge be-tween the oﬄine construction of the ONEX base and itsonline exploration. Offering a novel answer to the trade-offdilemma between the use of complex time warped distancesand timely responsiveness, ONEX has been shown to be sev-eral times faster than the fastest known method [6], whilestill delivering up to 19% more accurate results [5]. Ourweb-based visual analytics interface enable analysts to ex-plore and directly interact with time series data sets in anintuitive manner using rich classes of operations. Further,diverse visualizations capture the matched and/or clusteredtimes series data and their similarity relationships to enrichthe interpretation of the matches between sequences.Our live interactive demonstration on real world economicand social data sets illustrates how our ONEX visual webinterface enables analysts to interactively explore similarityand its recurring patterns.2. THE ONEX FRAMEWORKThe ONEX infrastructure is depicted in Fig. 1. Thepre-processing step (top) encodes similarity relationships be-tween time series into “ONEX similarity groups” using Eu-clidean Distance. Exploration of these groups is achieved byDTW-empowered operations via the query processor (mid-dle). The middle of the figure indicates the diverse ex-ploratory operations performed by ONEX which include re-trieving of the best match for a given sample sequence, find-ing repeating patterns of specific time series or seasonal sim-ilarity, and showing the changes in the similarity betweensequences for varying parameters. Lastly, the visual ana-lystics interface, depicted at the bottom, enables the user toview, analyze and interpret information for interactive timeseries exploration.3. KEY INNOVATIONSThe theoretical foundation of ONEX guarantees efficientFigure 1: ONEX Frameworkexploration of time series datasets by first creating a com-pact ONEX base using ED and then enabling time-warpedcomparisons between any sample sequence and this compactset of representative sequences in the ONEX base.3.1 ONEX Base: Using Clustering for DataReductionOur ONEX base preserves and compacts similarity rela-tionships between subsequences of the time series. We firstgroup subsequences of the same length that are similar usingthe ubiquitous and inexpensive Euclidean Distance into socalled “ONEX similarity groups”. We then summarize thesegroups by their centroid, or the average of all sequences ineach group, to be the representative of that group. Our con-struction methodology insures that theses similarity groupscontain sequences that are similar to each other within thesimilarity threshold ST, while each sequence is similar to therepresentative within half of the similarity threshold. Theuse of the compact ONEX base instead of the entire datasetfor exploration using DTW guarantees speed-up while as-suring highly accurate results.3.2 Time-Warped ExplorationModel Based onFundamental Similarity MappingOur ONEX time-warped retrieval framework rests uponthe proven insight of a triangle inequality between ED andDTW. Namely, the DTW similarity between a sample se-quence seq provided by the analyst and a match found inform of a representative of one of the ONEX similarity groupscan be proven to hold true for all subsequences in that re-spective group. This empowers ONEX to perform time-warped comparisons of the sample sequence over the com-pact ONEX base instead of the entire dataset. Namely, itguarantees that the best match to a sample sequence seq isfound in the group with the “best match representative” andthe DTW between seq and its best match is always withinthe similarity threshold ST.1596Figure 2: Similarity View with Overview Pane, Query Selection Pane, Query Preview Pane, and Similarity Results Paneshowed respectively in counter-clockwise order.3.3 ONEXExploratoryModel and ProcessingThe ONEX conceptual framework provides rich classesof exploratory operations as will be demonstrated using oureconomic analytics use case. These include similarity queriesthat retrieve the best match for a given sequence. Usingthese, referring back to our motivating example, an ana-lyst can find the state that has the most similar economicgrowth rate with that of MA. Seasonal similarity queriesfind repeated patterns within a given time series. Thus onecould find if a specific growth or decline in the economicalgrowth of MA has previously been experienced in this state.Threshold recommendations help analysts to select appro-priate parameter settings in a data-driven fashion. This isimportant as the similarity in growth rate percentages mayrequire very small thresholds, whereas similarity betweenunemployment figures is expressed in tens of thousands ofpeople uses higher thresholds. To efficiently support this di-versity of time-warped queries, our ONEX query processordoes not only leverage the compact ONEX base, but it alsoapplies several optimization strategies ranging from index-ing of time series using bounding envelopes to early pruningof unpromising candidates.3.4 ONEX Visual AnalyticsAn array of complementary visualization techniques fromstacked lines charts to connected scatter plots compose ourONEX web interface. Visualizations are critical for inter-active time series analytics as they allow for intuitive in-teractions by analysts with large data sets. For the robustalignment of sequences of different lengths or alignments, awarping path constructed by ONEX corresponds to a a setof indices marking which points are most similar, includingpossible multiple matchings, contrary to the use of point-wise distances. Our ONEX views uniquely display these“warped” points to highlight the shape matching. Our Mul-tiple Lines Charts display dotted lines between correspond-ing points of the sequences highlighting the role of the time-warped matching. Radial Plots compact the time series to aradial display that allows analysts to evaluate how close theshapes are aligned. Similarly, the Connected Scatter Plotsshowcase the ordering of a sequence by connecting consecu-tive points.4. ONEX DEMONSTRATIONIn our demonstration, the audience will be able to directlyinteract with ONEX via an intuitive visual web interface tounderstand how it assists analysts in addressing complex so-cietal and economical questions, such as the ones describedin our motivating examples. We use real datasets from di-verse domains such as economic, census and tax datasetsfrom MATTERS4 and a power usage dataset Electricity-Load5. Our interface empowers analysts to draw insightswith ease from these collections, as described below.Data Loading into ONEX. With a click of a but-ton, analysts can load new data sets into ONEX. Loadinga new dataset, such as the MATTERS GrowthRate, trig-gers the preprocessing of this data at the server side andits loading into the respective ONEX Base. Thereafter, theONEX server provides near real-time responsiveness to theanalyst exploring the data via a client-server architecture.Making Sense of Overall Time Series Trends. Tooffer an overview of the data, the Overview Pane (Fig. 2left top) displays the representatives of the similarity groups,color-coded such that the color intensity increases propor-tional with the cardinality of sequences in the group. Thisgives a quick sense of the typical patterns within the dataset as well as the overall data distribution. Each represen-tative is shown as a small graph that captures the generalshape of the group. This supports analysts in finding the4http://matters.mhtc.org/5 www.cs.ucr.edu/ eamonn/time series data/1597states with similar growth rates. Upon drill-down, the au-dience can scroll through the states in the Query SelectionPane, each visualized by its name and a small line graphdisplaying the growth rate over the last 6 years. In Fig. 2left bottom, MA is selected from this group.Honing in On Specific Temporal Trends. The Sim-ilarity View shown in Fig. 2 on the right assists the analystin finding states with a similar economic growth rate to thatof MA, while the Seasonal View in Fig. 4 enhances the un-derstanding of a specific time series by highlighting repeat-ing patterns. The Query Preview Pane displays the chosensample query in more detail. Brushing the second half ofthe graph will focus the attention on the recent trends inMA. As the first preview graph is brushed, the upper chartis updated to show the selected subsequence in more detail.Highlighting Time-Warped Shape Matching. Whenthe analyst performs a similarity search, the best match se-quence in the dataset is displayed along with the samplequery subsequence in the Results Pane (Fig. 2 top right).The default “multiple lines” chart displays both time serieson a single graph. The “matched points” are connected withdotted lines helping the analyst get a better intuition of howsimilar the time series shapes are and their relative warping.Contrasting Trends Across Multiple Linked Per-spectives. Different visualizations illustrate different as-pects of similarity. For example, to get a richer understand-ing of the similarity between MA and ARK, the analyst canswitch to different visuals by selecting the mode via the rightmenu bar of the Results Pane. The same pair of time se-(a) Radial Chart (b) Connect Scatter PlotFigure 3: Visualizing MATTERS Tech Employmentries can now be viewed in a compacted Radial Chart (Fig.3a). This view allows a consistent compression of the data,providing the analyst with alternative views to compare se-quences. Further, in the Connected Scatter Plot (Fig. 3b),the shape is close to a 45 degree angle. This indicates thatthe match is extremely close – when a point in such plotlies on the diagonal, it has the exact same value in both se-ries. This observation coupled with the fact that all valuesare very close in range indicates that the subsequences area close match.Exploring Re-occurrence of Motives Within TimeSeries. Our ONEX exploratory tool can work with datafrom diverse domains. We showcase now its use in explor-ing electrical usage data using the ElectricityLoad collection.The similarity view provides a wealth of information aboutrepeated patterns in electricity usage. Focusing on the elec-trical consumption of a single household, Fig. 4 shows asingle time series across one year in Portugal and finds re-peated patterns within it. The alternating blue and greencoloration are used to clarify instances of consecutive seg-ments. The top graph displays a monthly pattern indicat-Figure 4: Seasonal View displaying Patterns in the PowerUsage Dataseting that this household tends to use electricity in a consis-tent manner throughout the summer months. The bottompattern shows that winter months too have similar trends,empowering the analyst to determine that a few small habitchanges could have a large savings impact.5. CONCLUSIONONEX is a truly interactive time series exploration toolthat enables efficient exploration of time series datasets basedon the combination of two similarity distances. This leads toshorter time responses compared to the fastest known state-of-the-art method. Complemented by novel visual analytics,ONEX offers actionable insights into similarity through richclasses of operations.Acknowledgments We thank the Computer Science Dept.at WPI and Fulbright for financial support. We thank Dr.David Brown and Dr. Lane Harrison from WPI for goodfeedback and suggestions. We thank former undergraduatestudents Samantha Swartz and Michael Andrews from WPIfor help with the implementation.6. REFERENCES[1] V. Athitsos et al. Approximate embedding-basedsubsequence matching of time series. In ACMSIGMOD, pages 365–378. ACM, 2008.[2] D. J. Berndt and J. Clifford. Using dynamic timewarping to find patterns in time series. In KDDworkshop, volume 10, pages 359–370. Seattle, WA, 1994.[3] S. Chu, E. Keogh, et al. Iterative deepening dynamictime warping for time series. In SDM, pages 195–212.SIAM, 2002.[4] C. Faloutsos, R. M, et al. Fast subsequence matching intime-series databases. ACM, 1994.[5] R. Neamtu et al. Interactive time series explorationpowered by the marriage of similarity distances. VLDBEndowment, 10(3):169–180, 2016.[6] T. Rakthanmanon et al. Searching and mining trillionsof time series subsequences under dynamic timewarping. In ACM SIGKDD, pages 262–270. ACM,2012.[7] Y. Sakurai et al. Stream monitoring under the timewarping distance. In ICDE, pages 1046–1055. IEEE,2007.1598

Interactive time series analytics powered by ONEX

http://real.mtak.hu/74287/1/p1595_neamtu_u.pdf

Interactive time series analytics powered by ONEX

Abstract

Similar works

Full text

Available Versions

Crossref

Repository of the Academy's Library