FlashProfile: A Framework for Synthesizing Data Profiles

We address the problem of learning a syntactic profile for a collection of strings, i.e. a set of regex-like patterns that succinctly describe the syntactic variations in the strings. Real-world datasets, typically curated from multiple sources, often contain data in various syntactic formats. Thus, any data processing task is preceded by the critical step of data format identification. However, manual inspection of data to identify the different formats is infeasible in standard big-data scenarios. Prior techniques are restricted to a small set of pre-defined patterns (e.g. digits, letters, words, etc.), and provide no control over granularity of profiles. We define syntactic profiling as a problem of clustering strings based on syntactic similarity, followed by identifying patterns that succinctly describe each cluster. We present a technique for synthesizing such profiles over a given language of patterns, that also allows for interactive refinement by requesting a desired number of clusters. Using a state-of-the-art inductive synthesis framework, PROSE, we have implemented our technique as FlashProfile. Across $153$ tasks over $75$ large real datasets, we observe a median profiling time of only $\sim\,0.7\,$s. Furthermore, we show that access to syntactic profiles may allow for more accurate synthesis of programs, i.e. using fewer examples, in programming-by-example (PBE) workflows such as FlashFill.Comment: 28 pages, SPLASH (OOPSLA) 201

Sensor Relationship Inference in Single Resident Smart Homes Using Time Series

Determining sensor relationships in smart environments is complex due to the variety and volume of time series information they provide. Moreover, identifying sensor relationships to connect them with actuators is difficult for smart home users who may not have technical experience. Yet, gathering information on sensor relationships is a crucial intermediate step towards more advanced smart home applications such as advanced policy generation or automatic sensor configuration. Therefore, in this thesis, I propose a novel unsupervised learning approach, named SeReIn, to automatically group sensors by their inherent relationships solely using time series data for single resident smart homes. SeReIn extracts three features from smart home time series data - Frequent Next Event (FNE), Time Delta (TD), and Frequency (FQ). It then applies Spectral Clustering, K-Means clustering, and DBSCAN to group the related sensors. The application of unsupervised learning enables this approach to operate anywhere in the smart home domain regardless of the sensor types and deployment scenarios. SeReIn functions on both large deployments consisting of around 70 sensors and small deployments of only 10 sensors. Evaluation of SeReIn on real-world smart home datasets has shown that it can recognize inherent spatial relationships. Using three different unsupervised clustering evaluation metrics: Calinski-Harabasz Score, Silhouette Score, and Davies-Bouldin Score, I ensure that SeReIn successfully builds clusters based on sensor relationships