Exploring Unconventional Sources in Big Data: A Data Lifecycle Approach for Social and Economic Analysis with Machine Learning

This study delves into the realm of leveraging unconventional sources within the domain of Big Data for conducting insightful social and economic analyses. Employing a Data Lifecycle Approach, the research focuses on harnessing the potential of linear regression, random forest, and XGBoost techniques to extract meaningful insights from unconventional data sources. The study encompasses a structured methodology involving data collection, preprocessing, feature engineering, model selection, and iterative refinement. By applying these techniques to diverse datasets, encompassing sources like social media content, sensor data, and satellite imagery, the study aims to provide a comprehensive understanding of social and economic trends. The results obtained through these methods contribute to an enhanced comprehension of the intricate relationships within societal and economic systems, further highlighting the importance of unconventional data sources in driving valuable insights for decision-makers and researchers alike

Discovery Is Never By Chance: Designing for (Un)Serendipity

Serendipity has a long tradition in the history of science as having played a key role in many significant discoveries. Computer scientists, valuing the role of serendipity in discovery, have attempted to design systems that encourage serendipity. However, that research has focused primarily on only one aspect of serendipity: that of chance encounters. In reality, for serendipity to be valuable chance encounters must be synthesized into insight. In this paper we show, through a formal consideration of serendipity and analysis of how various systems have seized on attributes of interpreting serendipity, that there is a richer space for design to support serendipitous creativity, innovation and discovery than has been tapped to date. We discuss how ideas might be encoded to be shared or discovered by ‘association-hunting’ agents. We propose considering not only the inventor’s role in perceiving serendipity, but also how that inventor’s perception may be enhanced to increase the opportunity for serendipity. We explore the role of environment and how we can better enable serendipitous discoveries to find a home more readily and immediately