Statistical learning and big data applications

The amount of data generated in the field of laboratory medicine has grown to an extent that conventional laboratory information systems (LISs) are struggling to manage and analyze this complex, entangled information (“Big Data”). Statistical learning, a generalized framework from machine learning (ML) and artificial intelligence (AI) is predestined for processing “Big Data” and holds the potential to revolutionize the field of laboratory medicine. Personalized medicine may in particular benefit from AI-based systems, especially when coupled with readily available wearables and smartphones which can collect health data from individual patients and offer new, cost-effective access routes to healthcare for patients worldwide. The amount of personal data collected, however, also raises concerns about patient-privacy and calls for clear ethical guidelines for “Big Data” research, including rigorous quality checks of data and algorithms to eliminate underlying bias and enable transparency. Likewise, novel federated privacy-preserving data processing approaches may reduce the need for centralized data storage. Generative AI-systems including large language models such as ChatGPT currently enter the stage to reshape clinical research, clinical decision-support systems, and healthcare delivery. In our opinion, AI-based systems have a tremendous potential to transform laboratory medicine, however, their opportunities should be weighed against the risks carefully. Despite all enthusiasm, we advocate for stringent added-value assessments, just as for any new drug or treatment. Human experts should carefully validate AI-based systems, including patient-privacy protection, to ensure quality, transparency, and public acceptance. In this opinion paper, data prerequisites, recent developments, chances, and limitations of statistical learning approaches are highlighted

Privacy and Accountability in Black-Box Medicine

Black-box medicine—the use of big data and sophisticated machine learning techniques for health-care applications—could be the future of personalized medicine. Black-box medicine promises to make it easier to diagnose rare diseases and conditions, identify the most promising treatments, and allocate scarce resources among different patients. But to succeed, it must overcome two separate, but related, problems: patient privacy and algorithmic accountability. Privacy is a problem because researchers need access to huge amounts of patient health information to generate useful medical predictions. And accountability is a problem because black-box algorithms must be verified by outsiders to ensure they are accurate and unbiased, but this means giving outsiders access to this health information. This article examines the tension between the twin goals of privacy and accountability and develops a framework for balancing that tension. It proposes three pillars for an effective system of privacy-preserving accountability: substantive limitations on the collection, use, and disclosure of patient information; independent gatekeepers regulating information sharing between those developing and verifying black-box algorithms; and information-security requirements to prevent unintentional disclosures of patient information. The article examines and draws on a similar debate in the field of clinical trials, where disclosing information from past trials can lead to new treatments but also threatens patient privacy

Pyramid: Enhancing Selectivity in Big Data Protection with Count Featurization

Protecting vast quantities of data poses a daunting challenge for the growing number of organizations that collect, stockpile, and monetize it. The ability to distinguish data that is actually needed from data collected "just in case" would help these organizations to limit the latter's exposure to attack. A natural approach might be to monitor data use and retain only the working-set of in-use data in accessible storage; unused data can be evicted to a highly protected store. However, many of today's big data applications rely on machine learning (ML) workloads that are periodically retrained by accessing, and thus exposing to attack, the entire data store. Training set minimization methods, such as count featurization, are often used to limit the data needed to train ML workloads to improve performance or scalability. We present Pyramid, a limited-exposure data management system that builds upon count featurization to enhance data protection. As such, Pyramid uniquely introduces both the idea and proof-of-concept for leveraging training set minimization methods to instill rigor and selectivity into big data management. We integrated Pyramid into Spark Velox, a framework for ML-based targeting and personalization. We evaluate it on three applications and show that Pyramid approaches state-of-the-art models while training on less than 1% of the raw data