Probabilistic Branch-and-Bound for Clusterwise Linear Regression

Abstract

Clusterwise linear regression, a supervised learning technique that aims at finding latent groups with distinct linear relationships, has numerous applications across diverse scientific and applied domains. However, it often leads to optimization challenges due to the combinatorial nature of the problem. This paper introduces pclustreg, a novel approach based on probabilistic branch-and-bound optimization that maximizes the log-likelihood of a Gaussian mixture model. We show that, under suitable conditions, pclustreg guarantees (with a user-defined probability s<1) a solution with log-likelihood at least as good as the infeasible oracle estimator, which knows the true cluster assignments. Additionally, pclustreg can be used as a computationally lean heuristic, as showcased on simulated and real-world datasets