Atomic norms occur frequently in data science and engineering problems such as matrix completion, sparse linear regression, system identification and many more. These norms are often used to convexify non-convex optimization problems, which are convex apart from the solution lying in a non-convex set of so-called atoms. For the convex part being a linear constraint, the ability of several atomic norms to solve the original non-convex problem has been analyzed by means of tangent cones. This paper presents an alternative route for this analysis by showing that atomic norm convexifcations always provide an optimal convex relaxation for some related non-convex problems. As a result, we obtain the following benefits: (i) treatment of arbitrary convex constraints, (ii) potentially obtaining solutions to the non-convex problem with a posteriori success certificates, (iii) utilization of additional prior knowledge through the design or learning of the non-convex problem
