In this paper, we study sparse group Lasso for high-dimensional double sparse
linear regression, where the parameter of interest is simultaneously
element-wise and group-wise sparse. This problem is an important instance of
the simultaneously structured model -- an actively studied topic in statistics
and machine learning. In the noiseless case, we provide matching upper and
lower bounds on sample complexity for the exact recovery of sparse vectors and
for stable estimation of approximately sparse vectors, respectively. In the
noisy case, we develop upper and matching minimax lower bounds for estimation
error. We also consider the debiased sparse group Lasso and investigate its
asymptotic property for the purpose of statistical inference. Finally,
numerical studies are provided to support the theoretical results