In this paper, we investigate a self-sensing intelligent reflecting surface
(IRS) aided millimeter wave (mmWave) integrated sensing and communication
(ISAC) system. Unlike the conventional purely passive IRS, the self-sensing IRS
can effectively reduce the path loss of sensing-related links, thus rendering
it advantageous in ISAC systems. Aiming to jointly sense the
target/scatterer/user positions as well as estimate the sensing and
communication (SAC) channels in the considered system, we propose a two-phase
transmission scheme, where the coarse and refined sensing/channel estimation
(CE) results are respectively obtained in the first phase (using scanning-based
IRS reflection coefficients) and second phase (using optimized IRS reflection
coefficients). For each phase, an angle-based sensing turbo variational
Bayesian inference (AS-TVBI) algorithm, which combines the VBI, messaging
passing and expectation-maximization (EM) methods, is developed to solve the
considered joint location sensing and CE problem. The proposed algorithm
effectively exploits the partial overlapping structured (POS) sparsity and
2-dimensional (2D) block sparsity inherent in the SAC channels to enhance the
overall performance. Based on the estimation results from the first phase, we
formulate a Cram\'{e}r-Rao bound (CRB) minimization problem for optimizing IRS
reflection coefficients, and through proper reformulations, a low-complexity
manifold-based optimization algorithm is proposed to solve this problem.
Simulation results are provided to verify the superiority of the proposed
transmission scheme and associated algorithms