Adaptive grid semidefinite programming for finding optimal designs

We find optimal designs for linear models using anovel algorithm that iteratively combines a semidefinite programming(SDP) approach with adaptive grid techniques.The proposed algorithm is also adapted to find locally optimaldesigns for nonlinear models. The search space is firstdiscretized, and SDP is applied to find the optimal designbased on the initial grid. The points in the next grid set arepoints that maximize the dispersion function of the SDPgeneratedoptimal design using nonlinear programming. Theprocedure is repeated until a user-specified stopping rule isreached. The proposed algorithm is broadly applicable, andwe demonstrate its flexibility using (i) models with one ormore variables and (ii) differentiable design criteria, suchas A-, D-optimality, and non-differentiable criterion like Eoptimality,including the mathematically more challengingcasewhen theminimum eigenvalue of the informationmatrixof the optimal design has geometric multiplicity larger than 1. Our algorithm is computationally efficient because it isbased on mathematical programming tools and so optimalityis assured at each stage; it also exploits the convexity of theproblems whenever possible. Using several linear and nonlinearmodelswith one or more factors, we showthe proposedalgorithm can efficiently find optimal designs

Computing Optimal Experimental Designs via Interior Point Method

In this paper, we study optimal experimental design problems with a broad class of smooth convex optimality criteria, including the classical A-, D- and p th mean criterion. In particular, we propose an interior point (IP) method for them and establish its global convergence. Furthermore, by exploiting the structure of the Hessian matrix of the aforementioned optimality criteria, we derive an explicit formula for computing its rank. Using this result, we then show that the Newton direction arising in the IP method can be computed efficiently via Sherman-Morrison-Woodbury formula when the size of the moment matrix is small relative to the sample size. Finally, we compare our IP method with the widely used multiplicative algorithm introduced by Silvey et al. [29]. The computational results show that the IP method generally outperforms the multiplicative algorithm both in speed and solution quality

Secrecy Energy Efficiency of MIMOME Wiretap Channels with Full-Duplex Jamming

Full-duplex (FD) jamming transceivers are recently shown to enhance the information security of wireless communication systems by simultaneously transmitting artificial noise (AN) while receiving information. In this work, we investigate if FD jamming can also improve the systems secrecy energy efficiency (SEE) in terms of securely communicated bits-per- Joule, when considering the additional power used for jamming and self-interference (SI) cancellation. Moreover, the degrading effect of the residual SI is also taken into account. In this regard, we formulate a set of SEE maximization problems for a FD multiple-input-multiple-output multiple-antenna eavesdropper (MIMOME) wiretap channel, considering both cases where exact or statistical channel state information (CSI) is available. Due to the intractable problem structure, we propose iterative solutions in each case with a proven convergence to a stationary point. Numerical simulations indicate only a marginal SEE gain, through the utilization of FD jamming, for a wide range of system conditions. However, when SI can efficiently be mitigated, the observed gain is considerable for scenarios with a small distance between the FD node and the eavesdropper, a high Signal-to-noise ratio (SNR), or for a bidirectional FD communication setup.Comment: IEEE Transactions on Communication

Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning

For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any differentiable point. As a result, nonconvex machine learning is theoretically as supported as convex machine learning with a handcrafted basis in terms of the loss at differentiable local minima, except in the case when a preference is given to the handcrafted basis over the perturbable gradient basis. The proofs of these results are derived under mild assumptions. Accordingly, the proven results are directly applicable to many machine learning models, including practical deep neural networks, without any modification of practical methods. Furthermore, as special cases of our general results, this article improves or complements several state-of-the-art theoretical results on deep neural networks, deep residual networks, and overparameterized deep neural networks with a unified proof technique and novel geometric insights. A special case of our results also contributes to the theoretical foundation of representation learning.Comment: Neural computation, MIT pres