Positive Least Energy Solutions and Phase Separation for Coupled Schrodinger Equations with Critical Exponent: Higher Dimensional Case

We study the following nonlinear Schr\"{o}dinger system which is related to Bose-Einstein condensate: {displaymath} {cases}-\Delta u +\la_1 u = \mu_1 u^{2^\ast-1}+\beta u^{\frac{2^\ast}{2}-1}v^{\frac{2^\ast}{2}}, \quad x\in \Omega, -\Delta v +\la_2 v =\mu_2 v^{2^\ast-1}+\beta v^{\frac{2^\ast}{2}-1} u^{\frac{2^\ast}{2}}, \quad x\in \om, u\ge 0, v\ge 0 \,\,\hbox{in \om},\quad u=v=0 \,\,\hbox{on \partial\om}.{cases}{displaymath} Here \om\subset \R^N is a smooth bounded domain, $2^\ast:=\frac{2N}{N-2}$ is the Sobolev critical exponent, -\la_1(\om)0 and $\beta\neq 0$, where \lambda_1(\om) is the first eigenvalue of $-\Delta$ with the Dirichlet boundary condition. When \bb=0, this is just the well-known Brezis-Nirenberg problem. The special case N=4 was studied by the authors in (Arch. Ration. Mech. Anal. 205: 515-551, 2012). In this paper we consider {\it the higher dimensional case $N\ge 5$}. It is interesting that we can prove the existence of a positive least energy solution (u_\bb, v_\bb) {\it for any $\beta\neq 0$} (which can not hold in the special case N=4). We also study the limit behavior of (u_\bb, v_\bb) as $\beta\to -\infty$ and phase separation is expected. In particular, u_\bb-v_\bb will converge to {\it sign-changing solutions} of the Brezis-Nirenberg problem, provided $N\ge 6$. In case \la_1=\la_2, the classification of the least energy solutions is also studied. It turns out that some quite different phenomena appear comparing to the special case N=4.Comment: 48 pages. This is a revised version of arXiv:1209.2522v1 [math.AP