We study a superfluid in a rotating anharmonic trap and explicate a rigorous
proof of a transition from a vortex lattice to a giant vortex state as the
rotation is increased beyond a limiting speed determined by the interaction
strength. The transition is characterized by the disappearance of the vortices
from the annulus where the bulk of the superfluid is concentrated due to
centrifugal forces while a macroscopic phase circulation remains. The analysis
is carried out within two-dimensional Gross-Pitaevskii theory at large coupling
constant and reveals significant differences between 'soft' anharmonic traps
(like a quartic plus quadratic trapping potential) and traps with a fixed
boundary: In the latter case the transition takes place in a parameter regime
where the size of vortices is very small relative to the width of the annulus
whereas in 'soft' traps the vortex lattice persists until the width of the
annulus becomes comparable to the vortex cores. Moreover, the density profile
in the annulus where the bulk is concentrated is, in the 'soft' case,
approximately gaussian with long tails and not of the Thomas-Fermi type like in
a trap with a fixed boundary.Comment: Published version. Typos corrected, references adde