We consider nonlinear dynamics of a single vortex in a superconductor in a
strong rf magnetic field B0sinωt. Using the London theory, we
calculate the dissipated power Q(B0,ω), and the transient time scales
of vortex motion for the linear Bardeen-Stephen viscous drag force, which
results in unphysically high vortex velocities during vortex penetration
through the oscillating surface barrier. It is shown that penetration of a
single vortex through the ac surface barrier always involves penetration of an
antivortex and the subsequent annihilation of the vortex antivortex pairs.
Using the nonlinear Larkin-Ovchinnikov (LO) viscous drag force at higher vortex
velocities v(t) results in a jump-wise vortex penetration through the surface
barrier and a significant increase of the dissipated power. We calculate the
effect of dissipation on nonlinear vortex viscosity η(v) and the rf vortex
dynamics and show that it can also result in the LO-type behavior,
instabilities, and thermal localization of penetrating vortex channels. We
propose a thermal feedback model of η(v), which not only results in the LO
dependence of η(v) for a steady-state motion, but also takes into account
retardation of temperature field around rapidly accelerating vortex, and a
long-range interaction with the surface. We also address the effect of pinning
on the nonlinear rf vortex dynamics and the effect of trapped magnetic flux on
the surface resistance Rs calculated as a function or rf frequency and
field. It is shown that trapped flux can result in a temperature-independent
residual resistance Ri at low T, and a hysteretic low-field dependence of
Ri(B0), which can {\it decrease} as B0 is increased, reaching a minimum
at B0 much smaller than the thermodynamic critical field Bc.Comment: 18 figure