We study the de Haas--van Alphen (dHvA) oscillations in the magnetization of
a two-dimensional electron gas (2DEG) under the influence of the edge states
and/or the Rashba spin-orbit interaction (SOI). The boundaries of the systems
lift partially the degeneracies of Landau levels (LL's) and the resulting edge
states lead to the changes of both the center and the amplitude of the
sawtoothlike magnetization oscillation. The SOI mixes the spin-up and spin-down
states of neighboring LL's into two unequally spaced energy branches. The
inclusion of SOI changes the well-defined sawtooth pattern of the dHvA
oscillations in the magnetization. The weaker the magnetic field is, the larger
is the change of the dHvA oscillations due to the edge effect and/or the
spin-orbit coupling. Some theoretical results are compared with the
experimental data.Comment: 9 pages, 9 figure