Besides attenuation, dispersion is the major limiting factor in high data rate fiber optical transmission systems. Dispersion compensation techniques have to be deployed in order to increase the data bandwidth or the reach of fiber optical links. Typically fixed value dispersion compensators are used. However at channel bit rates of 40 GBit/s and beyond adjustable residual dispersion compensator modules (DCM) are needed to guarantee an error free transmission under changing environmental conditions. In this thesis dispersion techniques were investigated which exploit the special propagation properties of higher order modes in custom-designed optical fibers. After a short introduction of state-of-the-art dispersion techniques and their parameters (chapter 2) the modeling and calculation of propagation properties of a particular mode in an optical fiber with an arbitrary, rotation-symmetric refractive index profile is shown (chapter 3). A converter from the fundamental mode and back is needed in order to exploit the propagation properties of a higher order mode (HOM). In this work long-period gratings (LPG) were considered as mode converters (chapter 4) as they can excite selective and nearly lossless a higher order mode. The modeling und calculation of these gratings, based on the fiber calculation of chapter 3, is presented in the first part of chapter 4. Afterwards the manufacturing methods developed during this work are introduced. The spectral properties of realized long-period gratings are discussed and the influence of such factors as strain and temperature on tuning the mode conversion is shown. A dispersion compensator type utilizing only the waveguide dispersion of a certain mode in a custom few mode fiber (FMF) is the subject of chapter 5. The working principle, the fiber design process and first measurements of a realized HOM-DCM with almost completely coupling FMF-LPG are presented. Subsequently the principle of a novel dispersion compensator with an arbitrary dispersion function for a higher or the fundamental mode is explained. In chapter 6 another type of dispersion compensator is investigated consisting of equally distributed long-period gratings along an optical fiber. The fiber pieces between the gratings create a certain time delay between the fundamental mode and the considered higher order mode. It is shown in simulations and in an experiment, that by tuning the mode conversion of each grating and the optical phase relation between the two signal paths in each fiber piece this finite impulse filter structure is so adjusted to function as a tunable residual dispersion compensator
