The reflectivity of a semiconductor saturable absorber mirror (SESAM) is generally expected to increase with increasing pulse energy. However, for higher pulse energies the reflectivity can decrease again; we call this a ‘roll-over' of the nonlinear reflectivity curve caused by inverse saturable absorption. We show for several SESAMs that the measured roll-over is consistent with two-photon absorption only for short (femtosecond) pulses, while a stronger (yet unidentified) kind of nonlinear absorption is dominant for longer (picosecond) pulses. These inverse saturable absorption effects have important technological consequences, e.g. for the Q-switching dynamics of passively mode-locked lasers. A simple equation using only measurable SESAM parameters and including inverse saturable absorption is derived for the Q-switched mode-locking threshold. We present various data and discuss the sometimes detrimental effects of this roll-over for femtosecond high repetition rate lasers, as well as the potentially very useful consequences for passively mode-locked multi-GHz lasers. We also discuss strategies to enhance or reduce this induced absorption by using different SESAM designs or semiconductor material