We construct intersecting D-brane configurations that encode the gauge groups
and field content of dual N=4 supersymmetric gauge theories in three
dimensions. The duality which exchanges the Coulomb and Higgs branches and the
Fayet-Iliopoulos and mass parameters is derived from the SL(2,Z) symmetry of
the type IIB string. Using the D-brane configurations we construct explicitly
this mirror map between the dual theories and study the instanton corrections
in the D-brane worldvolume theory via open string instantons. A general
procedure to obtain mirror pairs is presented and illustrated. We encounter
transitions among different field theories that correspond to smooth movements
in the D-brane moduli space. We discuss the relation between the duality of the
gauge theories and the level-rank duality of affine Lie algebras. Examples of
other dual theories are presented and explained via T-duality and extremal
transitions in type II string compactifications. Finally we discuss a second
way to study instanton corrections in the gauge theory, by wrapping five-branes
around six-cycles in M-theory compactified on a Calabi-Yau 4-fold.Comment: 36 pages, 16 figures, LaTex