This thesis describes a face verification system that is smart-card-based. The objectives were to identify the key parameters that affect the design of such a system, to investigate the general optimisation problem and test its robustness when each key parameter is optimised. Some of these parameters have been coarsely investigated in the literature in the context of the general face recognition problem. However, the previous work only partially fulfilled the requirements of a smart-card-based system, in which the severe engineering constraints and limitations imposed by smart cards have to be taken into account in the overall design process. To address these problems on the proposed fully localised architecture of the smart card face verification system (SCFVS), the work starts with the selection of the client specific linear discriminant analysis (CS-LDA) algorithm, suitable to be ported to the target platform on which the biometric process can run. Then the main functional parts of the system are presented: face image geometric alignment, photometric normalisation, feature extraction, and on-card verification. Each part consists of a series of basic steps, where the role of each step is fixed. However, the algorithm is systematically varied in some steps to investigate the effect on system performance, and system complexity in terms of speed and memory management