This paper deals with the security vulnerabilities of the cryptographic algorithms A3, A8, and A5 existing in the GSM network. We review these algorithms and propose new secure algorithms named NewA3, NewA8, and NewA5 algorithms with respect to the A3, A8, and A5 algorithms. Our NewA5 algorithm is based on block ciphers, but we also propose NewA5 algorithm with Cipher Feedback, Counter, and Output Feedback modes to convert block cipher into stream cipher. However, stream cipher algorithms are slower than the block cipher algorithm. These new algorithms are proposed to use with a secure and efficient authentication and key agreement (AKA) protocol in the GSM network. The proposed architecture is secure against partition attack, narrow pipe attack, collision attack, interleaving attack, and man-in-the-middle attack. The security analysis of the proposed algorithms are discussed with respect to the cryptanalysis, brute force analysis, and operational analysis. We choose the NewA3 and NewA8 algorithms for challenge-response and key generation, respectively. Furthermore, the NewA5 is suitable for encryption as it is efficient than the existing A5/1 and A5/2 algorithms. In case when stream cipher algorithms are required to use, our new algorithms, NewA5-CTR, NewA5-CFB, and NewA5-OFB can be used for specific applications. These algorithms are completely secure and better than the existing A5/1 and A5/2 in terms of resistant to attacks
