Buffer Overflow Vulnerability Diagnosis For Commodity Software

Buffer overflow attacks have been a computer security threat in software-based systems andapplications for decades. The existence of buffer overflow vulnerabilities makes the systemsusceptible to Internet worms and denial of service (DDoS) attacks which can cause hugesocial and financial impacts. Due to its importance, buffer overflow problem has been intensively studied. Researchershave proposed different techniques to defend against unknown buffer overflow attacks. Theyhave also investigated various solutions, including automatic signature generation, automatic patch generation, etc., to automatically protect computer systems with known vulnerabilities. The effectiveness and efficiency of the automatic signature generation approaches andthe automatic patch generation approaches are all based on the accurate understanding ofthe vulnerabilities, the buffer overflow vulnerability diagnosis (BOVD). Currently, the results of automatic signature generation and automatic patch generation are far from satisfaction due to the insufficient research results from the automatic BOVD. This thesis defines the automatic buffer overflow vulnerability diagnosis (BOVD) problemand provides solutions towards automatic BOVD for commodity software. It targets oncommodity software when source code and symbol table are not available. The solutionscombine both of the dynamic analysis techniques and static analysis techniques to achievethe goal. Based on the observation that buffer overflow attack happens when the size of the destination buffer is smaller than the total number of writes after the data copy process if the buffer overflow attack happens through a data copy procedure, the diagnosis results return the information of the size of destination buffer, the total number of writes of a data copy procedure and how the user inputs are related with them. They are achieved through bound analysis, loop analysis and input analysis respectively. We demonstrate the effectiveness of this thesis approach using real world vulnerable applications including the buffer overflow vulnerabilities attacked by the record-setting Slammer and Blaster worms. This thesis also does the complete case study for buffer overflow vulnerabilities which may have independent interests to researchers. Our buffer overflow case study results can help other researchers to design more effective defense systems and debugging tools against buffer overflow attacks