One of the wide engineering fields is aircraft technologies and one of the most common needs for Airplane or UAV is estimating the altitude, which is some time difficult to estimate due to weather fluctuations and instability of the main parameters like pressure and speed. However, a combination of different sensors has been used to estimate altitude to guarantee an accurate reading and it is the method used these days. To overcome this problem is to use more capable technology such as machine vision based system to estimate the altitude, as advantages light weight, intelligence and accuracy, cheaper than commercial sensors as well as, computationally inexpensive. In this paper, we propose a vision-based system that can perform altitude estimation from aerial images. The satisfactory experimental results demonstrate the effectiveness of the proposed system