This paper proposes an impulse response modeling in presence of input and
noisy output of a linear time-invariant (LTI) system. The approach utilizes
Relative Entropy (RE) to choose the optimum impulse response estimate, optimum
time delay and optimum impulse response length. The desired RE is the
Kulback-Lielber divergence of the estimated distribution from its unknown true
distribution. A unique probabilistic validation approach estimates the desired
relative entropy and minimizes this criterion to provide the impulse response
estimate. Classical methods have approached this system modeling problem from
two separate angles for the time delay estimation and for the order selection.
Time delay methods focus on time delay estimate minimizing various proposed
criteria, while the existing order selection approaches choose the optimum
impulse response length based on their proposed criteria. The strength of the
proposed RE based method is in using the RE based criterion to estimate both
the time delay and impulse response length simultaneously. In addition,
estimation of the noise variance, when the Signal to Noise Ratio (SNR) is
unknown is also concurrent and is based on optimizing the same RE based
criterion. The RE based approach is also extended for online impulse response
estimations. The online method reduces the model estimation computational
complexity upon the arrival of a new sample. The introduced efficient stopping
criteria for this online approaches is extremely valuable in practical
applications. Simulation results illustrate precision and efficiency of the
proposed method compared to the conventional time delay or order selection
approaches.Comment: 13 pages, 11 figure