Estimator Selection: End-Performance Metric Aspects

Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs based on usual pointwise functional distances of the estimated system from the true one. The separation of the system estimator from the experiment design is done within this new framework by choosing and fixing the estimation method to either a maximum likelihood (ML) approach or a Bayesian estimator such as the minimum mean square error (MMSE). Since the MMSE estimator delivers a system estimate with lower mean square error (MSE) than the ML estimator for finite-length experiments, it is usually considered the best choice in practice in signal processing and control applications. Within the application-oriented framework a related meaningful question is: Are there end-performance metrics for which the ML estimator outperforms the MMSE when the experiment is finite-length? In this paper, we affirmatively answer this question based on a simple linear Gaussian regression example.Comment: arXiv admin note: substantial text overlap with arXiv:1303.428

API design for machine learning software: experiences from the scikit-learn project

Scikit-learn is an increasingly popular machine learning li- brary. Written in Python, it is designed to be simple and efficient, accessible to non-experts, and reusable in various contexts. In this paper, we present and discuss our design choices for the application programming interface (API) of the project. In particular, we describe the simple and elegant interface shared by all learning and processing units in the library and then discuss its advantages in terms of composition and reusability. The paper also comments on implementation details specific to the Python ecosystem and analyzes obstacles faced by users and developers of the library

Bandwidth selection for kernel estimation in mixed multi-dimensional spaces

Kernel estimation techniques, such as mean shift, suffer from one major drawback: the kernel bandwidth selection. The bandwidth can be fixed for all the data set or can vary at each points. Automatic bandwidth selection becomes a real challenge in case of multidimensional heterogeneous features. This paper presents a solution to this problem. It is an extension of \cite{Comaniciu03a} which was based on the fundamental property of normal distributions regarding the bias of the normalized density gradient. The selection is done iteratively for each type of features, by looking for the stability of local bandwidth estimates across a predefined range of bandwidths. A pseudo balloon mean shift filtering and partitioning are introduced. The validity of the method is demonstrated in the context of color image segmentation based on a 5-dimensional space

Accuracy versus simplicity in online battery model identification

This paper presents a framework for battery modeling in online, real-time applications where accuracy is important but speed is the key. The framework allows users to select model structures with the smallest number of parameters that is consistent with the accuracy requirements of the target application. The tradeoff between accuracy and speed in a battery model identification process is explored using different model structures and parameter-fitting algorithms. Pareto optimal sets are obtained, allowing a designer to select an appropriate compromise between accuracy and speed. In order to get a clearer understanding of the battery model identification problem, “identification surfaces” are presented. As an outcome of the battery identification surfaces, a new analytical solution is derived for battery model identification using a closed-form formula to obtain a battery’s ohmic resistance and open circuit voltage from measurement data. This analytical solution is used as a benchmark for comparison of other fitting algorithms and it is also used in its own right in a practical scenario for state-of-charge estimation. A simulation study is performed to demonstrate the effectiveness of the proposed framework and the simulation results are verified by conducting experimental tests on a small NiMH battery pack