23 research outputs found
Interpretable Meta-Measure for Model Performance
Full text linkMeasures for evaluation of model performance play an important role in
Machine Learning. However, the most common performance measures share several
limitations. The difference in performance for two models has no probabilistic
interpretation and there is no reference point to indicate whether they
represent a significant improvement. What is more, it makes no sense to compare
such differences between data sets. In this article, we introduce a new
meta-measure for performance assessment named Elo-based Predictive Power (EPP).
The differences in EPP scores have probabilistic interpretation and can be
directly compared between data sets. We prove the mathematical properties of
EPP and support them with empirical results of a~large scale benchmark on 30
classification data sets. Finally, we show applications of EPP to the selected
meta-learning problems and challenges beyond ML benchmarks.Comment: arXiv admin note: text overlap with arXiv:1908.0921
The use of analytic hierarchy process method in selecting an online poker tournament
Full text linkNa spletu lahko v katerem koli trenutku izbiramo med stotinami različnih spletnih poker-turnirjev (SPT). V diplomski nalogi smo odgovarjali na raziskovalno vprašanje, kako izbrati SPT z uporabo metode AHP glede na tip igralca. Glede na posameznikove motivacije za igranje SPT smo profilirali dva tipa igralcev ? rekreativnega in profesionalnega. Definiramo karakteristike, po katerih se ločijo SPT, in jim glede na tip igralca določimo pomembnost. Za pomoč pri odločanju uporabimo metodo AHP. To je ena od najpopularnejših metod večkriterijskega odločanja, s katero pridobimo rešitve s primerjavo parametrov po parih. S pomočjo metode AHP odločevalec strukturira svoje prioritete in jih uredi na način, ki je jasen in razumljiv. Metodo smo uporabili na praktičnem primeru izbire SPT v izbrani spletni sobi. Zgradili smo model AHP za izbiro SPT za oba tipa igralcev in v naboru turnirjev poiskali najprimernejšega. Z oblikovanjem modela AHP smo poiskali prednosti in slabosti ter ugotovili, da je metoda AHP koristno orodje za izbiro SPT.On the internet, we can play hundreds of different online poker tournaments at any moment. In this thesis, we analyze our main research question: how to pick the most suitable online poker tournament (OPT) with regard to the type of the player. According to individual motivations for playing OPT, we profile two types of players: recreational and professional. We define characteristics that differentiate tournaments and assign their importance according to the player type. For decision support, we use analytic hierarchy process (AHP) model. AHP is the one of the most popular methods of multi-criteria decision making, where the results are acquired by pairwise comparisons of criteria. With the AHP method, the decision makers can structure their thoughts and sort them out in a way that is intelligible and understandable. We demonstrate the use of AHP on a practical example of selecting OPT on a selected poker site. We design AHP model to select the most suitable OPT for both types of players from a set of tournaments. We analyzed the strengths and weaknesses of the AHP method, and concluded that it is a suitable tool for selecting an OPT
Pretending To Be Human: An Automated Theorem Prover To Write Mathematical Proofs
Get PDFThis project explores the theories of automated theorem proving and lambda calculus, and seeks to improve upon a current implementation of an automated theorem prover that produces human-like output. This software component is based on a published article titled “A Fully Automatic Theorem Prover with Human-Style Output” by M. Ganesalingam and W.T. Gowers [7], which outlines a program called robotone containing a first-order logic system (with a few modifications) able to automatically write simple direct proofs. It is designed to mimic different strategies that a human mathematician would use when facing a problem, and to write human-style proofs. The current prototype can solve and write eighteen proofs, seven of which are developed from this project with the addition of a mathematics library customized for the course MATH-332 Real-Analysis I at The College of Wooster. Future work includes further investigation on how human mathematicians construct their proofs, and a possible implementation of a more dynamic tactic mechanism to emulate humans’ thinking process better
