Assessing fun: young children as evaluators of interactive systems.

In this paper, we describe an exploratory study on the challenges of conducting usability tests with very young children aged 3 to 4 years old (nursery age) and the differences when working with older children aged 5 to 6 years old (primary school). A pilot study was conducted at local nursery and primary schools to understand and experience the challenges working with young children interacting with computer products. We report on the studies and compare the experiences of working with children of different age groups in evaluation studies of interactive systems

Searching by approximate personal-name matching

We discuss the design, building and evaluation of a method to access theinformation of a person, using his name as a search key, even if it has deformations. We present a similarity function, the DEA function, based on the probabilities of the edit operations accordingly to the involved letters and their position, and using a variable threshold. The efficacy of DEA is quantitatively evaluated, without human relevance judgments, very superior to the efficacy of known methods. A very efficient approximate search technique for the DEA function is also presented based on a compacted trie-tree structure.Postprint (published version

Evaluating epistemic uncertainty under incomplete assessments

The thesis of this study is to propose an extended methodology for laboratory based Information Retrieval evaluation under incomplete relevance assessments. This new methodology aims to identify potential uncertainty during system comparison that may result from incompleteness. The adoption of this methodology is advantageous, because the detection of epistemic uncertainty - the amount of knowledge (or ignorance) we have about the estimate of a system's performance - during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections. Across a series of experiments we demonstrate how this methodology can lead towards a finer grained analysis of systems. In particular, we show through experimentation how the current practice in Information Retrieval evaluation of using a measurement depth larger than the pooling depth increases uncertainty during system comparison

Analyzing the strength of ties of Retweet in health domain

Social Network (SN) is created whenever people interact with other people. Online SN gained considerable popularity in the last years such as Fa- cebook, Twitter and etc Twitter is SN and microblogging service that creates some interesting social network structures - follow relationships. Users follow someone mostly because they share common interests and they may exchange messages called tweets. If a user post a tweet, if their follower like it they repost it or retweet it. In this context, we aim to explore and study the topological structure of user‟s retweet network, as well, new scaling measures based on strength of retweet ties. The findings suggested that relations of “friendship” are important but not enough to find out how important users are. We uncovered other some principles that must be studied like, homophily phenomenon. Ho- mophily explores properties of social network relationships, i.e. the preference for associating with individuals of the same background. Last but not least, it is worth emphasizing that we uncovered a weak correlation between Degree Cen- trality and Betweenness Centrality (49 percent) in Retweet-network and strong correlation between Degree and Betweenness centrality in Follower-network (89 percent). These find suggests that retweet network may have some fractal properties

Tekstų nehomogeniškumo tyrimas naudojant žymeklius

The aim of the paper is to assess the distributional non-homogeneity of texts in the usage of functional words andother linguistic units. Our empirical study is based on recommended school fiction works taken from a digital library athttp://ebiblioteka.mkp.emokykla.lt. Sets of frequent word forms, called markers, are made, and their frequency counts in blocks of 50successive sentences are calculated. The frequency counts of the markers show significant excess variability (overdispersion) withrespect to a text homogeneity model usually assumed in linguistics. For chosen markers, different kinds of hierarchical binomiallogistic regression models with the author's identifier, the block length and the frequency counts of the remaining markers as explanatory variables are fitted to the block data in order to explain the observed overdispersion of the markers chosen.Straipsnio tikslas – įvertinti tekstų statistinį nehomogeniškumą pagal funkcinių žodžių ir kitų lingvistinių elementų vartoseną. Atliktas empirinis tyrimas remiasi mokykloms rekomenduojamų suskaitmenintų grožinės literatūros kūrinių biblioteka http://ebiblioteka.mkp.emokykla.lt. Apskaičiuojami sudarytų dažnų žodžių formų ar kitų kalbinių struktūrų rinkinių, juos vadinsime žymekliais, dažnumai tekstų blokuose, jungiančiuose 50 iš eilės einančių sakinių. Pastebėta, kad žymeklių dažnumai blokuose turi ženklią perteklinę sklaidą, palyginti su lingvistikoje įprastu homogeniškumo modeliu. Pasirinktoms žymeklių grupėms parinkti kelių tipų hierarchiniai binominės logistinės regresijos modeliai, naudojantys autoriaus identifikatorių, bloko ilgį ir likusių žymeklių dažnius blokuose kaip aiškinančiuosius kintamuosius, leido paaiškinti didelę dalį pasirinktųjų žymeklių perteklinės sklaidos

Open- vs. Restricted-Domain QA Systems in the Biomedical Field

Question Answering Systems (hereinafter QA systems) stand as a new alternative for Information Retrieval Systems. We conducted a study to evaluate the efficiency of QA systems as terminological sources for physicians, specialized translators, and users in general. To this end we analysed the performance of two open-domain and two restricted domain QA systems. The research entailed a collection of one hundred fifty definitional questions from WebMed. We studied the sources that QA systems used to retrieve the answers, and later applied a range of evaluation measures to mark the quality of answers. Through analysing the results obtained by asking the 150 questions in the QA systems MedQA, START, QuALiM and HONqa, it was possible to evaluate the systems’ operation through applying specific metrics. Despite the limitations demonstrated by these systems, as they are not accessible to everyone and they are not always completely developed, it has been confirmed that these four QA systems are valid and useful for obtaining definitional medical information in that they offer coherent and precise answers. The results are encouraging because they present this type of tool as a new possibility for gathering precise, reliable and specific information in a short period of time

Open- vs. Restricted-Domain QA Systems in the Biomedical Field

Question Answering Systems (hereinafter QA systems) stand as a new alternative for Information Retrieval Systems. We conducted a study to evaluate the efficiency of QA systems as terminological sources for physicians, specialized translators, and users in general. To this end we analysed the performance of two open-domain and two restricted domain QA systems. The research entailed a collection of one hundred fifty definitional questions from WebMed. We studied the sources that QA systems used to retrieve the answers, and later applied a range of evaluation measures to mark the quality of answers. Through analysing the results obtained by asking the 150 questions in the QA systems MedQA, START, QuALiM and HONqa, it was possible to evaluate the systems’ operation through applying specific metrics. Despite the limitations demonstrated by these systems, as they are not accessible to everyone and they are not always completely developed, it has been confirmed that these four QA systems are valid and useful for obtaining definitional medical information in that they offer coherent and precise answers. The results are encouraging because they present this type of tool as a new possibility for gathering precise, reliable and specific information in a short period of time