Ranking for Relevance and Display Preferences in Complex Presentation Layouts

Learning to Rank has traditionally considered settings where given the relevance information of objects, the desired order in which to rank the objects is clear. However, with today's large variety of users and layouts this is not always the case. In this paper, we consider so-called complex ranking settings where it is not clear what should be displayed, that is, what the relevant items are, and how they should be displayed, that is, where the most relevant items should be placed. These ranking settings are complex as they involve both traditional ranking and inferring the best display order. Existing learning to rank methods cannot handle such complex ranking settings as they assume that the display order is known beforehand. To address this gap we introduce a novel Deep Reinforcement Learning method that is capable of learning complex rankings, both the layout and the best ranking given the layout, from weak reward signals. Our proposed method does so by selecting documents and positions sequentially, hence it ranks both the documents and positions, which is why we call it the Double-Rank Model (DRM). Our experiments show that DRM outperforms all existing methods in complex ranking settings, thus it leads to substantial ranking improvements in cases where the display order is not known a priori

Representativeness and face-ism: Gender bias in image search

Implicit and explicit gender biases in media representations of individuals have long existed. Women are less likely to be represented in gender-neutral media content (representation bias), and their face-to-body ratio in images is often lower (face-ism bias). In this article, we look at representativeness and face-ism in search engine image results. We systematically queried four search engines (Google, Bing, Baidu, Yandex) from three locations, using two browsers and in two waves, with gender-neutral (person, intelligent person) and gendered (woman, intelligent woman, man, intelligent man) terminology, accessing the top 100 image results. We employed automatic identification for the individual’s gender expression (female/male) and the calculation of the face-to-body ratio of individuals depicted. We find that, as in other forms of media, search engine images perpetuate biases to the detriment of women, confirming the existence of the representation and face-ism biases. In-depth algorithmic debiasing with a specific focus on gender bias is overdue

Image search: an investigation of factors affecting search behaviour of users

Searching for images can be challenging. How users search for images is governed by their information need. Nevertheless, in fulfilling their information need, users are often affected by subjective factors. These factors include topic familiarity, task difficulty, relevance criteria and satisfaction. This thesis focuses on three research questions exploring how image information needs together with these factors affect online web users' searching behaviour. The questions are: 1. How does image information need affect the criteria users apply when selecting relevant images? 2. How do different factors in image retrieval affect users' image searching behaviour? 3. Can we identify image information needs solely from user queries? In addressing these challenges, we conducted both user studies and proxy log analysis to complement each other. User studies are conducted in a laboratory setting and the needs are artificial, while proxy log captures users' actual needs and behaviour in the wild. The main user study involved 48 students of various disciplines from RMIT University. In the study, we represent image information needs as types of tasks. Data were collected from questionnaires and screen capture recordings. The questionnaire was used to collect data on criteria users find important when judging image relevance and perception on the effects of subjective factors to their searching. Screen capture recordings of their search activities were observed and time stamped to identify and measure search and retrieval behaviour. These measures were used to evaluate the effects of subjective factors on users' image search behaviour. The results showed in judging image relevance, users may apply similar criteria, however, the importance of these criteria depend on the types of image. Similarly, ratings of users' perception on aspects of performing image search show they were task dependent and that effect of different aspects were related. Users were more affected by familiarity and satisfaction when performing difficult image search tasks. Results of correlation suggest that users' perception of aspects of performing image search did not always correspond with their actual search behaviour. However, for some subjective aspects of user search behaviour, we have identified particular objective measures that correlate well with that aspect. The examination of users' queries in proxy logs, shows that users search for unambiguous images more frequently compared to conceptual images. Their sessions are short with two to three terms per query. When analysing queries from logs, we are actually making a guess of what users were searching for. However, by examining the way users modify/reformulate their queries may give an indication of their information need. Results show, that users frequently submit new queries or replace terms from their previous query rather than revising the query into more depth or breadth. Similar findings were found when compared with the user study data, whereby users in both settings exhibit similarity in the number of queries, terms and reformulation type. This thesis concludes that given similar image information needs, ordinary users make relevance judgements similar to specialised users (such as journalists, art historians and medical doctors) despite giving attention to different criteria of relevance. Moreover, only certain measures of search behaviour used in text retrieval are applicable to image retrieval due to the difference in judging the relevance of textual information and image. In addition, visual information needs can be better inferred when analysing series of queries and their reformulation within a search session