Return On Contribution (ROC): A Metric for Enterprise Social Software

Abstract. The value of enterprise social media applications, components, and users is difficult to quantify in formal economic terms such as Return On Investment. In this work we propose a different approach, based on human service to other humans. We describe a family of metrics, Return On Contribution (ROC), to assist in managing social software systems. ROC focuses on human collaboration, namely the creation and consumption of information and knowledge among employees. We show how ROC can be used to track the performance of several types of social media applications, and how ROC can help to understand the usage patterns of items within those applications, and the performance of employees who use those applications. Design implications include the importance of “lurkers ” in organizational knowledge exchange, and specific types of measurements that may be of value to employees, managers, and system administrators

A user-centred approach to information retrieval

A user model is a fundamental component in user-centred information retrieval systems. It enables personalization of a user's search experience. The development of such a model involves three phases: collecting information about each user, representing such information, and integrating the model into a retrieval application. Progress in this area is typically met with privacy and scalability challenges that hinder the ability to synthesize collective knowledge from each user's search behaviour. In this thesis, I propose a framework that addresses each of these three phases. The proposed framework is based on social role theory from the social science literature and at the centre of this theory is the concept of a social position. A social position is a label for a group of users with similar behavioural patterns. Examples of such positions are traveller, patient, movie fan, and computer scientist. In this thesis, a social position acts as a label for users who are expected to have similar interests. The proposed framework does not require real users' data; rather it uses the web as a resource to model users. The proposed framework offers a data-driven and modular design for each of the three phases of building a user model. First, I present an approach to identify social positions from natural language sentences. I formulate this task as a binary classification task and develop a method to enumerate candidate social positions. The proposed classifier achieves an accuracy score of 85.8%, which indicates that social positions can be identified with good accuracy. Through an inter-annotator agreement study, I further show a reasonable level of agreement between users when identifying social positions. Second, I introduce a novel topic modelling-based approach to represent each social position as a multinomial distribution over words. This approach estimates a topic from a document collection for each position. To construct such a collection for a particular position, I propose a seeding algorithm that extracts a set of terms relevant to the social position. Coherence-based evaluation shows that the proposed approach learns significantly more coherent representations when compared with a relevance modelling baseline. Third, I present a diversification approach based on the proposed framework. Diversification algorithms aim to return a result list for a search query that would potentially satisfy users with diverse information needs. I propose to identify social positions that are relevant to a search query. These positions act as an implicit representation of the many possible interpretations of the search query. Then, relevant positions are provided to a diversification technique that proportionally diversifies results based on each social position's importance. I evaluate my approach using four test collections provided by the diversity task of the Text REtrieval Conference (TREC) web tracks for 2009, 2010, 2011, and 2012. Results demonstrate that my proposed diversification approach is effective and provides statistically significant improvements over various implicit diversification approaches. Fourth, I introduce a session-based search system under the framework of learning to rank. Such a system aims to improve the retrieval performance for a search query using previous user interactions during the search session. I present a method to match a search session to its most relevant social positions based on the session's interaction data. I then suggest identifying related sessions from query logs that are likely to be issued by users with similar information needs. Novel learning features are then estimated from the session's social positions, related sessions, and interaction data. I evaluate the proposed system using four test collections from the TREC session track. This approach achieves state-of-the-art results compared with effective session-based search systems. I demonstrate that such a strong performance is mainly attributed to features that are derived from social positions' data

Group recommendation with automatic detection and classification of groups

This PhD thesis presents ART (Automatic Recommendation Technologies), a set of group recommendation algorithms that detect groups of users with similar preferences. With respect to classic group recommendation, the first step that such systems have to compute is the detection of groups of people with similar preferences, in order to respect the constraint on the number of recommendations that can be produced and maximize users’ satisfaction

Cooperating search communities

Collaborative Web Search (CWS) seeks to exploit the high degree of natural query repetition and result selection regularity that is prevalent among communities of searchers. CWS reuses the search experiences of community members, to promote results that have previously been judged relevant for queries. This facilitates a better response to the type of vague queries that are commonplace in Web search and allows a generic search engine to adapt to the preferences of communities of individuals. CWS contemplates a society of search communities, each with its own repository of experience. In this paper we describe and evaluate a new technique for leveraging the search experiences of related communities as sources of additional search knowledge