An MPEG-7 scheme for semantic content modelling and filtering of digital video

Abstract Part 5 of the MPEG-7 standard specifies Multimedia Description Schemes (MDS); that is, the format multimedia content models should conform to in order to ensure interoperability across multiple platforms and applications. However, the standard does not specify how the content or the associated model may be filtered. This paper proposes an MPEG-7 scheme which can be deployed for digital video content modelling and filtering. The proposed scheme, COSMOS-7, produces rich and multi-faceted semantic content models and supports a content-based filtering approach that only analyses content relating directly to the preferred content requirements of the user. We present details of the scheme, front-end systems used for content modelling and filtering and experiences with a number of users

Persönliche Wege der Interaktion mit multimedialen Inhalten

Today the world of multimedia is almost completely device- and content-centered. It focuses it’s energy nearly exclusively on technical issues such as computing power, network specifics or content and device characteristics and capabilities. In most multimedia systems, the presentation of multimedia content and the basic controls for playback are main issues. Because of this, a very passive user experience, comparable to that of traditional TV, is most often provided. In the face of recent developments and changes in the realm of multimedia and mass media, this ”traditional” focus seems outdated. The increasing use of multimedia content on mobile devices, along with the continuous growth in the amount and variety of content available, make necessary an urgent re-orientation of this domain. In order to highlight the depth of the increasingly difficult situation faced by users of such systems, it is only logical that these individuals be brought to the center of attention. In this thesis we consider these trends and developments by applying concepts and mechanisms to multimedia systems that were first introduced in the domain of usercentrism. Central to the concept of user-centrism is that devices should provide users with an easy way to access services and applications. Thus, the current challenge is to combine mobility, additional services and easy access in a single and user-centric approach. This thesis presents a framework for introducing and supporting several of the key concepts of user-centrism in multimedia systems. Additionally, a new definition of a user-centric multimedia framework has been developed and implemented. To satisfy the user’s need for mobility and flexibility, our framework makes possible seamless media and service consumption. The main aim of session mobility is to help people cope with the increasing number of different devices in use. Using a mobile agent system, multimedia sessions can be transferred between different devices in a context-sensitive way. The use of the international standard MPEG-21 guarantees extensibility and the integration of content adaptation mechanisms. Furthermore, a concept is presented that will allow for individualized and personalized selection and face the need for finding appropriate content. All of which can be done, using this approach, in an easy and intuitive way. Especially in the realm of television, the demand that such systems cater to the need of the audience is constantly growing. Our approach combines content-filtering methods, state-of-the-art classification techniques and mechanisms well known from the area of information retrieval and text mining. These are all utilized for the generation of recommendations in a promising new way. Additionally, concepts from the area of collaborative tagging systems are also used. An extensive experimental evaluation resulted in several interesting findings and proves the applicability of our approach. In contrast to the ”lean-back” experience of traditional media consumption, interactive media services offer a solution to make possible the active participation of the audience. Thus, we present a concept which enables the use of interactive media services on mobile devices in a personalized way. Finally, a use case for enriching TV with additional content and services demonstrates the feasibility of this concept.Die heutige Welt der Medien und der multimedialen Inhalte ist nahezu ausschließlich inhalts- und geräteorientiert. Im Fokus verschiedener Systeme und Entwicklungen stehen oft primär die Art und Weise der Inhaltspräsentation und technische Spezifika, die meist geräteabhängig sind. Die zunehmende Menge und Vielfalt an multimedialen Inhalten und der verstärkte Einsatz von mobilen Geräten machen ein Umdenken bei der Konzeption von Multimedia Systemen und Frameworks dringend notwendig. Statt an eher starren und passiven Konzepten, wie sie aus dem TV Umfeld bekannt sind, festzuhalten, sollte der Nutzer in den Fokus der multimedialen Konzepte rücken. Um dem Nutzer im Umgang mit dieser immer komplexeren und schwierigen Situation zu helfen, ist ein Umdenken im grundlegenden Paradigma des Medienkonsums notwendig. Durch eine Fokussierung auf den Nutzer kann der beschriebenen Situation entgegengewirkt werden. In der folgenden Arbeit wird auf Konzepte aus dem Bereich Nutzerzentrierung zurückgegriffen, um diese auf den Medienbereich zu übertragen und sie im Sinne einer stärker nutzerspezifischen und nutzerorientierten Ausrichtung einzusetzen. Im Fokus steht hierbei der TV-Bereich, wobei die meisten Konzepte auch auf die allgemeine Mediennutzung übertragbar sind. Im Folgenden wird ein Framework für die Unterstützung der wichtigsten Konzepte der Nutzerzentrierung im Multimedia Bereich vorgestellt. Um dem Trend zur mobilen Mediennutzung Sorge zu tragen, ermöglicht das vorgestellte Framework die Nutzung von multimedialen Diensten und Inhalten auf und über die Grenzen verschiedener Geräte und Netzwerke hinweg (Session mobility). Durch die Nutzung einer mobilen Agentenplattform in Kombination mit dem MPEG-21 Standard konnte ein neuer und flexibel erweiterbarer Ansatz zur Mobilität von Benutzungssitzungen realisiert werden. Im Zusammenhang mit der stetig wachsenden Menge an Inhalten und Diensten stellt diese Arbeit ein Konzept zur einfachen und individualisierten Selektion und dem Auffinden von interessanten Inhalten und Diensten in einer kontextspezifischen Weise vor. Hierbei werden Konzepte und Methoden des inhaltsbasierten Filterns, aktuelle Klassifikationsmechanismen und Methoden aus dem Bereich des ”Textminings” in neuer Art und Weise in einem Multimedia Empfehlungssystem eingesetzt. Zusätzlich sind Methoden des Web 2.0 in eine als Tag-basierte kollaborative Komponente integriert. In einer umfassenden Evaluation wurde sowohl die Umsetzbarkeit als auch der Mehrwert dieser Komponente demonstriert. Eine aktivere Beteiligung im Medienkonsum ermöglicht unsere iTV Komponente. Sie unterstützt das Anbieten und die Nutzung von interaktiven Diensten, begleitend zum Medienkonsum, auf mobilen Geräten. Basierend auf einem Szenario zur Anreicherung von TV Sendungen um interaktive Dienste konnte die Umsetzbarkeit dieses Konzepts demonstriert werden

Mediating chance encounters through opportunistic social matching

Chance encounters, the unintended meeting between people unfamiliar with each other, serve as an important social lubricant helping people to create new social ties, such as making new friends or finding an activity, study or collaboration partner. Unfortunately, social barriers often prevent chance encounters in environments where people do not know each other and people have to rely on serendipity to meet or be introduced to interesting people around them. Little is known about the underlying dynamics of chance encounters and how systems could utilize contextual data to mediate chance encounters. This dissertation addresses this gap in research literature by exploring the design space of opportunistic social matching systems that aim to introduce relevant people to each other in the opportune moment and the opportune place in order to encourage face-to-face interaction. A theoretical framework of relational, social and personal context as predictors of encounter opportunities is proposed and validated through a mixed method approach using interviews, experience sampling and a field study of a design prototype. Key contributions of the field interview study (n=58) include novel context-aware social matching concepts such as: sociability of others as an indicator of opportune social context; activity involvement as an indicator of opportune personal context; and contextual rarity as an indicator of opportune relational context. The following study combining Experience Sampling Method (ESM) and participant interviews extends prior research on social matching by providing an empirical foundation for the design of opportunistic social matching systems. A generalized linear mixed model analysis (n=1781) shows that personal context (mood and busyness) together with the sociability of others nearby are the strongest predictors of people’s interest in a social match. Interview findings provide novel approaches on how to operationalize relational context based on social network rarity and discoverable rarity. Moreover, insights from this study highlight that additional meta-information about user interests is needed to operationalize relational context, such as users’ passion level for an interest and their skill levels for an activity. Based on these findings, the novel design concept of passive context-awareness for social matching is put forward. In the last study, Encount’r, an instantiation of an opportunistic social matching system, is designed and evaluated through a field study and participant interviews. A large-scale user profiling survey provides baseline rarity measures to operationalize relational context using rarity, passion levels, skills, needs, and offers. Findings show that attribute type, computed attribute rarity, self-reported passion levels for interest, and response time are associated with people’s interest in a match opportunity. Moreover, this study extends prior work by showing how the concept of passive context-awareness for opportunistic social matching is promising. Collectively, contributions of this work include a theoretical framework encompassing relational, social, and personal context; new innovative concepts to operationalize each of these aspects for opportunistic social matching; and field-tested design affordances for opportunistic social matching systems. This is important because opportunistic social matching systems can lead to new social ties and improved social capital

User decision improvement and trust building in product recommender systems

As online stores are offering an almost unlimited shelf space, users must increasingly rely on product search and recommender systems to find their most preferred products and decide which item is the truly best one to buy. However, much research work has emphasized on developing and improving the underlying algorithms whereas many of the user issues such as preference elicitation and trust formation received little attention. In this thesis, we aim at designing and evaluating various decision technologies, with emphases on how to improve users' decision accuracy with intelligent preference elicitation and revision tools, and how to build their competence-inspired subjective constructs via trustworthy recommender interfaces. Specifically, two primary technologies are proposed: one is called example critiquing agents aimed to stimulate users to conduct tradeoff navigation and freely specify feedback criteria to example products; another termed as preference-based organization interfaces designed to take two roles: explaining to users why and how the recommendations are computed and displayed, and suggesting critique suggestions to guide users to understand existing tradeoff potentials and to make concrete decision navigations from the top candidate for better choices. To evaluate the two technologies' true performance and benefits to real-users, an evaluation framework was first established, that includes important assessment standards such as the objective/subjective accuracy-effort measures and trust-related subjective aspects (e.g., competence perceptions and behavioral intentions). Based on the evaluation framework, a series of nine experiments has been conducted and most of them were participated by real-users. Three user studies focused on the example critiquing (EC) agent, which first identified the significant impact of tradeoff process with the help of EC on users' decision accuracy improvement, and then in depth explored the advantage of multi-item strategy (for critiquing coverage) against single-item display, and higher user-control level reflected by EC in supporting users to freely compose critiquing criteria for both simple and complex tradeoffs. Another three experiments studied the preference-based organization technique. Regarding its explanation role, a carefully conducted user survey and a significant-scale quantitative evaluation both demonstrated that it can be likely to increase users' competence perception and return intention, and reduce their cognitive effort in information searching, relative to the traditional "why" explanation method in ranked list views. In addition, a retrospective simulation revealed its superior algorithm accuracy in predicting critiques and product choices that real-users intended to make, in comparison with other typical critiquing generation approaches. Motivated by the empirically findings in terms of the two technologies' respective strengths, a hybrid system has been developed with the purpose of combining them into a single application. The final three experiments evaluated its two design versions and particularly validated the hybrid system's universal effectiveness among people from different types of cultural backgrounds: oriental culture and western culture. In the end, a set of design guidelines is derived from all of the experimental results. They should be helpful for the development of a preference-based recommender system, making it capable of practically benefiting its users in improving decision accuracy, expending effort they are willing to invest, and even promoting trust in the system with resulting behavioral intentions to purchase chosen products and return to the system for repeated uses

Yavaa: supporting data workflows from discovery to visualization

Recent years have witness an increasing number of data silos being opened up both within organizations and to the general public: Scientists publish their raw data as supplements to articles or even standalone artifacts to enable others to verify and extend their work. Governments pass laws to open up formerly protected data treasures to improve accountability and transparency as well as to enable new business ideas based on this public good. Even companies share structured information about their products and services to advertise their use and thus increase revenue. Exploiting this wealth of information holds many challenges for users, though. Oftentimes data is provided as tables whose sheer endless rows of daunting numbers are barely accessible. InfoVis can mitigate this gap. However, offered visualization options are generally very limited and next to no support is given in applying any of them. The same holds true for data wrangling. Only very few options to adjust the data to the current needs and barely any protection are in place to prevent even the most obvious mistakes. When it comes to data from multiple providers, the situation gets even bleaker. Only recently tools emerged to search for datasets across institutional borders reasonably. Easy-to-use ways to combine these datasets are still missing, though. Finally, results generally lack proper documentation of their provenance. So even the most compelling visualizations can be called into question when their coming about remains unclear. The foundations for a vivid exchange and exploitation of open data are set, but the barrier of entry remains relatively high, especially for non-expert users. This thesis aims to lower that barrier by providing tools and assistance, reducing the amount of prior experience and skills required. It covers the whole workflow ranging from identifying proper datasets, over possible transformations, up until the export of the result in the form of suitable visualizations

Smart Fitness System: Training Programming

Sistemas de recomendação no geral estão a ser cada vez mais usados por empresas que procuram oferecer uma experiência de utilização mais individual e personalizada aos seus clientes. Obter feedback em transações de negócio online nunca foi tão fácil e acessível, o que apenas ajuda a catalisar a evolução dos sistemas de recomendação. Adicionalmente, o uso de dispositivos tecnológicos como smartphones e computadores, juntamente com a conexão à internet, estão também a crescer a um ritmo acelerado sem sinal de paragem em vista. Juntando-se a este grupo de indústrias em crescimento está a indústria fitness, que está a ficar cada vez mais popular. Com isto, mais e mais pessoas estão a começar a usar os dispositivos mencionados anteriormente em combinação com as suas atividades fitness, para aumentar o seu desempenho, monitorizar progresso, definir objetivos, entre outros. Consequentemente, o mercado para sistemas fitness (p.e. aplicações fitness) está a aumentar e já é bastante denso. No entanto, a qualidade associada com tais sistemas fica um pouco aquém tanto em termos de inovação como de funcionalidades essenciais. Como resultado disto, este projeto propôs uma solução – um sistema fitness sob a forma de uma aplicação móvel aliada a um poderoso sistema de recomendação. Este sistema é pretendido que providencie uma experiência mais individual e personalizada para qualquer tipo de utilizador fitness através da oferta de funcionalidades essenciais como registo e monitorização de informação, análise de progresso, e também através de funcionalidades inovadoras como a implementação de um sistema de recomendação capaz de sugerir tópicos relacionados com fitness (p.e. regimes de treino ou exercícios específicos) baseado em múltiplos fatores como os objetivos, características individuais e historial de cada utilizador. Além do mais, deve também oferecer um assistente pessoal virtual, onde os utilizadores podem expressar as suas questões e dúvidas, e tê-las respondidas instantaneamente por um chatbot. Durante o desenvolvimento foi decidido que um segundo sistema de recomendação seria necessário para melhorar o sistema no geral. Este, o sistema, depois de implementado, foi avaliado e pode ser concluído que o resultado foi um sucesso, tendo passado em todas as métricas definidas, exceto uma, com classificações médias nos questionários de satisfação acima de 4/5. O feedback obtido por um especialista no sistema de recomendação foi altamente vantajoso e no geral decentemente positivo, apenas com algumas questões que necessitam de melhoramento. Embora o sistema de recomendação inteligente não tenha conseguido ser testado com informação aplicável, a investigação e trabalho feito constituem uma mais valia caso mais tarde exista a possibilidade de aplicar dados reais.Recommender systems in general are increasingly becoming more exploited by companies who seek to provide a more individual and personalized user-experience to their customers. The fact that providing feedback on online business transactions is also becoming ever so easier only helps to catalyze the evolution of recommender systems. Moreover, the use of technological devices such as smartphones and computers, in conjunction with an internet connection, are also continuing to grow at a fast pace, with no slowing down in sight. Joining on this group of growing industries is the fitness sector, which is becoming increasingly popular. With this, more and more people are starting to use the aforementioned devices in combination with their fitness activities, to boost performance, monitor progress, define goals, among other things. Consequently, the market for fitness systems (i.e. fitness applications) is expanding and is already very dense. However, the associated quality with such systems falls short both in terms of innovation and even crucial features. As a result, this dissertation proposes a solution - an innovative fitness system in the form of a mobile application allied with a powerful recommender system. The system is intended to provide a more individual and personalized experience to any type of fitness user through the offering of crucial features including the log and monitor of information, progress analysis, and also through innovative features such as the implementation of a recommender system capable of making fitness-related suggestions (i.e. training regimens or specific exercises) based on multiple factors like the user’s individual goals, characteristics, and history. Additionally, it should also provide a personal virtual assistant, where users can express their questions and doubts and have them answered instantly by a chatbot. During development, it was decided that a second recommender system was required to improve the system as a whole. This, the system, after being implemented, was evaluated and it can be concluded that the result was a success, having passed in all the defined metrics, except one, with average classifications of 4/5 on the satisfaction inquiries. The feedback obtained from the expert on the recommender system was highly useful and, in general, decently positive, having only a few questions that need improvement. Even though the intelligent recommender system couldn’t be tested with applicable data, the investigation and work done constitute a great asset in case there’s the opportunity to employ real data

Exploiting tag information for search and personalization

[no abstract

Effective decision support for semantic web service selection

The objective of this dissertation is to demonstrate the feasibility of the vision of the Internet of Services based on Semantic Web Services by suggesting an approach to end-user mediated Semantic Web Service selection. Our main contribution is an incremental and interactive approach to requirements elicitation and service selection that is inspired by example critiquing recommender systems. It alternates phases of intermediate service recommendation and phases of informal requirements specification. During that process, the user incrementally develops his service requirements and preferences and finally makes a selection decision. We demonstrate how the requirements elicitation and service selection process can be directed and focused to effectively reduce the system's uncertainty about the user's service requirements and thus to contribute to the efficiency of the service selection process. To acquire information about the actual performance of available services and thus about the risk that is associated with their execution, we propose a flexible feedback system, that leverages reported consumer experiences made in past service interactions. In particular, we provide means that allow to detailedly describe a service's performance with respect to its multiple facets. This is supplemented by a user-adaptive method that effectively assists service consumers in providing such feedback as well as a privacy-preserving technique for feedback propagation. We also demonstrate that available consumer feedback can be effectively exploited to assess the degree and kind of risk that is associated with the execution of an offered service and show how the user can be effectively made aware of this risk. In contrast to many other approaches related to Semantic Web Service technology, we performed an extensive and thorough evaluation of our contribution and documented its results. These show the effectiveness and efficiency of our approach