A paradigm of an interaction context-aware pervasive multimodal multimedia computing system

Communication is a very important aspect of human life; it is communication that helps human beings to connect with each other as individuals and as independent groups. Communication is the fulcrum that drives all human developments in all fields. In informatics, one of the main purposes of the existence of computer is information dissemination – to be able to send and receive information. Humans are quite successful in conveying ideas to one another, and reacting appropriately. This is due to the fact that we share the richness of the language, have a common understanding of how things work and an implicit understanding of everyday situations. When humans communicate with humans, they comprehend the information that is apparent to the current situation, or context, hence increasing the conversational bandwidth. This ability to convey ideas, however, does not transfer when humans interact with computers. On its own, computers do not understand our language, do not understand how the world works and cannot sense information about the current situation. In a typical computing set-up where we have an impoverished typical mechanism for providing computer with information using mouse, keyboard and screen, the end result is we explicitly provide information to computers, producing an effect that is contrary to the promise of transparency and calm technology in Weiser’s vision of ubiquitous computing (Weiser 1991; Weiser and Brown 1996). To reverse this trend, it is imperative that we researchers find ways that will enable computers to have access to context. It is through context-awareness that we can increase the richness of communication in human-computer interaction, through which we can reap the most likely benefit of more useful computational services. Context is a subjective idea as demonstrated by the state-of-the art in which each researcher has his own understanding of the term, which continues to evolve nonetheless. The acquisition of contextual information is essential but it is the end user, however, that will have the final say as to whether the envisioned context is correctly captured/acquired or not. Current literature informs us that some contextual information is already predefined by some researchers from the very beginning – this is correct if the application domain is fixed but is incorrect if we infer that a typical user does different computing tasks on different occasions. With the aim of coming up with more conclusive and inclusive design, we conjecture that what contextual information should be left to the judgment of the end user who is the one that has the knowledge determine which information is important to him and which is not. This leads us to the concept of incremental acquisition of context where context parameters are added, modified or deleted one context parameter at a time. In conjunction with our idea of inclusive context, we broaden the notion of context that it has become context of interaction. Interaction context is the term that is used to refer to the collective context of the user (i.e. user context), of his working environment (i.e. environmental context) and of his computing system (i.e. system context). Logically and mathematically, each of these interaction context elements – user context, environment context and system context – is composed of various parameters that describe the state of the user, of his workplace and his computing resources as he undertakes an activity in accomplishing his computing task, and each of these parameters may evolve over time. For example, user location is a user context parameter and its value will evolve as the user moves from one place to another. The same can be said about noise level as an environment context parameter; its value evolves over time. The same can be said with available bandwidth that continuously evolves which we consider as a system context parameter. To realize the incremental definition of incremental context, we have developed a tool called the virtual machine for incremental interaction context. This tool can be used to add, modify and delete a context parameter on one hand and determine the sensor-based context (i.e. context that is based on parameters whose values are obtained from raw data supplied by sensors) on the other. In order to obtain the full benefit of the richness of interaction context with regards to communication in human-machine interaction, the modality of interaction should not be limited to the traditional use of mouse-keyboard-screen alone. Multimodality allows for a much wider range of modes and forms of communication, selected and adapted to suit the given user’s context of interaction, by which the end user can transmit data to the computer and computer can respond or yield results to the user’s queries. In multimodal communication, the weaknesses of one mode of interaction, with regards to its suitability to a given situation, is compensated by replacing it with another mode of communication that is more suitable to the situation. For example, when the environment becomes disturbingly noisy, using voice may not be the ideal mode to input data; instead, the user may opt for transmitting text or visual information. Multimodality also promotes inclusive informatics as those with a permanent or temporary disability are given the opportunity to use and benefit from information technology advancement. For example, the work on presentation of mathematical expressions to visually-impaired users (Awdé 2009) would not have been made possible without multimodality. With mobile computing within our midst coupled with wireless communication that allows access to information and services, pervasive and adaptive multimodality is more than ever apt to enrich communication in human-computer interaction and in providing the most suitable modes for data input and output in relation to the evolving interaction context. A look back at the state of the art informs us that a great amount of effort was expended in finding the definition of context, in the acquisition of context, in the dissemination of context and the exploitation of context within a system that has a fixed domain of application (e.g. healthcare, education, etc.). Also, another close look tells us that much research efforts on ubiquitous computing were devoted to various application domains (e.g. identifying the user whereabouts, identifying services and tools, etc.) but there is rarely, if ever, an effort made to make multimodality pervasive and accessible to various user situations. In this regard, we come up with a research work that will provide for the missing link. Our work – the paradigm of an interaction context-sensitive pervasive multimodal multimedia computing system is an architectural design that exhibits adaptability to a much larger context called interaction context. It is intelligent and pervasive, meaning it is functional even when the end user is stationary or on the go. It is conceived with two purposes in mind. First, given an instance of interaction context, one which evolves over time, our system determines the optimal modalities that suit such interaction context. By optimal, we mean a selection decision on appropriate multimodality based on the given interaction context, available media devices that support the modalities and user preferences. We designed a mechanism (i.e. a paradigm) that will do this task and simulated its functionality with success. This mechanism employs machine learning (Mitchell 1997; Alpaydin 2004; Hina, Tadj et al. 2006) and uses case-based reasoning with supervised learning (Kolodner 1993; Lajmi, Ghedira et al. 2007). An input to this decision-making component is an instance of interaction context and its output is the optimal modality and its associated media devices that are for activation. This mechanism is continuously monitoring the user’s context of interaction and on behalf of the user continuously adapts accordingly. This adaptation is through dynamic reconfiguration of the pervasive multimodal system’s architecture. Second, given an instance of interaction context and the user’s task and preferences, we designed a mechanism that allows the automatic selection of user’s applications, the preferred suppliers to these applications and the preferred quality of service (QoS) dimensions’ configurations of these suppliers. This mechanism does its task in consultation with computing resources, sensing the available suppliers and possible configuration restrictions within the given computing set-up. Apart from the above-mentioned mechanisms, we also formulated scenarios as to how a computing system must provide the user interface given that we have already identified the optimal modalities that suit the user’s context of interaction. We present possible configurations of unimodal and bimodal interfaces based on the given interaction context as well as user preferences. Our work is different from previous work in that while other systems capture, disseminate and consume context to suit the preferred domain of application, ours captures the interaction context and reconfigures its architecture dynamically in generic fashion in order that the user could continue working on his task anytime, anywhere he wishes regardless of the application domain the user wishes to undertake. In effect, the system that we have designed along with all of its mechanisms, being generic in design, can be adapted or integrated with ease or with very little modification into various computing systems of various domains of applications. Simulations and mathematical formulations were provided to support our ideas and concepts related to the design of the paradigm. An actual program in Java was developed to support our concept of a virtual machine for incremental interaction context

Techniques d'interaction multimodales pour l'accès aux mathématiques par des personnes non-voyantes

Cette thèse s‟inscrit dans le domaine de l‟interaction Homme-Machine et plus précisément dans celui des interfaces multimodales destinées aux non-voyants. Elle a pour thème principal la présentation des expressions mathématiques aux non-voyants. Pour les étudiants non-voyants, apprendre les mathématiques est une tâche ardue et peut constituer une barrière, les séparant des disciplines techniques. Les travaux de recherche présentés ici décrivent les problèmes rencontrés dans la conception d‟un système permettant l‟accès aux mathématiques pour les utilisateurs déficients visuels. En effet, nous présentons une analyse des outils existants puis nous proposons des solutions pour combler leurs insuffisances. Nous exposons les techniques utilisées au sein de notre système pour répondre aux problèmes de la présentation des expressions mathématiques aux non-voyants. Nous exploitons la multimodalité comme technique d‟interaction pour développer des applications destinées à ce type d‟utilisateurs parce qu‟elle offre plusieurs opportunités grâce à sa richesse des interactions. Nous dotons également notre système d‟un comportement intelligent pour assurer une certaine autonomie à l‟utilisateur. En effet, le système est capable de présenter l‟information en fonction du contexte de l‟interaction (c-à-d. l‟utilisateur, son environnement et sa machine) et de la nature de l‟information. Le système est « pervasif » et adaptatif. L‟accès à l‟information est assuré n‟importe où n‟importe quand et il s‟adapte dynamiquement aux changements du contexte tout en fournissant continuellement des services à l‟utilisateur non-voyant, sans intervention humaine. Pour rendre le système adaptatif, nous avons élaboré un modèle qui détermine la complexité de l‟expression mathématique et nous avons intégré une technique d‟apprentissage automatique pour implémenter les mécanismes de décisions. Enfin, l‟architecture proposée est du type multi-agent. Ces techniques ont été validées par des études de cas et en utilisant les réseaux de Pétri et l‟outil de simulations JADE

Emerging research themes in services-oriented systems

n/

Entropy-Based Dynamic Ad Placement Algorithms for In-Video Advertising

With the evolution of the Internet and the increasing number of users over last years, online advertising has become one of the pillars models that sustains many of the Internet businesses. In this dissertation, we review the history of online advertising, will be made, as well as the state-of-the-art of the major scientific contributions in online advertising,in particularly in respect to in-video advertising. In in-video advertising, one of the major issues is to identify the best places for insertion of ads. In the literature, this problem is addressed in different ways. Some methods are designed for a specific genres of video, e.g., football or tennis, while others are independent of genre, trying to identify the meaningful video scenes (a set of continuous and related frames) where ads will be displayed. However, the vast majority of online videos in the Internet are not long enough to identify large scenes. So, in this dissertation we will address a new solution for advertisement insertion in online videos, a solution that can be utilized independently of the duration and genre of the video in question. When developing a solution for in-video advertising, a major challenge rests on the intrusiveness that the ad inserted will take upon the viewer. The intrusiveness is related to the place and timing used by the advertising to be inserted. For these reasons, the algorithm has to take in consideration the "where", "when" and "how" the advertisement should be inserted in the video, so that it is possible to reduce the intrusiveness of the ads to the viewer. In short, in addition to besides being independent of duration and genre, the proposed method for ad placement in video was developed taking in consideration the ad intrusiveness to the user.Com a evolução da Internet e o número crescente de utilizadores ao longo destes últimos anos, a publicidade on-line tornou-se um dos modelos base que tem sustentado muitos negócios na Internet. Da mesma forma, vídeos on-line constituem uma parte significativa do tráfego na Internet. É por isso possível entender desta forma, o potencial que ferramentas que possão explorar eficientemente ambas estas áreas possuem no mercado. Nesta dissertação será feita uma revisão da história da publicidade online, mas também será apresentado ao leitor uma revisão sobre o estado da arte das principais contribuições científicas para a publicidade on-line, em especial para a publicidade em video. Na publicidade em vídeo, uma das principais preocupações é identificar os melhores locais para a inserir os anúncios. Na literatura, este problema é abordado de diferentes maneiras, alguns criaram métodos para gêneros específicos de vídeo, por exemplo, futebol ou ténis, outros métodos são independentes do gênero, mas tentam identificar as cenas de vídeo (um conjunto contínuo de frames relacionadas) e apenas exibir anúncios neles. No entanto, a grande maioria dos vídeos on-line na Internet não são suficiente longos para serem identificadas cenas suficientemente longas para inserir os anúncios. Assim, nesta dissertação iremos abordar uma nova solução para a inserção de anúnicios em vídeos, uma solução que pode ser utilizada de forma independente da duração e gênero do vídeo em questão. Ao desenvolver uma solução para inserir anúncos em vídeos a grande preocupação recai sobre a intromissão que o anúncio inserido poderá ter sobre o utilizador. A intrusão está relacionada com o local e tempo utilizado pela publicidade quando é inserida. Por estas razões, o algoritmo tem que levar em consideração "onde", "quando" e "como" o anúncio deve ser inserido no vídeo, de modo que seja possível reduzir a intromissão dos anúncios para o utilizador. Em suma, para além de ser independente da duração e gênero do vídeo, o método proposto será também desenvolvido tendo em consideração a intromissáo do anúncio para o utilizador. Por fim, o método proposto será testado e comparado com outros métodos, de modo a que seja possivel perceber as suas capacidades

Risk Management for the Future

A large part of academic literature, business literature as well as practices in real life are resting on the assumption that uncertainty and risk does not exist. We all know that this is not true, yet, a whole variety of methods, tools and practices are not attuned to the fact that the future is uncertain and that risks are all around us. However, despite risk management entering the agenda some decades ago, it has introduced risks on its own as illustrated by the financial crisis. Here is a book that goes beyond risk management as it is today and tries to discuss what needs to be improved further. The book also offers some cases

Fake Malware Generation Using HMM and GAN

In the past decade, the number of malware attacks have grown considerably and, more importantly, evolved. Many researchers have successfully integrated state-of-the-art machine learning techniques to combat this ever present and rising threat to information security. However, the lack of enough data to appropriately train these machine learning models is one big challenge that is still present. Generative modelling has proven to be very efficient at generating image-like synthesized data that can match the actual data distribution. In this paper, we aim to generate malware samples as opcode sequences and attempt to differentiate them from the real ones with the goal to build fake malware data that can be used to effectively train the machine learning models. We use and compare different Generative Adversarial Networks (GAN) algorithms and Hidden Markov Models (HMM) to generate such fake samples obtaining promising results