Scalable Exploration of Complex Objects and Environments Beyond Plain Visual Replication​

Digital multimedia content and presentation means are rapidly increasing their sophistication and are now capable of describing detailed representations of the physical world. 3D exploration experiences allow people to appreciate, understand and interact with intrinsically virtual objects. Communicating information on objects requires the ability to explore them under different angles, as well as to mix highly photorealistic or illustrative presentations of the object themselves with additional data that provides additional insights on these objects, typically represented in the form of annotations. Effectively providing these capabilities requires the solution of important problems in visualization and user interaction. In this thesis, I studied these problems in the cultural heritage-computing-domain, focusing on the very common and important special case of mostly planar, but visually, geometrically, and semantically rich objects. These could be generally roughly flat objects with a standard frontal viewing direction (e.g., paintings, inscriptions, bas-reliefs), as well as visualizations of fully 3D objects from a particular point of views (e.g., canonical views of buildings or statues). Selecting a precise application domain and a specific presentation mode allowed me to concentrate on the well defined use-case of the exploration of annotated relightable stratigraphic models (in particular, for local and remote museum presentation). My main results and contributions to the state of the art have been a novel technique for interactively controlling visualization lenses while automatically maintaining good focus-and-context parameters, a novel approach for avoiding clutter in an annotated model and for guiding users towards interesting areas, and a method for structuring audio-visual object annotations into a graph and for using that graph to improve guidance and support storytelling and automated tours. We demonstrated the effectiveness and potential of our techniques by performing interactive exploration sessions on various screen sizes and types ranging from desktop devices to large-screen displays for a walk-up-and-use museum installation. KEYWORDS - Computer Graphics, Human-Computer Interaction, Interactive Lenses, Focus-and-Context, Annotated Models, Cultural Heritage Computing

Transmission adaptative de modèles 3D massifs

Avec les progrès de l'édition de modèles 3D et des techniques de reconstruction 3D, de plus en plus de modèles 3D sont disponibles et leur qualité augmente. De plus, le support de la visualisation 3D sur le web s'est standardisé ces dernières années. Un défi majeur est donc de transmettre des modèles massifs à distance et de permettre aux utilisateurs de visualiser et de naviguer dans ces environnements virtuels. Cette thèse porte sur la transmission et l'interaction de contenus 3D et propose trois contributions majeures. Tout d'abord, nous développons une interface de navigation dans une scène 3D avec des signets -- de petits objets virtuels ajoutés à la scène sur lesquels l'utilisateur peut cliquer pour atteindre facilement un emplacement recommandé. Nous décrivons une étude d'utilisateurs où les participants naviguent dans des scènes 3D avec ou sans signets. Nous montrons que les utilisateurs naviguent (et accomplissent une tâche donnée) plus rapidement en utilisant des signets. Cependant, cette navigation plus rapide a un inconvénient sur les performances de la transmission : un utilisateur qui se déplace plus rapidement dans une scène a besoin de capacités de transmission plus élevées afin de bénéficier de la même qualité de service. Cet inconvénient peut être atténué par le fait que les positions des signets sont connues à l'avance : en ordonnant les faces du modèle 3D en fonction de leur visibilité depuis un signet, on optimise la transmission et donc, on diminue la latence lorsque les utilisateurs cliquent sur les signets. Deuxièmement, nous proposons une adaptation du standard de transmission DASH (Dynamic Adaptive Streaming over HTTP), très utilisé en vidéo, à la transmission de maillages texturés 3D. Pour ce faire, nous divisons la scène en un arbre k-d où chaque cellule correspond à un adaptation set DASH. Chaque cellule est en outre divisée en segments DASH d'un nombre fixe de faces, regroupant des faces de surfaces comparables. Chaque texture est indexée dans son propre adaptation set à différentes résolutions. Toutes les métadonnées (les cellules de l'arbre k-d, les résolutions des textures, etc.) sont référencées dans un fichier XML utilisé par DASH pour indexer le contenu: le MPD (Media Presentation Description). Ainsi, notre framework hérite de la scalabilité offerte par DASH. Nous proposons ensuite des algorithmes capables d'évaluer l'utilité de chaque segment de données en fonction du point de vue du client, et des politiques de transmission qui décident des segments à télécharger. Enfin, nous étudions la mise en place de la transmission et de la navigation 3D sur les appareils mobiles. Nous intégrons des signets dans notre version 3D de DASH et proposons une version améliorée de notre client DASH qui bénéficie des signets. Une étude sur les utilisateurs montre qu'avec notre politique de chargement adaptée aux signets, les signets sont plus susceptibles d'être cliqués, ce qui améliore à la fois la qualité de service et la qualité d'expérience des utilisateur

NAVIGATION SUPPORT AND SOCIAL VISUALIZATION FOR PERSONALIZED E-LEARNING

A large number of educational resources is now made available on the Web to support both regular classroom learning and online learning. However, the abundance of available content produced at least two problems: how to help students to find the most appropriate resources and how to engage them into using these resources and benefit from them. Personalized and social learning have been suggested as potential ways to address these problems. This work attempts to combine the ideas of personalized and social learning by providing navigation support through an open social student modeling visualization. A series of classroom studies exploited the idea of the approach and revealed promising results, which demonstrated the personalized guidance and social visualization combined helped students to find the most relevant resources of parameterized self-assessment questions for Java programming. Thus, this dissertation extend the approach to a larger collection of learning objects for cross content navigation and verify its capability of supporting social visualization for personalized E-Learning. The study results confirm that working with the non-mandatory system, students enhanced the learning quality in increasing their motivation and engagement. They successfully achieved better learning results. Meanwhile, incorporating a mixed collection of content in the open social student modeling visualizations effectively led the students to work at the right level of questions. Both strong and weak student worked with the appropriate levels of questions for their readiness accordingly and yielded a consistent performance across all three levels of complexities. Additionally, providing a more realistic content collection on the navigation supported open social student modeling visualizations results in a uniform performance in the group. The classroom study revealed a clear pattern of social guidance, where the stronger students left the traces for weaker ones to follow. The subjective evaluation confirms the design of the interface in terms of the content organization. Students’ positive responses also compliment the objective system usage data

Using a Real-Time Object Detection Application to Illustrate Effectiveness of Offloading and Prefetching in Cloudlet Architecture

In this thesis, we designed and implemented two versions of a real-time object de- tection application: A stand alone version and a cloud version. Through applying the application to a cloudlet environment, we are able to perform experiments and uses the results to illustrate the potential improvement that a cloudlet architecture can bring to mobile applications that require access to large amounts of cloud data or intensive com- putation. Potential improvements include data access speed, reduced CPU and memory usages as well as reduced battery consumption on mobile devices

Content Discovery in Online Services: A Case Study on a Video on Demand System

Video-on-demand services have gained popularity in recent years for the large catalogue of content they offer and the ability to watch them at any desired time. Having many options to choose from may be overwhelming for the users and affect negatively the overall experience. The use of recommender systems has been proven to help users discover relevant content faster. However, content discovery is affected not only by the number of choices, but also by the way the content is displayed to the user. Moreover, the development of recommender systems has been commonly focused on increasing their prediction accuracy, rather than the usefulness and user experience. This work takes on a user-centric approach to designing an efficient content discovery experience for its users. The main contribution of this research is a set of guidelines for designing the user interface and recommender system for the aforementioned purpose, formulated based on a user study and existing research. The guidelines were additionally translated into interface designs, which were then evaluated with users. The results showed that users were satisfied with the proposed design and the goal of providing a better content discovery experience was achieved. Moreover, the guidelines were found feasible by the company in which the research was conducted and thus have a high potential to work in a real product. With this research, I aim to highlight the importance of improving the content discovery process both from the perspective of the user interface and a recommender system, and encourage researchers to consider the user experience in those aspects

Theory of the perceived motion direction of equal-spatial-frequency plaid stimuli.

At an early stage, 3 different systems independently extract visual motion information from visual inputs. At later stages, these systems combine their outputs. Here, we consider a much studied (>650 publications) class of visual stimuli, plaids, which are combinations of 2 sine waves. Currently, there is no quantitative theory that can account for the perceived motion of plaids. We consider only perceived plaid direction, not speed, and obtain a large set of data exploring the various dimensions in which same-spatial-frequency plaids differ. We find that only 2 of the 3 motion systems are active in plaid processing, and that plaids with temporal frequencies 10 Hz or greater typically stimulate only the first-order motion system, which combines the plaid components by vector summation: Each plaid component is represented by a contrast-strength vector whose length is contrast-squared times a factor representing the relative effectiveness of that component's temporal frequency. The third-order system, which becomes primary at low temporal frequencies, also represents a plaid as 2 vectors that sum according to their contrast strength: a pure plaid in which both components have equal contrast and a residual sine wave. Second-order motion is irrelevant for these plaids. These principles enable a contrast-strength-vector summation theory for the responses of the first-order and third-order motion systems. With zero parameters estimated from the data, the theory captures the essence of the full range of the plaid data and supports the counterintuitive hypothesis that motion direction is processed independently of speed at early stages of visual processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved)

Dynamic load balancing in image retargeting using pipeline architecture

In today’s smart world demand of efficient multimedia based communication has increased at a rapid rate. Diversity on display sizes of gadgets used for multimedia communication confines the quality of images. Image retargeting is used as the focal solution to this problem which results in images with appropriate sizes. Enormously mounting demand of image retargeting expedites the rate of increment in computational load. This research paper expatiate and experiments a dynamic load balancing based three phase image retargeting methodology using pipeline architecture. In the first phase of image retargeting resize operation is performed on input image which results in multiple sized image copies of the same image. In the second phase resized images undergo quantization operation. In the final phase lossless compression is performed to have an expedient image. In the proposed exhibit think, we have done statistical analysis of results obtained, to confirm an impartial dynamic load balancing with a better degree of underlying resource utilization. We extend the approach to achieve significant storage optimization using three phase image retargeting

Generating 3D thumbnails for 3D contents

Ankara : The Department of Computer Engineering and the Institute of Engineering and Science of Bilkent University, 2010.Thesis (Master's) -- Bilkent University, 2010.Includes bibliographical references leaves 74-80.Creating effective thumbnails is a challenging task because finding important features and preserving the quality of the content is difficult. Additionally, as the popularity of 3D technologies is increasing, the usage area of 3D contents and the 3D content databases are expanding. However, for these contents, 2D thumbnails are insufficient since the current methods for generating them do not maintain the important and recognizable features of the content and give the idea of the content. In this thesis, we introduce a new thumbnail format, 3D thumbnail that helps users to understand the content of the 3D graphical scene or 3D video + depth by preserving the recognizable features and qualities. For creating 3D thumbnails, we developed a framework that generates 3D thumbnails for different 3D contents in order to create meaningful and qualified thumbnails. Thus, the system selects the best viewpoint in order to capture the scene with a high amount of detail for 3D graphical scenes. On the other hand, this process is different for 3D videos. In this case, by a saliency-depth based approach, we find the important objects on the selected frame of the 3D video and preserve them. Finally, after some steps such as placement, 3D rendering etc., the resulting thumbnails give more glorified information about the content. Finally, several experiments are presented which show that our proposed 3D thumbnail format is statistically (p <0.05 ) better than 2D thumbnails.Yiğit, YelizM.S

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the user's need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the user's need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search