Accessible user interface support for multi-device ubiquitous applications: architectural modifiability considerations

The market for personal computing devices is rapidly expanding from PC, to mobile, home entertainment systems, and even the automotive industry. When developing software targeting such ubiquitous devices, the balance between development costs and market coverage has turned out to be a challenging issue. With the rise of Web technology and the Internet of things, ubiquitous applications have become a reality. Nonetheless, the diversity of presentation and interaction modalities still drastically limit the number of targetable devices and the accessibility toward end users. This paper presents webinos, a multi-device application middleware platform founded on the Future Internet infrastructure. Hereto, the platform's architectural modifiability considerations are described and evaluated as a generic enabler for supporting applications, which are executed in ubiquitous computing environments

Ambient hues and audible cues: An approach to automotive user interface design using multi-modal feedback

The use of touchscreen interfaces for in-vehicle information, entertainment, and for the control of comfort settings is proliferating. Moreover, using these interfaces requires the same visual and manual resources needed for safe driving. Guided by much of the prevalent research in the areas of the human visual system, attention, and multimodal redundancy the Hues and Cues design paradigm was developed to make touchscreen automotive user interfaces more suitable to use while driving. This paradigm was applied to a prototype of an automotive user interface and evaluated with respects to driver performance using the dual-task, Lane Change Test (LCT). Each level of the design paradigm was evaluated in light of possible gender differences. The results of the repeated measures experiment suggests that when compared to interfaces without both the Hues and the Cues paradigm applied, the Hues and Cues interface requires less mental effort to operate, is more usable, and is more preferred. However, the results differ in the degradation in driver performance with interfaces that only have visual feedback resulting in better task times and significant gender differences in the driving task with interfaces that only have auditory feedback. Overall, the results reported show that the presentation of multimodal feedback can be useful in design automotive interfaces, but must be flexible enough to account for individual differences

Connected Car: technologies, issues, future trends

The connected car -a vehicle capable of accessing to the Internet, of communicating with smart devices as well as other cars and road infrastructures, and of collecting real-time data from multiple sources- is likely to play a fundamental role in the foreseeable Internet Of Things. In a context ruled by very strong competitive forces, a significant amount of car manufacturers and software and hardware developers have already embraced the challenge of providing innovative solutions for new generation vehicles. Today’s cars are asked to relieve drivers from the most stressful operations needed for driving, providing them with interesting and updated entertainment functions. In the meantime, they have to comply to the increasingly stringent standards about safety and reliability. The aim of this paper is to provide an overview of the possibilities offered by connected functionalities on cars and the associated technological issues and problems, as well as to enumerate the currently available hardware and software solutions and their main features

Mobile Web applications

This document presents the work that was elaborated at the company Present Technologies as part of the academic discipline Internship/Industrial Project for the Master’s degree in Informatics and Systems, Software Development branch, at Instituto Superior de Engenharia de Coimbra. The area of the mobile web applications has grown exponentially over the last few years turning it into a very dynamic field where new development platforms and frameworks are constantly emerging. Thus, the internship consisted in the study of two new mobile operating systems, Tizen and Firefox OS, as well as two frameworks for packaging of mobile web applications – Adobe PhoneGap and Appcelerator Titanium. These platforms are in the direct interest of Present Technology since it pretends to use them in its future projects in general and in the Phune Gaming project in particular. Since Television is one of the Present Technologies’ business areas, during the course of the internship it was decided to perform additionally a study of two Smart TV platforms, namely Samsung Smart TV and Opera TV, which was considered as a valuable knowledge for the company. For each of the platforms was performed a study about its architecture, supported standards and the development tools that are provided, nevertheless the focus was on the applications and for this reason a practical case study was conducted. The case studies consisted in the creation of a prototype or packaging of an application, for the case of the packaging tools, in order to prove the feasibility of the applications for the Present Technologies’ needs. The outcome of the work performed during the internship is that it raised the awareness of Present Technology of the studied platforms, providing it with prototypes and written documentation for the platforms’ successful usage in future projects

Mobile Web applications

This document presents the work that was elaborated at the company Present Technologies as part of the academic discipline Internship/Industrial Project for the Master’s degree in Informatics and Systems, Software Development branch, at Instituto Superior de Engenharia de Coimbra. The area of the mobile web applications has grown exponentially over the last few years turning it into a very dynamic field where new development platforms and frameworks are constantly emerging. Thus, the internship consisted in the study of two new mobile operating systems, Tizen and Firefox OS, as well as two frameworks for packaging of mobile web applications – Adobe PhoneGap and Appcelerator Titanium. These platforms are in the direct interest of Present Technology since it pretends to use them in its future projects in general and in the Phune Gaming project in particular. Since Television is one of the Present Technologies’ business areas, during the course of the internship it was decided to perform additionally a study of two Smart TV platforms, namely Samsung Smart TV and Opera TV, which was considered as a valuable knowledge for the company. For each of the platforms was performed a study about its architecture, supported standards and the development tools that are provided, nevertheless the focus was on the applications and for this reason a practical case study was conducted. The case studies consisted in the creation of a prototype or packaging of an application, for the case of the packaging tools, in order to prove the feasibility of the applications for the Present Technologies’ needs. The outcome of the work performed during the internship is that it raised the awareness of Present Technology of the studied platforms, providing it with prototypes and written documentation for the platforms’ successful usage in future projects

A Distributed Service Delivery Platform for Automotive Environments: Enhancing Communication Capabilities of an M2M Service Platform for Automotive Application

Full version: Access restricted permanently due to 3rd party copyright restrictions. Restriction set on 11.04.2018 by SE, Doctoral CollegeThe automotive domain is changing. On the way to more convenient, safe, and efficient vehicles, the role of electronic controllers and particularly software has increased significantly for many years, and vehicles have become software-intensive systems. Furthermore, vehicles are connected to the Internet to enable Advanced Driver Assistance Systems and enhanced In-Vehicle Infotainment functionalities. This widens the automotive software and system landscape beyond the physical vehicle boundaries to presently include as well external backend servers in the cloud. Moreover, the connectivity facilitates new kinds of distributed functionalities, making the vehicle a part of an Intelligent Transportation System (ITS) and thus an important example for a future Internet of Things (IoT). Manufacturers, however, are confronted with the challenging task of integrating these ever-increasing range of functionalities with heterogeneous or even contradictory requirements into a homogenous overall system. This requires new software platforms and architectural approaches. In this regard, the connectivity to fixed side backend systems not only introduces additional challenges, but also enables new approaches for addressing them. The vehicle-to-backend approaches currently emerging are dominated by proprietary solutions, which is in clear contradiction to the requirements of ITS scenarios which call for interoperability within the broad scope of vehicles and manufacturers. Therefore, this research aims at the development and propagation of a new concept of a universal distributed Automotive Service Delivery Platform (ASDP), as enabler for future automotive functionalities, not limited to ITS applications. Since Machine-to-Machine communication (M2M) is considered as a primary building block for the IoT, emergent standards such as the oneM2M service platform are selected as the initial architectural hypothesis for the realisation of an ASDP. Accordingly, this project describes a oneM2M-based ASDP as a reference configuration of the oneM2M service platform for automotive environments. In the research, the general applicability of the oneM2M service platform for the proposed ASDP is shown. However, the research also identifies shortcomings of the current oneM2M platform with respect to the capabilities needed for efficient communication and data exchange policies. It is pointed out that, for example, distributed traffic efficiency or vehicle maintenance functionalities are not efficiently treated by the standard. This may also have negative privacy impacts. Following this analysis, this research proposes novel enhancements to the oneM2M service platform, such as application-data-dependent criteria for data exchange and policy aggregation. The feasibility and advancements of the newly proposed approach are evaluated by means of proof-of-concept implementation and experiments with selected automotive scenarios. The results show the benefits of the proposed enhancements for a oneM2M-based ASDP, without neglecting to indicate their advantages for other domains of the oneM2M landscape where they could be applied as well

Linux based mobile: operating systems

Trabalho de Projeto para obtenção do grau de Mestre em Engenharia Informática e de ComputadoresNos últimos quinze anos a industria móvel evolui de um Nokia 3310 que conseguia guardar vinte e quatro registos de chamadas para um iPhone que literalmenteconsegue salvaguardar uma vida inteira de chamadas. A industria móvel cresceu edescartou na maioria os sistema operativos proprietários, convergindo os seus esforços numa selecção de sistemas como Android, iOS e Windows Phone. Os sistemas operativos móveis estão em todo o lado, nos nossos telefones, relógiosou carros. Estes reestruturaram completamente a sociedade oferecendo a possibilidade de contactar qualquer pessoa no mundo inteiro a qualquer hora. Hoje emdia praticamente não conseguimos viver sem os nossos dispositivos móveis porqueos utilizamos para trabalhar, socializar, estudar e consumir informação. Apesar de serem os sistemas mais utilizados no mundo, os mecanismos internos,como é que eles executam, ou como trabalhar com eles continua a ser sujeito a taboo, devido à sua complexidade. Este projecto apresenta como é que um sistema operativo móvel moderno está organizado, como o compilar e como os executar num sistema embebido. Para o fazer foinecessário realizar um estudo para entender o kernel Linux, como é que este corree o que contém. Os sistemas Android e Tizen foram estudados e compreendidos deforma a entender o seu processo de compilação e execução. Finalmente, o projecto também descreve como executar num mesmo dispositivovários sistemas operativos e como os controlar. A prova de conceito foi realizadanuma placa de prototipagem ARM, usando um processador com a tecnologia maisrecente.Abstract: In the last fifteen years the mobile industry evolved from the Nokia 3310 that couldstore a hopping twenty-four phone records to an iPhone that literately can savea lifetime phone history. The mobile industry grew and thrown way most of theproprietary operating systems to converge their efforts in a selected few, such asAndroid, iOS and Windows Phone. Mobile operating systems are everywhere: on our phones, watches or cars. Theycompletely reshaped the worldwide society by having instant contact with virtuallyeveryone everywhere. Nowadays we almost can’t live without our mobile devicesbecause we use them to work, socialize, study and consume information. Although being the most used operating systems on the planet, the internal mechanisms, how they run and how towork them is still subject of taboo, mainly becausethe complexity that these systems have. This project presents how a modern mobile operating system is organized, howto build it and how to deploy into an embedded device. To accomplish that, thenecessary study was made to understand the Linux kernel, how it runs and what itcontains. The full fledged operating systems Android and Tizen were dismemberedto their core and analysed/studied on how to build and deploy them. Finally, the project also describes how to deploy on a single device, multiple operating systems and how can onemanage them. The proof of concept was built underan ARM board using the latest processor technology

웹 어플리케이션을 위한 스냅샷 활용

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 2. 문수묵.Web applications (apps) are programmed using HTML5, CSS, and JavaScript, and are distributed in the source code format. Web apps can be executed on any devices where a web browser is installed, allowing one-source, multi-platform environment. We can exploit this advantage of platform independence for a new user experience called app migration, which allows migrating an app in the middle of execution seamlessly between smart devices. We proposes such a migration framework for web apps where we can save the current state of a running app and resume its execution on a different device by restoring the saved state. We save the web apps state in the form of a snapshot, which is actually another web app whose execution can restore the saved state. In the snapshot, the state of the JavaScript variables and DOM trees are saved using the JSON format. We solved some of the saving/restoring problems related to event handlers and closures by accessing the browser and the JavaScript engine internals. Our framework does not require instrumenting an app or changing its source code, but works for the original app. We implemented the framework on the Chrome browser with the V8 JavaScript engine and successfully migrated non-trivial sample apps with reasonable saving and restoring overhead. We also discuss other usage of the snapshot for optimizations and user experiences for the web platform. Another issues of web app is its performance. Web apps are involved with a performance issue due to JavaScript, because its dynamic typing, function object, and prototype are difficult to execute efficiently, so even just-in-time compilers do not help much. We propose a new approach to accelerate a web app, especially its loading time. Generally, running an app is composed of app loading to initialize the app, followed by event-driven computation. If the same job needs to be done to load an app, especially the execution of the same JavaScript code, it will be better to save the JavaScript execution state in advance and to start the app from the saved state. In fact, app loading is often involved with the initialization of the web framework such as jQuery [1], Enyo [2] , or Ext JS [3] where many JavaScript objects are created. Also, app-specific objects are created during app loading. If we save the initialized state of these objects in the form of a snapshot and start app loading by restoring the objects from the snapshot, we would accelerate app loading. Snapshot which saves JavaScript execution state has limitation because it cannot save DOM state. By using DOM Log-Replay which saves DOM state, we can save whole app loading state. Thus, we can remove all JavaScript execution during web app loading and directly reach the state which loading is completed by DOM snapshot, resulting in acceleration of web application loading time. DOM Log-Replay saves every DOM-related action occurs during loading time as a log. When restoring snapshot, log is replayed to restore DOM tree and various DOM elements. We actually implemented the idea for the web applications based on two web framework. This can reduce the whole app loading time by at least 60%, which could be noticed tangibly. Because snapshot saves JavaScript state as file, there is size overhead. This can be an issue in device with small memory such as IoT. Thus, size of snapshot needs to be reduced. Firstly, we compress snapshot and hide decompression overhead by implementing pre-decompression which decompresses snapshot in another thread before snapshot deserialization. Secondly, we implemented framework snapshot sharing which shares snapshot of JavaScript framework between apps developed with the same framework. With these techniques, performance is degraded for about 1.7%, but memory overhead is dramatically reduced from 12x to 2.59x.Chapter 1. Introduction 1 1.1. Web Applications 1 1.2. Snapshot for Web Applications 2 1.3. Organization of the thesis 3 Chapter 2. How Web Apps Work 5 2.1. Web App Loading 5 2.2. JavaScript Execution State 9 Chapter 3. Migration of Web Applications 11 3.1. App Migration 11 3.2. App Migration Framework and Scenario 12 3.3. Saving and Restoring the App State 14 3.4. Issues for Saving and Restoring the App State 19 3.4.1. Object Reference Alias Problem 19 3.4.2. Saving Closure Function and Closure Variables 22 3.4.3. DOM Tree and DOM Reference Variable 26 3.4.4. Event Handler 28 3.5. Implementation 31 3.6. Evaluation 33 3.6.1. Experimental Environment 33 3.6.2. State Saving and Restoration Time Overhead 34 3.6.3. Snapshot File Size 35 3.6.4. Portion of State Save/Restoration 36 3.6.5. Number of Objects Saved and Restored 37 3.7. Other Usage of the Snapshot 38 Chapter 4. Loading Time Acceleration of Web Applications 40 4.1. Acceleration of Web Apps with Snapshot 40 4.2. Saving the JavaScript State 41 4.3. JavaScript Web Framework 43 4.4. Approach to Taking the Snapshot 46 4.5. Saving and Restoring the Snapshot 48 4.5.1. Approach to Save/Restore JavaScript State 48 4.5.2. Snapshot for JSC 49 4.5.3. Snapshot for V8 59 4.5.4. DOM Objects and Event Objects 66 4.6. Determinism for Snapshot 69 4.7. Experimental Results 72 4.7.1. Acceleration of Framework Initialization 73 4.7.2. Acceleration of Framework Initialization 75 Chapter 5. Enhanced Snapshot Optimization 78 5.1. Limitation of JavaScript Snapshot 78 5.2. Architecture for Enhanced Snapshot Optimization 79 5.3. DOM Log-Replay 81 5.3.1. Why DOM Log-Replay 81 5.3.2. Capturing DOM Log 84 5.3.3. Replay DOM 85 5.4. Pre-Decompressing Snapshot 86 5.5. Framework Snapshot Sharing 88 5.6. Evaluation 90 5.6.1. Environment 90 5.6.2. Benchmark Web App 91 5.6.3. Acceleration of Web Apps using DOM Log-Replay 91 5.6.4. Memory reduction by Pre-Decompressing 93 5.6.5. Memory reduction by Framework Sharing 94 Chapter 6. Related Works 96 Chapter 7. Conclusion 100 Bibliography 103 초록 106Docto

Contributions to Big Geospatial Data Rendering and Visualisations

Current geographical information systems lack features and components which are commonly found within rendering and game engines. When combined with computer game technologies, a modern geographical information system capable of advanced rendering and data visualisations are achievable. We have investigated the combination of big geospatial data, and computer game engines for the creation of a modern geographical information system framework capable of visualising densely populated real-world scenes using advanced rendering algorithms. The pipeline imports raw geospatial data in the form of Ordnance Survey data which is provided by the UK government, LiDAR data provided by a private company, and the global open mapping project of OpenStreetMap. The data is combined to produce additional terrain data where data is missing from the high resolution data sources of LiDAR by utilising interpolated Ordnance Survey data. Where data is missing from LiDAR, the same interpolation techniques are also utilised. Once a high resolution terrain data set which is complete in regards to coverage, is generated, sub datasets can be extracted from the LiDAR using OSM boundary data as a perimeter. The boundaries of OSM represent buildings or assets. Data can then be extracted such as the heights of buildings. This data can then be used to update the OSM database. Using a novel adjacency matrix extraction technique, 3D model mesh objects can be generated using both LiDAR and OSM information. The generation of model mesh objects created from OSM data utilises procedural content generation techniques, enabling the generation of GIS based 3D real-world scenes. Although only LiDAR and Ordnance Survey for UK data is available, restricting the generation to the UK borders, using OSM alone, the system is able to procedurally generate any place within the world covered by OSM. In this research, to manage the large amounts of data, a novel scenegraph structure has been generated to spatially separate OSM data according to OS coordinates, splitting the UK into 1kilometer squared tiles, and categorising OSM assets such as buildings, highways, amenities. Once spatially organised, and categorised as an asset of importance, the novel scenegraph allows for data dispersal through an entire scene in real-time. The 3D real-world scenes visualised within the runtime simulator can be manipulated in four main aspects; • Viewing at any angle or location through the use of a 3D and 2D camera system. • Modifying the effects or effect parameters applied to the 3D model mesh objects to visualise user defined data by use of our novel algorithms and unique lighting data-structure effect file with accompanying material interface. • Procedurally generating animations which can be applied to the spatial parameters of objects, or the visual properties of objects. • Applying Indexed Array Shader Function and taking advantage of the novel big geospatial scenegraph structure to exploit better rendering techniques in the context of a modern Geographical Information System, which has not been done, to the best of our knowledge. Combined with a novel scenegraph structure layout, the user can view and manipulate real-world procedurally generated worlds with additional user generated content in a number of unique and unseen ways within the current geographical information system implementations. We evaluate multiple functionalities and aspects of the framework. We evaluate the performance of the system, measuring frame rates with multi sized maps by stress testing means, as well as evaluating the benefits of the novel scenegraph structure for categorising, separating, manoeuvring, and data dispersal. Uniform scaling by n2 of scenegraph nodes which contain no model mesh data, procedurally generated model data, and user generated model data. The experiment compared runtime parameters, and memory consumption. We have compared the technical features of the framework against that of real-world related commercial projects; Google Maps, OSM2World, OSM-3D, OSM-Buildings, OpenStreetMap, ArcGIS, Sustainability Assessment Visualisation and Enhancement (SAVE), and Autonomous Learning Agents for Decentralised Data and Information (ALLADIN). We conclude that when compared to related research, the framework produces data-sets relevant for visualising geospatial assets from the combination of real-world data-sets, capable of being used by a multitude of external game engines, applications, and geographical information systems. The ability to manipulate the production of said data-sets at pre-compile time aids processing speeds for runtime simulation. This ability is provided by the pre-processor. The added benefit is to allow users to manipulate the spatial and visual parameters in a number of varying ways with minimal domain knowledge. The features of creating procedural animations attached to each of the spatial parameters and visual shading parameters allow users to view and encode their own representations of scenes which are unavailable within all of the products stated. Each of the alternative projects have similar features, but none which allow full animation ability of all parameters of an asset; spatially or visually, or both. We also evaluated the framework on the implemented features; implementing the needed algorithms and novelties of the framework as problems arose in the development of the framework. Examples of this is the algorithm for combining the multiple terrain data-sets we have (Ordnance Survey terrain data and Light Detection and Ranging Digital Surface Model data and Digital Terrain Model data), and combining them in a justifiable way to produce maps with no missing data values for further analysis and visualisation. A majority of visualisations are rendered using an Indexed Array Shader Function effect file, structured to create a novel design to encapsulate common rendering effects found in commercial computer games, and apply them to the rendering of real-world assets for a modern geographical information system. Maps of various size, in both dimensions, polygonal density, asset counts, and memory consumption prove successful in relation to real-time rendering parameters i.e. the visualisation of maps do not create a bottleneck for processing. The visualised scenes allow users to view large dense environments which include terrain models within procedural and user generated buildings, highways, amenities, and boundaries. The use of a novel scenegraph structure allows for the fast iteration and search from user defined dynamic queries. The interaction with the framework is allowed through a novel Interactive Visualisation Interface. Utilising the interface, a user can apply procedurally generated animations to both spatial and visual properties to any node or model mesh within the scene. We conclude that the framework has been a success. We have completed what we have set out to develop and create, we have combined multiple data-sets to create improved terrain data-sets for further research and development. We have created a framework which combines the real-world data of Ordnance Survey, LiDAR, and OpenStreetMap, and implemented algorithms to create procedural assets of buildings, highways, terrain, amenities, model meshes, and boundaries. for visualisation, with implemented features which allows users to search and manipulate a city’s worth of data on a per-object basis, or user-defined combinations. The successful framework has been built by the cross domain specialism needed for such a project. We have combined the areas of; computer games technology, engine and framework development, procedural generation techniques and algorithms, use of real-world data-sets, geographical information system development, data-parsing, big-data algorithmic reduction techniques, and visualisation using shader techniques