New data analytics and visualization methods in personal data mining, cancer data analysis and sports data visualization

In this dissertation, we discuss a reading profiling system, a biological data visualization system and a sports visualization system. Self-tracking is getting increasingly popular in the field of personal informatics. Reading profiling can be used as a personal data collection method. We present UUAT, an unintrusive user attention tracking system. In UUAT, we used user interaction data to develop technologies that help to pinpoint a users reading region (RR). Based on computed RR and user interaction data, UUAT can identify a readers reading struggle or interest. A biomarker is a measurable substance that may be used as an indicator of a particular disease. We developed CancerVis for visual and interactive analysis of cancer data and demonstrate how to apply this platform in cancer biomarker research. CancerVis provides interactive multiple views from different perspectives of a dataset. The views are synchronized so that users can easily link them to a same data entry. Furthermore, CancerVis supports data mining practice in cancer biomarker, such as visualization of optimal cutpoints and cutthrough exploration. Tennis match summarization helps after-live sports consumers assimilate an interested match. We developed TennisVis, a comprehensive match summarization and visualization platform. TennisVis offers chart- graph for a client to quickly get match facts. Meanwhile, TennisVis offers various queries of tennis points to satisfy diversified client preferences (such as volley shot, many-shot rally) of tennis fans. Furthermore, TennisVis offers video clips for every single tennis point and a recommendation rating is computed for each tennis play. A case study shows that TennisVis identifies more than 75% tennis points in full time match

Home-based physical therapy with an interactive computer vision system

In this paper, we present ExerciseCheck. ExerciseCheck is an interactive computer vision system that is sufficiently modular to work with different sources of human pose estimates, i.e., estimates from deep or traditional models that interpret RGB or RGB-D camera input. In a pilot study, we first compare the pose estimates produced by four deep models based on RGB input with those of the MS Kinect based on RGB-D data. The results indicate a performance gap that required us to choose the MS Kinect when we tested ExerciseCheck with Parkinson’s disease patients in their homes. ExerciseCheck is capable of customizing exercises, capturing exercise information, evaluating patient performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. We conclude that ExerciseCheck is a user-friendly computer vision application that can assist patients by providing motivation and guidance to ensure correct execution of the required exercises. Our results also suggest that while there has been considerable progress in the field of pose estimation using deep learning, current deep learning models are not fully ready to replace RGB-D sensors, especially when the exercises involved are complex, and the patient population being accounted for has to be carefully tracked for its “active range of motion.”Published versio

SNAP, Crackle, WebWindows!

We elaborate the SNAP---Scalable (ATM) Network and (PC) Platforms---view of computing in the year 2000. The World Wide Web will continue its rapid evolution, and in the future, applications will not be written for Windows NT/95 or UNIX, but rather for WebWindows with interfaces defined by the standards of Web servers and clients. This universal environment will support WebTop productivity tools, such as WebWord, WebLotus123, and WebNotes built in modular dynamic fashion, and undermining the business model for large software companies. We define a layered WebWindows software architecture in which applications are built on top of multi-use services. We discuss examples including business enterprise systems (IntraNets), health care, financial services and education. HPCC is implicit throughout this discussion for there is no larger parallel system than the World Wide metacomputer. We suggest building the MPP programming environment in terms of pervasive sustainable WebWindows technologies. In particular, WebFlow will support naturally dataflow integrating data and compute intensive applications on distributed heterogeneous systems

A framework for the development of tolerant real time applications

This work presents a framework architecture for the development of distributed real-time applications to be integrated into WWW clients. It assumes a WWW environment over networks providing a best-effort delivery service like the internets based on the IP protocol. The framework is that of an application programming interface (API) providing the program developer with the services needed by tolerant realtime applications. Once developed, an application is bundled together with the API to form a WWW plug-in which can subsequently be called from a WWW client interface or browser. The application is then perceived as being integrated into the WWW environment. The design aims to provide real-time applications with a transport service layer assuring near end-to-end isochronism despite the weak guaranties of the underlying network service. The implementation of the mechanisms that allow multistream real-time communications to adapt to the operational conditions of these networks are discussed. In this work, the RTP and RTCP protocols were also implemented as part of the API. Experience with this framework reports the development of a prototype real-time application for multimedia group communication and the analysis of the behaviour of RTP sessions in a real operational situation. The analysis uses protocol state data logged during their operation

A Web-Based Collaborative Multimedia Presentation Document System

With the distributed and rapidly increasing volume of data and expeditious development of modern web browsers, web browsers have become a possible legitimate vehicle for remote interactive multimedia presentation and collaboration, especially for geographically dispersed teams. To our knowledge, although there are a large number of applications developed for these purposes, there are some drawbacks in prior work including the lack of interactive controls of presentation flows, general-purpose collaboration support on multimedia, and efficient and precise replay of presentations. To fill the research gaps in prior work, in this dissertation, we propose a web-based multimedia collaborative presentation document system, which models a presentation as media resources together with a stream of media events, attached to associated media objects. It represents presentation flows and collaboration actions in events, implements temporal and spatial scheduling on multimedia objects, and supports real-time interactive control of the predefined schedules. As all events are represented by simple messages with an object-prioritized approach, our platform can also support fine-grained precise replay of presentations. Hundreds of kilobytes could be enough to store the events in a collaborative presentation session for accurate replays, compared with hundreds of megabytes in screen recording tools with a pixel-based replay mechanism

Proceedings of the 1st International Conference on Live Coding

Open Access peer reviewed papers on live coding published at the 1st International Conference on Live Coding (ICLC) in Leeds

Design and implementation of a modular controller for robotic machines

This research focused on the design and implementation of an Intelligent Modular Controller (IMC) architecture designed to be reconfigurable over a robust network. The design incorporates novel communication, hardware, and software architectures. This was motivated by current industrial needs for distributed control systems due to growing demand for less complexity, more processing power, flexibility, and greater fault tolerance. To this end, three main contributions were made. Most distributed control architectures depend on multi-tier heterogeneous communication networks requiring linking devices and/or complex middleware. In this study, first, a communication architecture was proposed and implemented with a homogenous network employing the ubiquitous Ethernet for both real-time and non real-time communication. This was achieved by a producer-consumer coordination model for real-time data communication over a segmented network, and a client-server model for point-to-point transactions. The protocols deployed use a Time-Triggered (TT) approach to schedule real-time tasks on the network. Unlike other TT approaches, the scheduling mechanism does not need to be configured explicitly when controller nodes are added or removed. An implicit clock synchronization technique was also developed to complement the architecture. Second, a reconfigurable mechanism based on an auto-configuration protocol was developed. Modules on the network use this protocol to automatically detect themselves, establish communication, and negotiate for a desired configuration. Third, the research demonstrated hardware/software co-design as a contribution to the growing discipline of mechatronics. The IMC consists of a motion controller board designed and prototyped in-house, and a Java microcontroller. An IMC is mapped to each machine/robot axis, and an additional IMC can be configured to serve as a real-time coordinator. The entire architecture was implemented in Java, thus reinforcing uniformity, simplicity, modularity, and openness. Evaluation results showed the potential of the flexible controller to meet medium to high performance machining requirements

Development of an open access system for remote operation of robotic manipulators

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáExploring the realms of research, training, and learning in the field of robotic systems poses obstacles for institutions lacking the necessary infrastructure. The significant investment required to acquire physical robotic systems often limits access and hinders progress in these areas. While robotic simulation platforms provide a virtual environment for experimentation, the potential of remote robotic environments surpasses this by enabling users to interact with real robotic systems during training and research activities. This way, users, including students and researchers, can engage in a virtual experience that transcends geographical boundaries, connecting them to real-world robotic systems though the Internet. By bridging the gap between virtual and physical worlds, remote environments offer a more practical and immersive experience, and open up new horizons for collaborative research and training. Democratizing access to these technologies means empower educational institutions and research centers to engage in practical and handson learning experiences. However, the implementation of remote robotic environments comes with its own set of technical challenges: communication, security, stability and access. In light of these challenges, a ROS-based system has been developed, providing open access with promising results (low delay and run-time visualization). This system enables remote control of robotic manipulators and has been successfully validated through the remote operation of a real UR3 manipulator.Explorar as áreas de pesquisa, treinamento e aprendizado no campo de sistemas robóticos apresenta obstáculos para instituições que não possuem a infraestrutura necessária. O investimento significativo exigido para adquirir sistemas robóticos físicos muitas vezes limita o acesso e dificulta o progresso nessas áreas. Embora as plataformas de simulação robótica forneçam um ambiente virtual para experimentação, o potencial dos ambientes robóticos remotos vai além disso, permitindo que os usuários interajam com sistemas robóticos reais durante atividades de treinamento e pesquisa. Dessa forma, os usuários, incluindo estudantes e pesquisadores, podem participar de uma experiência virtual que transcende as fronteiras geográficas, conectando-os a sistemas robóticos do mundo real por meio da Internet. Ao estabelecer uma ponte entre os mundos virtual e físico, os ambientes remotos oferecem uma experiência mais prática e imersiva, abrindo novos horizontes para a pesquisa colaborativa e o treinamento. Democratizar o acesso a essas tecnologias significa capacitar instituições educacionais e centros de pesquisa a se envolverem em experiências práticas e de aprendizado prático. No entanto, a implementação de ambientes robóticos remotos traz consigo um conjunto próprio de desafios técnicos: comunicação, segurança, estabilidade e acesso. Diante desses desafios, foi desenvolvida uma plataforma baseada em ROS, oferecendo acesso aberto com resultados promissores (baixo delay e visualização em run-time). Essa plataforma possibilita o controle remoto de manipuladores robóticos e foi validada com sucesso por meio da operação remota de um manipulador UR3 real

Secure Communication in Disaster Scenarios

Während Naturkatastrophen oder terroristischer Anschläge ist die bestehende Kommunikationsinfrastruktur häufig überlastet oder fällt komplett aus. In diesen Situationen können mobile Geräte mithilfe von drahtloser ad-hoc- und unterbrechungstoleranter Vernetzung miteinander verbunden werden, um ein Notfall-Kommunikationssystem für Zivilisten und Rettungsdienste einzurichten. Falls verfügbar, kann eine Verbindung zu Cloud-Diensten im Internet eine wertvolle Hilfe im Krisen- und Katastrophenmanagement sein. Solche Kommunikationssysteme bergen jedoch ernsthafte Sicherheitsrisiken, da Angreifer versuchen könnten, vertrauliche Daten zu stehlen, gefälschte Benachrichtigungen von Notfalldiensten einzuspeisen oder Denial-of-Service (DoS) Angriffe durchzuführen. Diese Dissertation schlägt neue Ansätze zur Kommunikation in Notfallnetzen von mobilen Geräten vor, die von der Kommunikation zwischen Mobilfunkgeräten bis zu Cloud-Diensten auf Servern im Internet reichen. Durch die Nutzung dieser Ansätze werden die Sicherheit der Geräte-zu-Geräte-Kommunikation, die Sicherheit von Notfall-Apps auf mobilen Geräten und die Sicherheit von Server-Systemen für Cloud-Dienste verbessert