Adaptive object-modeling : patterns, tools and applications

Tese de Programa Doutoral. Informática. Universidade do Porto. Faculdade de Engenharia. 201

Automatically Generating Websites from Hand-drawn Mockups

Designers often use physical hand-drawn mockups to convey their ideas to stakeholders. Unfortunately, these sketches do not depict the exact final look and feel of web pages, and communication errors will often occur, resulting in prototypes that do not reflect the stakeholder's vision. Multiple suggestions exist to tackle this problem, mainly in the translation of visual mockups to prototypes. Some authors propose end-to-end solutions by directly generating the final code from a single (black-box) Deep Neural Network. Others propose the use of object detectors, providing more control over the acquired elements but missing out on the mockup's layout. Our approach provides a real-time solution that explores: (1) how to achieve a large variety of sketches that would look indistinguishable from something a human would draw, (2) a pipeline that clearly separates the different responsibilities of extracting and constructing the hierarchical structure of a web mockup, (3) a methodology to segment and extract containers from mockups, (4) the usage of in-sketch annotations to provide more flexibility and control over the generated artifacts, and (5) an assessment of the synthetic dataset impact in the ability to recognize diagrams actually drawn by humans. We start by presenting an algorithm that is capable of generating synthetic mockups. We trained our model (N=8400, Epochs=400) and subsequently fine-tuned it (N=74, Epochs=100) using real human-made diagrams. We accomplished a mAP of 95.37%, with 90% of the tests taking less than 430ms on modest commodity hardware (approximate to 2.3fps). We further provide an ablation study with well-known object detectors to evaluate the synthetic dataset in isolation, showing that the generator achieves a mAP score of 95%, approximate to 1.5 x higher than training using hand-drawn mockups alone

Towards a Pattern Language for the Masters Student

Every year, thousands of new students begin their Masters in STEM related topics. Despite being regarded as a common occurrence by the faculty, it represents the culmination of years of studying and preparation for their professional life. Notwithstanding, these students face well-known recurrent problems: how to choose a topic, how to choose an advisor, how to start researching, and how to deal with all the unknowns associated with academic research. Although there are several books on how to write a thesis, most of them avoid prescriptive recommendations on topics beyond research per se or focus on doctoral students, for which the duration and motivation are significantly different. In this paper, we draft a pattern language comprised of thirty patterns that we have observed from supervising over a hundred masters students within the last decade

Programming for young children using tangible tiles and camera-enabled handheld devices

Schools are trying to teach programming at an earlier age, but there are some difficulties, namely the cost of having enough computer stations for the kids. We present the tangible system Tactode for young students to learn to program in the classroom, using handheld camera devices. The system was tested with a small focus group of students between 10 and 12 years old, that were asked to draw a regular polygon using the Scratch cat. All students completed the required task although some required help. Both students and teachers reported that they thoroughly enjoyed the experience and would like to repeat. In questionaries following the activities, the students declared that they found the language easy to use, with only 14% deeming it somewhat difficult. We consider these early results encouraging as well as informative for future developments

Blockchain-based PKI for Crowdsourced IoT Sensor Information

The Internet of Things is progressively getting broader, evol-ving its scope while creating new markets and adding more to the existing ones. However, both generation and analysis of large amounts of data, which are integral to this concept, may require the proper protection and privacy-awareness of some sensitive information. In order to control the access to this data, allowing devices to verify the reliability of their own interactions with other endpoints of the network is a crucial step to ensure this required safeness. Through the implementation of a blockchain-based Public Key Infrastructure connected to the Keybase platform, it is possible to achieve a simple protocol that binds devices' public keys to their owner accounts, which are respectively supported by identity proofs. The records of this blockchain represent digital signatures performed by this Keybase users on their respective devices' public keys, claiming their ownership. Resorting to this distributed and decentralized PKI, any device is able to autonomously verify the entity in control of a certain node of the network and prevent future interactions with unverified parties

Visually-defined Real-Time Orchestration of IoT Systems

In this work, we propose a method for extending Node-RED to allow the automatic decomposition and partitioning of the system towards higher decentralization. We provide a custom firmware for constrained devices to expose their resources, as well as new nodes and modifications in the Node-RED engine that allow automatic orchestration of tasks. The firmware is responsible for low-level management of health and capabilities, as well as executing MicroPython scripts on demand. Node-RED then takes advantage of this firmware by (1) providing a device registry allowing devices to announce themselves, (2) generating MicroPython code from dynamic analysis of flow and nodes, and (3) automatically (re-)assigning nodes to devices based on pre-specified properties and priorities. A mechanism to automatically detect abnormal run-time conditions and provide dynamic self-adaptation was also explored. Our solution was tested using synthetic home automation scenarios, where several experiments were conducted with both virtual and physical devices. We then exhaustively measured each scenario to allow further understanding of our proposal and how it impacts the system's resiliency, efficiency, and elasticity