Reverseorc:Reverse engineering of resizable user interface layouts with or-constraints

Reverse engineering (RE) of user interfaces (UIs) plays an important role in software evolution. However, the large diversity of UI technologies and the need for UIs to be resizable make this challenging. We propose ReverseORC, a novel RE approach able to discover diverse layout types and their dynamic resizing behaviours independently of their implementation, and to specify them by using OR constraints. Unlike previous RE approaches, ReverseORC infers flexible layout constraint specifications by sampling UIs at different sizes and analyzing the differences between them. It can create specifications that replicate even some non-standard layout managers with complex dynamic layout behaviours. We demonstrate that ReverseORC works across different platforms with very different layout approaches, e.g., for GUIs as well as for the Web. Furthermore, it can be used to detect and fix problems in legacy UIs, extend UIs with enhanced layout behaviours, and support the creation of flexible UI layouts.Comment: CHI2021 Full Pape

Reflow: Automatically Improving Touch Interactions in Mobile Applications through Pixel-based Refinements

Touch is the primary way that users interact with smartphones. However, building mobile user interfaces where touch interactions work well for all users is a difficult problem, because users have different abilities and preferences. We propose a system, Reflow, which automatically applies small, personalized UI adaptations, called refinements -- to mobile app screens to improve touch efficiency. Reflow uses a pixel-based strategy to work with existing applications, and improves touch efficiency while minimally disrupting the design intent of the original application. Our system optimizes a UI by (i) extracting its layout from its screenshot, (ii) refining its layout, and (iii) re-rendering the UI to reflect these modifications. We conducted a user study with 10 participants and a heuristic evaluation with 6 experts and found that applications optimized by Reflow led to, on average, 9% faster selection time with minimal layout disruption. The results demonstrate that Reflow's refinements useful UI adaptations to improve touch interactions

Artificial Intelligence (AI) and User Experience (UX) design: A systematic literature review and future research agenda

PurposeThe aim of this article is to map the use of AI in the user experience (UX) design process. Disrupting the UX process by introducing novel digital tools such as Artificial Intelligence (AI) has the potential to improve efficiency and accuracy, while creating more innovative and creative solutions. Thus, understanding how AI can be leveraged for UX has important research and practical implications.Design/Methodology/ApproachThis article builds on a systematic literature review approach and aims to understand how AI is used in UX design today, as well as uncover some prominent themes for future research. Through a process of selection and filtering, 46 research articles are analysed, with findings synthesized based on a user-centred design and development process.FindingsOur analysis shows how AI is leveraged in the UX design process at different key areas. Namely, these include understanding the context of use, uncovering user requirements, aiding solution design, and evaluating design, and for assisting development of solutions. We also highlight the ways in which AI is changing the UX design process through illustrative examples.Originality/valueWhile there is increased interest in the use of AI in organizations, there is still limited work on how AI can be introduced into processes that depend heavily on human creativity and input. Thus, we show the ways in which AI can enhance such activities and assume tasks that have been typically performed by humans

Automating Software Development for Mobile Computing Platforms

Mobile devices such as smartphones and tablets have become ubiquitous in today\u27s computing landscape. These devices have ushered in entirely new populations of users, and mobile operating systems are now outpacing more traditional desktop systems in terms of market share. The applications that run on these mobile devices (often referred to as apps ) have become a primary means of computing for millions of users and, as such, have garnered immense developer interest. These apps allow for unique, personal software experiences through touch-based UIs and a complex assortment of sensors. However, designing and implementing high quality mobile apps can be a difficult process. This is primarily due to challenges unique to mobile development including change-prone APIs and platform fragmentation, just to name a few. in this dissertation we develop techniques that aid developers in overcoming these challenges by automating and improving current software design and testing practices for mobile apps. More specifically, we first introduce a technique, called Gvt, that improves the quality of graphical user interfaces (GUIs) for mobile apps by automatically detecting instances where a GUI was not implemented to its intended specifications. Gvt does this by constructing hierarchal models of mobile GUIs from metadata associated with both graphical mock-ups (i.e., created by designers using photo-editing software) and running instances of the GUI from the corresponding implementation. Second, we develop an approach that completely automates prototyping of GUIs for mobile apps. This approach, called ReDraw, is able to transform an image of a mobile app GUI into runnable code by detecting discrete GUI-components using computer vision techniques, classifying these components into proper functional categories (e.g., button, dropdown menu) using a Convolutional Neural Network (CNN), and assembling these components into realistic code. Finally, we design a novel approach for automated testing of mobile apps, called CrashScope, that explores a given android app using systematic input generation with the intrinsic goal of triggering crashes. The GUI-based input generation engine is driven by a combination of static and dynamic analyses that create a model of an app\u27s GUI and targets common, empirically derived root causes of crashes in android apps. We illustrate that the techniques presented in this dissertation represent significant advancements in mobile development processes through a series of empirical investigations, user studies, and industrial case studies that demonstrate the effectiveness of these approaches and the benefit they provide developers

A methodology for aggregate assembly modelling and planning

The introduction of Concurrent Engineering highlights the need for a link between the early stages of product design and assembly planning. This thesis presents aggregate assembly process planning as a novel methodology to provide this link. The theory behind the research is to bring all aspects of product development together to consider assembly planning at the conceptual stage of design. Decisions taken during the early design stage not only have the greatest influence on production times and costs, but also should ensure that a design is easy to manufacture and assemble. An automated computer-based system has been developed to implement the methodology. The system generates aggregate assembly process plans which give details of feasible sequences, assembly process times and costs, resource requirements, and factory loadings. The Aggregate Assembly Modelling and Planning (AAMP) system employs object-oriented modelling techniques to represent designs, process planning knowledge, and assembly resources. The minimum information requirements have been identified, and a product model encompassing this data has been developed. An innovative factor of this thesis is to employ Assembly Feature Connections (AFCs) within the product model to represent assembly connectivity. Detailed generic assembly process models, functioning with limited design data, are used to calculate assembly criteria. The introduction of a detailed resource model to represent assembly facilities enables the system to calculate accurate assembly times, dependent on which resources are used within a factory, or even which factory is employed. A new algorithm uses the structure of the product model, process constraints and assembly rules to efficiently generate accurate assembly sequences. Another new algorithm loads the assembly operations onto workstations, ensuring that the capability and capacity are available. The aggregate assembly process planning functionality has been tested using products from industry, and has yielded accurate results that prove to be both technically feasible and realistic. Industrial response has been extremely favourable. Specific comments on the usefulness and simplicity of such a comprehensive system gives encouragement to the concept that aggregate assembly process planning provides the required link between the early stages of product design and assembly planning

UML Assisted Visual Debugging for Distributed Systems

The DOD is developing a Joint Battlespace Infosphere, linking a large number of data sources and user applications. To assist in this process, debugging and analysis tools are required. Software debugging is an extremely difficult cognitive process requiring comprehension of the overall application behavior, along with detailed understanding of specific application components. This is further complicated with distributed systems by the addition of other programs, their large size and synchronization issues. Typical debuggers provide inadequate support for this process, focusing primarily on the details accessible through source code. To overcome this deficiency, this research links the dynamic program execution state to a Unified Modeling Language (UML) class diagram that is reverse-engineered from data accessed within the Java Platform Debug Architecture. This research uses focus + context, graph layout, and color encoding techniques to enhance the standard UML diagram. These techniques organize and present objects and events in a manner that facilitates analysis of system behavior. High-level abstractions commonly used in system design support debugging while maintaining access to low-level details with an interactive display. The user is also able to monitor the control flow through highlighting of the relevant object and method in the display

Temporal meta-model framework for Enterprise Information Systems (EIS) development

This thesis has developed a Temporal Meta-Model Framework for semi-automated Enterprise System Development, which can help drastically reduce the time and cost to develop, deploy and maintain Enterprise Information Systems throughout their lifecycle. It proposes that the analysis and requirements gathering can also perform the bulk of the design phase, stored and available in a suitable model which would then be capable of automated execution with the availability of a set of specific runtime components

Complex VLSI Feature Comparison for Commercial Microelectronics Verification

Shortcomings in IC verification make for glaring vulnerabilities in the form of hardware backdoors, or extraneous operation modes that allow unauthorized, undetected access. The DARPA TRUST program addressed the need for verification of untrusted circuits using industry-standard and custom software. The process developed under TRUST and implemented at the AFRL Mixed Signal Design Center has not been tested using real-world circuits outside of the designated TRUST test cases. This research demonstrates the potential of applying software designed for TRUST test articles on microchips from questionable sources. A specific process is developed for both transistor-level library cell verification and gate-level circuit verification. The relative effectiveness and scalability of the process are assessed