EUDoptimizer: Assisting End Users in Composing IF-THEN Rules Through Optimization

Nowadays, several interfaces for end-user development (EUD) empower end users to jointly program the behavior of their smart devices and online services, typically through trigger-action rules. Despite their popularity, such interfaces often expose too much functionality and force the user to search among a large number of supported technologies disposed of confused grid menus. This paper contributes to the EUD with the aim of interactively assisting end users in composing IF-THEN rules with an optimizer in the loop. The goal, in particular, is to automatically redesign the layout of the EUD interfaces to facilitate the users in defining triggers and actions. For this purpose, we define a predictive model to characterize the composition of trigger-action rules on the basis of their final functionality, we adopt different optimization algorithms to explore the design space, and c) we present EUDoptimizer, the integration of our approach in IFTTT, one of the most popular EUD interfaces. We demonstrate that good layout solutions can be obtained in a reasonable amount of time. Furthermore, an empirical evaluation with 12 end users shows evidence that EUDoptimizer reduces the efforts needed to compose trigger-action rules

Moldable Items Packing Optimization

This research has led to the development of two mathematical models to optimize the problem of packing a hybrid mix of rigid and moldable items within a three-dimensional volume. These two developed packing models characterize moldable items from two perspectives: (1) when limited discrete configurations represent the moldable items and (2) when all continuous configurations are available to the model. This optimization scheme is a component of a lean effort that attempts to reduce the lead-time associated with the implementation of dynamic product modifications that imply packing changes. To test the developed models, they are applied to the dynamic packing changes of Meals, Ready-to-Eat (MREs) at two different levels: packing MRE food items in the menu bags and packing menu bags in the boxes. These models optimize the packing volume utilization and provide information for MRE assemblers, enabling them to preplan for packing changes in a short lead-time. The optimization results are validated by running the solutions multiple times to access the consistency of solutions. Autodesk Inventor helps visualize the solutions to communicate the optimized packing solutions with the MRE assemblers for training purposes

Optimization Approaches to Adaptive Menus

Graphical menus perform as vital components and offer essential controls in today’s graphical interface. However, few studies have been conducted to modelling the performance of a menu. Furthermore, menu optimization methods previously proposed have been largely concentrating on reshaping layout of the whole menu system. In order to model menu performance, this thesis extends the Search-Decision-Pointing model by introducing two additional factors, i.e. the cost function and semantic function. The cost function is a penalty function which decreases the user expertise regarding a menu layout according to the degree of modification done to the menu. The semantic function is a reward function which encourages items with strong relations be positioned close to each other. Centered on this menu performance model, several optimization methods have been implemented. Each method focuses on improving menu performance by applying distinctive strategies, such as increasing item size or reducing item pointing distance. Three test cases have been exercised to evaluate the optimization methods in a simulated software which displays graphical user interfaces and emulates the menu utilization of real users. The results of test cases reveal that the menu performance has been successfully improved in all test cases by the fundamental heuristic search algorithm. Moreover, other optimization methods have been able to further increase menu performance ranging from 3% to 8% depending on test cases. In addition, it is identified that increasing the size of an item offers surprisingly little benefit. Conversely, reducing item pointing distance has greatly improved menu performance. Moreover, positioning items by their semantic relations may also enhance group saliency. On the other hand, optimization methods may not always succeed in providing usable menus due to design constraints. Hence, menu performance optimization shall be carefully exercised by considering the entire graphical user interface

Experimental user interface design toolkit for interaction research (IDTR).

The research reported and discussed in this thesis represents a novel approach to User Interface evaluation and optimisation through cognitive modelling. This is achieved through the development and testing of a toolkit or platform titled Toolkit for Optimisation of Interface System Evolution (TOISE). The research is conducted in two main phases. In phase 1, the Adaptive Control of Thought Rational (ACT-R) cognitive architecture is used to design Simulated Users (SU) models. This allows models of user interaction to be tested on a specific User Interface (UI). In phase 2, an evolutionary algorithm is added and used to evolve and test an optimised solution to User Interface layout based on the original interface design. The thesis presents a technical background, followed by an overview of some applications in their respective fields. The core concepts behind TOISE are introduced through a discussion of the Adaptive Control of Thought “ Rational (ACT-R) architecture with a focus on the ACT-R models that are used to simulate users. The notion of adding a Genetic Algorithm optimiser is introduced and discussed in terms of the feasibility of using simulated users as the basis for automated evaluation to optimise usability. The design and implementation of TOISE is presented and discussed followed by a series of experiments that evaluate the TOISE system. While the research had to address and solve a large number of technical problems the resulting system does demonstrate potential as a platform for automated evaluation and optimisation of user interface layouts. The limitations of the system and the approach are discussed and further work is presented. It is concluded that the research is novel and shows considerable promise in terms of feasibility and potential for optimising layout for enhanced usability

An integrated network visualization framework towards metabolic engineering applications

Background Over the last years, several methods for the phenotype simulation of microorganisms, under specified genetic and environmental conditions have been proposed, in the context of Metabolic Engineering (ME). These methods provided insight on the functioning of microbial metabolism and played a key role in the design of genetic modifications that can lead to strains of industrial interest. On the other hand, in the context of Systems Biology research, biological network visualization has reinforced its role as a core tool in understanding biological processes. However, it has been scarcely used to foster ME related methods, in spite of the acknowledged potential. Results In this work, an open-source software that aims to fill the gap between ME and metabolic network visualization is proposed, in the form of a plugin to the OptFlux ME platform. The framework is based on an abstract layer, where the network is represented as a bipartite graph containing minimal information about the underlying entities and their desired relative placement. The framework provides input/output support for networks specified in standard formats, such as XGMML, SBGN or SBML, providing a connection to genome-scale metabolic models. An user-interface makes it possible to edit, manipulate and query nodes in the network, providing tools to visualize diverse effects, including visual filters and aspect changing (e.g. colors, shapes and sizes). These tools are particularly interesting for ME, since they allow overlaying phenotype simulation results or elementary flux modes over the networks. Conclusions The framework and its source code are freely available, together with documentation and other resources, being illustrated with well documented case studies.This work is partially funded by ERDF - European Regional Development Fund through the COMPETE Programme (operational programme for competitiveness) and by National Funds through the FCT (Portuguese Foundation for Science and Technology) within project ref. COMPETE FCOMP-01-0124-FEDER-015079 and the FCT Strategic Project PEst-OE/EQB/LA0023/2013. The work of PV is funded by PhD grant ref. SFRH/BDE/51442/2011

An Empirical Methodology for Engineering Human Systems Integration

The systems engineering technical processes are not sufficiently supported by methods and tools that quantitatively integrate human considerations into early system design. Because of this, engineers must often rely on qualitative judgments or delay critical decisions until late in the system lifecycle. Studies reveal that this is likely to result in cost, schedule, and performance consequences. This dissertation presents a methodology to improve the application of systems engineering technical processes for design. This methodology is mathematically rigorous, is grounded in relevant theory, and applies extant human subjects data to critical systems development challenges. The methodology is expressed in four methods that support early systems engineering activities: a requirements elicitation method, a function allocation method, an input device design method, and a display layout design method. These form a coherent approach to early system development. Each method is separately discussed and demonstrated using a prototypical system development program. In total, this original and significant work has a broad range of systems engineer applicability to improve the engineering of human systems integration

Layout Inference and Table Detection in Spreadsheet Documents

Spreadsheets have found wide use in many different domains and settings. They provide a broad range of both basic and advanced functionalities. In this way, they can support data collection, transformation, analysis, and reporting. Nevertheless, at the same time spreadsheets maintain a friendly and intuitive interface. Additionally, they entail no to very low cost. Well-known spreadsheet applications, such as OpenOffice, LibreOffice, Google Sheets, and Gnumeric, are free to use. Moreover, Microsoft Excel is widely available, with millions of users worldwide. Thus, spreadsheets are not only powerful tools, but also have a very low entrance barrier. Therefore, they have become very popular with novices and professionals alike. As a result, a large volume of valuable data resides in these documents. From spreadsheets, of particular interest are data coming in tabular form, since they provide concise, factual, and to a large extend structured information. One natural progression is to transfer tabular data from spreadsheets to databases. This would allow spreadsheets to become a direct source of data for existing or new business processes. It would be easier to digest them into data warehouses and to integrate them with other sources. Nevertheless, besides databases, there are other means to work with spreadsheet data. New paradigms, like NoDB, advocate querying directly from raw documents. Going one step further, spreadsheets together with other raw documents can be stored in a sophisticated centralized repository, i.e., a data lake. From then on they can serve (on-demand) various tasks and applications. All in all, by making spreadsheet data easily accessible, we can prevent information silos, i.e., valuable knowledge being isolated and scattered in multiple spreadsheet documents. Yet, there are considerable challenges to the automatic processing and understanding of these documents. After all, spreadsheets are designed primarily for human consumption, and as such, they favor customization and visual comprehension. Data are often intermingled with formatting, formulas, layout artifacts, and textual metadata, which carry domain-specific or even user-specific information (i.e., personal preferences). Multiple tables, with different layout and structure, can be found on the same sheet. Most importantly, the structure of the tables is not known, i.e., not explicitly given by the spreadsheet documents. Altogether, spreadsheets are better described as partially structured, with a significant degree of implicit information. In literature, the automatic understanding of spreadsheet data has only been scarcely investigated, often assuming just the same uniform table layout. However, due to the manifold possibilities to structure tabular data in spreadsheets, the assumption of a uniform layout either excludes a substantial number of tables from the extraction process or leads to inaccurate results. In this thesis, we primarily address two fundamental tasks that can lead to more accurate information extraction from spreadsheet documents. Namely, we propose intuitive and effective approaches for layout analysis and table detection in spreadsheets. Nevertheless, our overall solution is designed as a processing pipeline, where specialized steps build on top of each other to discover the tabular data. One of our main objectives is to eliminate most of the assumptions from related work. Instead, we target highly diverse sheet layouts, with one or multiple tables. On the same time, we foresee the presence of textual metadata and other non-tabular data in the sheet. Furthermore, we make use of sophisticated machine learning and optimization techniques. This brings flexibility to our approach, allowing it to work even with complex or malformed tables. Moreover, this intended flexibility makes our approaches transferable to new spreadsheet datasets. Thus, we are not bounded to specific domains or settings.:1 INTRODUCTION 1.1 Motivation 1.2 Contributions 1.3 Outline 2 FOUNDATIONS AND RELATED WORK 2.1 The Evolution of Spreadsheet Documents 2.1.1 Spreadsheet User Interface and Functionalities 2.1.2 Spreadsheet File Formats 2.1.3 Spreadsheets Are Partially-Structured 2.2 Analysis and Recognition in Electronic Documents 2.2.1 A General Overview of DAR 2.2.2 DAR in Spreadsheets 2.3 Spreadsheet Research Areas 2.3.1 Layout Inference and Table Recognition 2.3.2 Unifying Databases and Spreadsheets 2.3.3 Spreadsheet Software Engineering 2.3.4 Data Wrangling Approaches 3 AN EMPIRICAL STUDY OF SPREADSHEET DOCUMENTS 3.1 Available Corpora 3.2 Creating a Gold Standard Dataset 3.2.1 Initial Selection 3.2.2 Annotation Methodology 3.3 Dataset Analysis 3.3.1 Takeaways from Business Spreadsheets 3.3.2 Comparison Between Domains 3.4 Summary and Discussion 3.4.1 Datasets for Experimental Evaluation 3.4.2 A Processing Pipeline 4 LAYOUT ANALYSIS 4.1 A Method for Layout Analysis in Spreadsheets 4.2 Feature Extraction 4.2.1 Content Features 4.2.2 Style Features 4.2.3 Font Features 4.2.4 Formula and Reference Features 4.2.5 Spatial Features 4.2.6 Geometrical Features 4.3 Cell Classification 4.3.1 Classification Datasets 4.3.2 Classifiers and Assessment Methods 4.3.3 Optimum Under-Sampling 4.3.4 Feature Selection 4.3.5 Parameter Tuning 4.3.6 Classification Evaluation 4.4 Layout Regions 4.5 Summary and Discussions 5 CLASSIFICATION POST-PROCESSING 5.1 Dataset for Post-Processing 5.2 Pattern-Based Revisions 5.2.1 Misclassification Patterns 5.2.2 Relabeling Cells 5.2.3 Evaluating the Patterns 5.3 Region-Based Revisions 5.3.1 Standardization Procedure 5.3.2 Extracting Features from Regions 5.3.3 Identifying Misclassified Regions 5.3.4 Relabeling Misclassified Regions 5.4 Summary and Discussion 6 TABLE DETECTION 6.1 A Method for Table Detection in Spreadsheets 6.2 Preliminaries 6.2.1 Introducing a Graph Model 6.2.2 Graph Partitioning for Table Detection 6.2.3 Pre-Processing for Table Detection 6.3 Rule-Based Detection 6.3.1 Remove and Conquer 6.4 Genetic-Based Detection 6.4.1 Undirected Graph 6.4.2 Header Cluster 6.4.3 Quality Metrics 6.4.4 Objective Function 6.4.5 Weight Tuning 6.4.6 Genetic Search 6.5 Experimental Evaluation 6.5.1 Testing Datasets 6.5.2 Training Datasets 6.5.3 Tuning Rounds 6.5.4 Search and Assessment 6.5.5 Evaluation Results 6.6 Summary and Discussions 7 XLINDY: A RESEARCH PROTOTYPE 7.1 Interface and Functionalities 7.1.1 Front-end Walkthrough 7.2 Implementation Details 7.2.1 Interoperability 7.2.2 Efficient Reads 7.3 Information Extraction 7.4 Summary and Discussions 8 CONCLUSION 8.1 Summary of Contributions 8.2 Directions of Future Work BIBLIOGRAPHY LIST OF FIGURES LIST OF TABLES A ANALYSIS OF REDUCED SAMPLES B TABLE DETECTION WITH TIRS B.1 Tables in TIRS B.2 Pairing Fences with Data Regions B.3 Heuristics Framewor