Reconocimiento de widgets automático para aplicaciones Java/Swing en TESTAR

[ES] El grupo de Software Testing and Quality (STaQ) del centro de investigación PROS de la Universidad Politécnica de Valencia (UPV) ha desarrollado una herramienta, denominada TESTAR (www.testar.org) para el testing automatizada a nivel de Interfaz de Usuario (IU). TESTAR genera y ejecuta casos de prueba automáticamente basado en un modelo de árbol derivado automáticamente desde la IU de la aplicación bajo prueba. Este árbol es construido con ayuda del API de accesibilidad del sistema operativo que ayuda a reconocer todos los elementos gráficos de la IU (widgets). La herramienta no es del tipo capturar/reproducir ni usa reconocimientos de formas. Las empresas que han desplegado la herramienta son muy optimistas y lo ven como un cambio de paradigma del testing a largo plazo, pues tiene potencial para resolver muchos de los problemas de las herramientas existentes. En este proyecto el objetivo es extender e implementar la capacidad de reconocimiento de widgets (elementos gráficos de Interfaz de Usuario) de la herramienta TESTAR para aplicaciones Java en sistemas operativos Microsoft Windows. Esta herramienta facilita el testeo automático de aplicaciones software desde su Interfaz de Usuario (IU), pero actualmente se ha identificado una limitación en el reconocimiento de widgets cuando la tecnología Java utilizada es Swing, funcionando sin problemas para AWT y SWT. TESTAR se sustenta en tecnologías de accesibilidad que exponen los widgets de las aplicaciones software. El carácter “lightweight” en la implementación de Swing provoca que los widgets Swing no sean identificados por tecnologías de accesiblidad. Para apoyar la accesbilidad de aplicaciones Swing existe un puente denominado Java Access Bridge que expone el API de accesibilidad de Java en una librería dinámica (DLL) de Windows: http://www.oracle.com/technetwork/articles/javase/index-jsp-136191.html Por tanto, la labor del proyecto será: • Estudiar el puente Java Access Bridge para facilitar el reconocimiento de widgets de manera automática en aplicaciones Java/Swing existentes. • Implementar un plug-in para TESTAR, que dote a la herramienta de reconocimiento de widgets en Swing que permitirá automatizar la prueba de software bajo tecnología Swing, además del soporte actual a las tecnologías AWT y SWT. • Evaluar la capacidad de TESTAR en Java/Swing con dos casos prácticos con aplicaciones en empresas. (En este momento EVERIS y Clearone son empresas que han mostrado interés en tener disponible esta capacidad en TESTAR). • Documentar los resultados[EN] The Software Testing and Quality (STAQ) group of the PROS research center at the Polytechnic University of Valencia (UPV) has developed a tool, called TESTAR (www.testar.org) for automated testing at the user interface level (UI) . TESTAR generates and executes test cases automatically based on a tree model automatically derived from the UI of the application under test. This tree is built using the Accessibility API of the operating system that helps to recognize all graphical UI elements (widgets). The tool is not capture / replay nor uses image recognition. Companies that have deployed the tool are very positive and see it as a paradigm shift for testing. They believe that TESTAR has the potential to solve many problems with existing tools. In this project the aim is to extend and implement the recognizability of widgets (graphical elements of the User Interface) of the TESTAR tool for Java applications in Microsoft Windows operating systems. TESTAR has a limitation regarding the recognition of widgets when the Java technology Swing is used (it runs smoothly for AWT and SWT). TESTAR is based on accessibility technologies that expose widgets of the software application under test. The "lightweight" character of Swing makes that some Swing elements are not correctly identified by accessibility technologies . To support the application accesbilidad for Swing there is a bridge called Java Access Bridge exposes the Java Accessibility API in a dynamic link library (DLL) for Windows: http://www.oracle.com/technetwork/articles/javase/index-jsp-136191.html Therefore, the work of the project will be: • Study the Java Access Bridge bridge to facilitate recognition of widgets automatically in Java / Swing applications. • Implement a plug-in for TESTAR to enrich the tool with recognition of Swing widgets, in addition to the current support for AWT and SWT technologies. • Assess the capacity of TESTAR in Java / Swing with two case studies with industrial applications. (Currently EVERIS and Clearone are companies that have shown interest in having this capacity available in TESTAR). • Document the resultsPastor Ricos, F. (2017). Reconocimiento de widgets automático para aplicaciones Java/Swing en TESTAR. http://hdl.handle.net/10251/88838.TFG

DECODER - DEveloper COmpanion for Documented and annotatEd code Reference

This work has been developed with the financial support of the European Union's Horizon 2020 research and innovation programme under grant agreement No. 824231Gil Pascual, M.; Pastor-Ricós, F.; Torres Bosch, MV.; Vos, TE. (2020). DECODER - DEveloper COmpanion for Documented and annotatEd code Reference. Springer. 643-644. http://hdl.handle.net/10251/178910S64364

Evaluating TESTAR's effectiveness through code coverage

[EN] Testing is of paramount importance in assuring the quality of software products. Nevertheless, it is not easy to judge which techniques or tools are the most effective. A commonly used surrogate metric to evaluate the effectiveness of testing tools is code coverage, which has been widely used for unit and integration testing. However, for GUI testing approaches, this metric has not been sufficiently investigated. To fill this gap, we run experiments with the TESTAR tool, a scriptless testing tool that automatically generates test cases at the Graphical User Interface (GUI) level. In the experiment, we analyze and compare the obtained code coverage when using four different action selection mechanisms (ASMs) in TESTAR that are used to test three SUTsThis research has been funded by the following projects: H2020 EU project DECODER (www.decoder-project.eu), H2020 EU project iv4XR (www.iv4xrproject.eu) and ITEA project IVVES (www.ivves.eu).Van Der Brugge, A.; Pastor-Ricós, F.; Aho, P.; Marín, B.; Vos, TE. (2021). Evaluating TESTAR's effectiveness through code coverage. SISTEDES. 1-14. http://hdl.handle.net/10251/178270S11

Applying Scriptless Test Automation on Web Applications from the Financial Sector

[EN] This industry showcase presents experiences on application of testar, an open source tool for scriptless testing through graphical user interface (GUI), to the web applications of Kuveyt T¿urk Participation Bank in Turkey. Kuveyt T¿urk Bank uses Selenium and Appium for regression testing of mobile and internet banking, but the maintenance cost of the test scripts is increasing day by day. Therefore, scriptless GUI testing with testar was evaluated. To provide better support for testing web-based applications, testar was extended with Selenium WebDriver integration, JavaScript support, and other new features. Results show that testar detects GUI elements much better after the improvements, and it was able to find 2 relevant errors that were not identified by existing scripted test cases.This work has been partially funded by ITEA3 TESTOMAT Project, ITEA3 IVVES project7 and EU H2020 DECODER projecAho, P.; Buijs, G.; Akin, A.; Senturk, S.; Pastor-Ricós, F.; De Gouw, S.; Vos, TE. (2021). Applying Scriptless Test Automation on Web Applications from the Financial Sector. SISTEDES. 1-4. http://hdl.handle.net/10251/178254S1

Extended data for Grammar-Based Action Selection Rules for Scriptless Testing

<p>Extended data for the paper Grammar-Based Action Selection Rules for Scriptless Testing</p&gt

Using GUI Change Detection for Delta Testing

Current software development processes in the industry are designed to respond to rapid modification or changes in software features. Delta testing is a technique used to check that the identified changes are deliberate and neither compromise existing functionality nor result in introducing new defects. This paper proposes a technique for delta testing at the Graphical User Interface (GUI) level. We employ scriptless testing and state-model inference to automatically detect and visualize GUI changes between different versions of the same application. Our proposed offline change detection algorithm compares two existing GUI state models to detect changes. We present a proof of concept experiment with the open-source application Notepad++, which allows automatic inference and highlights GUI changes. The results show that our technique is a valuable amplification of scriptless testing tools for delta testing.</p

Distributed state model inference for scriptless GUI testing

State model inference of software applications through the Graphical User Interface (GUI) is a technique that identifies GUI states and transitions, and maps them into a model. Scriptless GUI testing tools can benefit substantially from the availability of these state models, for example, to improve the exploration, or have sophisticated test oracles. However, inferring models for large systems requires a long execution time. Our goal is to improve the speed of the state model inference process. To achieve this goal, this paper presents a distributed state model inference approach with an open source scriptless GUI testing tool. Moreover, in order to be able to infer a suitable model, we design a set of strategies to deal with abstraction challenges and to distinguish GUI states and transitions in the model. To validate it, we conduct an experiment with two open source web applications that have been tested with the distributed architecture using one to six Docker containers sharing the same state model. With the obtained results, we can conclude that it is feasible to infer a model with a distributed approach and that using the distributed approach reduces the time required for inferring a state model

Testar:scriptless testing through graphical user interface

Covering all the possible paths of the graphical user interface (GUI) with test scripts would take too much effort and result in serious maintenance issues. We propose complementing scripted testing with scriptless test automation using the open-source testar tool. This paper gives a comprehensive overview of testar and its latest extensions together with the ongoing and future research. With this paper, we hope we can help and encourage other researchers to use testar for their GUI testing-related research and pave the way for an international research agenda in GUI testing built upon stable and open-source infrastructure

State Model Inference Through the GUI Using Run-Time Test Generation

Software testing is an important part of engineering trustworthy information systems. End-to-end testing through Graphical User Interface (GUI) can be done manually, but it is a very time consuming and costly process. There are tools to capture or manually define scripts for automating regression testing through a GUI, but the main challenge is the high maintenance cost of the scripts when the GUI changes. In addition, GUIs tend to have a large state space, so creating scripts to cover all the possible paths and defining test oracles to check all the elements of all the states would be an enormous effort. This paper presents an approach to automatically explore a GUI while inferring state models that are used for action selection in run-time GUI test generation, implemented as an extension to the open source TESTAR tool. As an initial validation, we experiment on the impact of using various state abstraction mechanisms on the model inference and the performance of the implemented action selection algorithm based on the inferred model. Later, we analyse the challenges and provide future research directions on model inference and scriptless GUI testing

State Model Inference Through the GUI Using Run-Time Test Generation

Software testing is an important part of engineering trustworthy information systems. End-to-end testing through Graphical User Interface (GUI) can be done manually, but it is a very time consuming and costly process. There are tools to capture or manually define scripts for automating regression testing through a GUI, but the main challenge is the high maintenance cost of the scripts when the GUI changes. In addition, GUIs tend to have a large state space, so creating scripts to cover all the possible paths and defining test oracles to check all the elements of all the states would be an enormous effort. This paper presents an approach to automatically explore a GUI while inferring state models that are used for action selection in run-time GUI test generation, implemented as an extension to the open source TESTAR tool. As an initial validation, we experiment on the impact of using various state abstraction mechanisms on the model inference and the performance of the implemented action selection algorithm based on the inferred model. Later, we analyse the challenges and provide future research directions on model inference and scriptless GUI testing