An Empirical Investigation of Performance Overhead in Cross-Platform Mobile Development Frameworks

The heterogeneity of the leading mobile platforms in terms of user interfaces, user experience, programming language, and ecosystem have made cross-platform development frameworks popular. These aid the creation of mobile applications – apps – that can be executed across the target platforms (typically Android and iOS) with minimal to no platform-specific code. Due to the cost- and time-saving possibilities introduced through adopting such a framework, researchers and practitioners alike have taken an interest in the underlying technologies. Examining the body of knowledge, we, nonetheless, frequently encounter discussions on the drawbacks of these frameworks, especially with regard to the performance of the apps they generate. Motivated by the ongoing discourse and a lack of empirical evidence, we scrutinised the essential piece of the cross-platform frameworks: the bridge enabling cross-platform code to communicate with the underlying operating system and device hardware APIs. The study we present in the article benchmarks and measures the performance of this bridge to reveal its associated overhead in Android apps. The development of the artifacts for this experiment was conducted using five cross-platform development frameworks to generate Android apps, in addition to a baseline native Android app implementation. Our results indicate that – for Android apps – the use of cross-platform frameworks for the development of mobile apps may lead to decreased performance compared to the native development approach. Nevertheless, certain cross-platform frameworks can perform equally well or even better than native on certain metrics which highlights the importance of well-defined technical requirements and specifications for deliberate selection of a cross-platform framework or overall development approach.publishedVersio

An empirical investigation of performance overhead in cross-platform mobile development frameworks

The heterogeneity of the leading mobile platforms in terms of user interfaces, user experience, programming language, and ecosystem have made cross-platform development frameworks popular. These aid the creation of mobile applications – apps – that can be executed across the target platforms (typically Android and iOS) with minimal to no platform-specific code. Due to the cost- and time-saving possibilities introduced through adopting such a framework, researchers and practitioners alike have taken an interest in the underlying technologies. Examining the body of knowledge, we, nonetheless, frequently encounter discussions on the drawbacks of these frameworks, especially with regard to the performance of the apps they generate. Motivated by the ongoing discourse and a lack of empirical evidence, we scrutinised the essential piece of the cross-platform frameworks: the bridge enabling cross-platform code to communicate with the underlying operating system and device hardware APIs. The study we present in the article benchmarks and measures the performance of this bridge to reveal its associated overhead in Android apps. The development of the artifacts for this experiment was conducted using five cross-platform development frameworks to generate Android apps, in addition to a baseline native Android app implementation. Our results indicate that – for Android apps – the use of cross-platform frameworks for the development of mobile apps may lead to decreased performance compared to the native development approach. Nevertheless, certain cross-platform frameworks can perform equally well or even better than native on certain metrics which highlights the importance of well-defined technical requirements and specifications for deliberate selection of a cross-platform framework or overall development approach

On the Energy Footprint of Mobile Testing Frameworks

High energy consumption is a challenging issue that an ever increasing number of mobile applications face today. However, energy consumption is being tested in an ad hoc way, despite being an important non-functional requirement of an application. Such limitation becomes particularly disconcerting during software testing: on the one hand, developers do not really know how to measure energy; on the other hand, there is no knowledge as to what is the energy overhead imposed by the testing framework. In this paper, as we evaluate eight popular mobile UI automation frameworks, we have discovered that there are automation frameworks that increase energy consumption up to roughly 2200 percent. While limited in the interactions one can do, Espresso is the most energy efficient framework. However, depending on the needs of the tester, Appium, Monkeyrunner, or UIAutomator are good alternatives. In practice, results show that deciding which is the most suitable framework is vital. We provide a decision tree to help developers make an educated decision on which framework suits best their testing needs. Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Software Engineerin