Predicting the usability of mobile applications using AI tools: The rise of large user interface models, opportunities, and challenges

Abstract

This article proposes the so-called large user interface models (LUIMs) to enable the generation of user interfaces and prediction of usability using artificial intelligence in the context of mobile applications. To this end, we synergized an integrated framework for the effective testing of the usability of mobile applications following a selective review of the most influential models of mobile usability testing. Next, we identified and analysed 13 recent AI tools that generate user interfaces for mobile apps, and systematically tested these tools to identify their AI capabilities. Our striking findings demonstrate that current generative UI tools fail to address mobile usability attributes, such as efficiency, learnability, effectiveness, satisfaction, and memorability. Our large UI models' architecture proposes to leverage the capabilities of large language models, large vision models, and large code models to overcome the challenges of AI-driven UI/UX design and front-end implementations. This fascinating UI eco-system must be augmented with sufficient UI data and multi-sensory input regarding user behaviour to train the models. We anticipate LUIMs to create ample opportunities, like expedited frontend software development, enhanced personalised user experience, and wider accessibility of smart technologies. However, the research challenges hindering the UI generation and usability prediction of mobile apps include the seamless integration of complex generative AI models, semantic understanding of non-uniform visual designs, scarcity of UX datasets, and modelling of realistic user interactions.</p

    Similar works

    Full text

    thumbnail-image