An Extended Study of the Correlation of Cognitive Complexity-related Code Measures

Several measures have been proposed to represent various characteristics of code, such as size, complexity, cohesion, coupling, etc. These measures are deemed interesting because the internal characteristics they measure (which are not interesting per se) are believed to be correlated with external software qualities (like reliability, maintainability, etc.) that are definitely interesting for developers or users. Although many measures have been proposed for software code, new measures are continuously proposed. However, before starting using a new measure, we would like to ascertain that it is actually useful and that it provides some improvement with respect to well established measures that have been in use for a long time and whose merits have been widely evaluated. In 2018, a new code measure, named “Cognitive Complexity” was proposed. According to the proposers, this measure should correlate to code understandability much better than traditional code measures, such as McCabe Complexity, for instance. However, hardly any experimentation proved whether the “Cognitive Complexity” measure is better than other measures or not. Actually, it was not even verified whether the new measure provides different knowledge concerning code with respect to traditional measures. In this paper, we aim at evaluating experimentally to what extent the new measure is correlated with traditional measures. To this end, we measured the code from a set of open-source Java projects and derived models of “Cognitive Complexity” based on the traditional code measures yielded by a state-of-the-art code measurement tool. We found that fairly accurate models of “Cognitive Complexity” can be obtained using just a few traditional code measures. In this sense, the “Cognitive Complexity” measure does not appear to provide additional knowledge with respect to previously proposed measures

Analyzing the Effects of Display Characteristics and Cognitive Variables on Performance Using Keystroke and Eye Movement Data

Information about how operators use their eyes while interacting with visual displays is often an overlooked aspect of human-computer interaction. Such information is fundamental to assessing the quality of software interfaces and understanding the cognitive processes that underlie operator behavior. Other research evaluating information displays evolved from using reaction time and subjective data as dependent variables to using oculometric measures. In the current research conventional performance measures are coupled with oculometric measures to evaluate the influence display characteristics and cognitive variables have on performance. Twelve subjects used a software program to complete a series of specified tasks. Subjects were asked to search for 36 item from the database in a serial manner. Both keystroke and oculometric data were recorded while the subjects used the software database. Four dependent variables were derived from this data: task time, error rate, dwell time and dwell frequency. The four independent variables were information density, display layout, task complexity, and experience. Out of the four independent variables used in the current research, task complexity, a cognitive variable, clearly had the largest effect on both the time-based measures of performance and the oculometric measures of performance. Task complexity yielded a main effect in the task time data, the error rate data, the dwell time data and the dwell frequency data. Increases in task complexity yielded increases in task time, error rate, dwell time and dwell frequency. The results also showed that local information density had an effect on task time but only when overall density of the software interface was higher. While it was found that information density had a consistent effect on the frequency of dwells these results support other research that shows information density has a limited effect on performance. The display layout variable also had a limited influence on both performance and oculometric measures

Metrics and Tools to Guide Design of Graphical User Interfaces

User interface design metrics assist developers evaluate interface designs in early phase before delivering the software to end users. This dissertation presents a metric-based tool called GUIEvaluator for evaluating the complexity of the user interface based on its structure. The metrics-model consists of five modified structural measures of interface complexity: Alignment, grouping, size, density, and balance. The results of GUIEvaluator are discussed in comparison with the subjective evaluations of interface layouts and the existing complexity metrics-models. To extend this metrics-model, the Screen-Layout Cohesion (SLC) metric has been proposed. This metric is used to evaluate the usability of user interfaces. The SLC metric has been developed based on Aesthetic, structural, and semantic aspects of GUIs. To provide the SLC calculation, a complementary tool has been developed, which is called GUIExaminer. This dissertation demonstrates the potential of incorporating automated complexity and cohesion metrics into the user interface design process. The findings show that a strong positive correlation between the subjective evaluation and both the GUIEvaluator and GUIExaminer, at a significance level 0.05. Moreover, the findings provide evidence of the effectiveness of the GUIEvaluator and GUIExaminer to predict the best user interface design among a set of alternative user interfaces. In addition, the findings show that the GUIEvaluator and GUIExaminer can measure some usability aspects of a given user interface. However, the metrics validation proves the usefulness of GUIEvaluator and GUIExaminer for evaluating user interface designs

Evaluation Criteria for Object-oriented Metrics

In this paper an evaluation model for object-oriented (OO) metrics is proposed. We have evaluated the existing evaluation criteria for OO metrics, and based on the observations, a model is proposed which tries to cover most of the features for the evaluation of OO metrics. The model is validated by applying it to existing OO metrics. In contrast to the other existing criteria, the proposed model is simple in implementation and includes the practical and important aspects of evaluation; hence it suitable to evaluate and validate any OO complexity metric

A systematic methodology to assess the impact of human factors in ship design

Evaluating ship layout for human factors (HF) issues using simulation software such as maritimeEXODUS can be a long and complex process. The analysis requires the identification of relevant evaluation scenarios; encompassing evacuation and normal operations; the development of appropriate measures which can be used to gauge the performance of crew and vessel and finally; the interpretation of considerable simulation data. Currently, the only agreed guidelines for evaluating HFs performance of ship design relate to evacuation and so conclusions drawn concerning the overall suitability of a ship design by one naval architect can be quite different from those of another. The complexity of the task grows as the size and complexity of the vessel increases and as the number and type of evaluation scenarios considered increases. Equally, it can be extremely difficult for fleet operators to set HFs design objectives for new vessel concepts. The challenge for naval architects is to develop a procedure that allows both accurate and rapid assessment of HFs issues associated with vessel layout and crew operating procedures. In this paper we present a systematic and transparent methodology for assessing the HF performance of ship design which is both discriminating and diagnostic. The methodology is demonstrated using two variants of a hypothetical naval ship

Measuring the Pro-Activity of Software Agents

Despite having well-defined characteristics, software agents do not have a developed set of measures defining their quality. Attempts at evaluating software agent quality have focused on some agent aspects, like the development process, whereas others focusing on the agent as a software product have basically adopted measures associated with other software paradigms, like procedural and object-oriented concepts. Here we propose a set of measures for evaluating software agent pro-activity, the software agent's goal-driven behavioral ability to take the initiative and satisfy its goal

An Approach for the Empirical Validation of Software Complexity Measures

Software metrics are widely accepted tools to control and assure software quality. A large number of software metrics with a variety of content can be found in the literature; however most of them are not adopted in industry as they are seen as irrelevant to needs, as they are unsupported, and the major reason behind this is due to improper empirical validation. This paper tries to identify possible root causes for the improper empirical validation of the software metrics. A practical model for the empirical validation of software metrics is proposed along with root causes. The model is validated by applying it to recently proposed and well known metrics

Applicability of Weyuker’s Properties on OO Metrics: Some Misunderstandings

Weyuker’s properties have been suggested as a guiding tool in identification of a good and comprehensive complexity measure by several researchers. Weyuker proposed nine properties to evaluate complexity measure for traditional programming. However, they are extensively used for evaluating object-oriented (OO) metrics, although the object-oriented features are entirely different in nature. In this paper, two recently reported OO metrics were evaluated and, based on it; the usefulness and relevance of these properties for evaluation purpose for object-oriented systems is discussed

A Complexity Measure Based on Cognitive Weights

Cognitive Informatics plays an important role in understanding the fundamental characteristics of software. This paper proposes a model of the fundamental characteristics of software, complexity in terms of cognitive weights of basic control structures. Cognitive weights are degree of difficulty or relative time and effort required for comprehending a given piece of software, which satisfy the definition of complexity. An attempt has also been made to prove the robustness of proposed complexity measure by comparing it with the other measures based on cognitive informatics