SIMPLIFIED READABILITY METRICS

This paper describes a new approach to measuring the complexity of software systems with considering their readability. Readability Metrics were first proposed by Chung and Yung 181 in 1990. Software industry uses software metrics to measure the complexity of software systems for software cost estimation, software development control, software assurance, software testing, and software maintenance [3], [71, [9], 151, [18]. Most of the software metrics measure the software complexity by one or more of the software attributes. We usually class@ the software attributes that software metrics use for measuring complexity into three categories: size, control flow, and data flow [5], f71. All the three categories concern with the physical activities of software development. Readability Metrics have been outstanding among the existing software complexity metrics for taking nonphysical software attributes, like readability, into considerations [8]. The applications of Readability Metrics are good in indicating the additional efforts required for less readable software systems, and help in keeping the software systems maintainable. However, the numerous metrics and the complicated formulas in the family usually make it tedious to apply Readability Metrics to large scale software systems. In this paper, we propose a simplified approach to Readability Metrics. We reduce the number of required measures and keep the considerations on software readability. We introduce our Readability model in a more formal way. The Readability Metrics preprocesses algorithm is developed with compilers front-end techniques. The experiment results show that this simplified approach has good predictive power in measuring software complexity with software readability, in addition to its ease of applying. The applications of Readability Metrics indicate the readability of software systems and help in keeping the source code readable and maintainable.Information Systems Working Papers Serie

Features indicating readability in Swedish text

Studies have shown that modern methods of readability assessment, using automated linguistic analysis and machine learning (ML), is a viable road forward for readability classification and ranking. In this paper we present a study of different levels of analysis and a large number of features and how they affect an ML-system’s accuracy when it comes to readability assessment. We test a large number of features proposed for different languages (mainly English) and evaluate their usefulness for readability assessment for Swedish as well as comparing their performance to that of established metrics. We find that the best performing features are language models based on part-of-speech and dependency type

Readability as a basis for information security policy assessment

Most organisations now impose information security policies (ISPs) or 'conditions of use' agreements upon their employees. The need to ensure that employees are informed and aware of their obligations toward information security is apparent. Less apparent is the correlation between the provision of such policies and their compliance. In this paper, we report our research into the factors that determine the efficacy of information security policies (ISPs). Policies should comprise rules or principles that users can easily understand and follow. Presently, there is no ready mechanism for estimating the likely efficacy of such policies across an organisation. One factor that has a plausible impact upon the comprehensibility of policies is their readability. The present study investigates the effectiveness of applying readability metrics as an indicator of policy comprehensibility. Results from a preliminary study reveal variations in the comprehension test results attributable to the difficulty of the examined policies. The pilot study shows some correlation between the software readability formula results and human comprehension test results and supports our view that readability has an impact upon understanding ISPs. These findings have important implications for users’ compliance with information security policies and suggest that the application of suitably selected readability metrics may allow policy designers to evaluate their draft policies for ease of comprehension prior to policy release. Indeed, there may be grounds for a readability compliance test that future ISPs must satisfy

Better Summarization Evaluation with Word Embeddings for ROUGE

ROUGE is a widely adopted, automatic evaluation measure for text summarization. While it has been shown to correlate well with human judgements, it is biased towards surface lexical similarities. This makes it unsuitable for the evaluation of abstractive summarization, or summaries with substantial paraphrasing. We study the effectiveness of word embeddings to overcome this disadvantage of ROUGE. Specifically, instead of measuring lexical overlaps, word embeddings are used to compute the semantic similarity of the words used in summaries instead. Our experimental results show that our proposal is able to achieve better correlations with human judgements when measured with the Spearman and Kendall rank coefficients.Comment: Pre-print - To appear in proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP

BLEU is Not Suitable for the Evaluation of Text Simplification

BLEU is widely considered to be an informative metric for text-to-text generation, including Text Simplification (TS). TS includes both lexical and structural aspects. In this paper we show that BLEU is not suitable for the evaluation of sentence splitting, the major structural simplification operation. We manually compiled a sentence splitting gold standard corpus containing multiple structural paraphrases, and performed a correlation analysis with human judgments. We find low or no correlation between BLEU and the grammaticality and meaning preservation parameters where sentence splitting is involved. Moreover, BLEU often negatively correlates with simplicity, essentially penalizing simpler sentences.Comment: Accepted to EMNLP 2018 (Short papers

Readability of scientific papers : data analysis of RCC

A legibilidade de uma revista de divulgação científica é um problema central das comissões editoriais, dos revisores e em especial dos leitores. O vocabulário de cada artigo relaciona-se em larga medida com as palavras-chave próprias de cada área científica, contudo a legibilidade também depende de outros factores. Neste artigo apresentam-se métricas de legibilidade que são função do comprimento das palavras e do comprimento das frases. Propõe-se que as métricas de legibilidade sejam balizadas por um limite superior e por um limite inferior. Finalmente, encontra-se uma relação entre as duas métricas, com base nos dados extraídos dos artigos publicados na RCC.The readability of a scientific review is a very important issue for the editorial committees, for the reviewers and especially for the readers. The vocabulary of each article is largely related to the keywords of the specific scientific area; however, the readability depends also on other factors. In this article we present two readability metrics that are based on the length of the words and on the length of the sentences. We propose that the readability metrics must be bounded by a superior limit and an inferior limit. Finally, the correlation between the two metrics is presented, given the data extracted from the articles published in the RCC.peerreviewe