Improvement and Evaluation of Data Consistency Metric CIL for Software Engineering Data Sets

Software data sets derived from actual software products and their development processes are widely used for project planning, management, quality assurance and process improvement, etc. Although it is demonstrated that certain data sets are not fit for these purposes, the data quality of data sets is often not assessed before using them. The principal reason for this is that there are not many metrics quantifying fitness of software development data. In that respect, this study makes an effort to fill in the void in literature by devising a new and efficient assessment method of data quality. To that end, we start as a reference from Case Inconsistency Level (CIL), which counts the number of inconsistent project pairs in a data set to evaluate its consistency. Based on a follow-up evaluation with a large sample set, we depict that CIL is not effective in evaluating the quality of certain data sets. By studying the problems associated with CIL and eliminating them, we propose an improved metric called Similar Case Inconsistency Level (SCIL). Our empirical evaluation with 54 data samples derived from six large project data sets shows that SCIL can distinguish between consistent and inconsistent data sets, and that prediction models for software development effort and productivity built from consistent data sets achieve indeed a relatively higher accuracy

Artificial Neural Network Based Audio Reinforcement for Computer Assisted Rote Learning

The dual-channel assumption of the cognitive theory of multimedia learning suggests that importing a large amount of information through a single (visual or audio) channel overloads that channel, causing partial loss of information, while importing it simultaneously through multiple channels relieves the burden on them and leads to the registration of a larger amount of information. In light of such knowledge, this study investigates the possibility of reinforcing visual stimuli with audio for supporting e-learners in memorization tasks. Specifically, we consider three kinds of learning material and two kinds of audio stimuli and partially reinforce each kind of material with each kind of stimuli in an arbitrary way. In a series of experiments, we determine the particular type of audio, which offers the highest improvement for each kind of material. Our work stands out as being the first study investigating the differences in memory performance in relation to different combinations of learning content and stimulus. Our key findings from the experiments are: (i) E-learning is more effective in refreshing memory rather than studying from scratch, (ii) Non-informative audio is more suited to verbal content, whereas informative audio is better for numerical content, (iii) Constant audio triggering degrades learning performance and thus audio triggering should be handled with care. Based on these findings, we develop an ANN-based estimator to determine the proper moment for triggering audio (i.e. when memory performance is estimated to be declining) and carry out follow-up experiments for testing the integrated framework. Our contributions involve (i) determination of the most effective audio for each content type, (ii) estimation of memory deterioration based on learners' interaction logs, and (iii) the proposal of improvement of memory registration through auditory reinforcement. We believe that such findings constitute encouraging evidence the memory registration of e-learners can be enhanced with content-aware audio incorporation

Empirical Evaluation of Mimic Software Project Data Sets for Software Effort Estimation

To conduct empirical research on industry software development, it is necessary to obtain data of real software projects from industry. However, only few such industry data sets are publicly available; and unfortunately, most of them are very old. In addition, most of today's software companies cannot make their data open, because software development involves many stakeholders, and thus, its data confidentiality must be strongly preserved. To that end, this study proposes a method for artificially generating a “mimic” software project data set, whose characteristics (such as average, standard deviation and correlation coefficients) are very similar to a given confidential data set. Instead of using the original (confidential) data set, researchers are expected to use the mimic data set to produce similar results as the original data set. The proposed method uses the Box-Muller transform for generating normally distributed random numbers; and exponential transformation and number reordering for data mimicry. To evaluate the efficacy of the proposed method, effort estimation is considered as potential application domain for employing mimic data. Estimation models are built from 8 reference data sets and their concerning mimic data. Our experiments confirmed that models built from mimic data sets show similar effort estimation performance as the models built from original data sets, which indicate the capability of the proposed method in generating representative samples

An Algorithm for Automatic Collation of Vocabulary Decks Based on Word Frequency

This study focuses on computer based foreign language vocabulary learning systems. Our objective is to automatically build vocabulary decks with desired levels of relative difficulty relations. To realize this goal, we exploit the fact that word frequency is a good indicator of vocabulary difficulty. Subsequently, for composing the decks, we pose two requirements as uniformity and diversity. Namely, the difficulty level of the cards in the same deck needs to be uniform enough so that they can be grouped together and difficulty levels of the cards in different decks need to be diverse enough so that they can be grouped in different decks. To assess uniformity and diversity, we use rank-biserial correlation and propose an iterative algorithm, which helps in attaining desired levels of uniformity and diversity based on word frequency in daily use of language. In experiments, we employed a spaced repetition flashcard software and presented users various decks built with the proposed algorithm, which contain cards from different content types. From users' activity logs, we derived several behavioral variables and examined the polyserial correlation between these variables and difficulty levels across different word classes. This analysis confirmed that the decks compiled with the proposed algorithm induce an effect on behavioral variables in line with the expectations. In addition, a series of experiments with decks involving varying content types confirmed that this relation is independent of word class

Estimating Level of Engagement from Ocular Landmarks

E-learning offers many advantages like being economical, flexible and customizable, but also has challenging aspects such as lack of – social-interaction, which results in contemplation and sense of remoteness. To overcome these and sustain learners’ motivation, various stimuli can be incorporated. Nevertheless, such adjustments initially require an assessment of engagement level. In this respect, we propose estimating engagement level from facial landmarks exploiting the facts that (i) perceptual decoupling is promoted by blinking during mentally demanding tasks; (ii) eye strain increases blinking rate, which also scales with task disengagement; (iii) eye aspect ratio is in close connection with attentional state and (iv) users’ head position is correlated with their level of involvement. Building empirical models of these actions, we devise a probabilistic estimation framework. Our results indicate that high and low levels of engagement are identified with considerable accuracy, whereas medium levels are inherently more challenging, which is also confirmed by inter-rater agreement of expert coders

Evaluating the applicability of reliability prediction models between different software

Abstract The prediction of fault-prone modules in a large software system is an important part in software evolution. Since prediction models in past studies have been constructed and used for individual systems, it has not been practically investigated whether a prediction model based on one system can also predict fault-prone modules accurately in other systems. Our expectation is that if we could build a model applicable to different systems, it would be extremely useful for software companies because they do not need to invest manpower and time for gathering data to construct a new model for every system. In this study, we evaluated the applicability of prediction models between two software systems through two experiments. In the first experiment, a prediction model using 19 module metrics as predictor variables was constructed in each system and was applied to the opposite system mutually. The result showed predictors were too fit to the base data and could not accurately predict fault-prone modules in the different system. On the basis of this result, we focused on a set of predictors showing great effectiveness in every model; and, in consequent, we identified two metrics (Lines of Code and Maximum Nesting Level) as commonly effective predictors in all the models. In the second experiment, by constructing prediction models using only these two metrics, prediction performance were dramatically improved. This result suggests that the commonly effective model applicable to more than two systems can be constructed by focusing on commonly effective predictors

Probing Software Engineering Beliefs about System Testing Defects: Analyzing Data for Future Directions

Research findings are often expressed as short startling sentences or software engineering (SE) beliefs such as “about 80 percent of the defects come from 20 percent of the modules” and “peer reviews catch 60 percent of the defects” [2]. Such SE beliefs are particularly important in industry, as they are attention-getting, easily understandable, and thus practically useful. In this paper we examine the power of such SE beliefs to justify process improvement through empirical validation of selected beliefs related to the increase or decrease of defects in system testing. We explore four basic SE beliefs in data from two midsize embedded software development organizations in Japan, and based on this information, identify possible process improvement actions for each organization. Based on our study, even small and medium-sized enterprises (SMEs) can use this approach to find possible directions to improve their process, which will result in better products

Java Birthmarks--Detecting the Software Theft--

To detect the theft of Java class files efficiently, we propose a concept of Java birthmarks, which are unique and native characteristics of every class file. For a pair of class files p and q, if q has the same birthmark as p's, q is suspected as a copy of p. Ideally, the birthmarks should satisfy the following properties: (a) preservation - the birthmarks should be preserved even if the original class file is tampered with, and (b) distinction - independent class files must be distinguished by completely different birthmarks. Taking (a) and (b) into account, we propose four types of birthmarks for Java class files. To show the effectiveness of the proposed birthmarks, we conduct three experiments. In the first experiment, we demonstrate that the proposed birthmarks are sufficiently robust against automatic program transformation (93.3876% of the birthmarks were preserved). The second experiment shows that the proposed birthmarks successfully distinguish non-copied files in a practical Java application (97.8005% of given class files were distinguished). In the third experiment, we exploit different Java compilers to confirm that the proposed Java birthmarks are core characteristics independent of compiler-specific issues

Spin polarization gate device based on the chirality-induced spin selectivity and robust nonlocal spin polarization

Nonlocal spin polarization phenomena are thoroughly investigated in the devices made of chiral metallic single crystals of CrNb$_3$S$_6$ and NbSi$_2$ as well as of polycrystalline NbSi$_2$. We demonstrate that simultaneous injection of charge currents in the opposite ends of the device with the nonlocal setup induces the switching behavior of spin polarization in a controllable manner. Such a nonlocal spin polarization appears regardless of the difference in the materials and device dimensions, implying that the current injection in the nonlocal configuration splits spin-dependent chemical potentials throughout the chiral crystal even though the current is injected into only a part of the crystal. We show that the proposed model of the spin dependent chemical potentials explains the experimental data successfully. The nonlocal double-injection device may offer significant potential to control the spin polarization to large areas because of the nature of long-range nonlocal spin polarization in chiral materials.Comment: 8 pages, 8 figure