Evaluating prediction systems in software project estimation

This is the Pre-print version of the Article - Copyright @ 2012 ElsevierContext: Software engineering has a problem in that when we empirically evaluate competing prediction systems we obtain conflicting results. Objective: To reduce the inconsistency amongst validation study results and provide a more formal foundation to interpret results with a particular focus on continuous prediction systems. Method: A new framework is proposed for evaluating competing prediction systems based upon (1) an unbiased statistic, Standardised Accuracy, (2) testing the result likelihood relative to the baseline technique of random ‘predictions’, that is guessing, and (3) calculation of effect sizes. Results: Previously published empirical evaluations of prediction systems are re-examined and the original conclusions shown to be unsafe. Additionally, even the strongest results are shown to have no more than a medium effect size relative to random guessing. Conclusions: Biased accuracy statistics such as MMRE are deprecated. By contrast this new empirical validation framework leads to meaningful results. Such steps will assist in performing future meta-analyses and in providing more robust and usable recommendations to practitioners.Martin Shepperd was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/H050329

3D-printed Acoustic Directional Couplers

Acoustic Directional Couplers permit separation of forward and reverse sound pressure waves. This separation opens the way to traceable precision acoustic reflection measurements. In order to span the audio frequency range, multiple couplers will be required, as each operates over a frequency range of slightly more than one octave. To reach 20kHz or above requires vary small, mechanically precise construction. We achieve this by 3D printing techniques. We manufactured two otherwise-identical couplers, one made with a powder-type 3D printer with photopolymer support structure, the other made with an ABS-filament thermoplastic-type 3D printer. We compare the measured acoustic performance of these two couplers. The wavelength of sound at 20 kHz is comparable to that encountered at a microwave frequency of 18 GHz. We expect to be able to fabricate couplers that reach 55 kHz where the wavelength is 6 mm, corresponding to a frequency of 50 GHz in the electromagnetic spectrum

How reliable are systematic reviews in empirical software engineering?

BACKGROUND – the systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE – the aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular we wish to investigate the consistency of process and the stability of outcomes. METHOD – we compare the results of two independent reviews under taken with a common research question. RESULTS – the two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS – in addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method

Signaling the Availability of Varying Quality of Reinforcement in a Token Economy

Token economies are commonly used to both increase and decrease behavior. Salient signaling may enhance the discriminability within a token economy system, thus increasing treatment effectiveness. Varying the quality of reinforcement to match the behavior of an individual may also impact responding. The current research compared the effectiveness of a no-signal condition, within-stimulus prompt condition, and an extra-stimulus prompt condition to signal the varying quality of reinforcement within a token economy system. Tokens were delivered to participants contingent on correct responding in a mastered skill task. In addition, Xs were delivered contingent on target behavior during session, with varying magnitudes of reinforcement being made available depending on the occurrences of target behavior. Results for Alex and Tommy indicate slightly higher rates of target behaviour observed in the control condition compared to the treatment conditions. However, results from the treatment conditions were undifferentiated. Results for Nathan were undifferentiated. Results suggest that varying qualities of reinforcement may be effective within a token economy system however, a prevailing method of signaling was not established

An In-depth Examination of Personality and Aggression Across Different Contexts

Acts of aggression are associated with a variety of negative outcomes. Accordingly, research has aimed to identify the personality traits that give rise to different forms of aggressive behaviour. Recent work has indicated that the factor of Honesty-Humility is associated with a variety of deviant behaviours, including aggression towards others; however, the nuances of these relationships require further investigation. This dissertation aimed to address several gaps in this literature through three main studies. In Study 1, we extended previous findings to younger populations, examining the associations between Honesty-Humility and aggression longitudinally in a large sample of children and youth. These findings demonstrated a bidirectional relationship between Honesty-Humility and aggression over time, such that low levels of Honesty-Humility resulted in higher levels of aggression and vice versa. In Study 2, we explored the specific facets of Honesty-Humility to determine if they differentially predict proactive and reactive aggression. Despite the theoretical link between Modesty and reactive aggression, we found limited support for this association, especially when controlling for proactive aggression. Overall, the Sincerity and Fairness facets were found to strongly predict both forms of aggression. Lastly, Study 3 explored the associations between Honesty-Humility and deviance, aggression, exploitation, and victimization in a workplace context. Robust relationships were found between Honesty-Humility and several deviant behaviours, further emphasizing the importance of this trait. In particular, when provided with the opportunity to aggress, individuals low in Honesty-Humility were more likely to do so, regardless of their level of power in the situation. Collectively, these findings indicate that Honesty-Humility is the strongest predictor of aggressive and deviant behaviour among the broad factors of personality. However, this dissertation extends previous findings by demonstrating the applicability of Honesty-Humility across different contexts and by providing a nuanced understanding of the components responsible for this relationship

Integrate the GM(1,1) and Verhulst models to predict software stage effort

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2009 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Software effort prediction clearly plays a crucial role in software project management. In keeping with more dynamic approaches to software development, it is not sufficient to only predict the whole-project effort at an early stage. Rather, the project manager must also dynamically predict the effort of different stages or activities during the software development process. This can assist the project manager to reestimate effort and adjust the project plan, thus avoiding effort or schedule overruns. This paper presents a method for software physical time stage-effort prediction based on grey models GM(1,1) and Verhulst. This method establishes models dynamically according to particular types of stage-effort sequences, and can adapt to particular development methodologies automatically by using a novel grey feedback mechanism. We evaluate the proposed method with a large-scale real-world software engineering dataset, and compare it with the linear regression method and the Kalman filter method, revealing that accuracy has been improved by at least 28% and 50%, respectively. The results indicate that the method can be effective and has considerable potential. We believe that stage predictions could be a useful complement to whole-project effort prediction methods.National Natural Science Foundation of China and the Hi-Tech Research and Development Program of Chin