Feature weighting techniques for CBR in software effort estimation studies: A review and empirical evaluation

Context : Software effort estimation is one of the most important activities in the software development process. Unfortunately, estimates are often substantially wrong. Numerous estimation methods have been proposed including Case-based Reasoning (CBR). In order to improve CBR estimation accuracy, many researchers have proposed feature weighting techniques (FWT). Objective: Our purpose is to systematically review the empirical evidence to determine whether FWT leads to improved predictions. In addition we evaluate these techniques from the perspectives of (i) approach (ii) strengths and weaknesses (iii) performance and (iv) experimental evaluation approach including the data sets used. Method: We conducted a systematic literature review of published, refereed primary studies on FWT (2000-2014). Results: We identified 19 relevant primary studies. These reported a range of different techniques. 17 out of 19 make benchmark comparisons with standard CBR and 16 out of 17 studies report improved accuracy. Using a one-sample sign test this positive impact is significant (p = 0:0003). Conclusion: The actionable conclusion from this study is that our review of all relevant empirical evidence supports the use of FWTs and we recommend that researchers and practitioners give serious consideration to their adoption

Gaining Insight into Determinants of Physical Activity using Bayesian Network Learning

Contains fulltext : 228326pre.pdf (preprint version ) (Open Access) Contains fulltext : 228326pub.pdf (publisher's version ) (Open Access)BNAIC/BeneLearn 202

Acquiring and Harnessing Verb Knowledge for Multilingual Natural Language Processing

Advances in representation learning have enabled natural language processing models to derive non-negligible linguistic information directly from text corpora in an unsupervised fashion. However, this signal is underused in downstream tasks, where they tend to fall back on superficial cues and heuristics to solve the problem at hand. Further progress relies on identifying and filling the gaps in linguistic knowledge captured in their parameters. The objective of this thesis is to address these challenges focusing on the issues of resource scarcity, interpretability, and lexical knowledge injection, with an emphasis on the category of verbs. To this end, I propose a novel paradigm for efficient acquisition of lexical knowledge leveraging native speakers’ intuitions about verb meaning to support development and downstream performance of NLP models across languages. First, I investigate the potential of acquiring semantic verb classes from non-experts through manual clustering. This subsequently informs the development of a two-phase semantic dataset creation methodology, which combines semantic clustering with fine-grained semantic similarity judgments collected through spatial arrangements of lexical stimuli. The method is tested on English and then applied to a typologically diverse sample of languages to produce the first large-scale multilingual verb dataset of this kind. I demonstrate its utility as a diagnostic tool by carrying out a comprehensive evaluation of state-of-the-art NLP models, probing representation quality across languages and domains of verb meaning, and shedding light on their deficiencies. Subsequently, I directly address these shortcomings by injecting lexical knowledge into large pretrained language models. I demonstrate that external manually curated information about verbs’ lexical properties can support data-driven models in tasks where accurate verb processing is key. Moreover, I examine the potential of extending these benefits from resource-rich to resource-poor languages through translation-based transfer. The results emphasise the usefulness of human-generated lexical knowledge in supporting NLP models and suggest that time-efficient construction of lexicons similar to those developed in this work, especially in under-resourced languages, can play an important role in boosting their linguistic capacity.ESRC Doctoral Fellowship [ES/J500033/1], ERC Consolidator Grant LEXICAL [648909

AN INVESTIGATION INTO AN EXPERT SYSTEM FOR TELECOMMUNICATION NETWORK DESIGN

Many telephone companies, especially in Eastern-Europe and the 'third world', are developing new telephone networks. In such situations the network design engineer needs computer based tools that not only supplement his own knowledge but also help him to cope with situations where not all the information necessary for the design is available. Often traditional network design tools are somewhat removed from the practical world for which they were developed. They often ignore the significant uncertain and statistical nature of the input data. They use data taken from a fixed point in time to solve a time variable problem, and the cost formulae tend to be on an average per line or port rather than the specific case. Indeed, data is often not available or just plainly unreliable. The engineer has to rely on rules of thumb honed over many years of experience in designing networks and be able to cope with missing data. The complexity of telecommunication networks and the rarity of specialists in this area often makes the network design process very difficult for a company. It is therefore an important area for the application of expert systems. Designs resulting from the use of expert systems will have a measure of uncertainty in their solution and adequate account must be made of the risk involved in implementing its design recommendations. The thesis reviews the status of expert systems as used for telecommunication network design. It further shows that such an expert system needs to reduce a large network problem into its component parts, use different modules to solve them and then combine these results to create a total solution. It shows how the various sub-division problems are integrated to solve the general network design problem. This thesis further presents details of such an expert system and the databases necessary for network design: three new algorithms are invented for traffic analysis, node locations and network design and these produce results that have close correlation with designs taken from BT Consultancy archives. It was initially supposed that an efficient combination of existing techniques for dealing with uncertainty within expert systems would suffice for the basis of the new system. It soon became apparent, however, that to allow for the differing attributes of facts, rules and data and the varying degrees of importance or rank within each area, a new and radically different method would be needed. Having investigated the existing uncertainty problem it is believed that a new more rational method has been found. The work has involved the invention of the 'Uncertainty Window' technique and its testing on various aspects of network design, including demand forecast, network dimensioning, node and link system sizing, etc. using a selection of networks that have been designed by BT Consultancy staff. From the results of the analysis, modifications to the technique have been incorporated with the aim of optimising the heuristics and procedures, so that the structure gives an accurate solution as early as possible. The essence of the process is one of associating the uncertainty windows with their relevant rules, data and facts, which results in providing the network designer with an insight into the uncertainties that have helped produce the overall system design: it indicates which sources of uncertainty and which assumptions are were critical for further investigation to improve upon the confidence of the overall design. The windowing technique works by virtue of its ability to retain the composition of the uncertainty and its associated values, assumption, etc. and allows for better solutions to be attained.BRITISH TELECOMMUNICATIONS PL

Brain network approaches to characterise structure-function relationships after childhood head injury

Traumatic brain injury (TBI) is a common leading cause of disability for children and young adults. Such an insult during development leads to a cognitive-behavioural syndrome of impairments post-injury however, the trajectory and magnitude of this syndrome at an individual-patient level are unknown. Pathological effects of a TBI on neuroanatomy can be quasi-quantitatively measured using magnetic resonance imaging (MRI). The current thesis highlights the current state of research into neuroanatomical differences post-TBI and identifies previous investigations of neuroanatomical correlates of later functioning adopt mostly reductionist and univariate approaches. It was posited that utilising MRI methodologies that respect the complex, interrelated nature of neuroanatomy across the cortex would provide better understanding of the neuroanatomical correlates of later cognitive functioning post-injury. The current thesis investigates novel network-level analyses of neuroanatomy, specifically structural covariance and morphometric similarity approaches, and its relationship with neuropsychological functioning, with a focus on executive functioning (EF) at 2 years post-injury. The data used included 107 survivors of paediatric TBI and 36 typically developing controls. Using a structural covariance methodology the current thesis provides evidence that executive dysfunction is associated with atrophic neuroanatomical-changes to topologically important brain-regions within the network. Results also provided evidence that the magnitude of EF difficulties was associated with the extent to which an injury diverted the brain from the ‘typical’ organization of the neuroanatomical network. Using individual morphometric similarity approaches coupled with supervised machine learning, a pattern of morphology, centred in the pre-frontal cortex, predicted later EF. Overall, these findings fit with the hypothesis that network-level neuroanatomical correlates of EF would be found. The current thesis concludes that, regarding the neuropsychological sequalae post-neurological insult, quantification of the complex organisation of neuroanatomy across the cortex is a useful biomarker. Future investigations integrating neuropsychology and neuroimaging to understand brain structure-function relationships should continue to utilise modern network approaches which capture the diffuse, disconnecting nature of injury

Influence of ethnicity in optimizing antiepileptic drug dosing : a comparison of Malay, Chinese and Indian populations in Malaysia

Reports of inter-ethnic differences in metabolism for phenytoin and carbamazepineh have raised questions concerning the importance of monitoring serum levels to the standardised population therapeutic concentrations. Although the pharmacokinetics of phenytoin, carbamazepine, valproic acid and phenobarbitone displayed both intra and inter-individual variations, the influence of ethnicity is still unclear. This thesis has thus set its objectives of investigating the impact of ethnicity on the efficacy of these therapeutic ranges and pharmacokinetics of these drugs. A total 1554 serum concentrations were randomly selected by a set of criteria from 470 Malays, 423 Chinese and 322 Indian of adult and paediatric patients. The Mantel-Haenzel method was used to estimate for inter-ethnic differences in response to the defined therapeutic ranges. The influence of ethnicity on pharmacokinetics was examined by the test of heterogeneity of the slopes estimates in the linear relationship of either serum concentration or clearance to dose. Coefficient of variation on the ratios of the above relationships was used to measure for inter individual variation. The results showed a highly variable response to treatment within the defined therapeutic ranges. Therapeutic response is not dependent on ethnicity and age although the latter was determined on carbamazepine and valproic acid treated patients only. The pharmacokinetics of carbamazepine, valproic acid and phenobarbitone showed high inter-individual variations and were unaffected by weight, age or ethnicity. Similar high inter-individual variation for phenytoin pharmacokinetic parameters (Km and Vmax) were observed. However, Km and Vmax(mg/day) of adult Chinese patients were significantly lower than Malay or Indian patients. The relationship between Km and Vmax and age or weight were insignificant. These findings demonstrate that Malaysian patients only differed in handling phenytoin therapy and support the use of ethnic specific phenytoin pharmacokinetic parameters during therapy

Influence of ethnicity in optimizing antiepileptic drug dosing : a comparison of Malay, Chinese and Indian populations in Malaysia

Reports of inter-ethnic differences in metabolism for phenytoin and carbamazepineh have raised questions concerning the importance of monitoring serum levels to the standardised population therapeutic concentrations. Although the pharmacokinetics of phenytoin, carbamazepine, valproic acid and phenobarbitone displayed both intra and inter-individual variations, the influence of ethnicity is still unclear. This thesis has thus set its objectives of investigating the impact of ethnicity on the efficacy of these therapeutic ranges and pharmacokinetics of these drugs. A total 1554 serum concentrations were randomly selected by a set of criteria from 470 Malays, 423 Chinese and 322 Indian of adult and paediatric patients. The Mantel-Haenzel method was used to estimate for inter-ethnic differences in response to the defined therapeutic ranges. The influence of ethnicity on pharmacokinetics was examined by the test of heterogeneity of the slopes estimates in the linear relationship of either serum concentration or clearance to dose. Coefficient of variation on the ratios of the above relationships was used to measure for inter individual variation. The results showed a highly variable response to treatment within the defined therapeutic ranges. Therapeutic response is not dependent on ethnicity and age although the latter was determined on carbamazepine and valproic acid treated patients only. The pharmacokinetics of carbamazepine, valproic acid and phenobarbitone showed high inter-individual variations and were unaffected by weight, age or ethnicity. Similar high inter-individual variation for phenytoin pharmacokinetic parameters (Km and Vmax) were observed. However, Km and Vmax(mg/day) of adult Chinese patients were significantly lower than Malay or Indian patients. The relationship between Km and Vmax and age or weight were insignificant. These findings demonstrate that Malaysian patients only differed in handling phenytoin therapy and support the use of ethnic specific phenytoin pharmacokinetic parameters during therapy

Improving efficiency in stroke trials: an exploration of methods to improve the use of the modified Rankin Scale in acute stroke trials

The modified Rankin Scale (mRS) is the preferred outcome measure in stroke trials. Typically, mRS assessment is based on a clinician’s rating of a patient interview and interobserver variability is common. Meta-analysis suggests an overall reliability of k=0.46 but this may be less (k=0.25) in multi-centre studies. Mandatory training in mRS assessment is employed in most trials to mitigate this but the problem persists. Variability in assigning outcomes may lead to endpoint misclassification increasing the challenge of accurately demonstrating a treatment effect. We aimed to assess the impact of endpoint misclassification on trial power and explore methods to improve the use of the mRS in acute stroke trials. First we used the mRS outcome distributions of previous phase III randomised controlled trials (RCT) in stroke (NXY059 study and tPA NINDS study) to perform statistical simulations. We generated power estimates and sample sizes from simulated mRS studies under various combinations of sample size, mRS reliability and adjudication panel size. Simulations suggest that the potential benefit of improving mRS reliability from k 0.25 to k 0.5, k 0.7 or k 0.9 may allow a reduction in sample size of n= 386, n= 490 or n= 488 in a typical n=2000 RCT. We then developed a method for providing group adjudication of mRS endpoints and examined the feasibility, reliability and validity of its use in a multicentre clinical trial. We conducted a “virtual” acute stroke trial across 14 UK sites. Local mRS interviews were scored as normal but also recorded to digital video camera. Video clips were uploaded via secure web portal for scoring by adjudication committee reviewers. We demonstrated excellent technical success rates with acceptability to both participants and investigators. 370 participants were included in our “virtual” acute stroke trial and 563 mRS video assessments were uploaded for central review. 96% (538/563) of study visits resulted in an adjudicated mRS score. At 30 and 90 days respectively, 57.5% (161/280) and 50.8% (131/258) of clips were misclassified. Agreement was measured using kappa statistics (k/kw) and intraclass correlation coefficient. Agreement between the adjudication committee was very good (30 days kw 0.85 [95%CI 0.81-0.86], 90 days kw 0.86 [95% CI 0.82-0.88]) with no significant or systematic bias in mRS scoring in comparison to the local mRS. We demonstrated criterion and construct validity of centrally adjudicated mRS scores through comparison with the locally assigned mRS score and other measures known to affect stroke outcome including baseline NIHSS (bNIHSS), Systolic Blood Pressure (SBP), blood glucose and home time. We studied our cohort of mRS video clips to identify any features predictive of variability in mRS scoring. Patient specific variables included participant age, pre stroke mRS, baseline stroke severity as graded by baseline NIHSS (bNIHSS) and presence of language disorder. Interview specific variables included length of interview, poor sound quality, location of the interview, use of a proxy or discussion of prior disability. At both 30 and 90 days only “interview length” was a significant predictor of agreement in mRS scoring. Using a sample of mRS video clips in English and Mandarin, we conducted a pilot study to assess the effect of translation of mRS interviews on interobserver reliability. The interobserver reliability of the translated mRS assessments was similar to native language clips (Native (n=69) kw 0.91 [95%CI 0.86-0.99], Translated (n=89) kw 0.90 [95% CI 0.83-0.96]). We then incorporated a translation step into the central adjudication model using our existing web portal. Inter observer reliability seen in the modified clips (kw 0.85 [95% CI 0.74-0.95]) was similar to that seen in the original video files (kw 0.88 [95% CI 0.78-0.99]). Finally we aimed to investigate the ability of raters to detect more subtle degrees of disability within mRS ranks through blinded assessment of pairs of clips with matching mRS grades. These pairs contained either two clips with full agreement in mRS grade at initial group review or one clip with full agreement and one clip where scores were skewed in the direction of “more” or “less” disability. Pairs were randomly assigned to multiple raters. We could not identify any reliable pattern in identification of the “less disabled” mRS clip. More sensitive grading of the mRS with “good” or “bad” forms of each grade is not reliable on the basis of this exploratory study. Perhaps alternative methods of converting the ordinal ranks of the mRS scale into a more continuous distribution should be investigated; such as the use of a mean mRS score following multiple mRS ratings. Prior estimates of mRS reliability in multicentre studies are poor [k=0.25]. The risks of endpoint misclassification affecting trial power are substantial. Simulations suggest that the effect of improving interobserver reliability and multiple mRS assessments may reduce study sample size by 25%, resulting in substantial ethical and financial benefits. Agreement between our adjudication committee was good [k=0.59(95% CI:0.53-0.63), kw=0.86(95% CI:0.82-0.88)]. Central review may bring many additional potential benefits: “expert” review, quality control and improved blinding in complex trial design. Central adjudication of mRS assessments is feasible, reliable and valid, including the use of translated mRS assessments. This model of outcome assessment has been incorporated into four ongoing large clinical trials: CLEAR-3, MISTIE-3, EUROHYP-1 and SITS-OPEN

Tank gunnery prediction systems

This thesis is concerned with fire control prediction schemes for tanks employed in a defensive role against moving targets. The problem is considered in three parts: the determination of likely target movement patterns in an operational setting; the assessment and modelling of human operator response to those motions; and the utilisation of this response in optimal prediction schemes. In the first part the results from war games, tactical exercises and field trials are collated, and a method is devised for generating test target tracks for human operator study and prediction scheme evaluation. In the second part previous approaches to operator modelling are reviewed, laboratory experiments are described and a mathematical model of human response is developed. In the third part the general statistical properties of predictors are examined, a new class of predictive algorithm called the 'threshold' algorithm is devised, and this type of algorithm is then evaluated using the results of the previous two parts. The thesis ends with some consideration of further research requirements or possibilities, and of the steps needed to validate the results obtained so far.<p