On the evaluation of reference models for software engineering practice.

This paper argues that conceptual models and more specifically reference models play a key role in the specification and design of information systems. However, an effective evaluation strategy of such models is a relatively immature field. The paper presents the key challenges for this evaluation activity and articulates an approach for understanding how to evaluate models based on the information and cognitive theories of structuralism and conversation theory. An example of a reference model developed for the Higher Education domain is used as a case study to illustrate how the approach may be applied

Evaluation of fit for 3D printed retainers as compared to thermoform retainers

ABSTRACT EVALUATION OF FIT FOR 3D PRINTED RETAINERS AS COMPARED TO THERMOFORM RETAINERS By David Cole, D.M.D. A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Dentistry at Virginia Commonwealth University Thesis Directors: Eser Tüfekçi, D.D.S., M.S., Ph.D., M.S.H.A. Professor, Department of Orthodontics Sompop Bencharit, D.D.S., M.S., Ph.D. Associate Professor and Director of Digital Dentistry, Department of General Practice Introduction: Despite recent advances in three-dimensional (3D) printing, little information is available on 3D printed retainers Methods: Three reference models were used to fabricate traditional vacuum formed, commercially-available vacuum formed, and 3D printed retainers. For each model, three retainers were made using the three methods (a total of 27 retainers). To determine the trueness, the distances between the intaglio surface of the retainers and the occlusal surface of the reference models were measured using an engineering software. A small difference was indicative of a good fit. Results: Average differences of the traditional vacuum formed retainers ranged from 0.10 to 0.20mm. The commercially-available and 3D printed retainers had a range of 0.10 to 0.30mm and 0.10 to 0.40mm, respectively. Conclusions: The traditional vacuum formed retainers showed the least amount of deviation from the original reference models while the 3D printed retainers showed the greatest deviation

An Evaluation of a Linguistically Motivated Conversational Software Agent Framework

yesThis paper presents a critical evaluation framework for a linguistically motivated conversational software agent (CSA). The CSA prototype investigates the integration, intersection and interface of the language, knowledge, and speech act constructions (SAC) based on a grammatical object, and the sub-model of belief, desires and intention (BDI) and dialogue management (DM) for natural language processing (NLP). A long-standing issue within NLP CSA systems is refining the accuracy of interpretation to provide realistic dialogue to support human-to-computer communication. This prototype constitutes three phase models: (1) a linguistic model based on a functional linguistic theory – Role and Reference Grammar (RRG), (2) an Agent Cognitive Model with two inner models: (a) a knowledge representation model, (b) a planning model underpinned by BDI concepts, intentionality and rational interaction, and (3) a dialogue model. The evaluation strategy for this Java-based prototype is multi-approach driven by grammatical testing (English language utterances), software engineering and agent practice. A set of evaluation criteria are grouped per phase model, and the testing framework aims to test the interface, intersection and integration of all phase models. The empirical evaluations demonstrate that the CSA is a proof-of-concept, demonstrating RRG’s fitness for purpose for describing, and explaining phenomena, language processing and knowledge, and computational adequacy. Contrastingly, evaluations identify the complexity of lower level computational mappings of NL – agent to ontology with semantic gaps, and further addressed by a lexical bridging solution

‘How can one evaluate a conversational software agent framework?’

YesThis paper presents a critical evaluation framework for a linguistically orientated conversational software agent (CSA) (Panesar, 2017). The CSA prototype investigates the integration, intersection and interface of the language, knowledge, and speech act constructions (SAC) based on a grammatical object (Nolan, 2014), and the sub-model of belief, desires and intention (BDI) (Rao and Georgeff, 1995) and dialogue management (DM) for natural language processing (NLP). A long-standing issue within NLP CSA systems is refining the accuracy of interpretation to provide realistic dialogue to support the human-to-computer communication. This prototype constitutes three phase models: (1) a linguistic model based on a functional linguistic theory – Role and Reference Grammar (RRG) (Van Valin Jr, 2005); (2) Agent Cognitive Model with two inner models: (a) knowledge representation model employing conceptual graphs serialised to Resource Description Framework (RDF); (b) a planning model underpinned by BDI concepts (Wooldridge, 2013) and intentionality (Searle, 1983) and rational interaction (Cohen and Levesque, 1990); and (3) a dialogue model employing common ground (Stalnaker, 2002). The evaluation approach for this Java-based prototype and its phase models is a multi-approach driven by grammatical testing (English language utterances), software engineering and agent practice. A set of evaluation criteria are grouped per phase model, and the testing framework aims to test the interface, intersection and integration of all phase models and their inner models. This multi-approach encompasses checking performance both at internal processing, stages per model and post-implementation assessments of the goals of RRG, and RRG based specifics tests. The empirical evaluations demonstrate that the CSA is a proof-of-concept, demonstrating RRG’s fitness for purpose for describing, and explaining phenomena, language processing and knowledge, and computational adequacy. Contrastingly, evaluations identify the complexity of lower level computational mappings of NL – agent to ontology with semantic gaps, and further addressed by a lexical bridging consideration (Panesar, 2017)

HOW CAN ONE EVALUATE A CONVERSATIONAL SOFTWARE AGENT FRAMEWORK?

This paper presents a critical evaluation framework for a linguistically orientated conversational software agent (CSA) (Panesar, 2017). The CSA prototype investigates the integration, intersection and interface of the language, knowledge, and speech act constructions (SAC) based on a grammatical object (Nolan, 2014), and the sub-model of belief, desires and intention (BDI) (Rao and Georgeff, 1995) and dialogue management (DM) for natural language processing (NLP). A long-standing issue within NLP CSA systems is refining the accuracy of interpretation to provide realistic dialogue to support the human-to-computer communication. This prototype constitutes three phase models: (1) a linguistic model based on a functional linguistic theory – Role and Reference Grammar (RRG) (Van Valin Jr, 2005); (2) Agent Cognitive Model with two inner models: (a) knowledge representation model employing conceptual graphs serialised to Resource Description Framework (RDF); (b) a planning model underpinned by BDI concepts (Wooldridge, 2013) and intentionality (Searle, 1983) and rational interaction (Cohen and Levesque, 1990); and (3) a dialogue model employing common ground (Stalnaker, 2002). The evaluation approach for this Java-based prototype and its phase models is a multi-approach driven by grammatical testing (English language utterances), software engineering and agent practice. A set of evaluation criteria are grouped per phase model, and the testing framework aims to test the interface, intersection and integration of all phase models and their inner models. This multi-approach encompasses checking performance both at internal processing, stages per model and post-implementation assessments of the goals of RRG, and RRG based specifics tests. The empirical evaluations demonstrate that the CSA is a proof-of-concept, demonstrating RRG’s fitness for purpose for describing, and explaining phenomena, language processing and knowledge, and computational adequacy. Contrastingly, evaluations identify the complexity of lower level computational mappings of NL – agent to ontology with semantic gaps, and further addressed by a lexical bridging consideration (Panesar, 2017)

Evaluating cross-organizational ERP requirements engineering practices: a focus group study

This focus group study presents our first validation of practices for engineering the coordination requirements in cross-organizational Enterprise Resource Planning (ERP) projects. The study evaluates 13 practices addressing a variety of coordination aspects crucial to ERP projects. These practices are results in previously published research publications by the first author. The practices are formulated in response to practitioners' needs at ERP adopting organizations. The proposed practices have now reached the stage where we need some independent feedback as to the extent to which they fit the realities of practitioners. We perform this validation by means of a qualitative research approach, namely the focus group method. Current software engineering literature provides few examples of using focus groups in the evaluation of good software development practices. Because of this, providing reflections on our focus-group-based validation experiences will be of value to both the research community and practitioners

A Case Study on Artefact-based RE Improvement in Practice

Most requirements engineering (RE) process improvement approaches are solution-driven and activity-based. They focus on the assessment of the RE of a company against an external norm of best practices. A consequence is that practitioners often have to rely on an improvement approach that skips a profound problem analysis and that results in an RE approach that might be alien to the organisational needs. In recent years, we have developed an RE improvement approach (called \emph{ArtREPI}) that guides a holistic RE improvement against individual goals of a company putting primary attention to the quality of the artefacts. In this paper, we aim at exploring ArtREPI's benefits and limitations. We contribute an industrial evaluation of ArtREPI by relying on a case study research. Our results suggest that ArtREPI is well-suited for the establishment of an RE that reflects a specific organisational culture but to some extent at the cost of efficiency resulting from intensive discussions on a terminology that suits all involved stakeholders. Our results reveal first benefits and limitations, but we can also conclude the need of longitudinal and independent investigations for which we herewith lay the foundation

Model-driven Enterprise Systems Configuration

Enterprise Systems potentially lead to significant efficiency gains but require a well-conducted configuration process. A promising idea to manage and simplify the configuration process is based on the premise of using reference models for this task. Our paper continues along this idea and delivers a two-fold contribution: first, we present a generic process for the task of model-driven Enterprise Systems configuration including the steps of (a) Specification of configurable reference models, (b) Configuration of configurable reference models, (c) Transformation of configured reference models to regular build time models, (d) Deployment of the generated build time models, (e) Controlling of implementation models to provide input to the configuration, and (f) Consolidation of implementation models to provide input to reference model specification. We discuss inputs and outputs as well as the involvement of different roles and validation mechanisms. Second, we present an instantiation case of this generic process for Enterprise Systems configuration based on Configurable EPCs

A systematic review of software development cost estimation studies

This paper aims to provide a basis for the improvement of software estimation research through a systematic review of previous work. The review identifies 304 software cost estimation papers in 76 journals and classifies the papers according to research topic, estimation approach, research approach, study context and data set. A web-based library of these cost estimation papers is provided to ease the identification of relevant estimation research results. The review results combined with other knowledge provide support for recommendations for future software cost estimation research, including: 1) Increase the breadth of the search for relevant studies, 2) Search manually for relevant papers within a carefully selected set of journals when completeness is essential, 3) Conduct more studies on estimation methods commonly used by the software industry, and, 4) Increase the awareness of how properties of the data sets impact the results when evaluating estimation methods

Quality-aware model-driven service engineering

Service engineering and service-oriented architecture as an integration and platform technology is a recent approach to software systems integration. Quality aspects ranging from interoperability to maintainability to performance are of central importance for the integration of heterogeneous, distributed service-based systems. Architecture models can substantially influence quality attributes of the implemented software systems. Besides the benefits of explicit architectures on maintainability and reuse, architectural constraints such as styles, reference architectures and architectural patterns can influence observable software properties such as performance. Empirical performance evaluation is a process of measuring and evaluating the performance of implemented software. We present an approach for addressing the quality of services and service-based systems at the model-level in the context of model-driven service engineering. The focus on architecture-level models is a consequence of the black-box character of services