An Approach to Improve Existing Measurement Frameworks in Software Development Organizations

Measurement is a key mechanism to characterize, evaluate, and improve software development, management, and maintenance processes. Nowadays, software organizations use metrics for very different purposes. Data is collected to describe, monitor, understand, assess, compare, validate, and appraise very diverse attributes related to software processes or products. Improving data collection and better using the existing data are important problems for software organizations. This dissertation proposes an approach for improving measurement and data use when a large number of diverse metrics are already being collected by a software organization. The approach combines two methods. One looks at an organization's measurement framework in a top-down fashion and the other looks at it in a bottom-up fashion. The top-down method, based on the Goal-Question-Metric (GQM) Paradigm, is used to identify the measurement goals of data users and map them to the metrics being used by the organization. This allows the measurement practitioners to: (1)~identify which metrics are and are not useful to the organization; and (2)~check if the goals of data user groups can be satisfied by the data that is being collected by the organization. The bottom-up method is based on a data mining technique called Attribute Focusing (AF). It is used to identify useful information in the existing data that the data users were not aware of. To validate the approach and to assess its usefulness, a case study was performed in a real industrial environment. The top-down and bottom-up methods were applied in the customer satisfaction measurement framework at the IBM Toronto Laboratory. The top-down method was applied to improve the customer satisfaction (CUSTSAT) measurement from the point of view of three data user groups. The bottom-up method was used to gain new insights into the existing CUSTSAT data. The top-down method identified several new metrics for the interviewed user groups. It also contributed to better understanding the data user needs and led to modification of some of the data analyses and presentations done for those groups. The bottom-up method produced important insights on both the customer satisfaction domain and the measurement framework itself. Unexpected associations between key variables prompted new insights on their importance for the organization. Some of these associations have also revealed problems with the metrics being used to collect the data. (Also cross-referenced as UMIACS-TR-97-82

Human-Intelligence and Machine-Intelligence Decision Governance Formal Ontology

Since the beginning of the human race, decision making and rational thinking played a pivotal role for mankind to either exist and succeed or fail and become extinct. Self-awareness, cognitive thinking, creativity, and emotional magnitude allowed us to advance civilization and to take further steps toward achieving previously unreachable goals. From the invention of wheels to rockets and telegraph to satellite, all technological ventures went through many upgrades and updates. Recently, increasing computer CPU power and memory capacity contributed to smarter and faster computing appliances that, in turn, have accelerated the integration into and use of artificial intelligence (AI) in organizational processes and everyday life. Artificial intelligence can now be found in a wide range of organizational systems including healthcare and medical diagnosis, automated stock trading, robotic production, telecommunications, space explorations, and homeland security. Self-driving cars and drones are just the latest extensions of AI. This thrust of AI into organizations and daily life rests on the AI community’s unstated assumption of its ability to completely replicate human learning and intelligence in AI. Unfortunately, even today the AI community is not close to completely coding and emulating human intelligence into machines. Despite the revolution of digital and technology in the applications level, there has been little to no research in addressing the question of decision making governance in human-intelligent and machine-intelligent (HI-MI) systems. There also exists no foundational, core reference, or domain ontologies for HI-MI decision governance systems. Further, in absence of an expert reference base or body of knowledge (BoK) integrated with an ontological framework, decision makers must rely on best practices or standards that differ from organization to organization and government to government, contributing to systems failure in complex mission critical situations. It is still debatable whether and when human or machine decision capacity should govern or when a joint human-intelligence and machine-intelligence (HI-MI) decision capacity is required in any given decision situation. To address this deficiency, this research establishes a formal, top level foundational ontology of HI-MI decision governance in parallel with a grounded theory based body of knowledge which forms the theoretical foundation of a systemic HI-MI decision governance framework

Designing web-based adaptive learning environment : distils as an example

In this study, two components are developed for the Web-based adaptive learning: an online Intelligent Tutoring Tool (ITT) and an Adaptive Lecture Guidance (ALG). The ITT provides students timely problem-solving help in a dynamic Web environment. The ALG prevents students from being disoriented when a new domain is presented using Web technology. A prototype, Distributed Intelligent Learning System (DISTILS), has been implemented in a general chemistry laboratory domain. In DISTILS, students interact with the ITT through a Web browser. When a student selects a problem, the problem is formatted and displayed in the user interface for the student to solve. On the other side, the ITT begins to solve the problem simultaneously. The student can then request help from the ITT through the interface. The ITT interacts with the student, verifying those solution activities in an ascending order of the student knowledge status. In DISTILS, a Web page is associated with a HTML Learning Model (HLM) to describe its knowledge content. The ALG extracts the HLM, collects the status of students\u27 knowledge in HLM, and presents a knowledge map illustrating where the student is, how much proficiency he/she already has and where he/she is encouraged to explore. In this way, the ALG helps students to navigate the Web-based course material, protecting them from being disoriented and giving them guidance in need. Both the ITT and ALG components are developed under a generic Common Object Request Broker Architecture (CORBA)-driven framework. Under this framework, knowledge objects model domain expertise, a student modeler assesses student\u27s knowledge progress, an instruction engine includes two tutoring components, such as the ITT and the ALG, and the CORBA-compatible middleware serves as the communication infrastructure. The advantage of such a framework is that it promotes the development of modular and reusable intelligent educational objects. In DISTILS, a collection of knowledge objects were developed under CORBA to model general chemistry laboratory domain expertise. It was shown that these objects can be easily assembled in a plug-and-play manner to produce several exercises for different laboratory experiments. Given the platform independence of CORBA, tutoring objects developed under such a framework have the potential to be easily reused in different applications. Preliminary results showed that DISTILS effectively enhanced learning in Web environment. Three high school students and twenty-two NJIT students participated in the evaluation of DISTILS. In the final quiz of seven questions, the average correct answers of the students who studied in a Web environment with DISTILS (DISTILS Group) was 5.3, and the average correct answers of those who studied in the same Web environment without DISTILS (NoDISTILS Group) was 2.75. A t-test conducted on this small sample showed that the DISTILS group students significantly scored better than the NoDISTILS group students

An Approach to Guide Users Towards Less Revealing Internet Browsers

When browsing the Internet, HTTP headers enable both clients and servers send extra data in their requests or responses such as the User-Agent string. This string contains information related to the sender’s device, browser, and operating system. Previous research has shown that there are numerous privacy and security risks result from exposing sensitive information in the User-Agent string. For example, it enables device and browser fingerprinting and user tracking and identification. Our large analysis of thousands of User-Agent strings shows that browsers differ tremendously in the amount of information they include in their User-Agent strings. As such, our work aims at guiding users towards using less exposing browsers. In doing so, we propose to assign an exposure score to browsers based on the information they expose and vulnerability records. Thus, our contribution in this work is as follows: first, provide a full implementation that is ready to be deployed and used by users. Second, conduct a user study to identify the effectiveness and limitations of our proposed approach. Our implementation is based on using more than 52 thousand unique browsers. Our performance and validation analysis show that our solution is accurate and efficient. The source code and data set are publicly available and the solution has been deployed

SUPPORTING ENGINEERING DESIGN OF ADDITIVELY MANUFACTURED MEDICAL DEVICES WITH KNOWLEDGE MANAGEMENT THROUGH ONTOLOGIES

Medical environments pose a substantial challenge for engineering designers. They combine significant knowledge demands with large investment for new product development and severe consequences in the case of design failure. Engineering designers must contend with an often-chaotic environment to which they have limited access and familiarity, a user base that is difficult to engage and highly diverse in many attributes, and a market structure that often pits stakeholders against one another. As medical care in general moves towards personalized models and surgical tools towards less invasive options emerging manufacturing technologies in additive manufacturing offer significant potential for the design of highly innovative medical devices. At the same time however these same technologies also introduce yet more challenges to the design process. This dissertation presents a knowledge-based approach to addressing the existing and emerging challenges of medical device design. The approach aims to address these challenges using knowledge captured in a suite of modular ontologies modeling knowledge domains that must be considered in medical device design. These include ontologies for understanding clinical context, human factors, regulation, enterprise, and manufacturability. Together these ontologies support design ideation, knowledge capture, and design verification. These ontologies are subsequently used to formulate a comprehensive knowledge framework for medical device design, and to enable an innovative design process. Case studies analyzing the design of surgical tools in several medical specialties are used to assess the capabilities of this approach

A Structured Systemic Framework for Software Development

The purpose of this research was to develop and apply a systems-based framework for the analysis of software development project performance. Software development project performance is measured at the project level; that is, cost, schedule, and product quality that affect the overall project. To date, most performance improvement efforts have been focused on individual processes within the overall software development system. Making improvements to sub-elements, processes, or sub-systems without regard for the overall project is a classic misbehavior entered into by practitioners who fail to use a holistic, systemic approach. Attempts to improve sub-system behavior are at odds with The Principle of Sub-optimization. (van Gigch, 1974) The traditional method of predicting software development project performance, in terms of sub-system performance is too restrictive. A new holistic, systemic view based on systems principles offers a more robust way to look at performance. This research addressed this gap in the systems and software body of knowledge by developing a generalizable and transportable framework for software project performance that is based on systems principles. A rigorous mixed-method research methodology, employing both inductive and case study methods, was used to develop and validate the framework. Two research questions were identified as integral to increasing the understanding of a systems-based framework. (1) How does systems theory apply to the analysis of software development project performance? (2) What results from the application of a systems-based analysis framework for analyzing performance on a software development project? Using Discoverers\u27 Induction (Whewell, 1858), a systems-based framework for the analysis of software development project performance was constructed, adding to the systems and software body of knowledge and substantiating a comprehensive and unambiguous theoretical construct for software development. Then, the framework was applied to two completed software development projects to support validation. The structured systemic framework shows significant promise for contribution to software practitioners by indicating future software development project performance. The research also made a contribution in the area of research methodologies by resurrecting William Whewell\u27s Discoverers\u27 Induction (1858) and furthering the use of the case study method in the engineering management and systems engineering domain, areas where their application has been very limited