Testing in the incremental design and development of complex products

Testing is an important aspect of design and development which consumes significant time and resource in many companies. However, it has received less research attention than many other activities in product development, and especially, very few publications report empirical studies of engineering testing. Such studies are needed to establish the importance of testing and inform the development of pragmatic support methods. This paper combines insights from literature study with findings from three empirical studies of testing. The case studies concern incrementally developed complex products in the automotive domain. A description of testing practice as observed in these studies is provided, confirming that testing activities are used for multiple purposes depending on the context, and are intertwined with design from start to finish of the development process, not done after it as many models depict. Descriptive process models are developed to indicate some of the key insights, and opportunities for further research are suggested

Taking real-life seriously : an approach to decomposing context beyond 'environment' in living labs

The maturity of Living Labs has grown and several researchers have tried to create a uniform definition of what Living Labs are by emphasizing the multi-method and real-life, contextual approach. Although researchers thus recognize the importance of context in Living Labs, they do not provide insights into how context can be taken into account. The real-life context predominantly focuses on the in-situ use of a product during field trials where users are observed in their everyday life. The contribution of this paper will be twofold. By means of a case study we will show how context can be evaluated in the front end of design, so Living Lab researchers are no longer dependent on the readiness level of a product, and we will show how field trials can be evaluated in a more structured way to cover all components of context. By using a framework to evaluate the impact of context on product use, Living Lab researchers can improve the overall effectiveness of data gathering and analysis methods in a Living Lab project

Facilitating insight into a simulation model using visualization and dynamic model previews

This paper shows how model simplification, by replacing iterative steps with unitary predictive equations, can enable dynamic interaction with a complex simulation process. Model previews extend the techniques of dynamic querying and query previews into the context of ad hoc simulation model exploration. A case study is presented within the domain of counter-current chromatography. The relatively novel method of insight evaluation was applied, given the exploratory nature of the task. The evaluation data show that the trade-off in accuracy is far outweighed by benefits of dynamic interaction. The number of insights gained using the enhanced interactive version of the computer model was more than six times higher than the number of insights gained using the basic version of the model. There was also a trend for dynamic interaction to facilitate insights of greater domain importance

CubeSat Radiation Hardness Assurance Beyond Total Dose: Evaluating Single Event Effects

Radiation poses known and serious risks to smallsat survivability and mission duration, with effects falling into two categories: long-term total ionizing dose (TID) and instantaneous single event effects (SEE). Although literature exists on the topic of addressing TID in smallsats, few resources exist for addressing SEEs. Many varieties of SEEs exist, such as bit upsets and latch ups, which can occur in any electronic component containing active semiconductors (such as transistors). SEE consequences range from benign to destructive, so mission reliability can be enhanced by implementing fault protection strategies based on predicted SEE rates. Unfortunately, SEE rates are most reliably estimated through experimental testing that is often too costly for smallsat-scale missions. Prior test data published by larger programs exist, but may be sparse or incompatible with the environment of a particular mission. Despite these limitations, a process may be followed to gain insights and make informed design decisions for smallsats in the absence of hardware testing capabilities or similar test data. This process is: (1) Define the radiation environment; (2) identify the most critical and/or susceptible components on a spacecraft; (3) perform a search for compatible prior test data and/or component class data; (4) evaluate mission-specific SEE rates from available data; (5) study the rates alongside the mission requirements to identify high-risk areas of potential mitigation. The methodology developed in this work is based on the multi-institutional, National Science Foundation (NSF) Space Weather Atmospheric Reconfigurable Multiscale Experiment (SWARM-EX) mission. The steps taken during SWARM-EX’s radiation analysis alongside the detailed methodology serve as a case study for how these techniques can be applied to increasing the reliability of a university-scale smallsat mission

The Role of Trust in Explaining Food Choice: Combining Choice Experiment and Attribute Best−Worst Scaling

This paper presents empirical findings from a combination of two elicitation techniques—discrete choice experiment (DCE) and best–worst scaling (BWS)—to provide information about the role of consumers’ trust in food choice decisions in the case of credence attributes. The analysis was based on a sample of 459 Taiwanese consumers and focuses on red sweet peppers. DCE data were examined using latent class analysis to investigate the importance and the utility different consumer segments attach to the production method, country of origin, and chemical residue testing. The relevance of attitudinal and trust-based items was identified by BWS using a hierarchical Bayesian mixed logit model and was aggregated to five latent components by means of principal component analysis. Applying a multinomial logit model, participants’ latent class membership (obtained from DCE data) was regressed on the identified attitudinal and trust components, as well as demographic information. Results of the DCE latent class analysis for the product attributes show that four segments may be distinguished. Linking the DCE with the attitudinal dimensions reveals that consumers’ attitude and trust significantly explain class membership and therefore, consumers’ preferences for different credence attributes. Based on our results, we derive recommendations for industry and policy

The Development and Validation of a Generic Instrument, QoDoS, for Assessing the Quality of Decision Making.

Introduction: The impact of decision-making during the development and the regulatory review of medicines greatly influences the delivery of new medicinal products. Currently, there is no generic instrument that can be used to assess the quality of decision-making. This study describes the development of the Quality of Decision-Making Orientation Scheme QoDoS© instrument for appraising the quality of decision-making. Methods: Semi-structured interviews about decision-making were carried out with 29 senior decision makers from the pharmaceutical industry (10), regulatory authorities (9) and contract research organizations (10). The interviews offered a qualified understanding of the subjective decision-making approach, influences, behaviors and other factors that impact such processes for individuals and organizations involved in the delivery of new medicines. Thematic analysis of the transcribed interviews was carried out using NVivo8® software. Content validity was carried out using qualitative and quantitative data by an expert panel, which led to the developmental version of the QoDoS. Further psychometric evaluations were performed, including factor analysis, item reduction, reliability testing and construct validation. Results: The thematic analysis of the interviews yielded a 94-item initial version of the QoDoS© with a 5-point Likert scale. The instrument was tested for content validity using a panel of experts for language clarity, completeness, relevance and scaling, resulting in a favorable agreement by panel members with an intra-class correlation coefficient value of 0.89 (95% confidence interval = 0.56, 0.99). A 76-item QoDoS© (version 2) emerged from content validation. Factor analysis produced a 47-item measure with four domains. The 47-item QoDoS© (version 3) showed high internal consistency (n=120, Cronbach’s alpha = 0.89), high reproducibility (n=20, intra-class correlation =0.77) and a mean completion time of 10 min. Reliability testing and construct validation was successfully performed.Peer reviewedFinal Published versio

The Knowledge Application and Utilization Framework Applied to Defense COTS: A Research Synthesis for Outsourced Innovation

Purpose -- Militaries of developing nations face increasing budget pressures, high operations tempo, a blitzing pace of technology, and adversaries that often meet or beat government capabilities using commercial off-the-shelf (COTS) technologies. The adoption of COTS products into defense acquisitions has been offered to help meet these challenges by essentially outsourcing new product development and innovation. This research summarizes extant research to develop a framework for managing the innovative and knowledge flows. Design/Methodology/Approach – A literature review of 62 sources was conducted with the objectives of identifying antecedents (barriers and facilitators) and consequences of COTS adoption. Findings – The DoD COTS literature predominantly consists of industry case studies, and there’s a strong need for further academically rigorous study. Extant rigorous research implicates the importance of the role of knowledge management to government innovative thinking that relies heavily on commercial suppliers. Research Limitations/Implications – Extant academically rigorous studies tend to depend on measures derived from work in information systems research, relying on user satisfaction as the outcome. Our findings indicate that user satisfaction has no relationship to COTS success; technically complex governmental purchases may be too distant from users or may have socio-economic goals that supersede user satisfaction. The knowledge acquisition and utilization framework worked well to explain the innovative process in COTS. Practical Implications – Where past research in the commercial context found technological knowledge to outweigh market knowledge in terms of importance, our research found the opposite. Managers either in government or marketing to government should be aware of the importance of market knowledge for defense COTS innovation, especially for commercial companies that work as system integrators. Originality/Value – From the literature emerged a framework of COTS product usage and a scale to measure COTS product appropriateness that should help to guide COTS product adoption decisions and to help manage COTS product implementations ex post