A paired-comparison approach for fusing preference orderings from rank-ordered agents

The problem of aggregating multi-agent preference orderings has received considerable attention in many fields of research, such as multi-criteria decision aiding and social choice theory; nevertheless, the case in which the agents’ importance is expressed in the form of a rank-ordering, instead of a set of weights, has not been much debated. The aim of this article is to present a novel algorithm – denominated as ‘‘Ordered Paired-Comparisons Algorithm’’ (OPCA), which addresses this decision-making problem in a relatively simple and practical way. The OPCA is organized into three main phases: (i) turning multi- agent preference orderings into sets of paired comparisons, (ii) synthesizing the paired-comparison sets, and (iii) constructing a fused (or consensus) ordering. Particularly interesting is phase two, which introduces a new aggregation process based on a priority sequence, obtained from the agents’ importance rank-ordering. A detailed description of the new algorithm is supported by practical examples

A novel algorithm for fusing preference orderings by rank-ordered agents

Yager proposed an algorithm to combine multi-agent preference orderings of several alternatives into a single consensus fused ordering, when the agents’ importance is expressed through a rank-ordering and not a set of weights. This algorithm is simple and automatable but has some limitations which reduce its range of application, e.g., (i) preference orderings should not include incomparabilities between alternative and/or omissions of some of them, and (ii) the fused ordering may sometimes not reflect the majority of the multi-agent preference orderings. The aim of this article is to present an enhanced version of the Yager’s algorithm, which overcomes the above limitations. Some practical examples support the description of the new algorithm

Checking the consistency of the solution in ordinal semi-democratic decision making problems

An interesting decision-making problem is that of aggregating multi-agent preference orderings into a consensus ordering, in the case the agents’ importance is expressed in the form of a rank-ordering. Due to the specificity of the problem, the scientific literature encompasses a relatively small number of aggregation techniques. For the aggregation to be effective, it is important that the consensus ordering well reflects the input data, i.e., the agents’ preference orderings and importance rank-ordering. The aim of this paper is introducing a new quantitative tool – represented by the so-called p indicators – which allows to check the degree of consistency between consensus ordering and input data, from several perspectives. This tool is independent from the aggregation technique in use and applicable to a wide variety of practical contexts, e.g., problems in which preference orderings include omissions and/or incomparabilities between some alternatives. Also, the p indicators are simple, intuitive and practical for comparing the results obtained from different techniques. The description is supported by various application examples

Design decisions: concordance of designers and effects of the Arrow’s theorem on the collective preference ranking

The problem of collective decision by design teams has received considerable attention in the scientific literature of engineering design. A much debated problem is that in which multiple designers formulate their individual preference rankings of different design alternatives and these rankings should be aggregated into a collective one. This paper focuses the attention on three basic research questions: (i) “How can the degree of concordance of designer rankings be measured?”, (ii) “For a given set of designer rankings, which aggregation model provides the most coherent solution?”, and (iii) “To what extent is the collective ranking influenced by the aggregation model in use?”. The aim of this paper is to present a novel approach that addresses the above questions in a relatively simple and agile way. A detailed description of the methodology is supported by a practical application to a real-life case study

A new proposal to improve the customer competitive benchmarking in QFD

Quality Function Deployment (QFD) is a structured tool that supports the design of new products/services, translating customer requirements into technical and process characteristics. The so-called Customer Competitive Benchmarking is a module of the QFD’s House of Quality, in which a sample of (potential) customers express their perceptions on a set of competing products/services, within the same market segment of the one to be designed; this information is then elaborated by a cross-functional team of experts and used to define improvement and strategic goals. Despite the importance of this kind of benchmarking for the whole QFD process, the scientific literature reveals limited research. This paper critically analyzes the canonical procedure of customer-competitive benchmarking, highlighting its major weaknesses and problematic aspects. Additionally, it proposes an alternative procedure to overcome (at least partly) those weaknesses, without undermining the simplicity in data collection and processing of the canonical procedure. This alternative procedure utilizes the Thurstone’s Law of Comparative Judgment, which allows to transform subjective judgments by multiple respondents into a collective cardinal scaling. The description is supported by several pedagogical and real-life examples

Aggregating multiple ordinal rankings in engineering design: the best model according to the Kendall's coefficient of concordance

AbstractAggregating the preferences of a group of experts is a recurring problem in several fields, including engineering design; in a nutshell, each expert formulates an ordinal ranking of a set of alternatives and the resulting rankings should be aggregated into a collective one. Many aggregation models have been proposed in the literature, showing strengths and weaknesses, in line with the implications of Arrow's impossibility theorem. Furthermore, the coherence of the collective ranking with respect to the expert rankings may change depending on: (i) the expert rankings themselves and (ii) the aggregation model adopted. This paper assesses this coherence for a variety of aggregation models, through a recent test based on the Kendall's coefficient of concordance (W), and studies the characteristics of those models that are most likely to achieve higher coherence. Interestingly, the so-called Borda count model often provides best coherence, with some exceptions in the case of collective rankings with ties. The description is supported by practical examples

Aggregating multiple ordinal rankings in engineering design: the best model according to the Kendall’s coefficient of concordance

Aggregating the preferences of a group of experts is a recurring problem in several fields, including engineering design; in a nutshell, each expert formulates an ordinal ranking of a set of alternatives and the resulting rankings should be aggregated into a collective one. Many aggregation models have been proposed in the literature, showing strengths and weaknesses, in line with the implications of Arrow's impossibility theorem. Furthermore, the coherence of the collective ranking with respect to the expert rankings may change depending on: (i) the expert rankings themselves and (ii) the aggregation model adopted. This paper assesses this coherence for a variety of aggregation models, through a recent test based on the Kendall's coefficient of concordance (W), and studies the characteristics of those models that are most likely to achieve higher coherence. Interestingly, the so-called Borda count model often provides best coherence, with some exceptions in the case of collective rankings with ties. The description is supported by practical examples

Online Data Cleaning

Data-centric applications have never been more ubiquitous in our lives, e.g., search engines, route navigation and social media. This has brought along a new age where digital data is at the core of many decisions we make as individuals, e.g., looking for the most scenic route to plan a road trip, or as professionals, e.g., analysing customers’ transactions to predict the best time to restock different products. However, the surge in data generation has also led to creating massive amounts of dirty data, i.e., inaccurate or redundant data. Using dirty data to inform business decisions comes with dire consequences, for instance, an IBM report estimates that dirty data costs the U.S. $3.1 trillion a year. Dirty data is the product of many factors which include data entry errors and integration of several data sources. Data integration of multiple sources is especially prone to producing dirty data. For instance, while individual sources may not have redundant data, they often carry redundant data across each other. Furthermore, different data sources may obey different business rules (sometimes not even known) which makes it challenging to reconcile the integrated data. Even if the data is clean at the time of the integration, data updates would compromise its quality over time. There is a wide spectrum of errors that can be found in the data, e,g, duplicate records, missing values, obsolete data, etc. To address these problems, several data cleaning efforts have been proposed, e.g., record linkage to identify duplicate records, data fusion to fuse duplicate data items into a single representation and enforcing integrity constraints on the data. However, most existing efforts make two key assumptions: (1) Data cleaning is done in one shot; and (2) The data is available in its entirety. Those two assumptions do not hold in our age where data is highly volatile and integrated from several sources. This calls for a paradigm shift in approaching data cleaning: it has to be made iterative where data comes in chunks and not all at once. Consequently, cleaning the data should not be repeated from scratch whenever the data changes, but instead, should be done only for data items affected by the updates. Moreover, the repair should be computed effciently to support applications where cleaning is performed online (e.g. query time data cleaning). In this dissertation, we present several proposals to realize this paradigm for two major types of data errors: duplicates and integrity constraint violations. We frst present a framework that supports online record linkage and fusion over Web databases. Our system processes queries posted to Web databases. Query results are deduplicated, fused and then stored in a cache for future reference. The cache is updated iteratively with new query results. This effort makes it possible to perform record linkage and fusion effciently, but also effectively, i.e., the cache contains data items seen in previous queries which are jointly cleaned with incoming query results. To address integrity constraints violations, we propose a novel way to approach Functional Dependency repairs, develop a new class of repairs and then demonstrate it is superior to existing efforts, in runtime and accuracy. We then show how our framework can be easily tuned to work iteratively to support online applications. We implement a proof-ofconcept query answering system to demonstrate the iterative capability of our system

A Risk- and Fuzzy Set-Based Methodology for Advanced Concept Technology Demonstration Military Utility Assessment Design

The U.S. Department of Defense Advanced Concept Technology Demonstration (ACTD) and derivative, rapid acquisition programs offer timely solutions to critical military needs by assessing the utility of technologies mature enough to be fielded without application of traditional, defense system development processes. Military utility assessments (MUA) are ACTDs\u27 most critical features, but the lack of a standard for identifying assessment criteria tailored to specific demonstrations risks poorly informed acquisition decisions and the military operations those decisions are intended to support. The purpose of this research was to develop and deploy a methodology for identifying measures of effectiveness integral to advanced concept technology demonstration military utility assessment design. Within a context determined by attributes of complex systems, the research observed twin premises that ACTD assessment designs should accommodate: all risks possible when incorporating demonstration prototypes within superior and complex, joint military operations metasystems; and the ambiguities and other of what have been termed “fuzzy” manifestations of the cognition and language with which end-user, military operators craft and express perspectives required to identify measures of effectiveness fundamental to MUA designs. The effort pursued three research questions: (1) How might joint military operations metasystem models guide the identification of ACTD measures of effectiveness? (2) How might be developed and employed joint military metasystem models with which can be identified ACTD measures of effectiveness? (3) How useful might ACTD managers and analysts find the MUA design methodology developed and deployed with this research? The deployed methodology stimulated answers to these research questions by uniquely combining tailored versions of established risk assessment methods with a fuzzy method for resolving small group preferences. The risk assessment methods honored one research premise while enabling the identification and employment of a joint military operations metasystem model suited to MUA design needs of a simulated ACTD. The fuzzy preference method honored the second research premise as it, too, promoted metasystem model employment. The complete methodology was shown to hold favor with a large segment of a community expert in managing and assessing the utility of ACTDs emphasizing critical, joint military service needs