Health consensus: a digital adapted Delphi for healthcare

New tools are needed to facilitate the involvement of health professionals in healthcare participative processes, partially because a relevant segment of healthcare knowledge and decision-making is capillary distributed among them. A collaborative design strategy has been applied to the creation of an Internet tool to produce digitally adapted Delphi for healthcare purposes. During the period 2012-16 the prototype of the tool has been gradually improved through its application to 18 real cases. It is proposed the model Health Consensus as a digitally adapted Delphi supported by the various capabilities of Internet. The authors agree that Health Consensus is a useful and expandable tool for participative processes. The Internet provides several opportunities to overcome many of the limitations of conventional Delphi, as well as improving the final studies with new functionalities.Peer ReviewedPostprint (author's final draft

Playing Planning Poker in Crowds: Human Computation of Software Effort Estimates

Reliable cost effective effort estimation remains a considerable challenge for software projects. Recent work has demonstrated that the popular Planning Poker practice can produce reliable estimates when undertaken within a software team of knowledgeable domain experts. However, the process depends on the availability of experts and can be time-consuming to perform, making it impractical for large scale or open source projects that may curate many thousands of outstanding tasks. This paper reports on a full study to investigate the feasibility of using crowd workers supplied with limited information about a task to provide comparably accurate estimates using Planning Poker. We describe the design of a Crowd Planning Poker (CPP) process implemented on Amazon Mechanical Turk and the results of a substantial set of trials, involving more than 5000 crowd workers and 39 diverse software tasks. Our results show that a carefully organised and selected crowd of workers can produce effort estimates that are of similar accuracy to those of a single expert

On the application of artificial intelligence and human computation to the automation of agile software task effort estimation

Software effort estimation (SEE), as part of the wider project planning and product road mapping process, occurs throughout a software development life cycle. A variety of effort estimation methods have been proposed in the literature, including algorithmic methods, expert based methods, and more recently, methods based on techniques drawn from machine learning and natural language processing. In general, the consensus in the literature is that expert-based methods such as Planning Poker are more reliable than automated effort estimation. However, these methods are labour intensive and difficult to scale to large-scale projects. To address this limitation, this thesis investigates the feasibility of using human computation techniques to coordinate crowds of inexpert workers to predict expert-comparable effort estimates for a given software development task. The research followed an empirical methodology and used four different methods: literature review, replication, a series of laboratory experiments, and ethnography. The literature uncovered the lack of suitable datasets that include the attributes of descriptive text (corpus), actual cost, and expert estimates for a given software development task. Thus, a new dataset was developed to meet the necessary requirements. Next, effort estimation based on recent natural language processing advancements was evaluated and compared with expert estimates. The results suggest that there was no significant improvement, and the automated approach was still outperformed by expert estimates. Therefore, the feasibility of scaling the Planning Poker effort estimation method by using human computation in a micro-task crowdsourcing environment was explored. A series of pilot experiments were conducted to find the proper design for adapting Planning Poker to a crowd environment. This resulted in designing a new estimation method called Crowd Planning Poker (CPP). The pilot experiments revealed that a significant proportion of the crowd submitted poor quality assignments. Therefore, an approach to actively managing the quality of SEE work was proposed and evaluated before being integrated into the CPP method. A substantial overall evaluation was then conducted. The results demonstrated that crowd workers were able to discriminate between tasks of varying complexity and produce estimates that were comparable with those of experts and at substantially reduced cost compared with small teams of domain experts. It was further noted in the experiments that crowd workers provide useful insights as to the resolution of the task. Therefore, as a final step, fine-grained details about crowd workers’ behaviour, including actions taken and artifacts reviewed, were used in an ethnographic study to understand how crowd effort estimation takes place in a crowd. Four persona archetypes were developed to describe the crowd behaviours, and the results of the behaviour analysis were confirmed by surveying the crowd workers

Schätzwerterfüllung in Softwareentwicklungsprojekten

Effort estimates are of utmost economic importance in software development projects. Estimates bridge the gap between managers and the invisible and almost artistic domain of developers. They give a means to managers to track and control projects. Consequently, numerous estimation approaches have been developed over the past decades, starting with Allan Albrecht's Function Point Analysis in the late 1970s. However, this work neither tries to develop just another estimation approach, nor focuses on improving accuracy of existing techniques. Instead of characterizing software development as a technological problem, this work understands software development as a sociological challenge. Consequently, this work focuses on the question, what happens when developers are confronted with estimates representing the major instrument of management control? Do estimates influence developers, or are they unaffected? Is it irrational to expect that developers start to communicate and discuss estimates, conform to them, work strategically, hide progress or delay? This study shows that it is inappropriate to assume an independency of estimated and actual development effort. A theory is developed and tested, that explains how developers and managers influence the relationship between estimated and actual development effort. The theory therefore elaborates the phenomenon of estimation fulfillment.Schätzwerte in Softwareentwicklungsprojekten sind von besonderer ökonomischer Wichtigkeit. Sie überbrücken die Lücke zwischen Projektleitern und der unsichtbaren und beinahe künstlerischen Domäne der Entwickler. Sie stellen ein Instrument dar, welches erlaubt, Projekte zu verfolgen und zu kontrollieren. Daher wurden in den vergangenen vier Jahrzehnten diverse Schätzverfahren entwickelt, beginnend mit der "Function Point" Analyse von Allan Albrecht. Diese Arbeit versucht allerdings weder ein neues Schätzverfahren zu entwickeln noch bestehende Verfahren zu verbessern. Anstatt Softwareentwicklung als technologisches Problem zu charakterisieren, wird in dieser Arbeit eine soziologische Perspektive genutzt. Dementsprechend fokussiert diese Arbeit die Frage, was passiert, wenn Entwickler mit Schätzwerten konfrontiert werden, die das wichtigste Kontrollinstrument des Managements darstellen? Lassen sich Entwickler von diesen Werten beeinflussen oder bleiben sie davon unberührt? Wäre es irrational, zu erwarten, dass Entwickler Schätzwerte kommunizieren, diese diskutieren, sich diesen anpassen, strategisch arbeiten sowie Verzögerungen verschleiern? Die vorliegende Studie zeigt, dass die Unabhängigkeitsannahme von Schätzwerten und tatsächlichem Entwicklungsaufwand unbegründet ist. Es wird eine Theorie entwickelt, welche erklärt, wie Entwickler und Projektleiter die Beziehung von Schätzungen und Aufwand beeinflussen und dass das Phänomen der Schätzwerterfüllung auftreten kann

A framework for design assurance in developing embedded systems

Doctor of PhilosophyDepartment of Electrical and Computer EngineeringStephen A. DyerSteven WarrenEmbedded systems control nearly every device we encounter. Examples abound: appliances, scientific instruments, building environmental controls, avionics, communications, smart phones, and transportation subsystems. These embedded systems can fail in various ways: performance, safety, and meeting market needs. Design errors often cause failures in performance or safety. Market failures, particularly delayed schedule release or running over budget, arise from poor processes. Rigorous methods can significantly reduce the probability of failure. Industry has produced and widely published “best practices” that promote rigorous design and development of embedded systems. Unfortunately, 20 to 35% of development teams do not use them, which leads to operational failures or missed schedules and budgets. This dissertation increases the potential for success in designing and developing embedded systems through the following: 1. It identifies, through literature review, the reasons and factors that cause teams to avoid best practices, which in turn contribute to development failures. 2. It provides a framework, as a psychologically unbiased mediator, to help teams institute best practices. The framework is both straightforward to implement and use and simple to learn. 3. It examines the feasibility of both crowdsourcing and the Delphi method to aid, through anonymous comments on proposed projects, unbiased mediation and estimation within the framework. In two separate case studies, both approaches resulted in underestimation of both required time and required effort. The wide variance in the surveys’ results from crowdsourcing indicated that approach to not be particularly useful. On the other hand, convergence of estimates and forecasts in both projects resulted when employing the Delphi method. Both approaches required six or more weeks to obtain final results. 4. It develops a recommendation model, as a plug-in module to the framework, for the build-versus-buy decision in design of subsystems. It takes a description of a project, compares designing a custom unit with integrating a commercial unit into the final product, and generates a recommendation for the build-versus-buy decision. A study of 18 separate case studies examines the sensitivity of 14 parameters in making the build-versus-buy decision when developing embedded systems. Findings are as follows: team expertise and available resources are most important; partitioning tasks and reducing interdependence are next in importance; the quality and support of commercial units are less important; and finally, premiums and product lifecycles have the least effect on the cost of development. A recommendation model incorporates the results of the sensitivity study and successfully runs on 16 separate case studies. It shows the feasibility and features of a tool that can recommend a build-or-buy decision. 5. It develops a first-order estimation model as another plug-in module to the framework. It aids in planning the development of embedded systems. It takes a description of a project and estimates required time, required effort, and challenges associated with the project. It is simple to implement and easy to use; it can be a spreadsheet, a Matlab model or a webpage; each provides an output like the model for the build-versus-buy decision

A real time urban sustainability assessment framework for the smart city paradigm

Cities have proven to be a great source of concerns on their impact on the world environment and ecosystem. The objective, in a context where environmental concerns are growing rapidly, is no longer to develop liveable cities but to develop sustainable and responsive cities. This study investigates the currently available urban sustainability assessment (USA) schemes and outlines the main issues that the field is facing. After an extensive literature review, the author advocates for a scheme that would dynamically capture urban areas sustainability insights during their operation, a more user-centred and transparent scheme. The methodological approach has enabled the construction of a solid expertise on urban sustainability indicators, the essential role of the smart city and the Internet of Thing for a real-time key performance indicators determination and assessment, and technical and organisational challenges that such solution would encounter. Key domains such as sensing networks, remote sensing and GIS technologies, BIM technologies, Statistical databases and Open Governmental data platform, crowdsourcing and data mining that could support a real-time urban sustainability assessment have been studied. Additionally, the use of semantic web technologies has been investigated as a mean to deal with sources heterogeneity from diverse data structures and their interoperability. An USA ontology has been designed, integrating existing ontologies such as SSN, ifcOWL, cityGML and geoSPARQL. A web application back-end has then been built around this ontology. The application backbone is an Ontology-Based Data Access where a Relational Database is mapped to the USA ontology, enabling to link sensors data to pieces of information on the urban environment. Overall, this study has contributed to the body of knowledge by introducing an Ontology-Based Data Access (OBDA) approach to support real-time urban sustainability assessment leveraging sensors networks. It addresses both technical and organisational challenges that the smart systems domain is facing and is believed to be a valuable approach in the upcoming smart city paradigm. The solution proposed to tackle the research questions still faces some limitations such as a limited validation of the USA scheme, the OBDA limited intelligence, an improvable BIM and cityGML models conversion to RDF or the lack of user interface. Future work should be carried out to overcome those limitations and to provide stakeholders a high-hand service

Collective intelligence: creating a prosperous world at peace

XXXII, 612 p. ; 24 cmLibro ElectrónicoEn este documento se plantea un tema de interes general mas como lo es especificamente el tema de la evolucion de la sociedad en materia de industria y crecimiento de las actividades humanas en el aspecto de desarrollo de la creatividad enfocada a los mercadosedited by Mark Tovey ; foreword by Yochai Benkler (re-mixed by Hassan Masum) ; prefaces by Thomas Malone, Tom Atlee & Pierre Levy ; afterword by Paul Martin & Thomas Homer-Dixon.The era of collective intelligence has begun in earnest. While others have written about the wisdom of crowds, an army of Davids, and smart mobs, this collection of essays for the first time brings together fifty-five pioneers in the emerging discipline of collective intelligence. They provide a base of tools for connecting people, producing high-functioning teams, collaborating at multiple scales, and encouraging effective peer-production. Emerging models are explored for digital deliberative democracy, self-governance, legislative transparency, true-cost accounting, and the ethical use of open sources and methods. Collective Intelligence is the first of a series of six books, which will also include volumes on Peace Intelligence, Commercial Intelligence, Gift Intelligence, Cultural Intelligence, and Global Intelligence.Table of Contents Dedication i Publisher’s Preface iii Foreword by Yochai Benkler Remix Hassan Masum xi The Wealth of Networks: Highlights remixed Editor’s Preface xxi Table of Contents xxv A What is collective intelligence and what will we do 1 about it? (Thomas W. Malone, MIT Center for Collective Intelligence) B Co-Intelligence, collective intelligence, and conscious 5 evolution (Tom Atlee, Co-Intelligence Institute) C A metalanguage for computer augmented collective 15 intelligence (Prof. Pierre Lévy, Canada Research Chair in Collective Intelligence, FRSC) I INDIVIDUALS & GROUPS I-01 Foresight I-01-01 Safety Glass (Karl Schroeder, science fiction author 23 and foresight consultant) I-01-02 2007 State of the Future (Jerome C. Glenn & 29 Theodore J. Gordon, United Nations Millennium Project) I-02 Dialogue & Deliberation I-02-01 Thinking together without ego: Collective intelligence 39 as an evolutionary catalyst (Craig Hamilton and Claire Zammit, Collective-Intelligence.US) I-02-02 The World Café: Awakening collective intelligence 47 and committed action (Juanita Brown, David Isaacs and the World Café Community) I-02-03 Collective intelligence and the emergence of 55 wholeness (Peggy Holman, Nexus for Change, The Change Handbook) I-02-04 Knowledge creation in collective intelligence (Bruce 65 LaDuke, Fortune 500, HyperAdvance.com) I-02-05 The Circle Organization: Structuring for collective 75 wisdom (Jim Rough, Dynamic Facilitation & The Center for Wise Democracy) I-03 Civic Intelligence I-03-01 Civic intelligence and the public sphere (Douglas 83 Schuler, Evergreen State College, Public Sphere Project) I-03-02 Civic intelligence and the security of the homeland 95 (John Kesler with Carole and David Schwinn, IngeniusOnline) I-03-03 Creating a Smart Nation (Robert Steele, OSS.Net) 107 I-03-04 University 2.0: Informing our collective intelligence 131 (Nancy Glock-Grueneich, HIGHEREdge.org) I-03-05 Producing communities of communications and 145 foreknowledge (Jason “JZ” Liszkiewicz, Reconfigure.org) I-03-06 Global Vitality Report 2025: Learning to transform I-04 Electronic Communities & Distributed Cognition I-04-01 Attentional capital and the ecology of online social 163 conflict and think together effectively (Peter+Trudy networks (Derek Lomas, Social Movement Lab, Johnson-Lenz, Johnson-Lenz.com ) UCSD) I-04-02 A slice of life in my virtual community (Howard 173 Rheingold, Whole Earth Review, Author & Educator) I-04-03 Shared imagination (Dr. Douglas C. Engelbart, 197 Bootstrap) I-05 Privacy & Openness I-05-01 We’re all swimming in media: End-users must be able 201 to keep secrets (Mitch Ratcliffe, BuzzLogic & Tetriad) I-05-02 Working openly (Lion Kimbro, Programmer and 205 Activist) I-06 Integral Approaches & Global Contexts I-06-01 Meta-intelligence for analyses, decisions, policy, and 213 action: The Integral Process for working on complex issues (Sara Nora Ross, Ph.D. ARINA & Integral Review) I-06-02 Collective intelligence: From pyramidal to global 225 (Jean-Francois Noubel, The Transitioner) I-06-03 Cultivating collective intelligence: A core leadership 235 competence in a complex world (George Pór, Fellow at Universiteit van Amsterdam) II LARGE-SCALE COLLABORATION II-01 Altruism, Group IQ, and Adaptation II-01-01 Empowering individuals towards collective online 245 production (Keith Hopper, KeithHopper.com) II-01-02 Who’s smarter: chimps, baboons or bacteria? The 251 power of Group IQ (Howard Bloom, author) II-01-03 A collectively generated model of the world (Marko 261 A. Rodriguez, Los Alamos National Laboratory) II-02 Crowd Wisdom and Cognitive Bias II-02-01 Science of CI: Resources for change (Norman L 265 Johnson, Chief Scientist at Referentia Systems, former LANL) II-02-02 Collectively intelligent systems (Jennifer H. Watkins, 275 Los Alamos National Laboratory) II-02-03 A contrarian view (Jaron Lanier, scholar-in-residence, 279 CET, UC Berkeley & Discover Magazine) II-03 Semantic Structures & The Semantic Web II-03-01 Information Economy Meta Language (Interview with 283 Professor Pierre Lévy, by George Pór) II-03-02 Harnessing the collective intelligence of the World- 293 Wide Web (Nova Spivack, RadarNetworks, Web 3.0) II-03-03 The emergence of a global brain (Francis Heylighen, 305 Free University of Brussels) II-04 Information Networks II-04-01 Networking and mobilizing collective intelligence (G. Parker Rossman, Future of Learning Pioneer) II-04-02 Toward high-performance organizations: A strategic 333 role for Groupware (Douglas C. Engelbart, Bootstrap) II-04-03 Search panacea or ploy: Can collective intelligence 375 improve findability? (Stephen E. Arnold, Arnold IT, Inc.) II-05 Global Games, Local Economies, & WISER II-05-01 World Brain as EarthGame (Robert Steele and many 389 others, Earth Intelligence Network) II-05-02 The Interra Project (Jon Ramer and many others) 399 II-05-03 From corporate responsibility to Backstory 409 Management (Alex Steffen, Executive Editor, Worldchanging.com) II-05-04 World Index of Environmental & Social 413 Responsibility (WISER) By the Natural Capital Institute II-06 Peer-Production & Open Source Hardware II-06-01 The Makers’ Bill of Rights (Jalopy, Torrone, and Hill) 421 II-06-02 3D Printing and open source design (James Duncan, 423 VP of Technology at Marketingisland) II-06-03 REBEARTHTM: 425 II-07 Free Wireless, Open Spectrum, and Peer-to-Peer II-07-01 Montréal Community Wi-Fi (Île Sans Fil) (Interview 433 with Michael Lenczner by Mark Tovey) II-07-02 The power of the peer-to-peer future (Jock Gill, 441 Founder, Penfield Gill Inc.) Growing a world 6.6 billion people would want to live in (Marc Stamos, B-Comm, LL.B) II-07-03 Open spectrum (David Weinberger) II-08 Mass Collaboration & Large-Scale Argumentation II-08-01 Mass collaboration, open source, and social 455 entrepreneurship (Mark Tovey, Advanced Cognitive Engineering Lab, Institute of Cognitive Science, Carleton University) II-08-02 Interview with Thomas Homer-Dixon (Hassan 467 Masum, McLaughlin-Rotman Center for Global Health) II-08-03 Achieving collective intelligence via large-scale argumentation (Mark Klein, MIT Center for Collective Intelligence) II-08-04 Scaling up open problem solving (Hassan Masum & 485 Mark Tovey) D Afterword: The Internet and the revitalization of 495 democracy (The Rt. Honourable Paul Martin & Thomas Homer-Dixon) E Epilogue by Tom Atlee 513 F Three Lists 515 1. Strategic Reading Categories 2. Synopsis of the New Progressives 3. Fifty-Two Questions that Matter G Glossary 519 H Index 52