Key performance indicators for successful simulation projects

There are many factors that may contribute to the successful delivery of a simulation project. To provide a structured approach to assessing the impact various factors have on project success, we propose a top-down framework whereby 15 Key Performance Indicators (KPI) are developed that represent the level of successfulness of simulation projects from various perspectives. They are linked to a set of Critical Success Factors (CSF) as reported in the simulation literature. A single measure called Project’s Success Measure (PSM), which represents the project’s total success level, is proposed. The framework is tested against 9 simulation exemplar cases in healthcare and this provides support for its reliability. The results suggest that responsiveness to the customer’s needs and expectations, when compared with other factors, holds the strongest association with the overall success of simulation projects. The findings highlight some patterns about the significance of individual CSFs, and how the KPIs are used to identify problem areas in simulation projects.This study was supported by the Multidisciplinar Assessment of Technology Centre for Healthcare (MATCH) program (EPSRC Grant EP/F063822/1)

Key performance indicators for successful simulation projects

There are many factors that may contribute to the successful delivery of a simulation project. To provide a structured approach to assessing the impact various factors have on project success, we propose a top-down framework whereby 15 Key Performance Indicators (KPI) are developed that represent the level of successfulness of simulation projects from various perspectives. They are linked to a set of Critical Success Factors (CSF) as reported in the simulation literature. A single measure called Project’s Success Measure (PSM), which represents the project’s total success level, is proposed. The framework is tested against 9 simulation exemplar cases in healthcare and this provides support for its reliability. The results suggest that responsiveness to the customer’s needs and expectations, when compared with other factors, holds the strongest association with the overall success of simulation projects. The findings highlight some patterns about the significance of individual CSFs, and how the KPIs are used to identify problem areas in simulation projects

Rekomendasi Pengukuran Kinerja Menggunakan Information Technology Balanced Scorecard dan Strategy Maps

Dinas Komunikasi dan Informatika (DINKOMINFO) berkewajiban untuk mensukseskan Visi dan Misi Kabupaten Wonosobo yang termuat dalam RPJMD Tahun 2016-2012. Mensukseskan Visi dan Misi DINKOMINFO perlu adanya implementasi TI yang dikelola dengan baik dan peningkatan kinerja dari DINKOMINFO Kabupaten Wonosobo untuk mendukung perkembangan TI. Permasalahan yang ada DINKOMINFO Wonosobo dalam Rancangan Strategis (RENSTRA) 2016-2021 belum dispesifikasikan kebutuhan dan strategi dari setiap bidang khususnya bidang Informatika serta belum adanya penilaian kontribusi TI terhadap bisnis dan belum adanya penilaian pengukuran kinerja TI, maka perlu dilakukan rekomendasi pengukuran kinerja untuk menspesifikasikan kebutuhan serta strategi dan dapat mengetahui kontribusi TI terhadap bisnis dengan menggunakan Information Technology Balanced Scorecard untuk menyesuaikan perencanaan dan aktivitas-aktivitas sistem informasi yang dapat mengukur, mengevaluasi efektivitas organisasi sistem informasi, mendorong dan mempertahankan kinerja sistem informasi yang semakin meningkat. Tujuan dari penelitian ini menghasilkan rekomendasi pengukuran kinerja menggunakan IT Balanced Scorecard dari hasil analisis matriks SWOT, PEST dan CSF yang akan menghasilkan sasaran – sasaran untuk membentuk strategy maps. Penelitian ini menghasilkan 10 sasaran strategis yang dipetakan kedalam 4 Perspective IT Balanced Scorecard sebagai perancangan pengukuran kinerja IT Balanced Scorecard DINKOMINFO Kabupaten Wonosobo. Rekomendasi pengukuran kinerja DINKOMINFO Kabupaten Wonosobo hasil dari IT Balanced Scorecard adalah 15 KPI (Key Performance Indicators) yang dijadikan sebagai nilai ketercapaian program untuk mengetahui hasil pengukuran kinerja DINKOMINFO Kabupaten Wonosobo. Hasil 10 sasaran strategis dibentuk ke dalam strategy maps diterjemahkan ke dalam hubungan sebab akibat yang menggambarkan kondisi dari DINKOMINFO Kabupaten Wonosobo

Using LIWC to choose simulation approaches: A feasibility study

Can language usage help determine which model approach is best suited to provide decision makers with desired insights? This research addresses that question through an investigation of Linguistic Inquiry and Word Count (LIWC), which calculates the presence of more than 80 language dimensions in text samples, and permits construction of custom dictionaries. This article demonstrates use of LIWC to ensure better problem/model fit within the context of selecting a decision support tool. We selected two simulation tools as research instruments to investigate a broader question on the usefulness of LIWC to guide choice of DSS tool. The tools selected were System Dynamics (SD) and Discrete Event Simulation (DES). First, we tested LIWC to analyze practitioners’ language use when developing models. LIWC pointed out significant linguistic differences consistent with prior theoretical work, based on model development approach in a number of dimensions. These differences provided a basis for developing a custom dictionary for use on the second part of our study. The second part of the study focused on language used by decision makers in problem statements and used the linguistic clues identified in the first part of the study to ensure problem/model fit. Results indicated problem statements contained linguistic clues related to the type of information desired by problem solvers. The article concludes with a discussion about how LIWC and similar tools can help determine which DSS tools are suited to particular applications

A framework for characterising and evaluating the effectiveness of environmental modelling

Environmental modelling is transitioning from the traditional paradigm that focuses on the model and its quantitative performance to a more holistic paradigm that recognises successful model-based outcomes are closely tied to undertaking modelling as a social process, not just as a technical procedure. This paper redefines evaluation as a multi-dimensional and multi-perspective concept, and proposes a more complete framework for identifying and measuring the effectiveness of modelling that serves the new paradigm. Under this framework, evaluation considers a broader set of success criteria, and emphasises the importance of contextual factors in determining the relevance and outcome of the criteria. These evaluation criteria are grouped into eight categories: project efficiency, model accessibility, credibility, saliency, legitimacy, satisfaction, application, and impact. Evaluation should be part of an iterative and adaptive process that attempts to improve model-based outcomes and foster pathways to better futures

Utilization of Discrete Event Simulation in Business Processes Management Projects: a Literature Review

Discrete event simulation (DES) software is often used in business process management (BPM) projects, in order to support modeling, change and automatization of process modeling. However, the current research lacks systematic overview of the benefits and pitfalls of using DES software for BPM. The goal of this paper is to provide a systematic analysis of research paper related to the operations research, computer science, business and management that report the usage of DES in BPM projects, in order to shed some light in this area. Results of this analysis could be useful to academics and business practitioners in their projects

Facets of trust in simulation studies

This is the author accepted manuscript. The final version is available on open access from Elsevier via the DOI in this recordThe purpose of a modelling and simulation (M&S) study for real-world operations management applications is to support decision-making and inform potential action, therefore investigating the aspects of the modelling process which influence trust is important. Previous work has considered the question of trust through the lens of model validation. However, whilst a simulation model may be technically well executed, stakeholders’ trust in the results may also depend upon intangible factors such as interpersonal relationships. Existing literature has also focused on the credibility of the simulation practitioner, however the credibility attribute belongs to the stakeholder, and it ignores the trust aspects that may exist between the stakeholders and the model itself. In this paper, we argue that different facets of trust emerge throughout the stages of a simulation study, and both influence, and are influenced by, the interaction between the model, the modeller and the stakeholders of the study. We present a synthesis of existing literature and extend it by proposing a formative model of trust which presents a conceptualisation of this tripartite relationship. Our contribution is the identification of the different facets of trust in the lifecycle of a modelling and simulation study. We argue that these interacting facets converge via the three-way relationship between modeller, model and stakeholders toward epistemic trust in the knowledge generated by the simulation study and ultimately model acceptability and implementation. To the best of our knowledge, ours is the first study that focuses solely on the question of trust in an M&S study.Economic and Social Research Council (ESRC

Apuramento do custo e medidas de desempenho no apoio à gestão de uma organização social

As organizações sem fins lucrativos, assim como as restantes organizações, necessitam de ferramentas de gestão que se adaptem à sua realidade de modo a alcançarem um equilíbrio económico sustentável. O facto deste tipo de organização enfrentar vários obstáculos, como a escassez de recursos financeiros, faz com que este tema seja oportuno na medida em que pode ajudar a atenuar os problemas de sustentabilidade económica, através da implementação de uma análise de custo adequada e da utilização de ferramentas de gestão e suporte à decisão, nomeadamente, os indicadores de desempenho. O presente trabalho apresenta o resultado de uma análise dos custos relativa aos utentes de uma organização sem fins lucrativos, a Santa Casa da Misericórdia de Lamego, tendo por base a contabilidade analítica, e viabilizada pela análise das demonstrações financeiras e outros documentos da Instituição. Esta análise será seguida de uma avaliação do desempenho organizacional realizada através do estudo de indicadores de desempenho que são também uma importante ferramenta de apoio à gestão e ainda a sua apresentação através de um painel de visualização, o dashboard. Os desenvolvimentos propostos pretendem dar um contributo para apoiar a gestão da Instituição e, consequentemente, o seu desempenho organizacional. Os resultados finais falam por si na medida em que o apuramento do custo médio por utente e a aplicação de indicadores de desempenho, demonstrados visualmente em dashboards, se apresentam como uma mais-valia para esta Instituição ao nível do apoio à tomada de decisão

A Hybrid Modelling Framework for Real-time Decision-support for Urgent and Emergency Healthcare

In healthcare, opportunities to use real-time data to support quick and effective decision-making are expanding rapidly, as data increases in volume, velocity and variety. In parallel, the need for short-term decision-support to improve system resilience is increasingly relevant, with the recent COVID-19 crisis underlining the pressure that our healthcare services are under to deliver safe, effective, quality care in the face of rapidly-shifting parameters. A real-time hybrid model (HM) which combines real-time data, predictions, and simulation, has the potential to support short-term decision-making in healthcare. Considering decision-making as a consequence of situation awareness focuses the HM on what information is needed where, when, how, and by whom with a view toward sustained implementation. However the articulation between real-time decision-support tools and a sociotechnical approach to their development and implementation is currently lacking in the literature. Having identified the need for a conceptual framework to support the development of real-time HMs for short-term decision-support, this research proposed and tested the Integrated Hybrid Analytics Framework (IHAF) through an examination of the stages of a Design Science methodology and insights from the literature examining decision-making in dynamic, sociotechnical systems, data analytics, and simulation. Informed by IHAF, a HM was developed using real-time Emergency Department data, time-series forecasting, and discrete-event simulation. The application started with patient questionnaires to support problem definition and to act as a formative evaluation, and was subsequently evaluated using staff interviews. Evaluation of the application found multiple examples where the objectives of people or sub-systems are not aligned, resulting in inefficiencies and other quality problems, which are characteristic of complex adaptive sociotechnical systems. Synthesis of the literature, the formative evaluation, and the final evaluation found significant themes which can act as antecedents or evaluation criteria for future real-time HM studies in sociotechnical systems, in particular in healthcare. The generic utility of IHAF is emphasised for supporting future applications in similar domains