PaPaS: A Portable, Lightweight, and Generic Framework for Parallel Parameter Studies

The current landscape of scientific research is widely based on modeling and simulation, typically with complexity in the simulation's flow of execution and parameterization properties. Execution flows are not necessarily straightforward since they may need multiple processing tasks and iterations. Furthermore, parameter and performance studies are common approaches used to characterize a simulation, often requiring traversal of a large parameter space. High-performance computers offer practical resources at the expense of users handling the setup, submission, and management of jobs. This work presents the design of PaPaS, a portable, lightweight, and generic workflow framework for conducting parallel parameter and performance studies. Workflows are defined using parameter files based on keyword-value pairs syntax, thus removing from the user the overhead of creating complex scripts to manage the workflow. A parameter set consists of any combination of environment variables, files, partial file contents, and command line arguments. PaPaS is being developed in Python 3 with support for distributed parallelization using SSH, batch systems, and C++ MPI. The PaPaS framework will run as user processes, and can be used in single/multi-node and multi-tenant computing systems. An example simulation using the BehaviorSpace tool from NetLogo and a matrix multiply using OpenMP are presented as parameter and performance studies, respectively. The results demonstrate that the PaPaS framework offers a simple method for defining and managing parameter studies, while increasing resource utilization.Comment: 8 pages, 6 figures, PEARC '18: Practice and Experience in Advanced Research Computing, July 22--26, 2018, Pittsburgh, PA, US

Teaching rule‐based algorithmic composition: the PWGL library cluster rules

This paper presents software suitable for undergraduate students to implement computer programs that compose music. The software offers a low floor (students easily get started) but also a high ceiling (complex compositional theories can be modelled). Our students are particularly interested in tonal music: such aesthetic preferences are supported, without stylistically restricting users of the software. We use a rule‐based approach (constraint programming) to allow for great flexibility. Our software Cluster Rules implements a collection of compositional rules on rhythm, harmony, melody, and counterpoint for the new music constraint system Cluster Engine by Örjan Sandred. The software offers a low floor by observing several guidelines. The programming environment uses visual programming (Cluster Rules and Cluster Engine extend the algorithmic composition system PWGL). Further, music theory definitions follow a template, so students can learn from examples how to create their own definitions. Finally, students are offered a collection of predefined rules, which they can freely combine in their own definitions. Music Technology students, including students without any prior computer programming experience, have successfully used the software. Students used the musical results of their computer programs to create original compositions. The software is also interesting for postgraduate students, composers and researchers. Complex polyphonic constraint problems are supported (high ceiling). Users can freely define their own rules and combine them with predefined rules. Also, Cluster Engine’s efficient search algorithm makes advanced problems solvable in practice

Automatic coding of short text responses via clustering in educational assessment

Automatic coding of short text responses opens new doors in assessment. We implemented and integrated baseline methods of natural language processing and statistical modelling by means of software components that are available under open licenses. The accuracy of automatic text coding is demonstrated by using data collected in the Programme for International Student Assessment (PISA) 2012 in Germany. Free text responses of 10 items with Formula responses in total were analyzed. We further examined the effect of different methods, parameter values, and sample sizes on performance of the implemented system. The system reached fair to good up to excellent agreement with human codings Formula Especially items that are solved by naming specific semantic concepts appeared properly coded. The system performed equally well with Formula and somewhat poorer but still acceptable down to Formula Based on our findings, we discuss potential innovations for assessment that are enabled by automatic coding of short text responses. (DIPF/Orig.

On the adequacy of current empirical evaluations of formal models of categorization

Categorization is one of the fundamental building blocks of cognition, and the study of categorization is notable for the extent to which formal modeling has been a central and influential component of research. However, the field has seen a proliferation of noncomplementary models with little consensus on the relative adequacy of these accounts. Progress in assessing the relative adequacy of formal categorization models has, to date, been limited because (a) formal model comparisons are narrow in the number of models and phenomena considered and (b) models do not often clearly define their explanatory scope. Progress is further hampered by the practice of fitting models with arbitrarily variable parameters to each data set independently. Reviewing examples of good practice in the literature, we conclude that model comparisons are most fruitful when relative adequacy is assessed by comparing well-defined models on the basis of the number and proportion of irreversible, ordinal, penetrable successes (principles of minimal flexibility, breadth, good-enough precision, maximal simplicity, and psychological focus)