Hindustani raga and singer classification using 2D and 3D pose estimation from video recordings

Using pose estimation with video recordings, we apply an action recognition machine learning algorithm to demonstrate the use of the movement information to classify singers and the ragas (melodic modes) they perform. Movement information is derived from a specially recorded video dataset of solo Hindustani (North Indian) raga recordings by three professional singers each performing the same nine ragas, a smaller duo dataset (one singer with tabla accompaniment) as well as recordings of concert performances by the same singers. Data is extracted using pose estimation algorithms, both 2D (OpenPose) and 3D. A two-pathway convolutional neural network structure is proposed for skeleton action recognition to train a model to classify 12-second clips by singer and raga. The model is capable of distinguishing the three singers on the basis of movement information alone. For each singer, it is capable of distinguishing between the nine ragas with a mean accuracy of 38.2% (with the most successful model). The model trained on solo recordings also proved effective at classifying duo and concert recordings. These findings are consistent with the view that while the gesturing of Indian singers is idiosyncratic, it remains tightly linked to patterns of melodic movement: indeed we show that in some cases different ragas are distinguishable on the basis of movement information alone. A series of technical challenges are identified and addressed, with code shared alongside audiovisual data to accompany the paper

What do we (not) know about research software engineering?

As recognition of the vital importance of software for contemporary research is increasing, Research Software Engineering (RSE) is emerging as a discipline in its own right. We present an inventory of relevant research questions about RSE as a basis for future research and initiatives to advance the field, highlighting selected literature and initiatives. This work is the outcome of a RSE community workshop held as part of the 2020 International Series of Online Research Software Events (SORSE) which identified and prioritized key questions across three overlapping themes: people, policy and infrastructure. Almost half of the questions focus on the people theme, which addresses issues related to career paths, recognition and motivation; recruitment and retention; skills; and diversity, equity and inclusion. However, the people and policy themes have the same number of prioritized questions. We recommend that different types of stakeholders, such as RSE employers and policy makers, take responsibility for supporting or encouraging answering of these questions by organizations that have an interest. Initiatives such as the International Council of RSE Associations should also be engaged in this work

What Do We (Not) Know About Research Software Engineering?

As recognition of the vital importance of software for contemporary research is increasing, Research Software Engineering (RSE) is emerging as a discipline in its own right. We present an inventory of relevant research questions about RSE as a basis for future research and initiatives to advance the field, highlighting selected literature and initiatives. This work is the outcome of a RSE community workshop held as part of the 2020 International Series of Online Research Software Events (SORSE) which identified and prioritized key questions across three overlapping themes: people, policy and infrastructure. Almost half of the questions focus on the people theme, which addresses issues related to career paths, recognition and motivation; recruitment and retention; skills; and diversity, equity and inclusion. However, the people and policy themes have the same number of prioritized questions. We recommend that different types of stakeholders, such as RSE employers and policy makers, take responsibility for supporting or encouraging answering of these questions by organizations that have an interest. Initiatives such as the International Council of RSE Associations should also be engaged in this work

Quasi-matrix-free hybrid multigrid on dynamically adaptive Cartesian grids

We present a family of spacetree-based multigrid realizations using the tree’s multiscale nature to derive coarse grids. They align with matrix-free geometric multigrid solvers as they never assemble the system matrices, which is cumbersome for dynamically adaptive grids and full multigrid. The most sophisticated realizations use BoxMG to construct operator-dependent prolongation and restriction in combination with Galerkin/Petrov-Galerkin coarse-grid operators. This yields robust solvers for nontrivial elliptic problems. We embed the algebraic, problem-dependent, and grid-dependent multigrid operators as stencils into the grid and evaluate all matrix-vector products in situ throughout the grid traversals. Such an approach is not literally matrix-free as the grid carries the matrix. We propose to switch to a hierarchical representation of all operators. Only differences of algebraic operators to their geometric counterparts are held. These hierarchical differences can be stored and exchanged with small memory footprint. Our realizations support arbitrary dynamically adaptive grids while they vertically integrate the multilevel operations through spacetree linearization. This yields good memory access characteristics, while standard colouring of mesh entities with domain decomposition allows us to use parallel many-core clusters. All realization ingredients are detailed such that they can be used by other codes

Hybrid Geometric-Algebraic Matrix-Free Multigrid on Spacetrees

Linear solvers are the motor for many computer simulations that are based on partial differential equations (PDEs). For a wide range of problems, multigrid solvers belong to the most efficient ones. Their convergence rate is mostly independent of the mesh size of the underlying problem discretisation, and thus they have optimal complexity.During the last two decades, there has been a strong focus on algebraic multigrid, which can easily employ accurate unstructured grids and is very robust. But increased accuracy requirements, complex models, and huge amounts of data from engineering, scientific or, e.g., medical applications require the use of supercomputers. Their architecture forces researchers to rethink their algorithms and data handling. For example, algebraic multigrid suffers from a serious performance decrease on parallel architectures, due to high setup costs and communication overhead, unstructured data access, indirect addressing, and a large memory footprint. Therefore, the less robust geometric multigrid is now reconsidered. For the data, spacetrees have turned out to not only provide fast data access but also an efficient structure for performing the computations.This work combines the advantages of geometric and algebraic multigrid. It defines the solver on a geometrically coarsened structured grid, but instead of geometric multigrid operations the much more robust BoxMG by Dendy using operator dependent intergrid transfer operators and Petrov-Galerkin coarse-grid operators is used. The solver is implemented on a spacetree as underlying data- and computational structure, ensuring efficient data handling, data locality, and low communication overhead. It is integrated into and parallelised in the PDE solver framework Peano, which has a small memory footprint and is very memory-efficient. This results in a robust solver that is tailored to high performance computer

Impact of Non-potential Coronal Boundary Conditions on Solar Wind Prediction

Predictions of the solar wind at Earth are a central aspect of space weather prediction. The outcome of such a prediction, however, is highly sensitive to the method used for computing the magnetic field in the corona. We analyze the impact of replacing the potential field coronal boundary conditions, as used in operational space weather prediction tools, by non-potential conditions. For this, we compare the predicted solar wind plasma parameters with observations at 1 AU for two six-months intervals, one at solar maximum and one in the descending phase of the current cycle. As a baseline, we compare with the operational Wang-Sheeley-Arge model. We find that for solar maximum, the non-potential coronal model and an adapted solar wind speed formula lead to the best solar wind predictions in a statistical sense. For the descending phase, the potential coronal model performs best. The Wang-Sheeley-Arge model outperforms the others in predicting high speed enhancements and streamer interactions. A better parameter fitting for the adapted wind speed formula is expected to improve the performance of the non-potential model here

The Possible Impact of L5 Magnetograms on Non-Potential Solar Coronal Magnetic Field Simulations

The proposed Carrington-L5 mission would bring instruments to the L5 Lagrange point to provide us with crucial data for space weather prediction. To assess the importance of including a magnetograph, we consider the possible differences in non-potential solar coronal magnetic field simulations when magnetograph observations are available from the L5 point, compared with an L1-based field of view (FOV). A timeseries of synoptic radial magnetic field maps is constructed to capture the emergence of two active regions from the L5 FOV. These regions are initially absent in the L1 magnetic field maps, but are included once they rotate into the L1 FOV. Non-potential simulations for these two sets of input data are compared in detail. Within the bipolar active regions themselves, differences in the magnetic field structure can exist between the two simulations once the active regions are included in both. These differences tend to reduce within 5 days of the active region being included in L1. The delayed emergence in L1 can, however, lead to significant persistent differences in long-range connectivity between the active regions and the surrounding fields, and also in the global magnetic energy. In particular, the open magnetic flux and the location of open magnetic footpoints, are sensitive to capturing the real-time of emergence. These results suggest that a magnetograph at L5 could significantly improve predictions of the non-potential corona, the interplanetary magnetic field, and of solar wind source regions on the Sun