13 research outputs found
Music Synchronization, Audio Matching, Pattern Detection, and User Interfaces for a Digital Music Library System
Over the last two decades, growing efforts to digitize our cultural heritage could be observed. Most of these digitization initiatives pursuit either one or both of the following goals: to conserve the documents - especially those threatened by decay - and to provide remote access on a grand scale. For music documents these trends are observable as well, and by now several digital music libraries are in existence. An important characteristic of these music libraries is an inherent multimodality resulting from the large variety of available digital music representations, such as scanned score, symbolic score, audio recordings, and videos. In addition, for each piece of music there exists not only one document of each type, but many. Considering and exploiting this multimodality and multiplicity, the DFG-funded digital library initiative PROBADO MUSIC aimed at developing a novel user-friendly interface for content-based retrieval, document access, navigation, and browsing in large music collections. The implementation of such a front end requires the multimodal linking and indexing of the music documents during preprocessing. As the considered music collections can be very large, the automated or at least semi-automated calculation of these structures would be recommendable. The field of music information retrieval (MIR) is particularly concerned with the development of suitable procedures, and it was the goal of PROBADO MUSIC to include existing and newly developed MIR techniques to realize the envisioned digital music library system. In this context, the present thesis discusses the following three MIR tasks: music synchronization, audio matching, and pattern detection. We are going to identify particular issues in these fields and provide algorithmic solutions as well as prototypical implementations. In Music synchronization, for each position in one representation of a piece of music the corresponding position in another representation is calculated. This thesis focuses on the task of aligning scanned score pages of orchestral music with audio recordings. Here, a previously unconsidered piece of information is the textual specification of transposing instruments provided in the score. Our evaluations show that the neglect of such information can result in a measurable loss of synchronization accuracy. Therefore, we propose an OCR-based approach for detecting and interpreting the transposition information in orchestral scores. For a given audio snippet, audio matching methods automatically calculate all musically similar excerpts within a collection of audio recordings. In this context, subsequence dynamic time warping (SSDTW) is a well-established approach as it allows for local and global tempo variations between the query and the retrieved matches. Moving to real-life digital music libraries with larger audio collections, however, the quadratic runtime of SSDTW results in untenable response times. To improve on the response time, this thesis introduces a novel index-based approach to SSDTW-based audio matching. We combine the idea of inverted file lists introduced by Kurth and MĂźller (Efficient index-based audio matching, 2008) with the shingling techniques often used in the audio identification scenario. In pattern detection, all repeating patterns within one piece of music are determined. Usually, pattern detection operates on symbolic score documents and is often used in the context of computer-aided motivic analysis. Envisioned as a new feature of the PROBADO MUSIC system, this thesis proposes a string-based approach to pattern detection and a novel interactive front end for result visualization and analysis
An investigation into weighted data fusion for content-based multimedia information retrieval
Content Based Multimedia Information Retrieval (CBMIR) is characterised by the combination of noisy sources of information which, in unison, are able to achieve strong performance. In this thesis we focus on the combination of ranked results from the independent retrieval experts which comprise a CBMIR system through linearly weighted data fusion. The independent retrieval experts are low-level multimedia features, each of which contains an indexing function and ranking algorithm. This thesis is comprised of two halves. In the ďŹrst half, we perform a rigorous empirical investigation into the factors which impact upon performance in linearly weighted data fusion. In the second half, we leverage these ďŹnding to create a new class of weight generation algorithms for data fusion which are
capable of determining weights at query-time, such that the weights are topic dependent
Handbook of Digital Face Manipulation and Detection
This open access book provides the first comprehensive collection of studies dealing with the hot topic of digital face manipulation such as DeepFakes, Face Morphing, or Reenactment. It combines the research fields of biometrics and media forensics including contributions from academia and industry. Appealing to a broad readership, introductory chapters provide a comprehensive overview of the topic, which address readers wishing to gain a brief overview of the state-of-the-art. Subsequent chapters, which delve deeper into various research challenges, are oriented towards advanced readers. Moreover, the book provides a good starting point for young researchers as well as a reference guide pointing at further literature. Hence, the primary readership is academic institutions and industry currently involved in digital face manipulation and detection. The book could easily be used as a recommended text for courses in image processing, machine learning, media forensics, biometrics, and the general security area
Handbook of Digital Face Manipulation and Detection
This open access book provides the first comprehensive collection of studies dealing with the hot topic of digital face manipulation such as DeepFakes, Face Morphing, or Reenactment. It combines the research fields of biometrics and media forensics including contributions from academia and industry. Appealing to a broad readership, introductory chapters provide a comprehensive overview of the topic, which address readers wishing to gain a brief overview of the state-of-the-art. Subsequent chapters, which delve deeper into various research challenges, are oriented towards advanced readers. Moreover, the book provides a good starting point for young researchers as well as a reference guide pointing at further literature. Hence, the primary readership is academic institutions and industry currently involved in digital face manipulation and detection. The book could easily be used as a recommended text for courses in image processing, machine learning, media forensics, biometrics, and the general security area
Organising and structuring a visual diary using visual interest point detectors
As wearable cameras become more popular, researchers are increasingly focusing on novel applications to manage the large volume of data these devices produce. One such application is the construction of a Visual Diary from an individualâs photographs. Microsoftâs SenseCam, a
device designed to passively record a Visual Diary and cover a typical day of the user wearing the camera, is an example of one such device. The vast quantity of images generated by these devices means that the management and organisation of these collections is not a trivial matter.
We believe wearable cameras, such as SenseCam, will become more popular in the future and the management of the volume of data generated by these devices is a key issue.
Although there is a significant volume of work in the literature in the object detection and recognition
and scene classification fields, there is little work in the area of setting detection. Furthermore, few authors have examined the issues involved in analysing extremely large image collections (like a Visual Diary) gathered over a long period of time. An algorithm developed for setting
detection should be capable of clustering images captured at the same real world locations (e.g. in the dining room at home, in front of the computer in the office, in the park, etc.). This requires the selection and implementation of suitable methods to identify visually similar backgrounds in images using their visual features. We present a number of approaches to setting detection based on
the extraction of visual interest point detectors from the images. We also analyse the performance of two of the most popular descriptors - Scale Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF).We present an implementation of a Visual Diary application and evaluate
its performance via a series of user experiments. Finally, we also outline some techniques to allow the Visual Diary to automatically detect new settings, to scale as the image collection continues to grow substantially over time, and to allow the user to generate a personalised summary of their data
Security Analysis of System Behaviour - From "Security by Design" to "Security at Runtime" -
The Internet today provides the environment for novel applications and
processes which may evolve way beyond pre-planned scope and
purpose. Security analysis is growing in complexity with the increase
in functionality, connectivity, and dynamics of current electronic
business processes. Technical processes within critical
infrastructures also have to cope with these developments. To tackle
the complexity of the security analysis, the application of models is
becoming standard practice. However, model-based support for security
analysis is not only needed in pre-operational phases but also during
process execution, in order to provide situational security awareness
at runtime.
This cumulative thesis provides three major contributions to modelling
methodology.
Firstly, this thesis provides an approach for model-based analysis and
verification of security and safety properties in order to support
fault prevention and fault removal in system design or redesign.
Furthermore, some construction principles for the design of
well-behaved scalable systems are given.
The second topic is the analysis of the exposition of vulnerabilities
in the software components of networked systems to exploitation by
internal or external threats. This kind of fault forecasting allows
the security assessment of alternative system configurations and
security policies. Validation and deployment of security policies
that minimise the attack surface can now improve fault tolerance and
mitigate the impact of successful attacks.
Thirdly, the approach is extended to runtime applicability. An
observing system monitors an event stream from the observed system
with the aim to detect faults - deviations from the specified
behaviour or security compliance violations - at runtime.
Furthermore, knowledge about the expected behaviour given by an
operational model is used to predict faults in the near
future. Building on this, a holistic security management strategy is
proposed. The architecture of the observing system is described and
the applicability of model-based security analysis at runtime is
demonstrated utilising processes from several industrial scenarios.
The results of this cumulative thesis are provided by 19 selected
peer-reviewed papers
Research on Teaching and Learning In Biology, Chemistry and Physics In ESERA 2013 Conference
This paper provides an overview of the topics in educational research that were published in the ESERA 2013 conference proceedings. The aim of the research was to identify what aspects of the teacher-student-content interaction were investigated frequently and what have been studied rarely. We used the categorization system developed by Kinnunen, Lampiselkä, Malmi and Meisalo (2016) and altogether 184 articles were analyzed. The analysis focused on secondary and tertiary level biology, chemistry, physics, and science education. The results showed that most of the studies focus on either the teacherâs pedagogical actions or on the student - content relationship. All other aspects were studied considerably less. For example, the teachersâ thoughts about the studentsâ perceptions and attitudes towards the goals and the content, and the teachersâ conceptions of the studentsâ actions towards achieving the goals were studied only rarely. Discussion about the scope and the coverage of the research in science education in Europe is needed.Peer reviewe