Perceptual video quality estimation by regression with myopic experts

Objective video quality metrics can be viewed as 'myopic' expert systems that focus on particular aspects of visual information in video, such as image edges or motion parameters. We conjecture that the combination of many such high-level metrics leads to statistically-significant improvement in the prediction of reference-based perceptual video quality in comparison to each individual metric. To examine this hypothesis in a systematic and rigorous manner, we use: (i) the LIVE and the EPFL/PoliMi databases that provide the difference mean opinion scores (DMOS) for several video sequences under encoding and packet-loss errors; (ii) ten well-known metrics that range from mean-squared error based criteria to sophisticated visual quality estimators; (iii) five variants of regression-based supervised learning. For 400 experimental trials with random (non-overlapping) estimation and prediction subsets taken from both databases, we show that the best of our regression methods: (i) leads to statistically-significant improvement against the best individual metrics for DMOS prediction for more than 97% of the experimental trials; (ii) is statistically-equivalent to the performance of humans rating the video quality for 36.75% of the experiments with the EPFL/PoliMi database. On the contrary, no single metric achieves such statistical equivalence to human raters in any of the experimental trials

Open challenges for Machine Learning based Early Decision-Making research

More and more applications require early decisions, i.e. taken as soon as possible from partially observed data. However, the later a decision is made, the more its accuracy tends to improve, since the description of the problem to hand is enriched over time. Such a compromise between the earliness and the accuracy of decisions has been particularly studied in the field of Early Time Series Classification. This paper introduces a more general problem, called Machine Learning based Early Decision Making (ML-EDM), which consists in optimizing the decision times of models in a wide range of settings where data is collected over time. After defining the ML-EDM problem, ten challenges are identified and proposed to the scientific community to further research in this area. These challenges open important application perspectives, discussed in this paper

Probabilistic Human-Robot Information Fusion

This thesis is concerned with combining the perceptual abilities of mobile robots and human operators to execute tasks cooperatively. It is generally agreed that a synergy of human and robotic skills offers an opportunity to enhance the capabilities of today’s robotic systems, while also increasing their robustness and reliability. Systems which incorporate both human and robotic information sources have the potential to build complex world models, essential for both automated and human decision making. In this work, humans and robots are regarded as equal team members who interact and communicate on a peer-to-peer basis. Human-robot communication is addressed using probabilistic representations common in robotics. While communication can in general be bidirectional, this work focuses primarily on human-to-robot information flow. More specifically, the approach advocated in this thesis is to let robots fuse their sensor observations with observations obtained from human operators. While robotic perception is well-suited for lower level world descriptions such as geometric properties, humans are able to contribute perceptual information on higher abstraction levels. Human input is translated into the machine representation via Human Sensor Models. A common mathematical framework for humans and robots reinforces the notion of true peer-to-peer interaction. Human-robot information fusion is demonstrated in two application domains: (1) scalable information gathering, and (2) cooperative decision making. Scalable information gathering is experimentally demonstrated on a system comprised of a ground vehicle, an unmanned air vehicle, and two human operators in a natural environment. Information from humans and robots was fused in a fully decentralised manner to build a shared environment representation on multiple abstraction levels. Results are presented in the form of information exchange patterns, qualitatively demonstrating the benefits of human-robot information fusion. The second application domain adds decision making to the human-robot task. Rational decisions are made based on the robots’ current beliefs which are generated by fusing human and robotic observations. Since humans are considered a valuable resource in this context, operators are only queried for input when the expected benefit of an observation exceeds the cost of obtaining it. The system can be seen as adjusting its autonomy at run-time based on the uncertainty in the robots’ beliefs. A navigation task is used to demonstrate the adjustable autonomy system experimentally. Results from two experiments are reported: a quantitative evaluation of human-robot team effectiveness, and a user study to compare the system to classical teleoperation. Results show the superiority of the system with respect to performance, operator workload, and usability

Human amblyopia and its perceptual consequences

The presence of ocular defects of an optical or muscular nature during early childhood can cause amblyopia: a reduction in visual acuity of the defective eye. The research reported in this thesis investigated the impact of amblyopia on some aspects of visual perception by evaluating three main perceptual functions: precision of judgement of spatial relationships (in three-dimensional space), ability to detect depth in tests of stereopsis, and contrast sensitivity. In some experiments amblyopic subjects were paired with non-amblyopic subjects who had monocular acuity deficits owing to uncorrected refractive errors, in order to assess the importance of the acuity deficit as a determinant of other perceptual losses suffered by amblyopes. In an alignment task non-amblyopes with monocularly reduced acuity performed significantly better than amblyopes, suggesting that the acuity deficit was not solely responsible for amblyopes' perceptual deficit in this task. However, in another experiment in which a greater variety of spatial cues was provided amblyopes performed as well as non- amblyopes. Thus their perceptual skills would seem to be adequate for efficient functioning in most normal environments where spatial cues are abundant. Previous reports that amblyopes generally lack stereopsis were confirmed in two experiments with a few interesting exceptions, whose cases are discussed. The data obtained in the four experiments on space perception and stereopsis in amblyopia provided support for most current theories in these areas. Experiments on contrast sensitivity showed that the losses suffered by amblyopes, as measured by interocular comparison, varied between individuals, both in depth and in bandwidth (the range of spatial frequencies affected). This variation was not directly related to the extent of acuity deficit, or to the condition which originally gave rise to amblyopia, but did seem connected with the age at which the subject first received treatment for the primary causative ocular defect. A similarity between the contrast sensitivity functions of amblyopic eyes and those of infant eyes is considered as a basis for explaining the nature of contrast sensitivity loss in amblyopia. Some preliminary attempts to measure contrast sensitivity in infancy by methods suitable for screening purposes are described in the final chapter. The thesis includes a historical review of theories of amblyopia derived from clinical and experimental work on human subjects, and a critical evaluation of experimental work in which animals were visually deprived with a view to measuring the contributions of experience to visual development. The claims of some authors that such work may have clinical relevance for preventing or treating amblyopia are refuted, since clinical experience has already furnished sufficient evidence to achieve these ends. The perceptual consequences of human amblyopia, as characterised in the present research have important practical implications for the amblyope, and important theoretical implications for models and mechanisms of visual perception and its development

Constraints on movement variability during a discrete multi-articular action

The aim of this programme of work was to examine how the manipulation of organismic and task constraints affected movement variability during a basketball shooting task. The specific constraints that were manipulated included task expertise, state anxiety and dioptric blur (organismic constraints), and, shooting distance and attentional focus instruction (task constraints). The aim of Study 1 was to investigate the effect of shooting distance and task expertise on movement variability. Task expertise was characterised by decreased coordination variability and heightened compensatory variability between wrist, elbow and shoulder joints. However, no significant difference was found in joint angle variability at release as a function of task expertise. There was no significant change in movement variability with shooting distance, a finding that was consistent across all expertise groups. In Study 2, the aims were to examine the effect of induced dioptric blur on shooting performance and movement variability during basketball free-throw shooting, and, to ascertain whether task expertise plays a mediating role in the capacity to stabilise performance against impaired visual information. Significant improvements in shooting performance were noted with the introduction of moderate visual blur (+1.00 and +2.00 D). This performance change was evident in both expert and novice performers. Only with the onset of substantial dioptric blur (+3.00 D), equivalent to the legal blindness limit, was there a significant decrease in coordination variability. Despite the change in coordination variability at +3.00 D, there was no significant difference in shooting performance when compared to the baseline condition. The aims of Study 3 were to examine the effect of elevated anxiety on shooting performance and movement variability and, again, to determine whether task expertise plays a mediating role in stabilising performance and movement kinematics against perturbation from emotional fluctuations. Commensurate with the results of Study 2, both expert and novice performers were able to stabilise performance and movement kinematics, this time with elevated anxiety. Stabilisation was achieved through the allocation of additional attentional resources to the task. Study 4, had two aims. The first was to examine the interactive effects of practice and focus of attention on both performance and learning of an accuracy-based, discrete multi-articular action. The second was to identify potential focus-dependent changes on joint kinematics, intra-limb coordination and coordination variability. Support was found for the role of an external focus of attention on shooting performance during both acquisition and retention. However, there was evidence to suggest that internal focus instruction could play a pivotal role in shaping emerging patterns of intra-limb coordination and channelling the learners‟ search towards a smaller range of kinematic solutions within the perceptual-motor workspace. Collectively, this programme of work consistently highlighted the fundamental role that constraints play in governing shooting performance, movement variability and, more broadly, perceptual-motor organisation. For instance, task expertise was characterised by decreased coordination variability and heightened compensatory control. However, in light of the data pertaining to joint angle variability at release, general assumptions about expertise-variability relations cannot be made and should be viewed with caution. In addition, there is strong evidence to suggest that adaptation to constraints is, perhaps, a universal human response, and consequently not mediated by task expertise. Further research is needed to fully elucidate this proposition

Flaxseed (linum usitatissimum) ethanolic extract affects WNT signalling pathway-associated molecules; β-catenin and DKK1 expressions, during osteoblast differentiation of SHED

WNT signalling is important in regulating developmental process including bone development. Additionally, WNT signalling also involves in lineage differentiation of mesenchymal stem cells (MSC), including osteogenic differentiation, through canonical WNT pathway. Flaxseed (linum usitatissimum) is a plant with many health benefits including promoting bone health. Our previous study demonstrated that Flaxseed ethanolic extract (FEE) reduced the osteoblast differentiation potential of stem cells from human exfoliated deciduous teeth (SHED), a type of MSC. Hence, we aimed to analyse the effect of FEE on WNT signalling pathway-associated molecules; β-catenin, and DKK1 expressions, during the osteoblast differentiation of SHED. SHED cultured in osteoblast induction media (OIM) was treated with FEE at 4 mg/ml. RNA extracted from cells cultured at day 7, 14 and 21 was subjected to reverse-transcriptase PCR for β-catenin, and DKK1 gene expression analysis. FEE at 4 mg/ml significantly reduced β-catenin and DKK1 expression of SHED at day 7 (0.5293 ± 0.01, 1.0792 ± 0.02 respectively, p < 0.01) but induced their expression at day 14 (0.7675 ± 0.05, 1.7176 ± 0.07 respectively, p < 0.01). The expression was later reduced at day 21 (0.2592 ± 0.01, 1.0653 ± 0.04 respectively, p < 0.01). Changes in the β-catenin, and DKK1 expressions levels at different time frame might explain how FEE reduced the osteoblast differentiation potential of SHED. Overall, FEE modulates the expressions of WNT signalling pathway-associated molecules: β-catenin, and DKK1, during the process of osteoblast differentiation of SHED which could possibly interrupt the process of osteogenesis in the current environment

Mental health mobile apps during Covid-19 Pandemic to evaluate stress level in Selangor

The COVID-19 pandemic has impacted negatively on public mental health. As a result, monitoring the level of the population mental health is a priority during crises. This study aims to measure stress during the COVID-19 pandemic in Selangor. Cross-sectional study was done using SELANGKAH apps, where users are Selangor citizens. Data was collected from September 2021 until March 2022. This app was initially used as contact tracing and mental health modules (SEHAT) were added, consisting of a validated Perceived Stress Scale (PSS-10) questionnaire. Out of 42072 SEHAT users, 6411 people had completed the questionnaire. Majority were female (53.6%), Muslims (79.6%), had formal education up to secondary (49.0%), low income (89.9%), and young and middle- aged adults (59.7%). Majority have a moderate stress (66.8%), while 23.3% and 9.9% are low and high levels, respectively. High stress is significantly associated with females, high education, younger age groups, and low monthly income. Several factors could have contributed to this throughout the COVID-19 pandemic, such as online learning, uncertainty on study duration, financial constraints and limited social interactions. Moreover, as an effect of prolonged pandemic and MCO, a surge in the number of job terminations has also affected the source of income, which contributed to high levels of stress among the general population. The level of stress in Selangor was high during the pandemic as an effect of MCO

Giving eyes to ICT!, or How does a computer recognize a cow?

Het door Schouten en andere onderzoekers op het CWI ontwikkelde systeem berust op het beschrijven van beelden met behulp van fractale meetkunde. De menselijke waarneming blijkt mede daardoor zo efficiënt omdat zij sterk werkt met gelijkenissen. Het ligt dus voor de hand het te zoeken in wiskundige methoden die dat ook doen. Schouten heeft daarom beeldcodering met behulp van 'fractals' onderzocht. Fractals zijn zelfgelijkende meetkundige figuren, opgebouwd door herhaalde transformatie (iteratie) van een eenvoudig basispatroon, dat zich daardoor op steeds kleinere schalen vertakt. Op elk niveau van detaillering lijkt een fractal op zichzelf (Droste-effect). Met fractals kan men vrij eenvoudig bedrieglijk echte natuurvoorstellingen maken. Fractale beeldcodering gaat ervan uit dat het omgekeerde ook geldt: een beeld effectief opslaan in de vorm van de basispatronen van een klein aantal fractals, samen met het voorschrift hoe het oorspronkelijke beeld daaruit te reconstrueren. Het op het CWI in samenwerking met onderzoekers uit Leuven ontwikkelde systeem is mede gebaseerd op deze methode. ISBN 906196502

Predictive Modelling of Complex Urban Soundscapes: Enabling an engineering approach to soundscape design

Conventional noise control methods typically limit their focus to the reduction of unwanted noise, ignoring the benets of positive sounds and struggling to reflect the totality of noise impacts. Modern approaches to achieve improved health outcomes and public satisfaction aim to incorporate the perception of an acoustic environment, an approach known as ‘soundscape’. When attempting to apply soundscape in practice, it is apparent that new methods of analysing soundscape perception in urban spaces are required; in particular, a predictive model of the users’ perceptual response to the acoustic environment is necessary. This thesis is intended to enable a move towards applying engineering approaches to soundscape design. This is achieved by developing predictive models of soundscape perception through empirical studies examining a large scale soundscape assessment database. The results are presented in three parts: first, the data collection protocol and modelling methods developed for this work are presented; the second part demonstrates an initial development and application of a predictive soundscape model; the final section expands upon this initial model with two empirical studies exploring the potential for additional information to be included in the model. This thesis begins by establishing a protocol for large scale soundscape data collection based on ISO 12913-2 and the creation of a database containing 1,318 responses paired with 693 binaural recordings collected in 13 locations in London and Venice. The first study then presents an initial development and application of a model designed to predict soundscape perception based on psychoacoustic analysis of the binaural recordings. Through the collection of an additional 571 binaural recordings during the COVID-19 lockdowns, sound level reductions at every location are seen, ranging from a reduction of 1.27 dB(A) in Regents Park Japan to 17.33 dB(A) in Piazza SanMarco, with an average reduction across all locations of 7.27 dB(A). Multi-level models were developed to predict the overall soundscape pleasantness (R2 = 0.85) and eventfulness (R2 = 0.715) of each location and applied to the lockdown recordings to determine how the soundscape perception likely changed. The results demonstrated that perception shifted toward less eventful soundscapes and to more pleasant soundscapes for previously traffic-dominated locations but not for human- and natural-dominated locations. The modelling process also demonstrated that contextual information was important for predicting pleasantness but not for predicting eventfulness. The next stage of the thesis considers a series of expansions to the initial model. The second piece of empirical work makes use of a dataset of recordings collected from a Wireless Acoustic Sensor Network (WASN) which includes sound source labels and annoyance ratings collected from 100 participants in an online listening study. A multilevel model was constructed using a combination of psychoacoustic metrics and sound source labels to predict perceived annoyance, achieving an R2 of 0.64 for predicting individual responses. The sound source information is demonstrated to be a crucial factor, as the relationship between roughness, impulsiveness, and tonality and the predicted annoyance varies as a function of the sound source label. The third piece of empirical work uses multilevel models to examine the extent to which personal factors influence soundscape perception. The findings suggest that personal factors, including psychological wellbeing, age, gender, and occupational status, account for approximately 1.4% of the variance for pleasantness and 3.9% for eventfulness, while the influence of the locations accounted for approximately 34% and 14%, respectively. Drawing from the experience gained working with urban soundscape data, a new method of analysing and presenting the soundscape perception of urban spaces is developed. This method inherently considers the variety of perceptions within a group and provides an open-source visualisation tool to facilitate a nuanced approach to soundscape assessment and design. Based on this empirical evidence, a framework is established for developing future predictive soundscape models which can be integrated into an engineering approach. At each stage, the results of these studies is discussed in terms of how it can contribute to a generalisable predictive soundscape model