A COMPUTATION METHOD/FRAMEWORK FOR HIGH LEVEL VIDEO CONTENT ANALYSIS AND SEGMENTATION USING AFFECTIVE LEVEL INFORMATION

VIDEO segmentation facilitates e±cient video indexing and navigation in large digital video archives. It is an important process in a content-based video indexing and retrieval (CBVIR) system. Many automated solutions performed seg- mentation by utilizing information about the \facts" of the video. These \facts" come in the form of labels that describe the objects which are captured by the cam- era. This type of solutions was able to achieve good and consistent results for some video genres such as news programs and informational presentations. The content format of this type of videos is generally quite standard, and automated solutions were designed to follow these format rules. For example in [1], the presence of news anchor persons was used as a cue to determine the start and end of a meaningful news segment. The same cannot be said for video genres such as movies and feature films. This is because makers of this type of videos utilized different filming techniques to design their videos in order to elicit certain affective response from their targeted audience. Humans usually perform manual video segmentation by trying to relate changes in time and locale to discontinuities in meaning [2]. As a result, viewers usually have doubts about the boundary locations of a meaningful video segment due to their different affective responses. This thesis presents an entirely new view to the problem of high level video segmentation. We developed a novel probabilistic method for affective level video content analysis and segmentation. Our method had two stages. In the first stage, a®ective content labels were assigned to video shots by means of a dynamic bayesian 0. Abstract 3 network (DBN). A novel hierarchical-coupled dynamic bayesian network (HCDBN) topology was proposed for this stage. The topology was based on the pleasure- arousal-dominance (P-A-D) model of a®ect representation [3]. In principle, this model can represent a large number of emotions. In the second stage, the visual, audio and a®ective information of the video was used to compute a statistical feature vector to represent the content of each shot. Affective level video segmentation was achieved by applying spectral clustering to the feature vectors. We evaluated the first stage of our proposal by comparing its emotion detec- tion ability with all the existing works which are related to the field of a®ective video content analysis. To evaluate the second stage, we used the time adaptive clustering (TAC) algorithm as our performance benchmark. The TAC algorithm was the best high level video segmentation method [2]. However, it is a very computationally intensive algorithm. To accelerate its computation speed, we developed a modified TAC (modTAC) algorithm which was designed to be mapped easily onto a field programmable gate array (FPGA) device. Both the TAC and modTAC algorithms were used as performance benchmarks for our proposed method. Since affective video content is a perceptual concept, the segmentation per- formance and human agreement rates were used as our evaluation criteria. To obtain our ground truth data and viewer agreement rates, a pilot panel study which was based on the work of Gross et al. [4] was conducted. Experiment results will show the feasibility of our proposed method. For the first stage of our proposal, our experiment results will show that an average improvement of as high as 38% was achieved over previous works. As for the second stage, an improvement of as high as 37% was achieved over the TAC algorithm

Long-Range Autocorrelations of CpG Islands in the Human Genome

In this paper, we use a statistical estimator developed in astrophysics to study the distribution and organization of features of the human genome. Using the human reference sequence we quantify the global distribution of CpG islands (CGI) in each chromosome and demonstrate that the organization of the CGI across a chromosome is non-random, exhibits surprisingly long range correlations (10 Mb) and varies significantly among chromosomes. These correlations of CGI summarize functional properties of the genome that are not captured when considering variation in any particular separate (and local) feature. The demonstration of the proposed methods to quantify the organization of CGI in the human genome forms the basis of future studies. The most illuminating of these will assess the potential impact on phenotypic variation of inter-individual variation in the organization of the functional features of the genome within and among chromosomes, and among individuals for particular chromosomes

The Hierarchical Structure and Dynamics of Voids

Contrary to the common view voids have very complex internal structure and dynamics. Here we show how the hierarchy of structures in the density field inside voids is reflected by a similar hierarchy of structures in the velocity field. Voids defined by dense filaments and clusters can de described as simple expanding domains with coherent flows everywhere except at their boundaries. At scales smaller that the void radius the velocity field breaks into expanding sub-domains corresponding to sub- voids. These sub-domains break into even smaller sub-sub domains at smaller scales resulting in a nesting hierarchy of locally expanding domains. The ratio between the magnitude of the velocity field responsible for the expansion of the void and the velocity field defining the sub voids is approximately one order of magnitude. The small-scale components of the velocity field play a minor role in the shaping of the voids but they define the local dynamics directly affecting the processes of galaxy formation and evolution. The super-Hubble expansion inside voids makes them cosmic magnifiers by stretching their internal primordial density fluctuations allowing us to probe the small scales in the primordial density field. Voids also act like time machines by "freezing" the development of the medium-scale density fluctuations responsible for the formation of the tenuous web of structures seen connecting proto galaxies in computer simulations. As a result of this freezing haloes in voids can remain "connected" to this tenuous web until the present time. This may have an important effect in the formation and evolution of galaxies in voids by providing an efficient gas accretion mechanism via coherent low-velocity streams that can keep a steady inflow of matter for extended periods of time.Comment: High-res version are related media here: http://skysrv.pha.jhu.edu/~miguel/Papers/Hierarchy_voids/index.htm

Machine learning for flow field measurements: a perspective

Advancements in machine-learning (ML) techniques are driving a paradigm shift in image processing. Flow diagnostics with optical techniques is not an exception. Considering the existing and foreseeable disruptive developments in flow field measurement techniques, we elaborate this perspective, particularly focused to the field of particle image velocimetry. The driving forces for the advancements in ML methods for flow field measurements in recent years are reviewed in terms of image preprocessing, data treatment and conditioning. Finally, possible routes for further developments are highlighted.Stefano Discetti acknowledges funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 949085). Yingzheng Liu acknowledges financial support from the National Natural Science Foundation of China (11725209)

Information theoretic measures for encoding video

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1995.Includes bibliographical references (p. 75-78).by Giridharan Iyengar.M.S

Modelling chemical abundance distributions for dwarf galaxies in the Local Group: the impact of turbulent metal diffusion

We investigate stellar metallicity distribution functions (MDFs), including Fe and ${\alpha}$-element abundances, in dwarf galaxies from the Feedback in Realistic Environments (FIRE) project. We examine both isolated dwarf galaxies and those that are satellites of a Milky Way-mass galaxy. In particular, we study the effects of including a sub-grid turbulent model for the diffusion of metals in gas. Simulations that include diffusion have narrower MDFs and abundance ratio distributions, because diffusion drives individual gas and star particles toward the average metallicity. This effect provides significantly better agreement with observed abundance distributions of dwarf galaxies in the Local Group, including the small intrinsic scatter in [${\alpha}$/Fe] vs. [Fe/H] (less than 0.1 dex). This small intrinsic scatter arises in our simulations because the interstellar medium (ISM) in dwarf galaxies is well-mixed at nearly all cosmic times, such that stars that form at a given time have similar abundances to within 0.1 dex. Thus, most of the scatter in abundances at z = 0 arises from redshift evolution and not from instantaneous scatter in the ISM. We find similar MDF widths and intrinsic scatter for satellite and isolated dwarf galaxies, which suggests that environmental effects play a minor role compared with internal chemical evolution in our simulations. Overall, with the inclusion of metal diffusion, our simulations reproduce abundance distribution widths of observed low-mass galaxies, enabling detailed studies of chemical evolution in galaxy formation.Comment: 19 pages, 13 figures, published in MNRA

The mentalizing triangle: how interactions among self, other and object prompt mentalizing

To smoothly interact with other people requires individuals to generate appropriate responses based on other’s mental states. The ability we rely on is termed mentalizing. As humans it seems that we are endowed with the abilities to rapidly process other’s mental states, either by taking their perspectives or using mindreading skills. These abilities allow us to go beyond our direct experience of reality and to see or infer some of the contents of another’s mental world. Due to the complexity of social contexts, our mentalizing system needs to address a variety of challenges which put different requirements on either time or flexibility. During years of research, investigators have come up with various theories to explain how we cope with these challenges. Among them, the two-system account raised up by Apperly and colleagues (2010) has been favoured by many studies. Concisely, the two-system account claims that we have a fast-initiated mentalizing system which guarantees us to make quick judgments with limited cognitive resource; and a flexible system which allows deliberate thinking and enables mentalizing to generalize to multiple targets. Such a framework provides good explanations to debates such as whether preverbal young children can process mentalizing or not. But it is still largely unknown how healthy adults engage in mentalizing in everyday life. Specifically, why it seems easier for some targets to activate our mentalizing system, but with some others, we frequently fail to consider their perspectives or beliefs? To give an explanation to this question, I adopted a different research orientation in my PhD from the two-system account, which considers the dynamic interactions among three key elements in mentalizing: the self, agent(s), and object(s). I put forward a mentalizing triangle model and assume the interactions in these triadic relationships act as gateways triggering mentalizing. Thus, with some agents, we feel more intimate with them, which makes it easier for us to think about their minds. Similarly, in certain context, the agent may have frequent interactions with the object, thus we become more motivated to engage in mentalizing. In the following chapters, I first reviewed current literatures and illustrate evidence that could support or oppose the triangle model, then examined these triangle hypotheses both from behavioural and neuroimaging levels. In Study 1, I first measured mentalizing in the baseline condition where no interaction in the triangle relationships was provided. By adapting the false belief paradigm used by Kovacs, Teglas, & Endress (2010), I imported the Signal Detection theory to obtain more indices which could reflect participants mentalizing processes. Results of this study showed that people have a weak tendency to ascribe other’s beliefs when there is no interaction. Then, in Study 2, we added another condition which included the ‘agent-object’ interaction factor while using a similar paradigm in Study 1. Results in the noninteractiond condition replicated our findings of Study 1, but adding ‘agent-object’ interactions didn’t boost mentalizing. Study 3 and 4 tested the ‘self-agent’ interaction hypothesis in visual perspective taking (VPT), another basic mentalizing ability. In Study 3, I adopted virtual reality approach and for the first time investigated how people select which perspective to take when exposed to multiple conflicting perspectives. Importantly, I examined whether the propensity to engage in VPT is correlated with how we perceive other people as humans, i.e. the humanization process. Congruent with our hypotheses, participant exhibited stronger propensity to take a more humanised agent’s perspective. Then in Study 4, I used functional near-infrared spectroscopy (fNIRS) and investigated the neural mechanism underlying this finding. In general, the ‘selfagent’ hypothesis in the mentalizing triangle model was supported but not for the ‘agentobject’ hypothesis, which we consider may due to several approach limitations. The findings in this thesis are derived from applying novel approaches to classic experimental paradigms, and have shown the potentials of using new techniques, such as VR and fNIRS, in investigating the philosophical question of mentalizing. It also enlights social cognitive studies by considering classic psychological methods such as the Signal Detection Theory in future research