Visual analysis for spatio-temporal event correlation in manufacturing

The analysis of events with spatio-temporal context and their interdependencies is a crucial task in the manufacturing domain. In general, understanding this context, for example investigating error messages or alerts is important to take corrective actions. In the manufacturing domain, comprehending the relations of errors is often based on the technicians\u27 experience. Validation of cause-effect relations is necessary to understand if an effect has a preceding causality, e.g., if an error is the result of multiple issues from previous working steps. We present an approach to investigate spatio-temporal relations between such events. Based on a time-sensitive correlation measure, we provide multiple coordinated views to analyze and filter the data. In collaboration with an industry partner, we developed a visual analytics approach for error logs reported by machines that covers a multitude of analysis tasks. We present a case study based on real-world event logs of an assembly line with feedback from our industry partner\u27s domain experts. The findings show that experts can effectively identify error dependencies that impair the overall assembly line productivity using our technique. Furthermore, we discuss how our approach is applicable in other domains

Low-frequency oscillatory correlates of auditory predictive processing in cortical-subcortical networks: a MEG-study

Emerging evidence supports the role of neural oscillations as a mechanism for predictive information processing across large-scale networks. However, the oscillatory signatures underlying auditory mismatch detection and information flow between brain regions remain unclear. To address this issue, we examined the contribution of oscillatory activity at theta/alpha-bands (4–8/8–13 Hz) and assessed directed connectivity in magnetoencephalographic data while 17 human participants were presented with sound sequences containing predictable repetitions and order manipulations that elicited prediction-error responses. We characterized the spectro-temporal properties of neural generators using a minimum-norm approach and assessed directed connectivity using Granger Causality analysis. Mismatching sequences elicited increased theta power and phase-locking in auditory, hippocampal and prefrontal cortices, suggesting that theta-band oscillations underlie prediction-error generation in cortical-subcortical networks. Furthermore, enhanced feedforward theta/alpha-band connectivity was observed in auditory-prefrontal networks during mismatching sequences, while increased feedback connectivity in the alpha-band was observed between hippocampus and auditory regions during predictable sounds. Our findings highlight the involvement of hippocampal theta/alpha-band oscillations towards auditory prediction-error generation and suggest a spectral dissociation between inter-areal feedforward vs. feedback signalling, thus providing novel insights into the oscillatory mechanisms underlying auditory predictive processing

Dynamic Influence Networks for Rule-based Models

We introduce the Dynamic Influence Network (DIN), a novel visual analytics technique for representing and analyzing rule-based models of protein-protein interaction networks. Rule-based modeling has proved instrumental in developing biological models that are concise, comprehensible, easily extensible, and that mitigate the combinatorial complexity of multi-state and multi-component biological molecules. Our technique visualizes the dynamics of these rules as they evolve over time. Using the data produced by KaSim, an open source stochastic simulator of rule-based models written in the Kappa language, DINs provide a node-link diagram that represents the influence that each rule has on the other rules. That is, rather than representing individual biological components or types, we instead represent the rules about them (as nodes) and the current influence of these rules (as links). Using our interactive DIN-Viz software tool, researchers are able to query this dynamic network to find meaningful patterns about biological processes, and to identify salient aspects of complex rule-based models. To evaluate the effectiveness of our approach, we investigate a simulation of a circadian clock model that illustrates the oscillatory behavior of the KaiC protein phosphorylation cycle.Comment: Accepted to TVCG, in pres

Hyper-Document structure: maintaining discourse coherence in non-linear documents

The passage from linear text to hypertext poses the challenge of expressing discourse coherence in non-linear text, where linguistic discourse markers no longer work. While hypertext introduces new possibilities for discourse organisation, it also requires the use of new devices which can support the expression of coherence by exploiting the technical characteristics and expressive capabilities of the medium. In this paper we show how in hypertext the notion of abstract document structure encompasses animated graphics as a form of meta-language for discourse construction

Human Conscious Experience is Four-Dimensional and has a Neural Correlate Modeled by Einstein's Special Theory of Relativity

In humans, knowing the world occurs through spatial-temporal experiences and interpretations. Conscious experience is the direct observation of conscious events. It makes up the content of consciousness. Conscious experience is organized in four dimensions. It is an orientation in space and time, an understanding of the position of the observer in space and time. A neural correlate for four-dimensional conscious experience has been found in the human brain which is modeled by Einstein’s Special Theory of Relativity. Spacetime intervals are fundamentally involved in the organization of coherent conscious experiences. They account for why conscious experience appears to us the way it does. They also account for assessment of causality and past-future relationships, the integration of higher cognitive functions, and the implementation of goal-directed behaviors. Spacetime intervals in effect compose and direct our conscious life. The relativistic concept closes the explanatory gap and solves the hard problem of consciousness (how something subjective like conscious experience can arise in something physical like the brain). There is a place in physics for consciousness. We describe all physical phenomena through conscious experience, whether they be described at the quantum level or classical level. Since spacetime intervals direct the formation of all conscious experiences and all physical phenomena are described through conscious experience, the equation formulating spacetime intervals contains the information from which all observable phenomena may be deduced. It might therefore be considered expression of a theory of everything

Visualising Discourse Coherence in Non-Linear Documents

To produce coherent linear documents, Natural Language Generation systems have traditionally exploited the structuring role of textual discourse markers such as relational and referential phrases. These coherence markers of the traditional notion of text, however, do not work in non-linear documents: a new set of graphical devices is needed together with formation rules to govern their usage, supported by sound theoretical frameworks. If in linear documents graphical devices such as layout and formatting complement textual devices in the expression of discourse coherence, in non-linear documents they play a more important role. In this paper, we present our theoretical and empirical work in progress, which explores new possibilities for expressing coherence in the generation of hypertext documents

The Method of Contrast and the Perception of Causality in Audition

The method of contrast is used within philosophy of perception in order to demonstrate that a specific property could be part of our perception. The method is based on two passages. I argue that the method succeeds in its task only if the intuition of the difference, which constitutes the core of the first passage, has two specific traits. The second passage of the method consists in the evaluation of the available explanations of this difference. Among the three outlined options, I will demonstrate that only in the third option – as we shall see, the case of the scenario that remains the same but is perceived in two different ways by the same perceiver – the intuition purports a difference that posses the necessary characteristics, namely being immediately evident and extremely complex and multifaceted, which determine its tensive nature. The application within auditory perception of this third option will generate two cases, a diachronic one and a synchronic one, which clearly show that we can auditorily perceive causality as a link between two sonorous episodes. The causal explanation is the only possible explanation among the many evaluated within the second passage of the method of contrast

DramaQA: Character-Centered Video Story Understanding with Hierarchical QA

Despite recent progress on computer vision and natural language processing, developing video understanding intelligence is still hard to achieve due to the intrinsic difficulty of story in video. Moreover, there is not a theoretical metric for evaluating the degree of video understanding. In this paper, we propose a novel video question answering (Video QA) task, DramaQA, for a comprehensive understanding of the video story. The DramaQA focused on two perspectives: 1) hierarchical QAs as an evaluation metric based on the cognitive developmental stages of human intelligence. 2) character-centered video annotations to model local coherence of the story. Our dataset is built upon the TV drama "Another Miss Oh" and it contains 16,191 QA pairs from 23,928 various length video clips, with each QA pair belonging to one of four difficulty levels. We provide 217,308 annotated images with rich character-centered annotations, including visual bounding boxes, behaviors, and emotions of main characters, and coreference resolved scripts. Additionally, we provide analyses of the dataset as well as Dual Matching Multistream model which effectively learns character-centered representations of video to answer questions about the video. We are planning to release our dataset and model publicly for research purposes and expect that our work will provide a new perspective on video story understanding research.Comment: 21 pages, 10 figures, submitted to ECCV 202

Review of Edward Branigan, Narrative Comprehension and Film. London and New York: Routledge, 1992. (Distributed by the Law Book Company Ltd.). 325pp. ISBN 0415075114. (pbk), $45.00.

If Point of View in the Cinema introduced its author as one of the leading film analysts by its attention to the details of the process of cinematic presentation, Narrative Comprehension and Film establishes Edward Branigan as a creative theorist beyond the boundaries of film. The book's voice is that of a craftsman speaking from his workshop. No deconstructive symplok and certainly no rhetorical terrorism. Instead, we find a certain modesty of style, which is deceptive considering that Branigan offers a great deal of substance and a range of attractive speculative insights. The author's mastery of technical intricacies within the broader frame of general narrative makes Narrative Comprehension and Film an outstanding teaching book for film studies as well as other disciplines in the humanities. What makes it especially appealing is its careful elaboration of an inferential account of how we make sense of narrative. For this and other reasons, it is well worth paying closer attention than is perhaps usual for a review article to how the book's argument unfolds and in particular to how it manages to relate the double argument about narrative in film and human perception as interpretive construals