Continuous Health Interface Event Retrieval

Knowing the state of our health at every moment in time is critical for advances in health science. Using data obtained outside an episodic clinical setting is the first step towards building a continuous health estimation system. In this paper, we explore a system that allows users to combine events and data streams from different sources to retrieve complex biological events, such as cardiovascular volume overload. These complex events, which have been explored in biomedical literature and which we call interface events, have a direct causal impact on relevant biological systems. They are the interface through which the lifestyle events influence our health. We retrieve the interface events from existing events and data streams by encoding domain knowledge using an event operator language.Comment: ACM International Conference on Multimedia Retrieval 2020 (ICMR 2020), held in Dublin, Ireland from June 8-11, 202

On cyberbullying incidents and underlying online social relationships

Abstract Cyberbullying is an important social challenge that takes place over a technical substrate. Thus, it has attracted research interest across both computational and social science research communities. While the social science studies conducted via careful participant selection have shown the effect of personality, social relationships, and psychological factors on cyberbullying, they are often limited in scale due to manual survey or ethnographic study components. Computational approaches on the other hand have defined multiple automated approaches for detecting cyberbullying at scale, and have largely focused only on the textual content of the messages exchanged. There are no existing efforts aimed at testing, validating, and potentially refining the findings from traditional bullying literature as obtained via surveys and ethnographic studies at scale over online environments. By analyzing the social relationship graph between users in an online social network and deriving features such as out-degree centrality and the number of common friends, we find that multiple social characteristics are statistically different between the cyberbullying and non-bullying groups, thus supporting many, but not all, of the results found in previous survey-based bullying studies. The results pave way for better understanding of the cyberbullying phenomena at scale

Timeline-based information assimilation in multimedia surveillance and monitoring systems

Most surveillance and monitoring systems nowadays utilize multiple types of sensors. However, due to the asynchrony among and diversity of sensors, information assimilation-how to combine the information obtained from asynchronous and multifarious sources is an important and challenging research problem. In this paper, we propose a hierarchical probabilistic method for information assimilation in order to detect events of interest in a surveillance and monitoring environment. The proposed method adopts a bottom-up approach and performs assimilation of information at three different levels- media-stream level, atomic-event level and compound-event level. To detect an event, our method uses not only the current media streams but it also utilizes their two important properties- first, accumulated past history of whether they have been providing the concurring or contradictory evidences, and- second, the system designer’s confidence in them. A compound event, which comprises of two or more atomic-events, is detected by first estimating probabilistic decisions for the atomic-events based on individual streams, and then by aligning these decisions along a timeline and hierarchically assimilating them. The experimental results show the utility of our method

Coopetitive Multimedia Surveillance

Abstract. ‘Coopetitive ’ interaction strategy has been shown to give better results than similar strategies like ‘only cooperation’, ‘only competition’ etc [7]. However, this has been studied only in the context of visual sensors and for handling non-simultaneous events. In this paper, we study this ‘coopetitive ’ strategy from a multimedia surveillance system perspective, wherein the system needs to utilize multiple heterogeneous sensors and also handle multiple simultaneous events. Applying such an interaction strategy to multimedia surveillance systems is challenging because heterogeneous sensors have different capabilities for performing different sub-tasks as well as dissimilar response times. We adopt a merit-cum-availability based approach to allocate various sub-tasks to the competing sensors which eventually cooperate to achieve the specified system goal. Also, a ‘coopetition ’ based strategy is adopted for effectively utilizing the information coming asynchronously from different data sources. Multiple simultaneous events (e.g. multiple intrusions) are handled by adopting a predictive strategy which estimates the exit time for each intruder and then uses this information for enhanced scheduling. The results obtained for two sets of surveillance experiments conducted with two active cameras and a motion sensor grid are promising.

From Smart Camera to SmartHub: Embracing Cloud for Video Surveillance

Smart cameras were conceived to provide scalable solutions to automatic video analysis applications, such as surveillance and monitoring. Since then, many algorithms and system architectures have been proposed, which use smart cameras to distribute functionality and save bandwidth. Still, smart cameras are rarely used in commercial systems and real installations. In this paper, we investigate the reason behind the scarce commercial usage of smart cameras. We found that, in order to achieve scalability, smart cameras put additional constraints on the quality of input data to the vision algorithms, making it an unfavourable choice for future multicamera systems. We recognized that these constraints can be relaxed by following a cloud based hub architecture and propose a cloud entity, SmartHub, which provides a scalable solution with reduced constraints on the quality. A framework is proposed for designing SmartHub system for a given camera placement. Experiments show the efficacy of SmartHub based systems in multicamera scenarios

Modeling quality of information in multi-sensor surveillance systems

Current surveillance systems use multiple sensors and media processing techniques in order to record/detect information of interest in terms of events. Assessing the quality of information (QoI) of a surveillance system is an important task as any misleading information may lead to suspicion, undesired consequences, and unwanted invasion of privacy. In this paper, we propose a model to characterize QoI in multi-sensor surveillance systems in terms of four quality parameters, which are: accuracy, certainty, timeliness and integrity. The proposed model is extendable to include other quality parameters if deemed necessary for different task-specific scenarios, such as the ambient intelligence environment, which aims to provide context-aware personalized services to the poeple living in that environment. To demonstrate the utility of the proposed method, we provide experimental results in a surveillance system designed for identifying authorized entry in the observation area.

Goal-oriented optimal subset selection of correlated multimedia streams

A multimedia analysis system utilizes a set of correlated media streams, each of which, we assume, has a confidence level and a cost associated with it, and each of which partially helps in achieving the system goal. However, the fact that at any instant, not all of the media streams contribute towards a system goal brings up the issue of finding the best subset from the available set of media streams. For example, a subset of two video cameras and two microphones could be better than any other subset of sensors at some time instance to achieve a surveillance goal (e.g. event detection). This article presents a novel framework that finds the optimal subset of media streams so as to achieve the system goal under specified constraints. The proposed framework uses a dynamic programming approach to find the optimal subset of media streams based on three different criteria: first, by maximizing the probability of achieving the goal under the specified cost and confidence; second, by maximizing the confidence in the achieved goal under the specified cost and probability with which the goal is achieved; and third, by minimizing the cost to achieve the goal with a specified probability and confidence. Each of these problems is proven to be NP-Complete. From an AI point of view, the solution we propose is heuristic-based, and for each criterion, utilizes a heuristic function which for a given problem, combines optimal solutions of small-sized subproblems to yield a potential near-optimal solution to the original problem. The proposed framework allows for a tradeoff among the aforementioned three criteria, and offers the flexibility to compare whether any one set of media streams of low cost would be better than any other set of higher cost, or whether any one set of media streams of high confidence would be better than any other of low confidence. To show the utility of our framework

Goal-oriented optimal subset selection of correlated multimedia streams

10.1145/1198302.1198304ACM Transactions on Multimedia Computing, Communications and Applications3

Confidence Building Among Correlated Streams in Multimedia Surveillance Systems

Abstract. Multimedia surveillance systems utilize multiple correlated media streams, each of which has a different confidence level in accomplishing various surveillance tasks. For example, the system designer may have a higher confidence in the video stream compared to the audio stream for detecting humans running events. The confidence level of streams is usually precomputed based on their past accuracy. This traditional approach is cumbersome especially when we add a new stream in the system without the knowledge of its past history. This paper proposes a novel method which dynamically computes the confidence level of new streams based on their agreement/disagreement with the already trusted streams. The preliminary experimental results show the utility of our method.