MultiIoT: Towards Large-scale Multisensory Learning for the Internet of Things

The Internet of Things (IoT), the network integrating billions of smart physical devices embedded with sensors, software, and communication technologies for the purpose of connecting and exchanging data with other devices and systems, is a critical and rapidly expanding component of our modern world. The IoT ecosystem provides a rich source of real-world modalities such as motion, thermal, geolocation, imaging, depth, sensors, video, and audio for prediction tasks involving the pose, gaze, activities, and gestures of humans as well as the touch, contact, pose, 3D of physical objects. Machine learning presents a rich opportunity to automatically process IoT data at scale, enabling efficient inference for impact in understanding human wellbeing, controlling physical devices, and interconnecting smart cities. To develop machine learning technologies for IoT, this paper proposes MultiIoT, the most expansive IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 tasks. MultiIoT introduces unique challenges involving (1) learning from many sensory modalities, (2) fine-grained interactions across long temporal ranges, and (3) extreme heterogeneity due to unique structure and noise topologies in real-world sensors. We also release a set of strong modeling baselines, spanning modality and task-specific methods to multisensory and multitask models to encourage future research in multisensory representation learning for IoT

Human Behaviour Recognition using Fuzzy System in Videos

Human behavior can be detected and analyzed using video sequence is a latest research topic in computer vision & machine learning. Human behavior is used as a basis for many modern applications, such as video surveillance, content-based information retrieval from videos etc. HBA (Human behaviour analysis) is tricky to design and develop due to uncertainty and ambiguity involved in people’s daily activities. To address this gap, we propose hierarchical structure combining TDNN, tracking algorithms, and fuzzy systems. As a result, HBA system performance will be improved in terms of robustness, effectiveness and scalability

Attentive monitoring of multiple video streams driven by a Bayesian foraging strategy

In this paper we shall consider the problem of deploying attention to subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multi-stream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g. activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Dataset, a publicly available dataset, are presented to illustrate the utility of the proposed technique.Comment: Accepted to IEEE Transactions on Image Processin

Dynamic Bayesian Collective Awareness Models for a Network of Ego-Things

A novel approach is proposed for multimodal collective awareness (CA) of multiple networked intelligent agents. Each agent is here considered as an Internet-of-Things (IoT) node equipped with machine learning capabilities; CA aims to provide the network with updated causal knowledge of the state of execution of actions of each node performing a joint task, with particular attention to anomalies that can arise. Data-driven dynamic Bayesian models learned from multisensory data recorded during the normal realization of a joint task (agent network experience) are used for distributed state estimation of agents and detection of abnormalities. A set of switching dynamic Bayesian network (DBN) models collectively learned in a training phase, each related to particular sensorial modality, is used to allow each agent in the network to perform synchronous estimation of possible abnormalities occurring when a new task of the same type is jointly performed. Collective DBN (CDBN) learning is performed by unsupervised clustering of generalized errors (GEs) obtained from a starting generalized model. A growing neural gas (GNG) algorithm is used as a basis to learn the discrete switching variables at the semantic level. Conditional probabilities linking nodes in the CDBN models are estimated using obtained clusters. CDBN models are associated with a Bayesian inference method, namely, distributed Markov jump particle filter (D-MJPF), employed for joint state estimation and abnormality detection. The effects of networking protocols and of communications in the estimation of state and abnormalities are analyzed. Performance is evaluated by using a small network of two autonomous vehicles performing joint navigation tasks in a controlled environment. In the proposed method, first the sharing of observations is considered in ideal condition, and then the effects of a wireless communication channel have been analyzed for the collective abnormality estimation of the agents. Rician wireless channel and the usage of two protocols (i.e., IEEE 802.11p and IEEE 802.15.4) along with different channel conditions are considered as well

Real-time human ambulation, activity, and physiological monitoring:taxonomy of issues, techniques, applications, challenges and limitations

Automated methods of real-time, unobtrusive, human ambulation, activity, and wellness monitoring and data analysis using various algorithmic techniques have been subjects of intense research. The general aim is to devise effective means of addressing the demands of assisted living, rehabilitation, and clinical observation and assessment through sensor-based monitoring. The research studies have resulted in a large amount of literature. This paper presents a holistic articulation of the research studies and offers comprehensive insights along four main axes: distribution of existing studies; monitoring device framework and sensor types; data collection, processing and analysis; and applications, limitations and challenges. The aim is to present a systematic and most complete study of literature in the area in order to identify research gaps and prioritize future research directions

Towards the internet of agents: an analysis of the internet of things from the intelligence and autonomy perspective

Recently, the scientific community has demonstrated a special interest in the process related to the integration of the agent-oriented technology with Internet of Things (IoT) platforms. Then, it arises a novel approach named Internet of Agents (IoA) as an alternative to add an intelligence and autonomy component for IoT devices and networks. This paper presents an analysis of the main benefits derived from the use of the IoA approach, based on a practical point of view regarding the necessities that humans demand in their daily life and work, which can be solved by IoT networks modeled as IoA infrastructures. It has been presented 24 study cases of the IoA approach at different domains ––smart industry, smart city and smart health wellbeing–– in order to define the scope of these proposals in terms of intelligence and autonomy in contrast to their corresponding generic IoT applications.En los últimos años, la comunidad científica ha mostrado un interés especial en torno al proceso de integración de la tecnología orientada a agentes sobre plataformas de Internet de las Cosas (IoT, por sus siglas en inglés). Surge así, un nuevo enfoque denominado Internet de los Agentes (IoA, por sus siglas en inglés) como una alternativa para añadir un componente de inteligencia y autonomía sobre los dispositivos y redes de IoT. El presente trabajo muestra un análisis de los principales beneficios derivados del uso del enfoque del IoA, visto desde las actuales necesidades que el ser humano demanda en su trabajo y vida cotidiana, las cuales pueden ser resueltas por redes de IoT modeladas como infraestructuras de IoA. Se plantea un total de 24 casos prácticos de aplicaciones de IoA en diferentes dominios ––industria, ciudad, y salud y bienestar inteligente–– a fin de determinar el alcance de dichas aplicaciones en términos de inteligencia y autonomía respecto a sus correspondientes aplicaciones genéricas de IoT.This study was founded by the Ecuadorian Ministry of Higher Education, Science, Technology and Innovation (SENESCYT)

Inferential measurements for situation awareness: enhancing traffic surveillance by machine learning.

The paper proposes a generic approach to building inferential measurement systems. The large amount of data needed to be acquired and processed by such systems necessitates the use of machine learning techniques. In this study, an inferential measurement system aimed at enhancing situation awareness has been developed and tested on simulated traffic surveillance data. The performance of several Computational Intelligence techniques within this system has been examined and compared on the data containing anomalous driving patterns

Gunnislake Fish Counter Annual Report 2003

This is the Gunnislake Fish Counter, Annual Report 2003 produced by the Environment Agency South West Region on March 2004. The report presents the daily upstream counts of migratory salmonids recorded at Gunnislake weir fish counting station and trap (River Tamar SX 435 713) in 2003. Data contained within this report covers the period of the commercial migratory salmonid net buy-back scheme and the National Spring Salmon Bylaws. The total combined annual count of upstream migrating salmon and sea trout on the River Tamar in 2003 was 7% higher than the 9-year average. The minimum salmon count for 2003 was 3626. The 2003 upstream count for sea trout was 9913. Trap data for 2003 is consistent with historic trapping and net data in terms of the size split between salmon and sea trout stocks