Large–scale visual sensor networks have become progressively an essential part of our daily lives underpinning many technological, financial, and social advancements today, with applications in smart cities, traffic monitoring, environmental pollution control, public safety, and crime prevention

Aghajan, H

Bebis, G

Gong, S

Loutfi, A

Sigal, L

Spagnolo, P

Zheng, W-S

IEEE Transactions on Circuits and Systems for Video Technology

English

Queen Mary Research Online

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 1Guest Editorial Introduction to the Special Issue onLarge-scale Visual Sensor Networks: Architecturesand Applications.LARGE-scale visual sensor networks have become pro-gressively an essential part of our daily lives underpin-ning many technological, financial, and social advancementtoday, with applications in smart cities, traffic monitoring,environmental pollution control, public safety and crime pre-vention.Sensing technologies have made huge progress in the lasttwo decades, with the emerging opportunity to miniaturizeand communicate wirelessly in low power modalities at theedge. Motivated by several societal needs, such as security,natural interfaces, gaming, affective computing and assistedliving, there has been an increasing interest in developingvisual sensor networks for urban space, smart homes, andenvironmental monitoring in recent years.The integration of multiple devices in sensor networksis a hot topic in the scientific community. Thanks to boththe decreasing costs of visual sensors and the increasingperformance of distributed edge devices, the possibility ofperforming integration of a forest of both homogeneous andheterogeneous sensors is not only a theoretical study but alsoa makeable technological solution in a real world context.On one hand, the capability of processing data from multiplesensors opens up new scenarios, allowing the realization oflarge-scale sensor infrastructures. In this way, many socialactivities can be understood from a new point of view. Onthe other hand, large-scale sensor networks introduce newchallenges including data synchronization, communications,security, data fusion, real-time processing, and high-leveldecision making.This poses several research issues ranging from data storage,wireless communication coverage that allows self-organizationof the topology of the networked sensors, to their coordinationin terms of optimizing computation and where to focalizesensing and what to sense.In a sensor network, a key-role is played by large-scalevisual networks: the decreasing cost of smart devices enablessensor networks to scale to a large number of devices. Addingor removing sensors in a seamless way is an important require-ment for sensor network architectures of the future, whereeverybody should be able to modify the network architecturewithout compromising the operation of the network itself. Inaddition, the impact of deep learning on multi-source big datafrom distributed networks need be investigated. The capacityof managing large-scale sensor networks will become strategicin the coming years for most IT organizations.Sensor network research was initially driven by expensivemilitary applications such as battlefield surveillance and en-emy tracking. Nowadays, the diffusion of more accessiblefixed and mobile sensors has changed significantly our dailylife, e.g. smartphones. Many applications based on distributedsensing systems for civil applications have been developed.These applications can be classified into habitat monitoring,environment observation and forecast systems, human activitymonitoring for health, security, and surveillance. An interest-ing perspective on how to exploit sensor networks has alsoemerged from the COVID-19 pandemic: The availability ofcapillary distributed sensor networks can be exploited for adeep monitoring of specific conditions, in an ideal way at asingle-person level, by means of mobile phones specific apps.The availability of such detailed information can be used formassive tracing operations, that can be formally considered asborderline in terms of privacy or human rights, but that canbe considered as a possibility in specific critical periods, likea pandemic.This special issue highlights the latest developments in largescale visual sensor networks and their applications, rangingfrom the design and implementation to the processing andunderstanding of large amount of video data. It presentscontributions on new solutions addressing some of the chal-lenges and difficulties in this field, as well as prototypes,systems, tools and techniques. It also includes general surveypapers predicting future directions of research and technologydevelopment.Lu et al. propose an adaptive region proposal scheme withfeature channel regularization to facilitate robust object track-ing. Tracking is considered as a linear regression problem; anensemble of correlation filters is trained online to distinguishthe foreground target from the background. Authors integrateadaptively learned region proposals into an enhanced two-stream tracking framework based on correlation filters. Forthe tracking stream, a two-stage cascade correlation filters ondeep convolutional features is learned to ensure competitivetracking performance. For the detection stream, adaptive re-gion proposals have been employed, which are effective in re-covering target objects from tracking failures caused by heavyocclusion or out-of-view movement. In contrast to traditionaltracking-bydetection methods using random samples or slidingwindows, target re-detection over adaptively learned regionproposals is performed. Since region proposals naturally takethe objectness information into account, authors show that theproposed adaptive region proposals can handle the challengingscale estimation problem as well. Authors have extensivelytested their approach on OTB, VOT and UAV-123 datasetsdemonstrating that their method performs favorably againststate-of-the-art tracking algorithms.The contributions of the method proposed by Farinella etIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2al. are twofold. First, an approach for localization of shoppingcarts in a retail store from egocentric images is proposed.Second, a new database is presented to the scientific com-munity. Addressing the first task allows to infer informationon the behavior of the customers to understand how theymove in the store and what they pay more attention to. Tostudy the problem, a large dataset of images collected in areal retail store is proposed. The dataset comprises 19,531RGB images along with depth maps, ground truth cameraposes, as well as class labels specifying the areas of thestore in which each image has been acquired. The datasetcould surely encourages research in large-scale image-basedindoor localization; moreover, it addresses the scarcity of largedatasets to tackle the problem. Authors perform a benchmarkof several image-based localization techniques exploiting im-ages and depth information on the proposed dataset. In thiswork authors compare both localization performances andspace/time requirements. The results show that, while state-of-the-art approaches allow to achieve good results, there isspace for improvement.Zhang et al. propose an approach based on Siamese net-works; these have been successfully introduced into visualtracking, which match the best candidate and a target templatevia a couple of networks with shared parameters. However,most Siamese network-based trackers (SNTs) are tailored tobest match the canonical posture of the template and thesearch-region images, resulting in inferior performance whenthe target objects have large-scale pose variations. Besides,SNTs fail to accurately discriminate distractors because theyonly leverage high-level semantic features as target repre-sentations that cannot well tell from different targets of thesame category. To address these issues, authors introduce anefficient and effective SNT that is based on feature align-ment and aggregation networks. An effective feature align-ment network module to calibrate the search-region imageis designed. This module results in a more reliable matchingresponse that is robust to severe target pose variations. Then,an effective shallow-level and high-level feature aggregationnetwork module to complement the feature characteristics isdeveloped. This way, the learned feature representation notonly well differentiates the target from distractors, but alsointroduces robustness to target appearance variations. After-wards, a channel-attention mechanism to further strengthenthe discriminative capability of the aggregated feature rep-resentation is employed. Finally, both the alignment andthe aggregation modules are seamlessly integrated into theSiamese networks for robust tracking. Extensive evaluationson a variety of benchmarks including VOT-2017, OTB-100,UAV123 and GOT-10k demonstrate favorable performance ofthis tracker against state-of-the-art ones with a speed of 60fps.Fang et al. introduce an approach for future frame predictionin video, a traditional problem in computer vision, useful for arange of practical applications, such as intention prediction orvideo anomaly detection. However, this task is challengingbecause of the complex and dynamic evolution of scene.The difficulty of video frame prediction is to model theinherent spatio-temporal correlation between frames and posean adaptive and flexible framework for large motion change orappearance variation. Authors construct a deep multi-branchmask network (DMMNet) which adaptively fuses the advan-tages of optical flow warping and RGB pixel synthesizingmethods. In the procedure of DMMNet, mask layer is addedin each branch to adaptively adjust the magnitude range ofestimated optical flow and the weight of predicted frames byoptical flow warping and RGB pixel synthesizing, respectively.In other words, a more flexible masking network for motionand appearance fusion on video frame prediction is provided.Exhaustive experiments on Caltech pedestrian and UCF101datasets show that this model can obtain favorable videoframe prediction performance compared with the state-of-the-art methods.Paolo SpagnoloNational Research Council of ItalyVia Monteroni - Lecce, Italyemail: paolo.spagnolo@cnr.itGeorge BebisUniversity of Nevada, USAReno, NV 89557email: bebis@cse.unr.eduHamid AghajanDepartment of Electrical EngineeringSharif University of Technology, Iranemail: aghajan@ee.sharif.eduShaogang GongSchool of Electronic Engineering andComputer ScienceQueen Mary University of London, UKemail: s.gong@qmul.ac.ukAmy LoutfiSchool of Science and TechnologyOrebro University, Swedenemail: amy.loutfi@oru.seLeonid SigalDepartment of Computer ScienceUniversity of British ColumbiaVancouver, Canadaemail: lsigal@cs.ubc.caWei-Shi ZhengSchool of Data and Computer ScienceSun Yat-sen University, Chinaemail: wszheng@ieee.orgIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 3Paolo Spagnolo received the engineering degreein computer science from the University of Lecce,Lecce, Italy, in 2002. Since then he has been withthe Italian National Research Council. He has beenworking on several research topics regarding Ar-tificial Intelligence and Computer Vision. He hasstudied techniques and methodologies for multidi-mensional digital signal processing; linear and non-linear signal characterization; signal features extrac-tion; supervised and unsupervised classification ofsignals; deep neural network (CNN). He is author ofover 80 papers on Artificial Intelligence. He also acts as a reviewer for severalinternational journals. He participated in a number of international projects inthe area of image and video analysis. He has been regularly invited to takepart in the Scientific Committees of national and international conferences.Hamid Aghajan received his B.S. degree fromSharif University of Technology, Tehran, Iran, in1989, and M.S. and Ph.D. degrees from StanfordUniversity in 1991 and 1995, respectively, all inelectrical engineering. After gaining industry expe-rience in the silicon valley from 1995 to 2002 inthe fields of semiconductors, wireless and opticalcommunications, and genetics, he established andserved as director of a research lab on wirelesssensor networks and ambient intelligence at StanfordUniversity in 2003, where he supervised the researchof numerous students and international visitors until 2013 on topics involvingmethods and applications in smart homes, multi-camera networks, behaviorlearning and adaptive services in home and office, smart transportation, and el-derly monitoring. He served as founder and steering committee member of theInt. Conf. on Distributed Smart Cameras from 2007 to 2019. He co-foundedJournal of Ambient Intelligence and Smart Environments in 2009 and has beenserving as its co-editor-in-chief. He has been associated with the departmentof telecommunication and information processing in Gent University since2009, Belgium, and with the department of electrical engineering at SharifUniversity of Technology, Iran, since 2013. His recent research interests arein the field of neuroscience with an emphasis on characterizing the brain’sresponse to external stimuli for understanding network deficiencies causedby neurodegenerative diseases such as Alzheimer’s and designing stimulationtechniques for therapeutic brain entrainment.George Bebis received the B.S. degree in mathemat-ics and the M.S. degree in computer science fromthe University of Crete, Crete, Greece, in 1987 and1991, respectively, and the Ph.D. degree in electricaland computer engineering from the University ofCentral Florida, Orlando, FL, USA, in 1996. He iscurrently a Foundation Professor with the Depart-ment of Computer Science and Engineering (CSE),University of Nevada, Reno (UNR), Reno, NV, USA,and the Director of the Computer Vision Laboratory.From 2013 to 2018, he served as a Department Chairof CSE, UNR. His research has been funded by NSF, NASA, ONR, NIJ, andFord Motor Company. His research interests include computer vision, imageprocessing, pattern recognition, machine learning, and evolutionary comput-ing. Dr. Bebis is an Associate Editor of the Machine Vision and ApplicationsJournal and serves on the Editorial Board of the International Journal onArtificial Intelligence Tools, and the Computer Methods in Biomechanics andBiomedical Engineering: Imaging and Visualization. He has served on theprogram committees of various national and international conferences and theFounder/main Organizer of the International Symposium on Visual Computing(ISVC) and the International Symposium on Mathematical and ComputationalOncology (ISMCO).Shaogang Gong received the DPhil degree in com-puter vision from Keble College, Oxford Univer-sity, in 1989. He has been the Professor of VisualComputation with the Queen Mary University ofLondon since 2001. He is an elected fellow ofthe Institution of Electrical Engineers, a fellow ofthe British Computer Society, a member of theUK Computing Research Committee, and a TuringFellow of the Alan Turing Institute. He served onthe Steering Panel of the UK Government ChiefScientific Advisor’s Science Review. His researchinterests include computer vision, machine learning, and video analysis. Hehas published over 400 research papers and 7 books on topics includingPerson Re-Identification, Visual Analysis of Behaviour, Video Analytics forBusiness Intelligence, Dynamic Vision from Images to Face Recognition,Analysis and Modelling of Faces and Gestures. He is the inventor of 42international patents. He won the Institution of Engineering and Technology2020 Achievement Medal for Vision Engineering, for outstanding achievementand superior performance in contributing to public safety.Amy Loutfi is a Professor in Information Tech-nology at Örebro University and leads the researchgroup - AASS Machine Perception and InteractionLab. Amy Loutfi received her Ph.D. in ComputerScience in 2006 with a research topic in machineperception. Specifically, she researched about howgas sensors could be integrated onto robotic plat-forms and how these robots can interact with humansin order to solve a range of problems that requiredsensing and perception. She has since broadened herresearch interests to include general research direc-tions within machine perception, where AI methods like Machine learning areused for the interpretation of sensor data. She also has broadened her researchin the area of Human Robot Interaction where she has studied HRI in variousplatforms that include fully autonomous robots, but also teleoperated robots.She has a long experience working with industry and the public sector onresearch projects dealing with AI, robotics and human-robot interaction.Leonid Sigal is an Associate Professor in the De-partment of Computer Science at the University ofBritish Columbia. He is also a Canada ResearchChair (CRC II) in Computer Vision and MachineLearning and a remote Faculty Member of the VectorInstitute for AI in Toronto. In addition, he serves asan Academic Advisor to Borealis AI. His researchfocuses on problems of visual understanding andreasoning. This includes object recognition, sceneunderstanding, articulated motion capture, motionmodeling, action recognition, motion perception,manifold learning, transfer learning, character and cloth animation and anumber of other directions on the intersection of computer vision, machinelearning, and computer graphicsWei-Shi Zheng Dr. Wei-Shi Zheng is now a Pro-fessor with Sun Yat-sen University. His researchinterests include person/object association and ac-tivity understanding in visual surveillance, and therelated large-scale machine learning algorithm. Hehas ever served as area chairs of ICCV, CVPR,BMVC, IJCAI and AAAI. He is an IEEE MSATC member. He is an associate editor of PatternRecognition. He has ever joined Microsoft ResearchAsia Young Faculty Visiting Programme. He is arecipient of Excellent Young Scientists Fund of theNational Natural Science Foundation of China, and a recipient of RoyalSociety-Newton Advanced Fellowship of United Kingdom.

Introduction to the Special Issue on Large-Scale Visual Sensor Networks: Architectures and Applications

https://qmro.qmul.ac.uk/xmlui/bitstream/123456789/72704/2/Gong%20Guest%20editorial%20introduction%202021%20Accepted.pdf

Introduction to the Special Issue on Large-Scale Visual Sensor Networks: Architectures and Applications

Abstract

Similar works

Full text

Available Versions

Queen Mary Research Online