Automatic semantic parsing of the ground-plane in scenarios recorded with multiple moving cameras

Nowadays, video surveillance scenarios usually rely on manually annotated focus areas to constrain automatic video analysis tasks. Whereas manual annotation simplifies several stages of the analysis, its use hinders the scalability of the developed solutions and might induce operational problems in scenarios recorded with Multiple and Moving Cameras (MMC). To tackle these problems, an automatic method for the cooperative extraction of Areas of Interest (AoIs) is proposed. Each captured frame is segmented into regions with semantic roles using a stateof- the-art method. Semantic evidences from different junctures, cameras and points-of-view are then spatio-temporally aligned on a common ground plane. Experimental results on widely-used datasets recorded with multiple but static cameras suggest that this process provides broader and more accurate AoIs than those manually defined in the datasets. Moreover, the proposed method naturally determines the projection of obstacles and functional objects in the scene, paving the road towards systems focused on the automatic analysis of human behaviour. To our knowledge, this is the first study dealing with this problematic, as evidenced by the lack of publicly available MMC benchmarks. To also cope with this issue, we provide a new MMC dataset with associated semantic scene annotationsThis study has been partially supported by the Spanish Government through its TEC2014-53176-R HAVideo projec

Exploitation of Data Correlation and Performance Enhancement in Wireless Sensor Networks

With the combination of wireless communications and embedded system, lots of progress has been made in the area of wireless sensor networks (WSNs). The networks have already been widely deployed, due to their self-organization capacity and low-cost advantage. However, there are still some technical challenges needed to be addressed. In the thesis, three algorithms are proposed in improving network energy efficiency, detecting data fault and reducing data redundancy. The basic principle behind the proposed algorithms is correlation in the data collected by WSNs. The first sensor scheduling algorithm is based on the spatial correlation between neighbor sensor readings. Given the spatial correlation, sensor nodes are clustered into groups. At each time instance, only one node within each group works as group representative, namely, sensing and transmitting sensor data. Sensor nodes take turns to be group representative. Therefore, the energy consumed by other sensor nodes within the same group can be saved. Due to the continuous nature of the data to be collected, temporal and spatial correlation of sensor data has been exploited to detect the faulty data. By exploitation of temporal correlation, the normal range of upcoming sensor data can be predicted by the historical observations. Based on spatial correlation, weighted neighbor voting can be used to diagnose whether the value of sensor data is reliable. The status of the sensor data, normal or faulty, is decided by the combination of these two proposed detection procedures. Similar to the sensor scheduling algorithm, the recursive principal component analysis (RPCA) based algorithm has been studied to detect faulty data and aggregate redundant data by exploitation of spatial correlation as well. The R-PCA model is used to process the sensor data, with the help of squared prediction error (SPE) score and cumulative percentage formula. When SPE score of a collected datum is distinctly larger than that of normal data, faults can be detected. The data dimension is reduced according to the calculation result of cumulative percentage formula. All the algorithms are simulated in OPNET or MATLAB based on practical and synthetic datasets. Performances of the proposed algorithms are evaluated in each chapter

Energy-Efficient Data Management in Wireless Sensor Networks

Wireless Sensor Networks (WSNs) are deployed widely for various applications. A variety of useful data are generated by these deployments. Since WSNs have limited resources and unreliable communication links, traditional data management techniques are not suitable. Therefore, designing effective data management techniques for WSNs becomes important. In this dissertation, we address three key issues of data management in WSNs. For data collection, a scheme of making some nodes sleep and estimating their values according to the other active nodes’ readings has been proved energy-efficient. For the purpose of improving the precision of estimation, we propose two powerful estimation models, Data Estimation using a Physical Model (DEPM) and Data Estimation using a Statistical Model (DESM). Most of existing data processing approaches of WSNs are real-time. However, historical data of WSNs are also significant for various applications. No previous study has specifically addressed distributed historical data query processing. We propose an Index based Historical Data Query Processing scheme which stores historical data locally and processes queries energy-efficiently by using a distributed index tree. Area query processing is significant for various applications of WSNs. No previous study has specifically addressed this issue. We propose an energy-efficient in-network area query processing scheme. In our scheme, we use an intelligent method (Grid lists) to describe an area, thus reducing the communication cost and dropping useless data as early as possible. With a thorough simulation study, it is shown that our schemes are effective and energy- efficient. Based on the area query processing algorithm, an Intelligent Monitoring System is designed to detect various events and provide real-time and accurate information for escaping, rescuing, and evacuation when a dangerous event happened

Fine-grained performance analysis of massive MTC networks with scheduling and data aggregation

Abstract. The Internet of Things (IoT) represents a substantial shift within wireless communication and constitutes a relevant topic of social, economic, and overall technical impact. It refers to resource-constrained devices communicating without or with low human intervention. However, communication among machines imposes several challenges compared to traditional human type communication (HTC). Moreover, as the number of devices increases exponentially, different network management techniques and technologies are needed. Data aggregation is an efficient approach to handle the congestion introduced by a massive number of machine type devices (MTDs). The aggregators not only collect data but also implement scheduling mechanisms to cope with scarce network resources. This thesis provides an overview of the most common IoT applications and the network technologies to support them. We describe the most important challenges in machine type communication (MTC). We use a stochastic geometry (SG) tool known as the meta distribution (MD) of the signal-to-interference ratio (SIR), which is the distribution of the conditional SIR distribution given the wireless nodes’ locations, to provide a fine-grained description of the per-link reliability. Specifically, we analyze the performance of two scheduling methods for data aggregation of MTC: random resource scheduling (RRS) and channel-aware resource scheduling (CRS). The results show the fraction of users in the network that achieves a target reliability, which is an important aspect to consider when designing wireless systems with stringent service requirements. Finally, the impact on the fraction of MTDs that communicate with a target reliability when increasing the aggregators density is investigated

A Sparse Sensor Placement Strategy based on Information Entropy and Data Reconstruction for Ocean Monitoring

Sparse sensor placement strategies are applied to reconstruct a region’s full-state data conditioned to a limited number of sensors, particularly crucial to ocean monitoring systems. In maritime systems, existing sparse sensor placement methods consider the reconstruction error of data or rely on specific requirements. Considering how sensors acquire essential information for monitoring systems, the utilization of entropy from information theory becomes quite interesting. In this article, we show that entropy measurements on different quantities of information are sensitive to indicate the border areas, thus requiring a balance between the number of sensors needed and the amount of information collected by them in coastal areas. Due to such, we propose (i) a novel sparse sensor placement strategy based on entropy, where the entropy measurements in temporal dimension are utilized for sample selection, so portions of samples selected are utilized for training data, significantly improving the training efficiency without sacrificing accuracy of subsequent data reconstruction. In the proposed strategy, (ii) we use orthogonal triangle decomposition from linear algebra, where a low-cost sensor is employed as a pivot. In terms of spatial dimension, the entropy of each location is adopted as entropy weight to reconstruct full-state data. Additionally, (iii) the strategy employs a greedy algorithm of weighted column pivoting for the orthogonal triangle decomposition, which is designed to suit yet effectively seek additional information and minimal reconstruction error in each iteration processing step. Experimental results using Sea Surface Temperature (SST) data show that the proposed strategy outperforms existing methods, acquiring more information, ensuring higher efficiency, and reducing costs while minimizing reconstruction errors

Energy-efficient node selection algorithms with correlation optimization in wireless sensor networks

The sensing data of nodes is generally correlated in dense wireless sensor networks, and the active node selection problem aims at selecting a minimum number of nodes to provide required data services within error threshold so as to efficiently extend the network lifetime. In this paper, we firstly propose a new Cover Sets Balance (CSB) algorithm to choose a set of active nodes with the partially ordered tuple (data coverage range, residual energy). Then, we introduce a new Correlated Node Set Computing (CNSC) algorithm to find the correlated node set for a given node. Finally, we propose a High Residual Energy First (HREF) node selection algorithm to further reduce the number of active nodes. Extensive experiments demonstrate that HREF significantly reduces the number of active nodes, and CSB and HREF effectively increase the lifetime of wireless sensor networks compared with related works.This work is supported by the National Science Foundation of China under Grand nos. 61370210 and 61103175, Fujian Provincial Natural Science Foundation of China under Grant nos. 2011J01345, 2013J01232, and 2013J01229, and the Development Foundation of Educational Committee of Fujian Province under Grand no. 2012JA12027. It has also been partially supported by the "Ministerio de Ciencia e Innovacion," through the "Plan Nacional de I+D+i 2008-2011" in the "Subprograma de Proyectos de Investigacion Fundamental," Project TEC2011-27516, and by the Polytechnic University of Valencia, though the PAID-15-11 multidisciplinary Projects.Cheng, H.; Su, Z.; Zhang, D.; Lloret, J.; Yu, Z. (2014). Energy-efficient node selection algorithms with correlation optimization in wireless sensor networks. International Journal of Distributed Sensor Networks. 2014:1-14. https://doi.org/10.1155/2014/576573S1142014Yick, J., Mukherjee, B., & Ghosal, D. (2008). Wireless sensor network survey. Computer Networks, 52(12), 2292-2330. doi:10.1016/j.comnet.2008.04.002Sendra, S., Lloret, J., Garcia, M., & Toledo, J. F. (2011). Power Saving and Energy Optimization Techniques for Wireless Sensor Neworks (Invited Paper). Journal of Communications, 6(6). doi:10.4304/jcm.6.6.439-459Diallo, O., Rodrigues, J. J. P. C., Sene, M., & Lloret, J. (2015). Distributed Database Management Techniques for Wireless Sensor Networks. IEEE Transactions on Parallel and Distributed Systems, 26(2), 604-620. doi:10.1109/tpds.2013.207Oliveira, L. M. L., Rodrigues, J. J. P. C., Elias, A. G. F., & Zarpelão, B. B. (2014). Ubiquitous Monitoring Solution for Wireless Sensor Networks with Push Notifications and End-to-End Connectivity. Mobile Information Systems, 10(1), 19-35. doi:10.1155/2014/270568Diallo, O., Rodrigues, J. J. P. C., & Sene, M. (2012). Real-time data management on wireless sensor networks: A survey. Journal of Network and Computer Applications, 35(3), 1013-1021. doi:10.1016/j.jnca.2011.12.006Boyinbode, O., Le, H., & Takizawa, M. (2011). A survey on clustering algorithms for wireless sensor networks. International Journal of Space-Based and Situated Computing, 1(2/3), 130. doi:10.1504/ijssc.2011.040339Aslam, N., Phillips, W., Robertson, W., & Sivakumar, S. (2011). A multi-criterion optimization technique for energy efficient cluster formation in wireless sensor networks. Information Fusion, 12(3), 202-212. doi:10.1016/j.inffus.2009.12.005Karaboga, D., Okdem, S., & Ozturk, C. (2012). Cluster based wireless sensor network routing using artificial bee colony algorithm. Wireless Networks, 18(7), 847-860. doi:10.1007/s11276-012-0438-zNaeimi, S., Ghafghazi, H., Chow, C.-O., & Ishii, H. (2012). A Survey on the Taxonomy of Cluster-Based Routing Protocols for Homogeneous Wireless Sensor Networks. Sensors, 12(6), 7350-7409. doi:10.3390/s120607350Lloret, J., Garcia, M., Bri, D., & Diaz, J. (2009). A Cluster-Based Architecture to Structure the Topology of Parallel Wireless Sensor Networks. Sensors, 9(12), 10513-10544. doi:10.3390/s91210513Rajagopalan, R., & Varshney, P. (2006). Data-aggregation techniques in sensor networks: a survey. IEEE Communications Surveys & Tutorials, 8(4), 48-63. doi:10.1109/comst.2006.283821Al-Karaki, J. N., Ul-Mustafa, R., & Kamal, A. E. (2009). Data aggregation and routing in Wireless Sensor Networks: Optimal and heuristic algorithms. Computer Networks, 53(7), 945-960. doi:10.1016/j.comnet.2008.12.001Tan, H. O., Korpeoglu, I., & Stojmenovic, I. (2011). Computing Localized Power-Efficient Data Aggregation Trees for Sensor Networks. IEEE Transactions on Parallel and Distributed Systems, 22(3), 489-500. doi:10.1109/tpds.2010.68Gao, Q., Zuo, Y., Zhang, J., & Peng, X.-H. (2010). Improving Energy Efficiency in a Wireless Sensor Network by Combining Cooperative MIMO With Data Aggregation. IEEE Transactions on Vehicular Technology, 59(8), 3956-3965. doi:10.1109/tvt.2010.2063719Wei, G., Ling, Y., Guo, B., Xiao, B., & Vasilakos, A. V. (2011). Prediction-based data aggregation in wireless sensor networks: Combining grey model and Kalman Filter. Computer Communications, 34(6), 793-802. doi:10.1016/j.comcom.2010.10.003Xiang, L., Luo, J., & Vasilakos, A. (2011). Compressed data aggregation for energy efficient wireless sensor networks. 2011 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks. doi:10.1109/sahcn.2011.5984932Xu, Y., & Choi, J. (2012). Spatial prediction with mobile sensor networks using Gaussian processes with built-in Gaussian Markov random fields. Automatica, 48(8), 1735-1740. doi:10.1016/j.automatica.2012.05.029Min, J.-K., & Chung, C.-W. (2010). EDGES: Efficient data gathering in sensor networks using temporal and spatial correlations. Journal of Systems and Software, 83(2), 271-282. doi:10.1016/j.jss.2009.08.004Jianzhong Li, & Siyao Cheng. (2012). (ε, δ)-Approximate Aggregation Algorithms in Dynamic Sensor Networks. IEEE Transactions on Parallel and Distributed Systems, 23(3), 385-396. doi:10.1109/tpds.2011.193Hung, C.-C., Peng, W.-C., & Lee, W.-C. (2012). Energy-Aware Set-Covering Approaches for Approximate Data Collection in Wireless Sensor Networks. IEEE Transactions on Knowledge and Data Engineering, 24(11), 1993-2007. doi:10.1109/tkde.2011.224Liu, C., Wu, K., & Pei, J. (2007). An Energy-Efficient Data Collection Framework for Wireless Sensor Networks by Exploiting Spatiotemporal Correlation. IEEE Transactions on Parallel and Distributed Systems, 18(7), 1010-1023. doi:10.1109/tpds.2007.1046Xiaobo Zhang, Heping Wang, Nait-Abdesselam, F., & Khokhar, A. A. (2009). Distortion Analysis for Real-Time Data Collection of Spatially Temporally Correlated Data Fields in Wireless Sensor Networks. IEEE Transactions on Vehicular Technology, 58(3), 1583-1594. doi:10.1109/tvt.2008.928906Karasabun, E., Korpeoglu, I., & Aykanat, C. (2013). Active node determination for correlated data gathering in wireless sensor networks. Computer Networks, 57(5), 1124-1138. doi:10.1016/j.comnet.2012.11.018Gupta, H., Navda, V., Das, S., & Chowdhary, V. (2008). Efficient gathering of correlated data in sensor networks. ACM Transactions on Sensor Networks, 4(1), 1-31. doi:10.1145/1325651.1325655Campobello, G., Leonardi, A., & Palazzo, S. (2012). Improving Energy Saving and Reliability in Wireless Sensor Networks Using a Simple CRT-Based Packet-Forwarding Solution. IEEE/ACM Transactions on Networking, 20(1), 191-205. doi:10.1109/tnet.2011.2158442Tseng, L.-C., Chien, F.-T., Zhang, D., Chang, R. Y., Chung, W.-H., & Huang, C. (2013). Network Selection in Cognitive Heterogeneous Networks Using Stochastic Learning. IEEE Communications Letters, 17(12), 2304-2307. doi:10.1109/lcomm.2013.102113.131876Rodrigues, J. J. P. C., & Neves, P. A. C. S. (2010). A survey on IP-based wireless sensor network solutions. International Journal of Communication Systems, n/a-n/a. doi:10.1002/dac.1099Aziz, A. A., Sekercioglu, Y. A., Fitzpatrick, P., & Ivanovich, M. (2013). A Survey on Distributed Topology Control Techniques for Extending the Lifetime of Battery Powered Wireless Sensor Networks. IEEE Communications Surveys & Tutorials, 15(1), 121-144. doi:10.1109/surv.2012.031612.00124Mehlhorn, K. (1988). A faster approximation algorithm for the Steiner problem in graphs. Information Processing Letters, 27(3), 125-128. doi:10.1016/0020-0190(88)90066-xCheng, H., Liu, Q., & Jia, X. (2006). Heuristic algorithms for real-time data aggregation in wireless sensor networks. Proceeding of the 2006 international conference on Communications and mobile computing - IWCMC ’06. doi:10.1145/1143549.1143774Cheng, H., Guo, R., & Chen, Y. (2013). Node Selection Algorithms with Data Accuracy Guarantee in Service-Oriented Wireless Sensor Networks. International Journal of Distributed Sensor Networks, 9(4), 527965. doi:10.1155/2013/52796

Optimizing 3D Convolutions on Stereo Matching for Resource Efficient Computation

13301甲第5510号博士(工学)金沢大学博士論文本文Full 以下に掲載：Sensors 21(20) pp.6808 2021. MDPI. 共著者：Jianqiang Xiao, Dianbo Ma, Satoshi Yaman

Security of the Internet of Things: Vulnerabilities, Attacks and Countermeasures

Wireless Sensor Networks (WSNs) constitute one of the most promising third-millennium technologies and have wide range of applications in our surrounding environment. The reason behind the vast adoption of WSNs in various applications is that they have tremendously appealing features, e.g., low production cost, low installation cost, unattended network operation, autonomous and longtime operation. WSNs have started to merge with the Internet of Things (IoT) through the introduction of Internet access capability in sensor nodes and sensing ability in Internet-connected devices. Thereby, the IoT is providing access to huge amount of data, collected by the WSNs, over the Internet. Hence, the security of IoT should start with foremost securing WSNs ahead of the other components. However, owing to the absence of a physical line-of-defense, i.e., there is no dedicated infrastructure such as gateways to watch and observe the flowing information in the network, security of WSNs along with IoT is of a big concern to the scientific community. More specifically, for the application areas in which CIA (confidentiality, integrity, availability) has prime importance, WSNs and emerging IoT technology might constitute an open avenue for the attackers. Besides, recent integration and collaboration of WSNs with IoT will open new challenges and problems in terms of security. Hence, this would be a nightmare for the individuals using these systems as well as the security administrators who are managing those networks. Therefore, a detailed review of security attacks towards WSNs and IoT, along with the techniques for prevention, detection, and mitigation of those attacks are provided in this paper. In this text, attacks are categorized and treated into mainly two parts, most or all types of attacks towards WSNs and IoT are investigated under that umbrella: “Passive Attacks” and “Active Attacks”. Understanding these attacks and their associated defense mechanisms will help paving a secure path towards the proliferation and public acceptance of IoT technology

Data Analytics and Performance Enhancement in Edge-Cloud Collaborative Internet of Things Systems

Based on the evolving communications, computing and embedded systems technologies, Internet of Things (IoT) systems can interconnect not only physical users and devices but also virtual services and objects, which have already been applied to many different application scenarios, such as smart home, smart healthcare, and intelligent transportation. With the rapid development, the number of involving devices increases tremendously. The huge number of devices and correspondingly generated data bring critical challenges to the IoT systems. To enhance the overall performance, this thesis aims to address the related technical issues on IoT data processing and physical topology discovery of the subnets self-organized by IoT devices. First of all, the issues on outlier detection and data aggregation are addressed through the development of recursive principal component analysis (R-PCA) based data analysis framework. The framework is developed in a cluster-based structure to fully exploit the spatial correlation of IoT data. Specifically, the sensing devices are gathered into clusters based on spatial data correlation. Edge devices are assigned to the clusters for the R-PCA based outlier detection and data aggregation. The outlier-free and aggregated data are forwarded to the remote cloud server for data reconstruction and storage. Moreover, a data reduction scheme is further proposed to relieve the burden on the trunk link for data uploading by utilizing the temporal data correlation. Kalman filters (KFs) with identical parameters are maintained at the edge and cloud for data prediction. The amount of data uploading is reduced by using the data predicted by the KF in the cloud instead of uploading all the practically measured data. Furthermore, an unmanned aerial vehicle (UAV) assisted IoT system is particularly designed for large-scale monitoring. Wireless sensor nodes are flexibly deployed for environmental sensing and self-organized into wireless sensor networks (WSNs). A physical topology discovery scheme is proposed to construct the physical topology of WSNs in the cloud server to facilitate performance optimization, where the physical topology indicates both the logical connectivity statuses of WSNs and the physical locations of WSN nodes. The physical topology discovery scheme is implemented through the newly developed parallel Metropolis-Hastings random walk based information sampling and network-wide 3D localization algorithms, where UAVs are served as the mobile edge devices and anchor nodes. Based on the physical topology constructed in the cloud, a UAV-enabled spatial data sampling scheme is further proposed to efficiently sample data from the monitoring area by using denoising autoencoder (DAE). By deploying the encoder of DAE at the UAV and decoder in the cloud, the data can be partially sampled from the sensing field and accurately reconstructed in the cloud. In the final part of the thesis, a novel autoencoder (AE) neural network based data outlier detection algorithm is proposed, where both encoder and decoder of AE are deployed at the edge devices. Data outliers can be accurately detected by the large fluctuations in the squared error generated by the data passing through the encoder and decoder of the AE