Ground truth annotation of traffic video data

This paper presents a software application to generate ground-truth data on video files from traffic surveillance cameras used for Intelligent Transportation Systems (IT systems). The computer vision system to be evaluated counts the number of vehicles that cross a line per time unit intensity-, the average speed and the occupancy. The main goal of the visual interface presented in this paper is to be easy to use without the requirement of any specific hardware. It is based on a standard laptop or desktop computer and a Jog shuttle wheel. The setup is efficient and comfortable because one hand of the annotating person is almost all the time on the space key of the keyboard while the other hand is on the jog shuttle wheel. The mean time required to annotate a video file ranges from 1 to 5 times its duration (per lane) depending on the content. Compared to general purpose annotation tool a time factor gain of about 7 times is achieved.This work was funded by the Spanish Government project MARTA under the CENIT program and CICYT contract TEC2009-09146.Mossi García, JM.; Albiol Colomer, AJ.; Albiol Colomer, A.; Oliver Moll, J. (2014). Ground truth annotation of traffic video data. Multimedia Tools and Applications. 1-14. https://doi.org/10.1007/s11042-013-1396-xS114Albiol A et al (2011) Detection of parked vehicles using spatiotemporal maps. IEEE Trans Intell Transport Syst 12(4):1277–1291Blunsden SJ, Fisher R (2010) The BEHAVE video dataset: ground truthed video for multi-person behavior classification. Annal British Mach Vis Assoc 4:1–12Bradski G, Kaehler A (2008) Learning OpenCV: Computer vision with the OpenCV library. O'Reilly Media, IncorporatedBrooke J. SUS: a “quick and dirty” usability scale. Usability evaluation in industry. Taylor and FrancisBrostow GJ et al (2009) Semantic object classes in video: a high-definition ground truth database. Pattern Recognit Lett 30(2):88–97Buch N et al (2011) A review of computer vision techniques for the analysis of urban traffic. IEEE Trans Intell Transp Syst 12(3):920–939D’Orazio T et al. (2009) A semi-automatic system for ground truth generation of soccer video sequences. Advanced Video and Signal Based Surveillance, 2009. AVSS’09. Sixth IEEE International Conference on (Sep. 2009), 559–564Dollar P et al (2012) Pedestrian detection: an evaluation of the state of the art. IEEE Trans Pattern Anal Mach Intell 34(4):743–761Faro A et al (2011) Adaptive background modeling integrated with luminosity sensors and occlusion processing for reliable vehicle detection. IEEE Trans Intell Transport Syst 12(4):1398–1412Giro-i-Nieto X et al (2010) GAT: a graphical annotation tool for semantic regions. Multimed Tool Appl 46(2–3):155–174i-LIDS. Image Library for Intelligent Detection Systems: www.ilids.co.uk . Home Office Scientific Development Branch, United Kingdom. Last Accessed February 2013Kasturi R et al (2009) Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol. IEEE Trans Pattern Anal Mach Intell 31(2):319–336Laganière R (2011) OpenCV 2 computer vision application programming cookbook. Packt Pub LimitedLorist MM et al (2000) Mental fatigue and task control: planning and preparation. Psychophysiology 37(5):614–625Russell B et al (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1):157–173Serrano M, Gracía J, Patricio M, Molina J (2010). Interactive video annotation tool. Distributed Computing and Artificial Intelligence, 325–332Traffic City Cameras. Ajuntament de València, Spain. http://camaras.valencia.es . Last Accessed February 2013TREC video retrieval evaluation. http://www-nlpir.nist.gov/projects/trecvid/Vezzani R, Cucchiara R (2010) Video Surveillance Online Repository (ViSOR): an integrated framework. Multimed Tool Appl 50(2):359–380ViPER: the video performance evaluation resource: http://viper-toolkit.sourceforge.net/Volkmer T et al. (2005) A web-based system for collaborative annotation of large image and video collections: an evaluation and user study. Proceedings of the 13th annual ACM international conference on Multimedia (New York, NY, USA, 2005), 892–901Zhang HB, Li SA, Chen SY, Su SZ, Duh DJ, Li SZ (2012) Adaptive photograph retrieval method. Multimedia Tools and Applications, Published online September 2012.Zou Y et al (2011) Traffic incident classification at intersections based on image sequences by HMM/SVM classifiers. Multimed Tool Appl 52(1):133–14

Using Keystroke Dynamics in a Multi-Agent System for User Guiding in Online Social Networks

[EN] Nowadays there is a strong integration of online social platforms and applications with our daily life. Such interactions can make risks arise and compromise the information we share, thereby leading to privacy issues. In this work, a proposal that makes use of a software agent that performs sentiment analysis and another performing stress analysis on keystroke dynamics data has been designed and implemented. The proposal consists of a set of new agents that have been integrated into a multi-agent system (MAS) for guiding users interacting in online social environments, which has agents for sentiment and stress analysis on text. We propose a combined analysis using the different agents. The MAS analyzes the states of the users when they are interacting, and warns them if the messages they write are deemed negative. In this way, we aim to prevent potential negative outcomes on social network sites (SNSs). We performed experiments in the laboratory with our private SNS Pesedia over a period of one month, so we gathered data about text messages and keystroke dynamics data, and used the datasets to train the artificial neural networks (ANNs) of the agents. A set of experiments was performed for discovering which analysis is able to detect a state of the user that propagates more in the SNS, so it may be more informative for the MAS. Our study will help develop future intelligent systems that utilize user data in online social environments for guiding or helping them in their social experience.This work was funded by the project TIN2017-89156-R of the Spanish government.Aguado-Sarrió, G.; Julian Inglada, VJ.; García-Fornes, A.; Espinosa Minguet, AR. (2020). Using Keystroke Dynamics in a Multi-Agent System for User Guiding in Online Social Networks. Applied Sciences. 10(11):1-20. https://doi.org/10.3390/app10113754S1201011O’Keeffe, G. S., & Clarke-Pearson, K. (2011). The Impact of Social Media on Children, Adolescents, and Families. PEDIATRICS, 127(4), 800-804. doi:10.1542/peds.2011-0054George, J. M., & Dane, E. (2016). Affect, emotion, and decision making. Organizational Behavior and Human Decision Processes, 136, 47-55. doi:10.1016/j.obhdp.2016.06.004Thelwall, M. (2017). TensiStrength: Stress and relaxation magnitude detection for social media texts. Information Processing & Management, 53(1), 106-121. doi:10.1016/j.ipm.2016.06.009Aguado, G., Julian, V., & Garcia-Fornes, A. (2018). Towards Aiding Decision-Making in Social Networks by Using Sentiment and Stress Combined Analysis. Information, 9(5), 107. doi:10.3390/info9050107Schouten, K., & Frasincar, F. (2016). Survey on Aspect-Level Sentiment Analysis. IEEE Transactions on Knowledge and Data Engineering, 28(3), 813-830. doi:10.1109/tkde.2015.2485209Lee, P.-M., Tsui, W.-H., & Hsiao, T.-C. (2015). The Influence of Emotion on Keyboard Typing: An Experimental Study Using Auditory Stimuli. PLOS ONE, 10(6), e0129056. doi:10.1371/journal.pone.0129056Vizer, L. M., Zhou, L., & Sears, A. (2009). Automated stress detection using keystroke and linguistic features: An exploratory study. International Journal of Human-Computer Studies, 67(10), 870-886. doi:10.1016/j.ijhcs.2009.07.005Huang, F., Zhang, X., Zhao, Z., Xu, J., & Li, Z. (2019). Image–text sentiment analysis via deep multimodal attentive fusion. Knowledge-Based Systems, 167, 26-37. doi:10.1016/j.knosys.2019.01.019Mehrabian, A. (1996). Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament. Current Psychology, 14(4), 261-292. doi:10.1007/bf02686918Ulinskas, M., Damaševičius, R., Maskeliūnas, R., & Woźniak, M. (2018). Recognition of human daytime fatigue using keystroke data. Procedia Computer Science, 130, 947-952. doi:10.1016/j.procs.2018.04.09

Passphrase and keystroke dynamics authentication: security and usability

It was found that employees spend a total 2.25 days within a 60 day period on password related activities. Another study found that over 85 days an average user will create 25 accounts with an average of 6.5 unique passwords. These numbers are expected to increase over time as more systems become available. In addition, the use of 6.5 unique passwords highlight that passwords are being reused which creates security concerns as multiple systems will be accessible by an unauthorised party if one of these passwords is leaked. Current user authentication solutions either increase security or usability. When security increases, usability decreases, or vice versa. To add to this, stringent security protocols encourage unsecure behaviours by the user such as writing the password down on a piece of paper to remember it. It was found that passphrases require less cognitive effort than passwords and because passphrases are stronger than passwords, they don’t need to be changed as frequently as passwords. This study aimed to assess a two-tier user authentication solution that increases security and usability. The proposed solution uses passphrases in conjunction with keystroke dynamics to address this research problem. The design science research approach was used to guide this study. The study’s theoretical foundation includes three theories. The Shannon entropy formula was used to calculate the strength of passwords, passphrases and keystroke dynamics. The chunking theory assisted in assessing password and passphrase memorisation issues and the keystroke-level model was used to assess password and passphrase typing issues. Two primary data collection methods were used to evaluate the findings and to ensure that gaps in the research were filled. A login assessment experiment collected data on user authentication and user-system interaction for passwords and passphrases. Plus, an expert review was conducted to verify findings and assess the research artefact in the form of a model. The model can be used to assist with the implementation of a two-tier user authentication solution which involves passphrases and keystroke dynamics. There are a number of components that need to be considered to realise the benefits of this solution and ensure successful implementation

Modified SqueezeNet Architecture for Parkinson's Disease Detection Based on Keypress Data

Parkinson’s disease (PD) is the most common form of Parkinsonism, which is a group of neurological disorders with PD-like motor impairments. The disease affects over 6 million people worldwide and is characterized by motor and non-motor symptoms. The affected person has trouble in controlling movements, which may affect simple daily-life tasks, such as typing on a computer. We propose the application of a modified SqueezeNet convolutional neural network (CNN) for detecting PD based on the subject’s key-typing patterns. First, the data are pre-processed using data standardization and the Synthetic Minority Oversampling Technique (SMOTE), and then a Continuous Wavelet Transformation is applied to generate spectrograms used for training and testing a modified SqueezeNet model. The modified SqueezeNet model achieved an accuracy of 90%, representing a noticeable improvement in comparison to other approaches

USER AUTHENTICATION ACROSS DEVICES, MODALITIES AND REPRESENTATION: BEHAVIORAL BIOMETRIC METHODS

Biometrics eliminate the need for a person to remember and reproduce complex secretive information or carry additional hardware in order to authenticate oneself. Behavioral biometrics is a branch of biometrics that focuses on using a person’s behavior or way of doing a task as means of authentication. These tasks can be any common, day to day tasks like walking, sleeping, talking, typing and so on. As interactions with computers and other smart-devices like phones and tablets have become an essential part of modern life, a person’s style of interaction with them can be used as a powerful means of behavioral biometrics. In this dissertation, we present insights from the analysis of our proposed set of contextsensitive or word-specific keystroke features on desktop, tablet and phone. We show that the conventional features are not highly discriminatory on desktops and are only marginally better on hand-held devices for user identification. By using information of the context, our proposed word-specific features offer superior discrimination among users on all devices. Classifiers, built using our proposed features, perform user identification with high accuracies in range of 90% to 97%, average precision and recall values of 0.914 and 0.901 respectively. Analysis of the word-based impact factors reveal that four or five character words, words with about 50% vowels, and those that are ranked higher on the frequency lists might give better results for the extraction and use of the proposed features for user identification. We also examine a large umbrella of behavioral biometric data such as; keystroke latencies, gait and swipe data on desktop, phone and tablet for the assumption of an underlying normal distribution, which is common in many research works. Using suitable nonparametric normality tests (Lilliefors test and Shapiro-Wilk test) we show that a majority of the features from all activities and all devices, do not follow a normal distribution. In most cases less than 25% of the samples that were tested had p values \u3e 0.05. We discuss alternate solutions to address the non-normality in behavioral biometric data. Openly available datasets did not provide the wide range of modalities and activities required for our research. Therefore, we have collected and shared an open access, large benchmark dataset for behavioral biometrics on IEEEDataport. We describe the collection and analysis of our Syracuse University and Assured Information Security - Behavioral Biometrics Multi-device and multi -Activity data from Same users (SU-AIS BB-MAS) Dataset. Which is an open access dataset on IEEEdataport, with data from 117 subjects for typing (both fixed and free text), gait (walking, upstairs and downstairs) and touch on Desktop, Tablet and Phone. The dataset consists a total of about: 3.5 million keystroke events; 57.1 million data-points for accelerometer and gyroscope each; 1.7 million datapoints for swipes and is listed as one of the most popular datasets on the portal (through IEEE emails to all members on 05/13/2020 and 07/21/2020). We also show that keystroke dynamics (KD) on a desktop can be used to classify the type of activity, either benign or adversarial, that a text sample originates from. We show the inefficiencies of popular temporal features for this task. With our proposed set of 14 features we achieve high accuracies (93% to 97%) and low Type 1 and Type 2 errors (3% to 8%) in classifying text samples of different sizes. We also present exploratory research in (a) authenticating users through musical notes generated by mapping their keystroke latencies to music and (b) authenticating users through the relationship between their keystroke latencies on multiple devices

A data-driven fatigue prediction using recurrent neural networks

Industrial revolution 4.0 has marked the era of advances in interaction among machines and humans and cultivate automation. However, manufacturing industries still have tasks which are labor intensive for humans with lots of repetitive actions. These actions along with other factors can cause the worker to be fatigued or exhausted. These in the long term can develop into work-related musculoskeletal disorders (WMSD). Nevertheless, comprehending fatigue in a quantifiable and objective manner is yet an open problem due to the heterogeneity of subjects involved for data collection.In this study a benchmarking dataset comprising of physical fatigue attributes. They are used to perform fatigue prediction for manual material handling task. It includes data collected from Inertial Measurement unit (IMU) and Heart Rate (HR) sensor which is then pre-processed to extract to be used to run the model. The data serves as an input to a time-series prediction model called as Recurrent Neural Network (RNN)

Engineering data compendium. Human perception and performance. User's guide

The concept underlying the Engineering Data Compendium was the product of a research and development program (Integrated Perceptual Information for Designers project) aimed at facilitating the application of basic research findings in human performance to the design and military crew systems. The principal objective was to develop a workable strategy for: (1) identifying and distilling information of potential value to system design from the existing research literature, and (2) presenting this technical information in a way that would aid its accessibility, interpretability, and applicability by systems designers. The present four volumes of the Engineering Data Compendium represent the first implementation of this strategy. This is the first volume, the User's Guide, containing a description of the program and instructions for its use

Towards higher sense of presence: a 3D virtual environment adaptable to confusion and engagement

Virtual Reality scenarios where emitters convey information to receptors can be used as a tool for distance learning and to enable virtual visits to company physical headquarters. However, immersive Virtual Reality setups usually require visualization interfaces such as Head-mounted Displays, Powerwalls or CAVE systems, supported by interaction devices (Microsoft Kinect, Wii Motion, among others), that foster natural interaction but are often inaccessible to users. We propose a virtual presentation scenario, supported by a framework, that provides emotion-driven interaction through ubiquitous devices. An experiment with 3 conditions was designed involving: a control condition; a less confusing text script based on its lexical, syntactical, and bigram features; and a third condition where an adaptive lighting system dynamically acted based on the user’s engagement. Results show that users exposed to the less confusing script reported higher sense of presence, albeit without statistical significance. Users from the last condition reported lower sense of presence, which rejects our hypothesis without statistical significance. We theorize that, as the presentation was given orally and the adaptive lighting system impacts the visual channel, this conflict may have overloaded the users’ cognitive capacity and thus reduced available resources to address the presentation content.info:eu-repo/semantics/publishedVersio