Using association rule mining to enrich semantic concepts for video retrieval

In order to achieve true content-based information retrieval on video we should analyse and index video with high-level semantic concepts in addition to using user-generated tags and structured metadata like title, date, etc. However the range of such high-level semantic concepts, detected either manually or automatically, usually limited compared to the richness of information content in video and the potential vocabulary of available concepts for indexing. Even though there is work to improve the performance of individual concept classifiers, we should strive to make the best use of whatever partial sets of semantic concept occurrences are available to us. We describe in this paper our method for using association rule mining to automatically enrich the representation of video content through a set of semantic concepts based on concept co-occurrence patterns. We describe our experiments on the TRECVid 2005 video corpus annotated with the 449 concepts of the LSCOM ontology. The evaluation of our results shows the usefulness of our approach

Non classical concept representation and reasoning in formal ontologies

Formal ontologies are nowadays widely considered a standard tool for knowledge representation and reasoning in the Semantic Web. In this context, they are expected to play an important role in helping automated processes to access information. Namely: they are expected to provide a formal structure able to explicate the relationships between different concepts/terms, thus allowing intelligent agents to interpret, correctly, the semantics of the web resources improving the performances of the search technologies. Here we take into account a problem regarding Knowledge Representation in general, and ontology based representations in particular; namely: the fact that knowledge modeling seems to be constrained between conflicting requirements, such as compositionality, on the one hand and the need to represent prototypical information on the other. In particular, most common sense concepts seem not to be captured by the stringent semantics expressed by such formalisms as, for example, Description Logics (which are the formalisms on which the ontology languages have been built). The aim of this work is to analyse this problem, suggesting a possible solution suitable for formal ontologies and semantic web representations. The questions guiding this research, in fact, have been: is it possible to provide a formal representational framework which, for the same concept, combines both the classical modelling view (accounting for compositional information) and defeasible, prototypical knowledge ? Is it possible to propose a modelling architecture able to provide different type of reasoning (e.g. classical deductive reasoning for the compositional component and a non monotonic reasoning for the prototypical one)? We suggest a possible answer to these questions proposing a modelling framework able to represent, within the semantic web languages, a multilevel representation of conceptual information, integrating both classical and non classical (typicality based) information. Within this framework we hypothesise, at least in principle, the coexistence of multiple reasoning processes involving the different levels of representation

DelosDLMS: From the DELOS vision to the implementation of a future digital library management system

DelosDLMS is a novel digital library management system (DLMS) that has been developed as an integration effort within the DELOS Network of Excellence, a European Commission initiative funded under its fifth and sixth framework programs. In this paper, we describe DelosDLMS that takes into account the recommendations of several activities that were initiated by DELOS including the DELOS vision for digital libraries (DLs). A key aspect of DelosDLMS is its novel generic infrastructure that allows the generation of digital library systems out of a set of basic system services and DL services in a modular and extensible way. DL services like feature extraction, visualization, intelligent browsing, media-type-specific indexing, support for multilinguality, relevance feedback and many others can easily be incorporated or replaced. A further key aspect of DelosDLMS is its robustness against failures and its scalability for large collections and many parallel user requests. We discuss the current status of an effort to build DelosDLMS, a Digital Library Management System that integrates in various ways several components developed by DELOS members and showcases a great variety of functionality that is outlined as part of the DELOS visio

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Using conceptual graphs for clinical guidelines representation and knowledge visualization

The intrinsic complexity of the medical domain requires the building of some tools to assist the clinician and improve the patient’s health care. Clinical practice guidelines and protocols (CGPs) are documents with the aim of guiding decisions and criteria in specific areas of healthcare and they have been represented using several languages, but these are difficult to understand without a formal background. This paper uses conceptual graph formalism to represent CGPs. The originality here is the use of a graph-based approach in which reasoning is based on graph-theory operations to support sound logical reasoning in a visual manner. It allows users to have a maximal understanding and control over each step of the knowledge reasoning process in the CGPs exploitation. The application example concentrates on a protocol for the management of adult patients with hyperosmolar hyperglycemic state in the Intensive Care Unit

Fuzzy-description logic for supporting the rehabilitation of the elderly

[EN] According to the latest statistics, the proportion of the elderly (+65) is increasing and is expected to double within the European Union in a period of 50 years. This ageing is due to the improvement of quality of life and advances in medicine in the last decades. Gerontechnology is receiving a great deal of attention as a way of providing the elderly with sustainable products, environments, and services combining gerontology and technology. One of the most important aspects to consider by gerontechnology is the mobility/rehabilitation technologies, because there is an important relationship between mobility and the elderly's quality of life. Telerehabilitation systems have emerged to allow the elderly to perform their rehabilitation exercises remotely. However, in many cases, the proposed systems assist neither the patients nor the experts about the progress of the rehabilitation. To address this problem, we propose in this paper, a fuzzy-semantic system for evaluating patient's physical state during the rehabilitation process based on well-known standard for patients' evaluation. Moreover, a tool called FINE has been developed that facilitates the evaluation be accomplished in a semi-automatic way first asking patients to carry out a set of standard tests and then inferencing their state by means of a fuzzy-semantic approach using the data captured during the rehabilitation tasks.This research was funded by the Spanish Ministry of Economy and Competitiveness and by EU FEDER funds under project grants TIN2016-79100-R and TIN2015-72931-EXP. It has also been funded by the Junta de Comunidades de Castilla¿La Mancha scholarship 2018-UCLM1-9131Moya, A.; Navarro, E.; Jaén Martínez, FJ.; González, P. (2020). Fuzzy-description logic for supporting the rehabilitation of the elderly. Expert Systems. 37(2):1-16. https://doi.org/10.1111/exsy.12464116372Alamri, A., Cha, J., & El Saddik, A. (2010). AR-REHAB: An Augmented Reality Framework for Poststroke-Patient Rehabilitation. IEEE Transactions on Instrumentation and Measurement, 59(10), 2554-2563. doi:10.1109/tim.2010.2057750Antoniou, G., & van Harmelen, F. (2004). Web Ontology Language: OWL. Handbook on Ontologies, 67-92. doi:10.1007/978-3-540-24750-0_4Bobillo F.(2008).Managing vagueness in ontologies. Universidad de Granada.Bobillo F. (2015).The fuzzyDL system. Retrieved July 10 2018 fromhttp://www.umbertostraccia.it/cs/software/fuzzyDL/fuzzyDL.htmlBobillo, F., Delgado, M., & Gómez-Romero, J. (2012). DeLorean: A reasoner for fuzzy OWL 2. Expert Systems with Applications, 39(1), 258-272. doi:10.1016/j.eswa.2011.07.016Bobillo, F., & Straccia, U. (2016). The fuzzy ontology reasoner fuzzyDL. Knowledge-Based Systems, 95, 12-34. doi:10.1016/j.knosys.2015.11.017Boucenna, S., Narzisi, A., Tilmont, E., Muratori, F., Pioggia, G., Cohen, D., & Chetouani, M. (2014). Interactive Technologies for Autistic Children: A Review. Cognitive Computation, 6(4), 722-740. doi:10.1007/s12559-014-9276-xCarter J. E. L.(2002).The Heath‐Carter anthropometric somatotype—Instruction manual. San Diego:State University.Chiu, Y.-H., Chen, T.-W., Chen, Y. J., Su, C.-I., Hwang, K.-S., & Ho, W.-H. (2018). Fuzzy logic-based mobile computing system for hand rehabilitation after neurological injury. Technology and Health Care, 26(1), 17-27. doi:10.3233/thc-171403Fernández-Caballero, A., González, P., & Navarro, E. (2017). Gerontechnologies - Current achievements and future trends. Expert Systems, 34(2), e12203. doi:10.1111/exsy.12203Giles, R. (1976). Łukasiewicz logic and fuzzy set theory. International Journal of Man-Machine Studies, 8(3), 313-327. doi:10.1016/s0020-7373(76)80003-xHsieh, Y.-W., Hsueh, I.-P., Chou, Y.-T., Sheu, C.-F., Hsieh, C.-L., & Kwakkel, G. (2007). Development and Validation of a Short Form of the Fugl-Meyer Motor Scale in Patients With Stroke. Stroke, 38(11), 3052-3054. doi:10.1161/strokeaha.107.490730Karime, A., Eid, M., Alja’am, J. M., Saddik, A. E., & Gueaieb, W. (2014). A Fuzzy-Based Adaptive Rehabilitation Framework for Home-Based Wrist Training. IEEE Transactions on Instrumentation and Measurement, 63(1), 135-144. doi:10.1109/tim.2013.2277536Krynicki, K., Jaen, J., & Navarro, E. (2016). An ACO-based personalized learning technique in support of people with acquired brain injury. Applied Soft Computing, 47, 316-331. doi:10.1016/j.asoc.2016.04.039Leap Motion INC. (2018).Leap Motion. Retrieved July 10 2018 fromhttps://www.leapmotion.com/Lukasiewicz, T., & Straccia, U. (2008). Managing uncertainty and vagueness in description logics for the Semantic Web. Journal of Web Semantics, 6(4), 291-308. doi:10.1016/j.websem.2008.04.001Metz, D. . (2000). Mobility of older people and their quality of life. Transport Policy, 7(2), 149-152. doi:10.1016/s0967-070x(00)00004-4Nassabi M. H. Den Akker H. &Vollenbroek‐Hutten M. (2014).An ontology‐based recommender system to promote physical activity for pre‐frail elderly 181–184.Navarro, E., González, P., López-Jaquero, V., Montero, F., Molina, J. P., & Romero-Ayuso, D. (2018). Adaptive, Multisensorial, Physiological and Social: The Next Generation of Telerehabilitation Systems. Frontiers in Neuroinformatics, 12. doi:10.3389/fninf.2018.00043OpenNI Pioneering Members. (2018).OpenNI. Retrieved July 10 2018 fromhttp://openni.ru/about/index.htmlOrbbec 3D. (2018).Orbbec Astra Pro. Retrieved July 10 2018 fromhttps://orbbec3d.com/product‐astra‐pro/Rodríguez, A. C., Roda, C., Montero, F., González, P., & Navarro, E. (2015). An Interactive Fuzzy Inference System for Teletherapy of Older People. Cognitive Computation, 8(2), 318-335. doi:10.1007/s12559-015-9356-6Shaughnessy, M., Resnick, B. M., & Macko, R. F. (2006). Testing a Model of Post-Stroke Exercise Behavior. Rehabilitation Nursing, 31(1), 15-21. doi:10.1002/j.2048-7940.2006.tb00005.xSu, C.-J., Chiang, C.-Y., & Huang, J.-Y. (2014). Kinect-enabled home-based rehabilitation system using Dynamic Time Warping and fuzzy logic. Applied Soft Computing, 22, 652-666. doi:10.1016/j.asoc.2014.04.020Velozo, C. A., & Woodbury, M. L. (2011). Translating measurement findings into rehabilitation practice: An example using Fugl-Meyer Assessment-Upper Extremity with patients following stroke. The Journal of Rehabilitation Research and Development, 48(10), 1211. doi:10.1682/jrrd.2010.10.0203W3C. (2012).OWL 2 web ontology language. Retrieved July 10 2018 from https://www.w3.org/TR/owl2‐overview/Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338-353. doi:10.1016/s0019-9958(65)90241-xZhang, Z., Fang, Q., & Gu, X. (2014). Fuzzy inference system based automatic Brunnstrom stage classification for upper-extremity rehabilitation. Expert Systems with Applications, 41(4), 1973-1980. doi:10.1016/j.eswa.2013.08.09

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results