Automatic Super-Surface Removal in Complex 3D Indoor Environments Using Iterative Region-Based RANSAC

Removing bounding surfaces such as walls, windows, curtains, and floor (i.e., super-surfaces) from a point cloud is a common task in a wide variety of computer vision applications (e.g., object recognition and human tracking). Popular plane segmentation methods such as Random Sample Consensus (RANSAC), are widely used to segment and remove surfaces from a point cloud. However, these estimators easily result in the incorrect association of foreground points to background bounding surfaces because of the stochasticity of randomly sampling, and the limited scene-specific knowledge used by these approaches. Additionally, identical approaches are generally used to detect bounding surfaces and surfaces that belong to foreground objects. Detecting and removing bounding surfaces in challenging (i.e., cluttered and dynamic) real-world scene can easily result in the erroneous removal of points belonging to desired foreground objects such as human bodies. To address these challenges, we introduce a novel super-surface removal technique for 3D complex indoor environments. Our method was developed to work with unorganized data captured from commercial depth sensors and supports varied sensor perspectives. We begin with preprocessing steps and dividing the input point cloud into four overlapped local regions. Then, we apply an iterative surface removal approach to all four regions to segment and remove the bounding surfaces. We evaluate the performance of our proposed method in terms of four conventional metrics: specificity, precision, recall, and F1 score, on three generated datasets representing different indoor environments. Our experimental results demonstrate that our proposed method is a robust super-surface removal and size reduction approach for complex 3D indoor environments while scoring the four evaluation metrics between 90% and 99%

Relational Approach to Knowledge Engineering for POMDP-based Assistance Systems as a Translation of a Psychological Model

Assistive systems for persons with cognitive disabilities (e.g. dementia) are difficult to build due to the wide range of different approaches people can take to accomplishing the same task, and the significant uncertainties that arise from both the unpredictability of client's behaviours and from noise in sensor readings. Partially observable Markov decision process (POMDP) models have been used successfully as the reasoning engine behind such assistive systems for small multi-step tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modelling assistance that can deal with uncertainty and utility. Unfortunately, POMDPs usually require a very labour intensive, manual procedure for their definition and construction. Our previous work has described a knowledge driven method for automatically generating POMDP activity recognition and context sensitive prompting systems for complex tasks. We call the resulting POMDP a SNAP (SyNdetic Assistance Process). The spreadsheet-like result of the analysis does not correspond to the POMDP model directly and the translation to a formal POMDP representation is required. To date, this translation had to be performed manually by a trained POMDP expert. In this paper, we formalise and automate this translation process using a probabilistic relational model (PRM) encoded in a relational database. We demonstrate the method by eliciting three assistance tasks from non-experts. We validate the resulting POMDP models using case-based simulations to show that they are reasonable for the domains. We also show a complete case study of a designer specifying one database, including an evaluation in a real-life experiment with a human actor

Human Part Segmentation in Depth Images with Annotated Part Positions

We present a method of segmenting human parts in depth images, when provided the image positions of the body parts. The goal is to facilitate per-pixel labelling of large datasets of human images, which are used for training and testing algorithms for pose estimation and automatic segmentation. A common technique in image segmentation is to represent an image as a two-dimensional grid graph, with one node for each pixel and edges between neighbouring pixels. We introduce a graph with distinct layers of nodes to model occlusion of the body by the arms. Once the graph is constructed, the annotated part positions are used as seeds for a standard interactive segmentation algorithm. Our method is evaluated on two public datasets containing depth images of humans from a frontal view. It produces a mean per-class accuracy of 93.55% on the first dataset, compared to 87.91% (random forest and graph cuts) and 90.31% (random forest and Markov random field). It also achieves a per-class accuracy of 90.60% on the second dataset. Future work can experiment with various methods for creating the graph layers to accurately model occlusion

Affectively Aligned Cognitive Assistance Using Bayesian Affect Control Theory

Abstract. This paper describes a novel emotionally intelligent cognitive assistant to engage and help older adults with Alzheimer’s disease (AD) to complete activities of daily living (ADL) more independently. Our new system combines two research streams. First, the development of cognitive assistants with artificially intelligent controllers using partially observable Markov decision processes (POMDPs). Second, a model of the dynamics of emotion and identity called Affect Control Theory that arises from the sociological literature on culturally shared sentiments. We present background material on both of these research streams, and then demonstrate a prototype assistive technology that combines the two. We discuss the affective reasoning, the probabilistic and decision-theoretic reasoning, the computer-vision based activity monitoring, the embodied prompting, and we show results in proof-of-concept tests.

Automated recognition of individual performers from de-identified video sequences

dentification of individual humans from RGB image data is well-established. However, in many domains, such as in healthcare or applications involving children, ethical issues have been raised around using traditional RGB image data because individuals can be identified from these data. The widespread availability of reliable depth data, and the associated human skeleton data derived from these data, presents an opportunity to differentiate between individuals while potentially avoiding individually identifiable features. Using skeleton data only, we developed a unique 20-dimensional bone segment length feature vector for 1,761 trials (1,759,980 image frames) of data, captured from 14 participants who engaged in a one-hour group intervention playing Xbox One Kinect Bowling twice-weekly for 24 weeks. We then evaluated our novel feature using representative batch processing (k-nearest neighbour) and real-time (multi-layer perceptron) models, validated against manually-labelled ground-truth data. Our results suggest that our skeleton feature can differentiate between instances (i.e., individuals) with an accuracy over all participants of 100% for batch processing and 96.57% in real-time, and deals well with class imbalances. Our results suggest that we can reliably differentiate between individual persons using only skeleton data derived from depth image data in medical research

CCWORK protocol: a longitudinal study of Canadian Correctional Workers' Well-being, Organizations, Roles and Knowledge.

IntroductionKnowledge about the factors that contribute to the correctional officer's (CO) mental health and well-being, or best practices for improving the mental health and well-being of COs, have been hampered by the dearth of rigorous longitudinal studies. In the current protocol, we share the approach used in the Canadian Correctional Workers' Well-being, Organizations, Roles and Knowledge study (CCWORK), designed to investigate several determinants of health and well-being among COs working in Canada's federal prison system.Methods and analysis CCWORK is a multiyear longitudinal cohort design (2018-2023, with a 5-year renewal) to study 500 COs working in 43 Canadian federal prisons. We use quantitative and qualitative data collection instruments (ie, surveys, interviews and clinical assessments) to assess participants' mental health, correctional work experiences, correctional training experiences, views and perceptions of prison and prisoners, and career aspirations. Our baseline instruments comprise two surveys, one interview and a clinical assessment, which we administer when participants are still recruits in training. Our follow-up instruments refer to a survey, an interview and a clinical assessment, which are conducted yearly when participants have become COs, that is, in annual 'waves'. Ethics and dissemination CCWORK has received approval from the Research Ethics Board of the Memorial University of Newfoundland (File No. 20190481). Participation is voluntary, and we will keep all responses confidential. We will disseminate our research findings through presentations, meetings and publications (e.g., journal articles and reports). Among CCWORK's expected scientific contributions, we highlight a detailed view of the operational, organizational and environmental stressors impacting CO mental health and well-being, and recommendations to prison administrators for improving CO well-being

Advancing the COACH Automated Prompting System toward an Unsupervised, Real-world Deployment

The COACH is a prototype automated prompting system designed to encourage independent participation in daily tasks by older adults with dementia. Supervised, clinical trials with the COACH were promising, yet little was known about the capabilities of the system in an unsupervised, real-world environment or the needs of the system's users in such a deployment. To continue development of the COACH toward an unsupervised, real-world in-home deployment I employed an informed approach. First, the needs of the users of the system were identified through a national questionnaire completed by family caregivers of older adults with dementia and validated using a follow-up focus group. Participants indicated that the COACH should provide unobtrusive support of private or personal tasks by leveraging the remaining abilities of the older adults with dementia using the system, promoting independent task completion. Next, the performance of the COACH was evaluated in an unsupervised, real-world clinical deployment. One of the COACH modules responsible for tracking the interactions between the system users and objects in the environment was implicated as the cause of the majority of the failures, successfully identifying only 46.9% of the task steps completed by the trial participants. The technical factors limiting system performance in the real-world clinical deployment were synthesized with the needs of the users to develop technical design criteria which were used to guide the development of an improved COACH prototype. The performance of the new COACH prototype was evaluated in a second supervised deployment, resulting in the successful identification of 93.7% of the task steps completed by trial participants. Encouraged by these results, the COACH system is again ready for evaluation in an unsupervised, real-world environment.Ph.D

Predicting the role of assistive technologies in the lives of people with dementia using objective care recipient factors

Abstract Background The population of people with dementia is not homogeneous. People with dementia exhibit a wide range of needs, each characterized by diverse factors including age, sex, ethnicity, and place of residence. These needs and characterizing factors may influence the applicability, and ultimately the acceptance, of assistive technologies developed to support the independence of people with dementia. Accordingly, predicting the needs of users before developing the technologies may increase the applicability and acceptance of assistive technologies. Current methods of prediction rely on the difficult collection of subjective, potentially invasive information. We propose a method of prediction that uses objective, unobtrusive, easy to collect information to help inform the development of assistive technologies. Methods We develop a set of models that can predict the level of independence of people with dementia during 20 activities of daily living using simple, objective information. Using data collected from a Canadian survey conducted with caregivers of people with dementia, we create an ordered logistic regression model for each of the twenty daily tasks in the Bristol ADL scale. Results Data collected from 430 Canadian caregivers of people with dementia were analyzed to reveal: most care recipients were mothers or husbands, married, living in private housing with their caregivers, English-speaking, Canadian born, clinically diagnosed with dementia 1 to 6 years prior to the study, and were dependent on their caregiver. Next, we developed models that use 13 factors to predict a person with dementia’s ability to complete the 20 Bristol activities of daily living independently. The 13 factors include caregiver relation, age, marital status, place of residence, language, housing type, proximity to caregiver, service use, informal primary caregiver, diagnosis of Alzheimer’s disease or dementia, time since diagnosis, and level of dependence on caregiver. The resulting models predicted the aggregate level of independence correctly for 88 of 100 total responses categories, marginally for nine, and incorrectly for three. Conclusions Objective, easy to collect information can predict caregiver-reported level of task independence for a person with dementia. Knowledge of task independence can then inform the development of assistive technologies for people with dementia, improving their applicability and acceptance