2D Slice-driven Physics-based 3D Motion Estimation Framework for Pancreatic Radiotherapy

Pancreatic diseases are difficult to treat with high doses of radiation, as they often present both periodic and aperiodic deformations. Nevertheless, we expect that these difficulties can be overcome, and treatment results may be improved with the practical use of a device that can capture 2D slices of organs during irradiation. However, since only a few 2D slices can be taken, the 3D motion needs to be estimated from partially observed information. In this study, we propose a physics-based framework for estimating the 3D motion of organs, regardless of periodicity, from motion information obtained by 2D slices in one or more directions and a regression model that estimates the accuracy of the proposed framework to select the optimal slice. Using information obtained by slice-to-slice registration and setting the surrounding organs as boundaries, the framework drives the physical models for estimating 3D motion. The R2 score of the proposed regression model was greater than 0.9, and the RMSE was 0.357 mm. The mean errors were 5.11 $\pm$ 1.09 mm using an axial slice and 2.13 $\pm$ 0.598 mm using concurrent axial, sagittal, and coronal slices. Our results suggest that the proposed framework is comparable to volume-to-volume registration, and is feasible

Spatially Continuous Non-Contact Cold Sensation Presentation Based on Low-Temperature Airflows

Our perception of cold enriches our understanding of the world and allows us to interact with it. Therefore, the presentation of cold sensations will be beneficial in improving the sense of immersion and presence in virtual reality and the metaverse. This study proposed a novel method for spatially continuous cold sensation presentation based on low-temperature airflows. We defined the shortest distance between two airflows perceived as different cold stimuli as a local cold stimulus group discrimination threshold (LCSGDT). By setting the distance between airflows within the LCSGDT, spatially continuous cold sensations can be achieved with an optimal number of cold airflows. We hypothesized that the LCSGDTs are related to the heat-transfer capability of airflows and developed a model to relate them. We investigated the LCSGDTs at a flow rate of 25 L/min and presentation distances ranging from 10 to 50 mm. The results showed that under these conditions, the LCSGDTs are 131.4 $\pm$ 1.9 mm, and the heat-transfer capacity of the airflow corresponding to these LCSGDTs is an almost constant value, that is, 0.92.Comment: 7 page

Integration of Independent Heat Transfer Mechanisms for Non-Contact Cold Sensation Presentation With Low Residual Heat

Thermal sensation is crucial to enhancing our comprehension of the world and enhancing our ability to interact with it. Therefore, the development of thermal sensation presentation technologies holds significant potential, providing a novel method of interaction. Traditional technologies often leave residual heat in the system or the skin, affecting subsequent presentations. Our study focuses on presenting thermal sensations with low residual heat, especially cold sensations. To mitigate the impact of residual heat in the presentation system, we opted for a non-contact method, and to address the influence of residual heat on the skin, we present thermal sensations without significantly altering skin temperature. Specifically, we integrated two highly responsive and independent heat transfer mechanisms: convection via cold air and radiation via visible light, providing non-contact thermal stimuli. By rapidly alternating between perceptible decreases and imperceptible increases in temperature on the same skin area, we maintained near-constant skin temperature while presenting continuous cold sensations. In our experiments involving 15 participants, we observed that when the cooling rate was -0.2 to -0.24 degree celsius per second and the cooling time ratio was 30 to 50 %, more than 86.67 % of the participants perceived only persistent cold without any warmth

Looking represents choosing in toddlers: Exploring the equivalence between multimodal measures in forced‐choice tasks

In the two-alternative forced-choice (2AFC) paradigm, manual responses such as pointing have been widely used as measures to estimate cognitive abilities. While pointing measurements can be easily collected, coded, analyzed, and interpreted, absent responses are often observed particularly when adopting these measures for toddler studies, which leads to an increase of missing data. Although looking responses such as preferential looking can be available as alternative measures in such cases, it is unknown how well looking measurements can be interpreted as equivalent to manual ones. This study aimed to answer this question by investigating how accurately pointing responses (i.e., left or right) could be predicted from concurrent preferential looking. Using pre-existing videos of toddlers aged 18-23 months engaged in an intermodal word comprehension task, we developed models predicting manual from looking responses. Results showed substantial prediction accuracy for both the Simple Majority Vote and Machine Learning-Based classifiers, which indicates that looking responses would be reasonable alternative measures of manual ones. However, the further exploratory analysis revealed that when applying the created models for data of toddlers who did not produce clear pointing responses, the estimation agreement of missing pointing between the models and the human coders slightly dropped. This indicates that looking responses without pointing were qualitatively different from those with pointing. Bridging two measurements in forced-choice tasks would help researchers avoid wasting collected data due to the absence of manual responses and interpret results from different modalities comprehensively

Semi-automation of gesture annotation by machine learning and human collaboration

none6siGesture and multimodal communication researchers typically annotate video data manually, even though this can be a very time-consuming task. In the present work, a method to detect gestures is proposed as a fundamental step towards a semi-automatic gesture annotation tool. The proposed method can be applied to RGB videos and requires annotations of part of a video as input. The technique deploys a pose estimation method and active learning. In the experiment, it is shown that if about 27% of the video is annotated, the remaining parts of the video can be annotated automatically with an F-score of at least 0.85. Users can run this tool with a small number of annotations first. If the predicted annotations for the remainder of the video are not satisfactory, users can add further annotations and run the tool again. The code has been released so that other researchers and practitioners can use the results of this research. This tool has been confirmed to work in conjunction with ELAN.openIenaga, Naoto; Cravotta, Alice; Terayama, Kei; Scotney, Bryan W.; Saito, Hideo; Busà, M. GraziaIenaga, Naoto; Cravotta, Alice; Terayama, Kei; Scotney, Bryan W.; Saito, Hideo; Busà, M. Grazi

Two-stage video-based convolutional neural networks for adult spinal deformity classification

IntroductionAssessment of human gait posture can be clinically effective in diagnosing human gait deformities early in life. Currently, two methods—static and dynamic—are used to diagnose adult spinal deformity (ASD) and other spinal disorders. Full-spine lateral standing radiographs are used in the standard static method. However, this is a static assessment of joints in the standing position and does not include information on joint changes when the patient walks. Careful observation of long-distance walking can provide a dynamic assessment that reveals an uncompensated posture; however, this increases the workload of medical practitioners. A three-dimensional (3D) motion system is proposed for the dynamic method. Although the motion system successfully detected dynamic posture changes, access to the facilities was limited. Therefore, a diagnostic approach that is facility-independent, has low practice flow, and does not involve patient contact is required.MethodsWe focused on a video-based method to classify patients with spinal disorders either as ASD, or other forms of ASD. To achieve this goal, we present a video-based two-stage machine-learning method. In the first stage, deep learning methods are used to locate the patient and extract the area where the patient is located. In the second stage, a 3D CNN (convolutional neural network) device is used to capture spatial and temporal information (dynamic motion) from the extracted frames. Disease classification is performed by discerning posture and gait from the extracted frames. Model performance was assessed using the mean accuracy, F1 score, and area under the receiver operating characteristic curve (AUROC), with five-fold cross-validation. We also compared the final results with professional observations.ResultsOur experiments were conducted using a gait video dataset comprising 81 patients. The experimental results indicated that our method is effective for classifying ASD and other spinal disorders. The proposed method achieved a mean accuracy of 0.7553, an F1 score of 0.7063, and an AUROC score of 0.7864. Additionally, ablation experiments indicated the importance of the first stage (detection stage) and transfer learning of our proposed method.DiscussionThe observations from the two doctors were compared using the proposed method. The mean accuracies observed by the two doctors were 0.4815 and 0.5247, with AUROC scores of 0.5185 and 0.5463, respectively. We proved that the proposed method can achieve accurate and reliable medical testing results compared with doctors' observations using videos of 1 s duration. All our code, models, and results are available at https://github.com/ChenKaiXuSan/Walk_Video_PyTorch. The proposed framework provides a potential video-based method for improving the clinical diagnosis for ASD and non-ASD. This framework might, in turn, benefit both patients and clinicians to treat the disease quickly and directly and further reduce facility dependency and data-driven systems

Natural Gesture Extraction Based on Hand Trajectory

Automatic natural gesture recognition can be useful both for the development of human-robot applications and as an aid in the study of human gesture. The goal of this study is to recognize natural gestures using only an RGB video without machine learning methods. To develop and test the proposed method we recorded videos in which a speaker gestured naturally but in a controlled way. The advantage of using this method over lab-recorded data is that the data contain variations in gestures that are typically encountered when analyzing gestures of TV news or speech videos on the Internet. The hand positions are computed by a pose estimation method, and we recognize the gestures based on the hand trajectories, assuming that the gesturing hand(s) do(es) not change its direction abruptly during each phase of a gesture. Based on ground-truth annotations provided by linguistic experts, the accuracies were 92.15%, 91.76% and 75.81% for three natural gestures selected

Computer Vision-Based Approach for Quantifying Occupational Therapists’ Qualitative Evaluations of Postural Control

This study aimed to leverage computer vision (CV) technology to develop a technique for quantifying postural control. A conventional quantitative index, occupational therapists’ qualitative clinical evaluations, and CV-based quantitative indices using an image analysis algorithm were applied to evaluate the postural control of 34 typically developed preschoolers. The effectiveness of the CV-based indices was investigated relative to current methods to explore the clinical applicability of the proposed method. The capacity of the CV-based indices to reflect therapists’ qualitative evaluations was confirmed. Furthermore, compared to the conventional quantitative index, the CV-based indices provided more detailed quantitative information with lower costs. CV-based evaluations enable therapists to quantify details of motor performance that are currently observed qualitatively. The development of such precise quantification methods will improve the science and practice of occupational therapy and allow therapists to perform to their full potential