1,142 research outputs found
Towards Multi-modal Interpretable Video Understanding
This thesis introduces an innovative approach to video comprehension, which simulates human perceptual mechanisms and establishes a comprehensible and coherent narrative representation of video content. At the core of this approach lies the creation of a Visual-Linguistic (VL) feature for an interpretable video portrayal and an adaptive attention mechanism (AAM) aimed at concentrating solely on principal actors or pertinent objects while modeling their interconnections. Taking cues from the way humans disassemble scenes into visual and non-visual constituents, the proposed VL feature characterizes a scene via three distinct modalities: (i) a global visual environment, providing a broad contextual comprehension of the scene; (ii) local visual key entities, focusing on pivotal elements within the video; and (iii) linguistic scene elements, incorporating semantically pertinent language-based information for an all-encompassing grasp of the scene. Through the integration of these multimodal traits, the VL representation presents an extensive, diverse, and explicable perspective of video content, effectively bridging the divide between visual perception and linguistic depiction. In our study, we suggest a method for modeling these interactions using a multi-modal representation network. This network consists of two main components: a perception-based multi-modal representation (PMR) and a boundary-matching module (BMM). Additionally, we introduce an adaptive attention mechanism (AAM) within the PMR to focus on primary actors or relevant objects while showing their connections. The PMR module represents each video segment by combining visual and linguistic features. It represents primary actors and their immediate surroundings with visual elements and conveys information about relevant objects through language attributes, using an image-text model. The BMM module takes a sequence of these visual-linguistic features as input and generates action recommendations. Extensive experiments and thorough investigations were carried out on the ActivityNet-1.3 and THUMOS-14 datasets to showcase the superiority of our proposed network over previous cutting-edge methods. It displayed impressive performance and adaptability in both Temporal Action Proposal Generation (TAPG) and temporal action detection. These findings provide strong evidence for the effectiveness of our approach. To demonstrate the robustness and efficiency of our network, we conducted an additional ablation study on egocentric videos, focusing on the EPIC-KITCHENS 100 dataset. This underscores the network\u27s potential to advance the field of video comprehension.s In conclusion, this thesis delineates a promising path toward the development of interpretable video comprehension models. By emulating human perceptual processes and harnessing multimodal attributes, we contribute a fresh perspective to the discipline, opening the door for more advanced and intuitive video comprehension systems in the future
Dielectric modelling of human skin and breast tissue in terahertz frequencies : potential application to cancer detection
University of Technology Sydney. Faculty of Engineering and Information Technology.Growing developments in the generation and detection of terahertz (THz) radiation over more than two decades have created a strong incentive for researchers to study the biomedical applications of terahertz imaging. Contrasts in the THz images of various types of cancer, especially skin and breast cancer, are associated with changes in the dielectric properties of cancerous tissues. In fact, dielectric models can explain the interaction between terahertz radiation and human tissue at a molecular level just as their parameters have the potential for becoming indicators of cancer. However, dielectric modelling of various forms of human tissue remains limited due to a number of factors, especially suboptimal fitting algorithms and tissue heterogeneity.
Thanks to the high water content of human skin, its dielectric response to terahertz radiation can be described by the double Debye model. The existing fitting method using a nonlinear least square algorithm can extract the model parameters which track their measurements accurately at frequencies higher than one THz but poorly at lower frequencies. However, the majority of dielectric contrast between normal and cancerous skin tissues has been observed in the low THz range. Accordingly, this research has developed two global optimization algorithms which are capable of globally accurate tracking thereby supporting the full validity of the double Debye model in simulating the dielectric spectra of human skin in the THz frequencies. Numerical results confirm their superiority over the conventional methods. Furthermore, the next goal of the study is to apply statistical analysis to the parameters of the double Debye model in order to test their discrimination capability of skin cancer from normal tissue. Linear programming and support vector machine algorithms have also been employed using these parameters to classify normal skin tissue and basal cell carcinoma. By combining the double Debye parameters, the classification accuracy has shown significant improvement. The encouraging outcomes confirm the classification potential of the double Debye parameters.
The double Debye model, however, has been shown to be not suitable for simulating human breast tissue due to its low water content and heterogeneous structure, thus limiting the understanding of the THz dielectric response of breast tissue. To overcome this problem, this study proposes a new non-Debye dielectric model to fit the dielectric spectra of human breast tissue. Due to the mathematical complexity of the fitting procedure, a sampling gradient algorithm of non-smooth optimization is used to optimize the fitting solution. Simulation results confirm applicability of the non-Debye model through its exceptional ability to fit the examined data. Statistical measures have also been used to analyse the possibility of using the parameters of this model to differentiate breast tumours from healthy breast tissue. Based on the statistical analysis, popular classification methods such as support vector machines and Bayesian neural network have also been applied to examine these parameters and their combinations for breast cancer classification. The obtained classification accuracies indicate the classification potential of the model parameters as well as highlighting several valuable features of the parameter combinations
Generalizing Topological Graph Neural Networks with Paths
While Graph Neural Networks (GNNs) have made significant strides in diverse
areas, they are hindered by a theoretical constraint known as the
1-Weisfeiler-Lehmann test. Even though latest advancements in higher-order GNNs
can overcome this boundary, they typically center around certain graph
components like cliques or cycles. However, our investigation goes a different
route. We put emphasis on paths, which are inherent in every graph. We are able
to construct a more general topological perspective and form a bridge to
certain established theories about other topological domains. Interestingly,
without any assumptions on graph sub-structures, our approach surpasses earlier
techniques in this field, achieving state-of-the-art performance on several
benchmarks
Torque vectoring based drive assistance system for turning an electric narrow tilting vehicle
The increasing number of cars leads to traffic congestion and limits parking issue in urban area. The narrow tilting vehicles therefore can potentially become the next generation of city cars due to its narrow width. However, due to the difficulty in leaning a narrow tilting vehicle, a drive assistance strategy is required to maintain its roll stability during a turn. This article presents an effective approach using torque vectoring method to assist the rider in balancing the narrow tilting vehicles, thus reducing the counter-steering requirements. The proposed approach is designed as the combination of two torque controllers: steer angle–based torque vectoring controller and tilting compensator–based torque vectoring controller. The steer angle–based torque vectoring controller reduces the counter-steering process via adjusting the vectoring torque based on the steering angle from the rider. Meanwhile, the tilting compensator–based torque vectoring controller develops the steer angle–based torque vectoring with an additional tilting compensator to help balancing the leaning behaviour of narrow tilting vehicles. Numerical simulations with a number of case studies have been carried out to verify the performance of designed controllers. The results imply that the counter-steering process can be eliminated and the roll stability performance can be improved with the usage of the presented approach
Managing risks and system performance in supply network: a conceptual framework
Examining a certain risk will provide an insight into a single dimension, but a picture of different risks in the supply chain (SC) is still lacking, as risks do not take place independently, but typically simultaneously. This research aims to propose and validate a conceptual framework for linking various dimensions of risk to system performance in the SC by applying SC mapping - a new approach in the SC risk body of literature. In the model, risks were classified into three categories with regard to their level of impact on performance: 1) core risks, e.g., supply risk, investor-related operational risks, contractor-related operational risks and demand risks; 2) infrastructure risks, e.g., finance risk, information risk and time risk; 3) external risks, e.g., human-made risks; 4) natural risks. Using the framework, companies will have a systematic view of risks in the whole SC network whereby they can define risks in their own context and ascertain critical SC risks that cause negative effects on SC performance
Numerical solution of the problems for plates on some complex partial internal supports
In the recent works, Dang and Truong proposed an iterative method for solving some problems of plates on one, two and three line partial internal supports (LPISs), and a cross internal support. In nature they are problems with strongly mixed boundary conditions for biharmonic equation. For this reason the method combines a domain decomposition technique with the reduction of the order of the equation from four to two. In this study, the method is developed for plates on internal supports of more complex configurations. Namely, we examine the cases of symmetric rectangular and H-shape supports, where the computational domain after reducing to the first quadrant of the plate is divided into three subdomains. Also, we consider the case of asymmetric rectangular support where the computational domain needs to be divided into 9 subdomains. The problems under consideration are reduced to sequences of weak mixed boundary value problems for the Poisson equation, which are solved by difference method. The performed numerical experiments show the effectiveness of the iterative method
Automatic Crack Detection in Built Infrastructure Using Unmanned Aerial Vehicles
This paper addresses the problem of crack detection which is essential for
health monitoring of built infrastructure. Our approach includes two stages,
data collection using unmanned aerial vehicles (UAVs) and crack detection using
histogram analysis. For the data collection, a 3D model of the structure is
first created by using laser scanners. Based on the model, geometric properties
are extracted to generate way points necessary for navigating the UAV to take
images of the structure. Then, our next step is to stick together those
obtained images from the overlapped field of view. The resulting image is then
clustered by histogram analysis and peak detection. Potential cracks are
finally identified by using locally adaptive thresholds. The whole process is
automatically carried out so that the inspection time is significantly improved
while safety hazards can be minimised. A prototypical system has been developed
for evaluation and experimental results are included.Comment: In proceeding of The 34th International Symposium on Automation and
Robotics in Construction (ISARC), pp. 823-829, Taipei, Taiwan, 201
- …