A Novel Multi-Focus Image Fusion Method Based on Stochastic Coordinate Coding and Local Density Peaks Clustering

abstract: The multi-focus image fusion method is used in image processing to generate all-focus images that have large depth of field (DOF) based on original multi-focus images. Different approaches have been used in the spatial and transform domain to fuse multi-focus images. As one of the most popular image processing methods, dictionary-learning-based spare representation achieves great performance in multi-focus image fusion. Most of the existing dictionary-learning-based multi-focus image fusion methods directly use the whole source images for dictionary learning. However, it incurs a high error rate and high computation cost in dictionary learning process by using the whole source images. This paper proposes a novel stochastic coordinate coding-based image fusion framework integrated with local density peaks. The proposed multi-focus image fusion method consists of three steps. First, source images are split into small image patches, then the split image patches are classified into a few groups by local density peaks clustering. Next, the grouped image patches are used for sub-dictionary learning by stochastic coordinate coding. The trained sub-dictionaries are combined into a dictionary for sparse representation. Finally, the simultaneous orthogonal matching pursuit (SOMP) algorithm is used to carry out sparse representation. After the three steps, the obtained sparse coefficients are fused following the max L1-norm rule. The fused coefficients are inversely transformed to an image by using the learned dictionary. The results and analyses of comparison experiments demonstrate that fused images of the proposed method have higher qualities than existing state-of-the-art methods

Exploring space situational awareness using neuromorphic event-based cameras

The orbits around earth are a limited natural resource and one that hosts a vast range of vital space-based systems that support international systems use by both commercial industries, civil organisations, and national defence. The availability of this space resource is rapidly depleting due to the ever-growing presence of space debris and rampant overcrowding, especially in the limited and highly desirable slots in geosynchronous orbit. The field of Space Situational Awareness encompasses tasks aimed at mitigating these hazards to on-orbit systems through the monitoring of satellite traffic. Essential to this task is the collection of accurate and timely observation data. This thesis explores the use of a novel sensor paradigm to optically collect and process sensor data to enhance and improve space situational awareness tasks. Solving this issue is critical to ensure that we can continue to utilise the space environment in a sustainable way. However, these tasks pose significant engineering challenges that involve the detection and characterisation of faint, highly distant, and high-speed targets. Recent advances in neuromorphic engineering have led to the availability of high-quality neuromorphic event-based cameras that provide a promising alternative to the conventional cameras used in space imaging. These cameras offer the potential to improve the capabilities of existing space tracking systems and have been shown to detect and track satellites or ‘Resident Space Objects’ at low data rates, high temporal resolutions, and in conditions typically unsuitable for conventional optical cameras. This thesis presents a thorough exploration of neuromorphic event-based cameras for space situational awareness tasks and establishes a rigorous foundation for event-based space imaging. The work conducted in this project demonstrates how to enable event-based space imaging systems that serve the goals of space situational awareness by providing accurate and timely information on the space domain. By developing and implementing event-based processing techniques, the asynchronous operation, high temporal resolution, and dynamic range of these novel sensors are leveraged to provide low latency target acquisition and rapid reaction to challenging satellite tracking scenarios. The algorithms and experiments developed in this thesis successfully study the properties and trade-offs of event-based space imaging and provide comparisons with traditional observing methods and conventional frame-based sensors. The outcomes of this thesis demonstrate the viability of event-based cameras for use in tracking and space imaging tasks and therefore contribute to the growing efforts of the international space situational awareness community and the development of the event-based technology in astronomy and space science applications

IEEE Access Special Section Editorial: Big Data Technology and Applications in Intelligent Transportation

During the last few years, information technology and transportation industries, along with automotive manufacturers and academia, are focusing on leveraging intelligent transportation systems (ITS) to improve services related to driver experience, connected cars, Internet data plans for vehicles, traffic infrastructure, urban transportation systems, traffic collaborative management, road traffic accidents analysis, road traffic flow prediction, public transportation service plan, personal travel route plans, and the development of an effective ecosystem for vehicles, drivers, traffic controllers, city planners, and transportation applications. Moreover, the emerging technologies of the Internet of Things (IoT) and cloud computing have provided unprecedented opportunities for the development and realization of innovative intelligent transportation systems where sensors and mobile devices can gather information and cloud computing, allowing knowledge discovery, information sharing, and supported decision making. However, the development of such data-driven ITS requires the integration, processing, and analysis of plentiful information obtained from millions of vehicles, traffic infrastructures, smartphones, and other collaborative systems like weather stations and road safety and early warning systems. The huge amount of data generated by ITS devices is only of value if utilized in data analytics for decision-making such as accident prevention and detection, controlling road risks, reducing traffic carbon emissions, and other applications which bring big data analytics into the picture

A Systematic Review of Convolutional Neural Network-Based Structural Condition Assessment Techniques

With recent advances in non-contact sensing technology such as cameras, unmanned aerial and ground vehicles, the structural health monitoring (SHM) community has witnessed a prominent growth in deep learning-based condition assessment techniques of structural systems. These deep learning methods rely primarily on convolutional neural networks (CNNs). The CNN networks are trained using a large number of datasets for various types of damage and anomaly detection and post-disaster reconnaissance. The trained networks are then utilized to analyze newer data to detect the type and severity of the damage, enhancing the capabilities of non-contact sensors in developing autonomous SHM systems. In recent years, a broad range of CNN architectures has been developed by researchers to accommodate the extent of lighting and weather conditions, the quality of images, the amount of background and foreground noise, and multiclass damage in the structures. This paper presents a detailed literature review of existing CNN-based techniques in the context of infrastructure monitoring and maintenance. The review is categorized into multiple classes depending on the specific application and development of CNNs applied to data obtained from a wide range of structures. The challenges and limitations of the existing literature are discussed in detail at the end, followed by a brief conclusion on potential future research directions of CNN in structural condition assessment

Quantification of Interactions between Influenza Hemagglutinin and Host Cell Phosphoinositides by Super-Resolution Microscopy

The influenza viral membrane protein hemagglutinin (HA) forms dense nanoscale clusters on host cell plasma membranes (PM), but the mechanisms that direct HA clustering are not well understood. Previous studies have observed HA associated with actin rich regions of the PM, but there are no known direct interactions between HA and actin. Phosphatidylinositol 4,5-biphosphate (PIP2) is a signaling lipid in the PM which can regulate the actin cytoskeleton, and actin comets initiated by PIP2 are known to be exploited by HA to reach the PM of infected cells. PIP2 is also used by other viruses, such as HIV and Ebola, to form clusters of viral proteins on the PM. Using diffraction-limited and super-resolution FPALM methods, we observed that HA and PH domain, a protein marker for PIP2, are closely spatially related at the PM. Clusters of PIP2 are also significantly altered in both density and area in the presence of high levels of HA, while HA clusters are significantly altered in the presence of high levels of PIP2, suggestive of an interaction between the two. Although HA mutates rapidly, there are 3 cysteines and 1-2 basic residues in the cytoplasmic tail domain (CTD) which remain highly conserved among HA subtypes. These cysteines are known to undergo palmitoylation in the Golgi, a post-translational modification where hydrophobic palmitic acids are attached. Using HA mutants and super-resolution FPALM, we examined the role of both palmitoylation and charge on the clustering properties of HA and spatial association with PIP2. Mutation of the cysteines or basic residues causes significant reductions to cluster densities (relative to cell average), while mutation of the charges appears to modulate association with PIP2. The greatest changes were observed when both the cysteines and net charge of the HA CTD were changed, causing a maximal 22% ± 6% reduction in the radial distribution functions (RDF) of clusters and a maximal 30% ± 15% increase in associated PH domain RDF amplitude. Cluster properties, density, perimeter, and circularity were also significantly affected. Even though clusters were not eliminated through CTD mutations, these findings suggest that the CTD of HA does play a role in the clustering of HA and spatial association with PIP2

Identifying markers of cell identity from single-cell omics data

Einzelzell-Omics-Daten stehen derzeit im Fokus der Entwicklung computergestützter Methoden in der Molekularbiologie und Genetik. Einzelzellexperimenten lieferen dünnbesetzte, hochdimensionale Daten über zehntausende Gene oder hunderttausende regulatorische Regionen in zehntausenden Zellen. Diese Daten bieten den Forschenden die Möglichkeit, Gene und regulatorische Regionen zu identifizieren, welche die Bestimmung und Aufrechterhaltung der Zellidentität koordinieren. Die gängigste Strategie zur Identifizierung von Zellidentitätsmarkern besteht darin, die Zellen zu clustern und dann Merkmale zu finden, welche die Cluster unterscheiden, wobei davon ausgegangen wird, dass die Zellen innerhalb eines Clusters die gleiche Identität haben. Diese Annahme ist jedoch nicht immer zutreffend, insbesondere nicht für Entwicklungsdaten bei denen sich die Zellen in einem Kontinuum befinden und die Definition von Clustergrenzen biologisch gesehen potenziell willkürlich ist. Daher befasst sich diese Dissertation mit Clustering-unabhängigen Strategien zur Identifizierung von Markern aus Einzelzell-Omics-Daten. Der wichtigste Beitrag dieser Dissertation ist SEMITONES, eine auf linearer Regression basierende Methode zur Identifizierung von Markern. SEMITONES identifiziert (Gruppen von) Markern aus verschiedenen Arten von Einzelzell-Omics-Daten, identifiziert neue Marker und übertrifft bestehende Marker-Identifizierungsansätze. Außerdem ermöglicht die Identifizierung von regulatorischen Markerregionen durch SEMITONES neue Hypothesen über die Regulierung der Genexpression während dem Erwerb der Zellidentität. Schließlich beschreibt die Dissertation einen Ansatz zur Identifizierung neuer Markergene für sehr ähnliche, dennoch underschiedliche neurale Vorlauferzellen im zentralen Nervensystem von Drosphila melanogaster. Ingesamt zeigt die Dissertation, wie Cluster-unabhängige Ansätze zur Aufklärung bisher uncharakterisierter biologischer Phänome aus Einzelzell-Omics-Daten beitragen.Single-cell omics approaches are the current frontier of computational method development in molecular biology and genetics. A single single-cell experiment provides sparse, high-dimensional data on tens of thousands of genes or hundreds of thousands of regulatory regions (i.e. features) in tens of thousands of cells (i.e. samples). This data provides researchers with an unprecedented opportunity to identify those genes and regulatory regions that determine and coordinate cell identity acquisition and maintenance. The most common strategy for identifying cell identity markers consists of clustering the cells and then identifying differential features between these clusters, assuming that cells within a cluster share the same identity. This assumption is, however, not guaranteed to hold, particularly for developmental data where cells lie along a continuum and inferring cluster boundaries becomes non-trivial and potentially biologically arbitrary. In response, this thesis presents clustering-independent strategies for marker feature identification from single-cell omics data. The primary contribution of this thesis is a linear regression-based method for marker feature identification from single-cell omics data called SEMITONES. SEMITONES can identify markers or marker sets from diverse single-cell omics data types, identifies novel markers, outperforms existing marker identification approaches. The thesis also describes how the identification of marker regulatory regions by SEMITONES enables the generation of novel hypotheses regarding gene regulation during cell identity acquisition. Lastly, the thesis describes the clustering-independent identification of novel marker genes for highly similar yet distinct neural progenitor cells in the Drosophila melanogaster central nervous system. Altogether, the thesis demonstrates how clustering-independent approaches aid the elucidation of yet uncharacterised biological patterns from single cell-omics data

Visual Human Tracking and Group Activity Analysis: A Video Mining System for Retail Marketing

Thesis (PhD) - Indiana University, Computer Sciences, 2007In this thesis we present a system for automatic human tracking and activity recognition from video sequences. The problem of automated analysis of visual information in order to derive descriptors of high level human activities has intrigued computer vision community for decades and is considered to be largely unsolved. A part of this interest is derived from the vast range of applications in which such a solution may be useful. We attempt to find efficient formulations of these tasks as applied to the extracting customer behavior information in a retail marketing context. Based on these formulations, we present a system that visually tracks customers in a retail store and performs a number of activity analysis tasks based on the output from the tracker. In tracking we introduce new techniques for pedestrian detection, initialization of the body model and a formulation of the temporal tracking as a global trans-dimensional optimization problem. Initial human detection is addressed by a novel method for head detection, which incorporates the knowledge of the camera projection model.The initialization of the human body model is addressed by newly developed shape and appearance descriptors. Temporal tracking of customer trajectories is performed by employing a human body tracking system designed as a Bayesian jump-diffusion filter. This approach demonstrates the ability to overcome model dimensionality ambiguities as people are leaving and entering the scene. Following the tracking, we developed a two-stage group activity formulation based upon the ideas from swarming research. For modeling purposes, all moving actors in the scene are viewed here as simplistic agents in the swarm. This allows to effectively define a set of inter-agent interactions, which combine to derive a distance metric used in further swarm clustering. This way, in the first stage the shoppers that belong to the same group are identified by deterministically clustering bodies to detect short term events and in the second stage events are post-processed to form clusters of group activities with fuzzy memberships. Quantitative analysis of the tracking subsystem shows an improvement over the state of the art methods, if used under similar conditions. Finally, based on the output from the tracker, the activity recognition procedure achieves over 80% correct shopper group detection, as validated by the human generated ground truth results