373 research outputs found
Spherical Transformer: Adapting Spherical Signal to CNNs
Convolutional neural networks (CNNs) have been widely used in various vision
tasks, e.g. image classification, semantic segmentation, etc. Unfortunately,
standard 2D CNNs are not well suited for spherical signals such as panorama
images or spherical projections, as the sphere is an unstructured grid. In this
paper, we present Spherical Transformer which can transform spherical signals
into vectors that can be directly processed by standard CNNs such that many
well-designed CNNs architectures can be reused across tasks and datasets by
pretraining. To this end, the proposed method first uses locally structured
sampling methods such as HEALPix to construct a transformer grid by using the
information of spherical points and its adjacent points, and then transforms
the spherical signals to the vectors through the grid. By building the
Spherical Transformer module, we can use multiple CNN architectures directly.
We evaluate our approach on the tasks of spherical MNIST recognition, 3D object
classification and omnidirectional image semantic segmentation. For 3D object
classification, we further propose a rendering-based projection method to
improve the performance and a rotational-equivariant model to improve the
anti-rotation ability. Experimental results on three tasks show that our
approach achieves superior performance over state-of-the-art methods
Transcending Grids: Point Clouds and Surface Representations Powering Neurological Processing
In healthcare, accurately classifying medical images is vital, but
conventional methods often hinge on medical data with a consistent grid
structure, which may restrict their overall performance. Recent medical
research has been focused on tweaking the architectures to attain better
performance without giving due consideration to the representation of data. In
this paper, we present a novel approach for transforming grid based data into
its higher dimensional representations, leveraging unstructured point cloud
data structures. We first generate a sparse point cloud from an image by
integrating pixel color information as spatial coordinates. Next, we construct
a hypersurface composed of points based on the image dimensions, with each
smooth section within this hypersurface symbolizing a specific pixel location.
Polygonal face construction is achieved using an adjacency tensor. Finally, a
dense point cloud is generated by densely sampling the constructed
hypersurface, with a focus on regions of higher detail. The effectiveness of
our approach is demonstrated on a publicly accessible brain tumor dataset,
achieving significant improvements over existing classification techniques.
This methodology allows the extraction of intricate details from the original
image, opening up new possibilities for advanced image analysis and processing
tasks
- …