This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonCapsules (vector-valued neurons) have recently become a more active area of research
in neural networks. However, existing formulations have several drawbacks including
the large number of trainable parameters that they require as well as the reliance on
routing mechanisms between layers of capsules.
The primary aim of this project is to demonstrate the benefits of a new formulation
of capsules called Homogeneous Vector Capsules (HVCs) that overcome these
drawbacks.
Using HVCs, new state-of-the-art accuracies for the MNIST dataset are established
for multiple individual models as well as multiple ensembles.
This work additionally presents a dataset consisting of high-resolution images of
13 micro-PCBs captured in various rotations and perspectives relative to the camera,
with each sample labeled for PCB type, rotation category, and perspective categories.
Experiments performed and elucidated in this work examine classification accuracy of
rotations and perspectives that were not trained on as well as the ability to artificially
generate missing rotations and perspectives during training. The results of these
experiments include showing that using HVCs is superior to using fully connected
layers.
This work also showed that certain training samples are more informative of class
membership than others. These samples can be identified prior to training by analyzing
their position in reduced dimensional space relative to the classes’ centroids in that
space. And a definition and calculation both for class density and dataset completeness
based on the distribution of data in the reduced dimensional space has been put forth.
Experimentation using the dataset completeness calculation shows that those datasets
that meet a certain completeness threshold can be trained on a subset of the total
dataset, based on each class’s density, while improving upon or maintaining validation
accuracy