Revisiting Embeddings for Graph Neural Networks

Current graph representation learning techniques use Graph Neural Networks (GNNs) to extract features from dataset embeddings. In this work, we examine the quality of these embeddings and assess how changing them can affect the accuracy of GNNs. We explore different embedding extraction techniques for both images and texts; and find that the performance of different GNN architectures is dependent on the embedding style used. We see a prevalence of bag of words (BoW) embeddings and text classification tasks in available graph datasets. Given the impact embeddings has on GNN performance. this leads to a phenomenon that GNNs being optimised for BoW vectors

Exploiting tightly-coupled cores

This is the published manuscript. It was first published by Springer in the Journal of Signal Processing Systems here: http://link.springer.com/article/10.1007%2Fs11265-014-0944-6.The individual processors of a chip-multiprocessor traditionally have rigid boundaries. Inter-core communication is only possible via memory and control over a core’s resources is localised. Specialisation necessary to meet today’s challenging energy targets is typically provided through the provision of a range of processor types and accelerators. An alternative approach is to permit specialisation by tailoring the way a large number of homogeneous cores are used. The approach here is to relax processor boundaries, create a richer mix of intercore communication mechanisms and provide finer-grain control over, and access to, the resources of each core. We evaluate one such design, called Loki, that aims to support specialisation in software on a homogeneous many-core architecture. We focus on the design of a single 8-core tile, conceived as the building block for a larger many-core system. We explore the tile’s ability to support a range of parallelisation opportunities and detail the control and communication mechanisms needed to exploit each core’s resources in a flexible manner. Performance and a detailed breakdown of energy usage is provided for a range of benchmarks and configurations.This work was supported by EPSRC grant EP/G033110/1

Lane Compression: A Lightweight Lossless Compression Method for Machine Learning on Embedded Systems

This article presents Lane Compression, a lightweight lossless compression technique for machine learning that is based on a detailed study of the statistical properties of machine learning data. The proposed technique profiles machine learning data gathered ahead of run-time and partitions values bit-wise into different lanes with more distinctive statistical characteristics. Then the most appropriate compression technique is chosen for each lane out of a small number of low-cost compression techniques. Lane Compression’s compute and memory requirements are very low and yet it achieves a compression rate comparable to or better than Huffman coding. We evaluate and analyse Lane Compression on a wide range of machine learning networks for both inference and re-training. We also demonstrate the profiling prior to run-time and the ability to configure the hardware based on the profiling guarantee robust performance across different models and datasets. Hardware implementations are described and the scheme’s simplicity makes it suitable for compressing both on-chip and off-chip traffic. Samsung Advanced Institute of Technology (SAIT

Lattice Model of Sweeping Interface for Drying Process in Water-Granule Mixture

Based on the invasion percolation model, a lattice model for the sweeping interface dynamics is constructed to describe the pattern forming process by a sweeping interface upon drying the water-granule mixture. The model is shown to produce labyrinthine patterns similar to those found in the experiment[Yamazaki and Mizuguchi, J. Phys. Soc. Jpn. \textbf{69} (2000) 2387]. Upon changing the initial granular density, resulting patterns undergo the percolation transition, but estimated critical exponents are different from those of the conventional percolation. Loopless structure of clusters in the patterns produced by the sweeping dynamics seems to influence the nature of the transition.Comment: 6 pages, 7 figure

WH2 and proline-rich domains of WASP-family proteins collaborate to accelerate actin filament elongation.

WASP-family proteins are known to promote assembly of branched actin networks by stimulating the filament-nucleating activity of the Arp2/3 complex. Here, we show that WASP-family proteins also function as polymerases that accelerate elongation of uncapped actin filaments. When clustered on a surface, WASP-family proteins can drive branched actin networks to grow much faster than they could by direct incorporation of soluble monomers. This polymerase activity arises from the coordinated action of two regulatory sequences: (i) a WASP homology 2 (WH2) domain that binds actin, and (ii) a proline-rich sequence that binds profilin-actin complexes. In the absence of profilin, WH2 domains are sufficient to accelerate filament elongation, but in the presence of profilin, proline-rich sequences are required to support polymerase activity by (i) bringing polymerization-competent actin monomers in proximity to growing filament ends, and (ii) promoting shuttling of actin monomers from profilin-actin complexes onto nearby WH2 domains. Unoccupied WH2 domains transiently associate with free filament ends, preventing their growth and dynamically tethering the branched actin network to the WASP-family proteins that create it. Collaboration between WH2 and proline-rich sequences thus strikes a balance between filament growth and tethering. Our work expands the number of critical roles that WASP-family proteins play in the assembly of branched actin networks to at least three: (i) promoting dendritic nucleation; (ii) linking actin networks to membranes; and (iii) accelerating filament elongation

Focused quantization for sparse CNNs

Deep convolutional neural networks (CNNs) are powerful tools for a wide range of vision tasks, but the enormous amount of memory and compute resources required by CNNs pose a challenge in deploying them on constrained devices. Existing compression techniques, while excelling at reducing model sizes, struggle to be computationally friendly. In this paper, we attend to the statistical properties of sparse CNNs and present focused quantization, a novel quantization strategy based on power-of-two values, which exploits the weight distributions after fine-grained pruning. The proposed method dynamically discovers the most effective numerical representation for weights in layers with varying sparsities, significantly reducing model sizes. Multiplications in quantized CNNs are replaced with much cheaper bit-shift operations for efficient inference. Coupled with lossless encoding, we built a compression pipeline that provides CNNs with high compression ratios (CR), low computation cost and minimal loss in accuracy. In ResNet-50, we achieved a 18.08x CR with only 0.24% loss in top-5 accuracy, outperforming existing compression methods. We fully compressed a ResNet-18 and found that it is not only higher in CR and top-5 accuracy, but also more hardware efficient as it requires fewer logic gates to implement when compared to other state-of-the-art quantization methods assuming the same throughput.This work is supported in part by the National Key R&D Program of China (No. 2018YFB1004804), the National Natural Science Foundation of China (No. 61806192). We thank EPSRC for providing Yiren Zhao his doctoral scholarship

Early stages of ramified growth in quasi-two-dimensional electrochemical deposition

I have measured the early stages of the growth of branched metal aggregates formed by electrochemical deposition in very thin layers. The growth rate of spatial Fourier modes is described qualitatively by the results of a linear stability analysis [D.P. Barkey, R.H. Muller, and C.W. Tobias, J. Electrochem. Soc. {\bf 136}, 2207 (1989)]. The maximum growth rate is proportional to $(I/c)^\delta$ where $I$ is the current through the electrochemical cell, $c$ the electrolyte concentration, and $\delta = 1.37 \pm 0.08$. Differences between my results and the theoretical predictions suggest that electroconvection in the electrolyte has a large influence on the instability leading to ramified growth.Comment: REVTeX, four ps figure

Mopra CO Observations of the Bubble HII Region RCW120

We use the Mopra radio telescope to test for expansion of the molecular gas associated with the bubble HII region RCW120. A ring, or bubble, morphology is common for Galactic HII regions, but the three-dimensional geometry of such objects is still unclear. Detected near- and far-side expansion of the associated molecular material would be consistent with a three-dimensional spherical object. We map the $J = 1\rightarrow 0$ transitions of $^{12}$CO, $^{13}$CO, C$^{18}$O, and C$^{17}$O, and detect emission from all isotopologues. We do not detect the $0_0\rightarrow 1_{-1} E$ masing lines of CH$_3$OH at 108.8939 GHz. The strongest CO emission is from the photodissociation region (PDR), and there is a deficit of emission toward the bubble interior. We find no evidence for expansion of the molecular material associated with RCW120 and therefore can make no claims about its geometry. The lack of detected expansion is roughly in agreement with models for the time-evolution of an HII region like RCW120, and is consistent with an expansion speed of $< 1.5\, {\rm km\, s^{-1}}$. Single-position CO spectra show signatures of expansion, which underscores the importance of mapped spectra for such work. Dust temperature enhancements outside the PDR of RCW120 coincide with a deficit of emission in CO, confirming that these temperature enhancements are due to holes in the RCW120 PDR. H$\alpha$ emission shows that RCW120 is leaking $\sim5\%$ of the ionizing photons into the interstellar medium (ISM) through PDR holes at the locations of the temperature enhancements. H-alpha emission also shows a diffuse "halo" from leaked photons not associated with discrete holes in the PDR. Overall $25\pm10\%$ of all ionizing photons are leaking into the nearby ISM.Comment: 35 pages, 14 figures. Accepted to Ap