Sparsely Aggregated Convolutional Networks

We explore a key architectural aspect of deep convolutional neural networks: the pattern of internal skip connections used to aggregate outputs of earlier layers for consumption by deeper layers. Such aggregation is critical to facilitate training of very deep networks in an end-to-end manner. This is a primary reason for the widespread adoption of residual networks, which aggregate outputs via cumulative summation. While subsequent works investigate alternative aggregation operations (e.g. concatenation), we focus on an orthogonal question: which outputs to aggregate at a particular point in the network. We propose a new internal connection structure which aggregates only a sparse set of previous outputs at any given depth. Our experiments demonstrate this simple design change offers superior performance with fewer parameters and lower computational requirements. Moreover, we show that sparse aggregation allows networks to scale more robustly to 1000+ layers, thereby opening future avenues for training long-running visual processes.Comment: Accepted to ECCV 201

PockEngine: Sparse and Efficient Fine-tuning in a Pocket

On-device learning and efficient fine-tuning enable continuous and privacy-preserving customization (e.g., locally fine-tuning large language models on personalized data). However, existing training frameworks are designed for cloud servers with powerful accelerators (e.g., GPUs, TPUs) and lack the optimizations for learning on the edge, which faces challenges of resource limitations and edge hardware diversity. We introduce PockEngine: a tiny, sparse and efficient engine to enable fine-tuning on various edge devices. PockEngine supports sparse backpropagation: it prunes the backward graph and sparsely updates the model with measured memory saving and latency reduction while maintaining the model quality. Secondly, PockEngine is compilation first: the entire training graph (including forward, backward and optimization steps) is derived at compile-time, which reduces the runtime overhead and brings opportunities for graph transformations. PockEngine also integrates a rich set of training graph optimizations, thus can further accelerate the training cost, including operator reordering and backend switching. PockEngine supports diverse applications, frontends and hardware backends: it flexibly compiles and tunes models defined in PyTorch/TensorFlow/Jax and deploys binaries to mobile CPU/GPU/DSPs. We evaluated PockEngine on both vision models and large language models. PockEngine achieves up to 15 $\times$ speedup over off-the-shelf TensorFlow (Raspberry Pi), 5.6 $\times$ memory saving back-propagation (Jetson AGX Orin). Remarkably, PockEngine enables fine-tuning LLaMav2-7B on NVIDIA Jetson AGX Orin at 550 tokens/s, 7.9$\times$ faster than the PyTorch

Global genome expression analysis of rice in response to drought and high-salinity stresses in shoot, flag leaf, and panicle

To elucidate genome-level responses to drought and high-salinity stress in rice, a 70mer oligomer microarray covering 36,926 unique genes or gene models was used to profile genome expression changes in rice shoot, flag leaf and panicle under drought or high-salinity conditions. While patterns of gene expression in response to drought or high-salinity stress within a particular organ type showed significant overlap, comparison of expression profiles among different organs showed largely organ-specific patterns of regulation. Moreover, both stresses appear to alter the expression patterns of a significant number of genes involved in transcription and cell signaling in a largely organ-specific manner. The promoter regions of genes induced by both stresses or induced by one stress in more than one organ types possess relative enrichment of two cis-elements (ABRE core and DRE core) known to be associated with water stress. An initial computational analysis indicated that novel promoter motifs are present in the promoters of genes involved in rehydration after drought. This analysis suggested that rice might possess a mechanism that actively detects rehydration and facilitates rapid recovery. Overall, our data supports a notion that organ-specific gene regulation in response to the two abiotic stresses may primarily be mediated by organ-specific transcription responses. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s11103-006-9111-1) contains supplementary material, which is available to authorized users

Colourization of Dichromatic Images

This paper explores the colour information dichromatic vision provides in terms of its potential for colourization. Given a greyscale image as input, colourization generates an RGB image as output. Since colourization works well for luminance images, how well they might work for dichromatic images? Dichromatic images are colourized using a modification of the colourization method of Iizuka et al. (Proc. SIGGRAPH 2016, 35(4):110:1-110:11). In particular, an sRGB image is converted to cone LMS and M is discarded to yield a LS image. During training, the colourization neural network is provided LS images and their corresponding LMS images, and it adjusts its weights so that M is predicted from the L and S. One does not easily recognize that a colourized dichromatic image is, in fact, based on only L and S, and is not a regular full-colour image. This is stark contrast to the dichromatic simulations of Brettel et al. (Brettel, Viénot, Mollon, JOSA A 14, 2647-2655, 1997)