Impact Of Fines On Gas Relative Permeability Through Sand Using Pore Networks From 3d Synchrotron Micro-Computed Tomography

Fines migration and transport in sand systems have huge influence on vital applications, including the storage and recovery of water and energy resources from the subsurface. Multi-phase flow of gas through saturated unconsolidated media takes place between the pores of sediments, physical phenomenon at the pore-scale control the flow properties. Given a sandy sediment media, gas permeability is highly affected by fine particles due to migration, clogging and bridging reducing gas flow or causing sand particles to displace creating fractures. There is a knowledge gap of fines effects on gas production from sandy sediments, especially at the pore-scale. Therefore, there is a need to model and quantify effects of fines in multi-phase flow using pore networks to better understand gas recovery systems. Three-dimensional, synchrotron micro-computed tomography images of sand sediments were obtained at Argonne National Laboratory at a resolution of 3.89 micron per voxel. Kaolinite and Montmorillonite fine particles were added in varied concentrations in six soil specimens, each system was scanned at four stages with varied saturations of brine and CO2, resulting in 20 systems. Micro-computed tomography images were processed for 3D visualization, quantification and pore network modeling. Pore Network Models were generated, and relative permeability properties were then computed for each system. Findings revealed that fines accumulate at sand-brine and brine-gas interfaces. As fines concentration increased, gas percolation decreased. Further increase in fines concentrations resulted in blocking local gas flow causing pressure variations enough to create fractures that allows gas to escape and permeability to increase back. Pore Networks and Computer-Based Two-Phase Flow Simulations can effectively be used to characterize flow in porous media. In unconsolidated media the pore space geometry will change due to sand grains movements. At high concentrations, different fines type produces altered gas flow regimes, Kaolinite resulted in fractures while montmorillonite resulted in detached gas ganglia. Generally, increasing fines reduces gas percolation and further injection of gas reduced permeability. The finds herein are critical in understanding the impact of fines migration during gas flow in sand, they can be applied to characterizing and predicting two phase properties of unconsolidated sediments

Transfer Learning for Structured Pruning under Limited Task Data

Large, pre-trained models are problematic to use in resource constrained applications. Fortunately, task-aware structured pruning methods offer a solution. These approaches reduce model size by dropping structural units like layers and attention heads in a manner that takes into account the end-task. However, these pruning algorithms require more task-specific data than is typically available. We propose a framework which combines structured pruning with transfer learning to reduce the need for task-specific data. Our empirical results answer questions such as: How should the two tasks be coupled? What parameters should be transferred? And, when during training should transfer learning be introduced? Leveraging these insights, we demonstrate that our framework results in pruned models with improved generalization over strong baselines.Comment: 8 pages, 7 figures and 3 table

Self-Training for End-to-End Speech Recognition

We revisit self-training in the context of end-to-end speech recognition. We demonstrate that training with pseudo-labels can substantially improve the accuracy of a baseline model. Key to our approach are a strong baseline acoustic and language model used to generate the pseudo-labels, filtering mechanisms tailored to common errors from sequence-to-sequence models, and a novel ensemble approach to increase pseudo-label diversity. Experiments on the LibriSpeech corpus show that with an ensemble of four models and label filtering, self-training yields a 33.9% relative improvement in WER compared with a baseline trained on 100 hours of labelled data in the noisy speech setting. In the clean speech setting, self-training recovers 59.3% of the gap between the baseline and an oracle model, which is at least 93.8% relatively higher than what previous approaches can achieve.Comment: To be published in the 45th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 202

Adaptive Training Distributions with Scalable Online Bilevel Optimization

Large neural networks pretrained on web-scale corpora are central to modern machine learning. In this paradigm, the distribution of the large, heterogeneous pretraining data rarely matches that of the application domain. This work considers modifying the pretraining distribution in the case where one has a small sample of data reflecting the targeted test conditions. We propose an algorithm motivated by a recent formulation of this setting as an online, bilevel optimization problem. With scalability in mind, our algorithm prioritizes computing gradients at training points which are likely to most improve the loss on the targeted distribution. Empirically, we show that in some cases this approach is beneficial over existing strategies from the domain adaptation literature but may not succeed in other cases. We propose a simple test to evaluate when our approach can be expected to work well and point towards further research to address current limitations