Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening

In the feature space, the collapse between features invokes critical problems in representation learning by remaining the features undistinguished. Interpolation-based augmentation methods such as mixup have shown their effectiveness in relieving the collapse problem between different classes, called inter-class collapse. However, intra-class collapse raised in coarse-to-fine transfer learning has not been discussed in the augmentation approach. To address them, we propose a better feature augmentation method, asymptotic midpoint mixup. The method generates augmented features by interpolation but gradually moves them toward the midpoint of inter-class feature pairs. As a result, the method induces two effects: 1) balancing the margin for all classes and 2) only moderately broadening the margin until it holds maximal confidence. We empirically analyze the collapse effects by measuring alignment and uniformity with visualizing representations. Then, we validate the intra-class collapse effects in coarse-to-fine transfer learning and the inter-class collapse effects in imbalanced learning on long-tailed datasets. In both tasks, our method shows better performance than other augmentation methods

Revisiting Softmax Masking for Stability in Continual Learning

In continual learning, many classifiers use softmax function to learn confidence. However, numerous studies have pointed out its inability to accurately determine confidence distributions for outliers, often referred to as epistemic uncertainty. This inherent limitation also curtails the accurate decisions for selecting what to forget and keep in previously trained confidence distributions over continual learning process. To address the issue, we revisit the effects of masking softmax function. While this method is both simple and prevalent in literature, its implication for retaining confidence distribution during continual learning, also known as stability, has been under-investigated. In this paper, we revisit the impact of softmax masking, and introduce a methodology to utilize its confidence preservation effects. In class- and task-incremental learning benchmarks with and without memory replay, our approach significantly increases stability while maintaining sufficiently large plasticity. In the end, our methodology shows better overall performance than state-of-the-art methods, particularly in the use with zero or small memory. This lays a simple and effective foundation of strongly stable replay-based continual learning

Feature Structure Distillation for BERT Transferring

Knowledge distillation is an approach to transfer information on representations from a teacher to a student by reducing their difference. A challenge of this approach is to reduce the flexibility of the student's representations inducing inaccurate learning of the teacher's knowledge. To resolve it in BERT transferring, we investigate distillation of structures of representations specified to three types: intra-feature, local inter-feature, global inter-feature structures. To transfer them, we introduce \textit{feature structure distillation} methods based on the Centered Kernel Alignment, which assigns a consistent value to similar features structures and reveals more informative relations. In particular, a memory-augmented transfer method with clustering is implemented for the global structures. In the experiments on the nine tasks for language understanding of the GLUE dataset, the proposed methods effectively transfer the three types of structures and improve performance compared to state-of-the-art distillation methods. Indeed, the code for the methods is available in https://github.com/maroo-sky/FSDComment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

CFASL: Composite Factor-Aligned Symmetry Learning for Disentanglement in Variational AutoEncoder

Symmetries of input and latent vectors have provided valuable insights for disentanglement learning in VAEs.However, only a few works were proposed as an unsupervised method, and even these works require known factor information in training data. We propose a novel method, Composite Factor-Aligned Symmetry Learning (CFASL), which is integrated into VAEs for learning symmetry-based disentanglement in unsupervised learning without any knowledge of the dataset factor information.CFASL incorporates three novel features for learning symmetry-based disentanglement: 1) Injecting inductive bias to align latent vector dimensions to factor-aligned symmetries within an explicit learnable symmetry codebook 2) Learning a composite symmetry to express unknown factors change between two random samples by learning factor-aligned symmetries within the codebook 3) Inducing group equivariant encoder and decoder in training VAEs with the two conditions. In addition, we propose an extended evaluation metric for multi-factor changes in comparison to disentanglement evaluation in VAEs. In quantitative and in-depth qualitative analysis, CFASL demonstrates a significant improvement of disentanglement in single-factor change, and multi-factor change conditions compared to state-of-the-art methods.Comment: 21 pages, 14 figure

Dynamic Interactions between Carbon and Energy Prices in the U.S. Regional Greenhouse Gas Initiative Region

Numerous studies have investigated the dynamic interrelationship between carbon emission trading market and energy markets. Previous studies focused on the European Union Emissions Trading Scheme ascertain that carbon market and energy markets are closely attached, and find that electricity market is the main driver of the system. Our research on U.S. Regional Greenhouse Gas Initiative (RGGI) using Lag Augmented Vector Autoregression reveals that the RGGI market and electricity market in the region are tied but not strongly, unlike the EU-ETS. This loose relationship between the two markets might be explained by the recent weak carbon credit demand stemming from fuel switching and low electricity demand. Another finding is that natural gas is the main driver of the RGGI system, which is possibly due to from the recent shale gas boom. Keywords: Carbon emission trading; Lag Augmented Vector Autoregression; Regional Greenhouse Gas Initiative  JEL Classifications: C32; Q52; Q5

Stretchable, Patch-Type, Wireless, 6-axis Inertial Measurement Unit for Mobile Health Monitoring

Wearable devices which measure and transfer signals from the human body can provide useful biometric data for various biomedical applications. In this paper, we present an implementation of the advanced Inertial Measurement Unit (IMU) with wireless communication technology for mobile health monitoring. The device consists of rigid silicon-based components on a flexible/stretchable substrate for applications in epidermal electronic devices to collect precise data from the human body. Using the Bluetooth Low Energy (BLE) System-on-a-chip (SoC), the device can be miniaturized and portable, and the collected data can be processed with low power consumption. The dimensions of the implemented system are approximately 40 mm × 40 mm × 100 mm. Also, the device can be attached closely to human skin, which results in minimized signal distortion due to body movements or skin deformations. In order to achieve device flexibility and stretch ability, the interconnection wires are designed as serpentine-shaped structures on a stretchable substrate. The previously reported “cut-and-paste” method is utilized to fabricate the device that produces complex, twisty interconnections with thin metal sheets. The implemented patch-type, wireless, 6-axis IMU is expected to have potential in various applications, such as health monitoring, dependency care, and daily lifelogging

Participation “In the Heavenlies” in Christ: Deification in Ephesians

Paul’s expression “in the heavenlies” provides an intriguing showcase of the power dynamics of the divine–human relationship (e.g., 1:3, 20; 2:6; 3:10; 6:12). While scholars have identified the theme of union with Christ as an interpretive key for understanding believers’ position in the heavenlies, few have provided adequate attention to “in the heavenlies” according to the significance of theosis. I argue that a patristic idea of theosis offers an interpretive lens in understanding believers’ lives in the heavenlies. Thus, this study aims to situate the discussion on the heavenlies vis-à-vis the conversation around theosis in the New Testament