Universal Metric Learning with Parameter-Efficient Transfer Learning

A common practice in metric learning is to train and test an embedding model for each dataset. This dataset-specific approach fails to simulate real-world scenarios that involve multiple heterogeneous distributions of data. In this regard, we introduce a novel metric learning paradigm, called Universal Metric Learning (UML), which learns a unified distance metric capable of capturing relations across multiple data distributions. UML presents new challenges, such as imbalanced data distribution and bias towards dominant distributions. To address these challenges, we propose Parameter-efficient Universal Metric leArning (PUMA), which consists of a pre-trained frozen model and two additional modules, stochastic adapter and prompt pool. These modules enable to capture dataset-specific knowledge while avoiding bias towards dominant distributions. Additionally, we compile a new universal metric learning benchmark with a total of 8 different datasets. PUMA outperformed the state-of-the-art dataset-specific models while using about 69 times fewer trainable parameters

Aligning Large Language Models through Synthetic Feedback

Aligning large language models (LLMs) to human values has become increasingly important as it enables sophisticated steering of LLMs, e.g., making them follow given instructions while keeping them less toxic. However, it requires a significant amount of human demonstrations and feedback. Recently, open-sourced models have attempted to replicate the alignment learning process by distilling data from already aligned LLMs like InstructGPT or ChatGPT. While this process reduces human efforts, constructing these datasets has a heavy dependency on the teacher models. In this work, we propose a novel framework for alignment learning with almost no human labor and no dependency on pre-aligned LLMs. First, we perform reward modeling (RM) with synthetic feedback by contrasting responses from vanilla LLMs with various sizes and prompts. Then, we use the RM for simulating high-quality demonstrations to train a supervised policy and for further optimizing the model with reinforcement learning. Our resulting model, Aligned Language Model with Synthetic Training dataset (ALMoST), outperforms open-sourced models, including Alpaca, Dolly, and OpenAssistant, which are trained on the outputs of InstructGPT or human-annotated instructions. Our 7B-sized model outperforms the 12-13B models in the A/B tests using GPT-4 as the judge with about 75% winning rate on average.Comment: Preprint, 9 pages (with 10 pages of supplementary

Turbulent Jet-Assisted Microfiltration for Energy Efficient Harvesting of Microalgae

For energy-efficient harvesting of microalgae using a hollow fiber membrane, a turbulent jet was implemented to induce local high crossflow velocity near the membrane surface for fouling reduction during microfiltration. The performance of the turbulent jet-assisted module was evaluated and compared to that of a control group that represented other types of flow conditions including the conventional-type hollow fiber membrane module. When assisted by the turbulent jet, permeate flux at the steady-state increased by 126% and the specific energy for filtrating out a unit volume of permeate was reduced by 38% relative to the conventional type. In the results of a computational fluid dynamics analysis, the wall jet created after impingement of the jet flew along the membrane surface with a reduced boundary layer, and it is expected that this provided a scouring phenomenon. Shear stress on the membrane surface increased 3.7-fold on average, and was highest at the point of impingement. With regard to energy efficiency, concentrating on increasing the local fluid velocity near the membrane via turbulent jets rather than increasing the entire feed recirculation is more practical to improve the filtration performance for microalgae harvesting with low power consumption

Stand-off radiation detection techniques

Remote detection of radioactive materials is extremely challenging, yet it is important to realize the technique for safe usage of radioactive materials. Gamma rays are the most far distant penetrating photons that are involved with the radiation decay process. Herein, we overview the gamma-ray detection techniques that are material-based and vacuum tube-based. A muon detector is also reviewed as a radioactive material imager. We overview versatile detectors that are currently being widely used and new concepts that may pave the way for promising remote detectability up to several kilometers

Solvent Screening and Process Optimization for High Shear-Assisted Lipid Extraction from Wet Cake of Nannochloropsis Sp.

Microalgae are regarded as a promising feedstock for biofuels and value-added products but still suffer from an inefficient lipid extraction process. In the present study, a simple and energy-efficient extraction method is demonstrated to extract oil directly from the wet cake (260 g/L) of Nannochloropsis sp. with an assist from the high shear mixer (HSM). After the initial solvent screening, the composition of co-solvent and operating conditions were optimized according to lipid composition and extraction yield. The high shear-assisted extraction process was found to achieve 83% lipid extraction yield (94% for EPA) in 5 min and 95% yield (100% for EPA) in 30 min with minimal amounts of solvents (0.9 ml hexane, 0.39 ml ethanol, and 0.057 ml sulfuric acid for 1 g of wet cell) at 8000 rpm, 55 °C. In comparison with various two-step wet extraction methods, the HSM offers the most economical extraction in terms of specific energy consumption of 1.38 MJ/kg dry cell. Therefore, the HSM can be considered as an attractive alternative to conventional extraction methods, providing a new paradigm of wet extraction for microalgae

Utilizing inactive storage in a dam reservoir during extreme drought periods

The purpose of this study is to suggest a structural plan for improving the utilization of inactive storage in dam reservoirs, to mitigate extreme drought. Inactive storage in the dam is composed of emergency storage and dead storage. The emergency storage can be used in emergencies such as drought. But, in general, the dead storage for sedimentation is not used, even in an emergency. Therefore, we developed a methodology to determine how the dead storage space can be partially used during extreme drought periods when the sedimentation has not occurred yet. We call this partial space in a dam reservoir “drought storage”. An accurate analysis of sediment levels needs to be performed before calculating drought storage, and so the present sediment level in the dam reservoir was estimated using SED-2D linked with the RMA-2 model of SMS. After considering the additional available storage capacity based on the estimated sediment level, drought storage was finally determined. We also predicted future sediment levels after 100 years and suggest the amount of drought storage available in the future. As a result, we found that the available drought storage will be lower in the future compared to present drought storage, due to the gradual increase in reservoir sedimentation over time in the dam. Further research may be needed to effectively reduce sedimentation in order to increase the drought storage capacity

The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training

Visual dialog (VisDial) is a task of answering a sequence of questions grounded in an image, using the dialog history as context. Prior work has trained the dialog agents solely on VisDial data via supervised learning or leveraged pre-training on related vision-and-language datasets. This paper presents a semi-supervised learning approach for visually-grounded dialog, called Generative Self-Training (GST), to leverage unlabeled images on the Web. Specifically, GST first retrieves in-domain images through out-of-distribution detection and generates synthetic dialogs regarding the images via multimodal conditional text generation. GST then trains a dialog agent on the synthetic and the original VisDial data. As a result, GST scales the amount of training data up to an order of magnitude that of VisDial (1.2M to 12.9M QA data). For robust training of the generated dialogs, we also propose perplexity-based data selection and multimodal consistency regularization. Evaluation on VisDial v1.0 and v0.9 datasets shows that GST achieves new state-of-the-art results on both datasets. We further observe strong performance gains in the low-data regime (up to 9.35 absolute points on NDCG).Comment: 16 pages, 4 figure