OBJ2TEXT: Generating Visually Descriptive Language from Object Layouts

Generating captions for images is a task that has recently received considerable attention. In this work we focus on caption generation for abstract scenes, or object layouts where the only information provided is a set of objects and their locations. We propose OBJ2TEXT, a sequence-to-sequence model that encodes a set of objects and their locations as an input sequence using an LSTM network, and decodes this representation using an LSTM language model. We show that our model, despite encoding object layouts as a sequence, can represent spatial relationships between objects, and generate descriptions that are globally coherent and semantically relevant. We test our approach in a task of object-layout captioning by using only object annotations as inputs. We additionally show that our model, combined with a state-of-the-art object detector, improves an image captioning model from 0.863 to 0.950 (CIDEr score) in the test benchmark of the standard MS-COCO Captioning task.Comment: Accepted at EMNLP 201

Learning Globally Optimized Language Structure via Adversarial Training

Recent work has explored integrating autoregressive language models with energy-based models (EBMs) to enhance text generation capabilities. However, learning effective EBMs for text is challenged by the discrete nature of language. This work proposes an adversarial training strategy to address limitations in prior efforts. Specifically, an iterative adversarial attack algorithm is presented to generate negative samples for training the EBM by perturbing text from the autoregressive model. This aims to enable the EBM to suppress spurious modes outside the support of the data distribution. Experiments on an arithmetic sequence generation task demonstrate that the proposed adversarial training approach can substantially enhance the quality of generated sequences compared to prior methods. The results highlight the promise of adversarial techniques to improve discrete EBM training. Key contributions include: (1) an adversarial attack strategy tailored to text to generate negative samples, circumventing MCMC limitations; (2) an adversarial training algorithm for EBMs leveraging these attacks; (3) empirical validation of performance improvements on a sequence generation task

Adversarial Example Detection and Classification With Asymmetrical Adversarial Training

The vulnerabilities of deep neural networks against adversarial examples have become a significant concern for deploying these models in sensitive domains. Devising a definitive defense against such attacks is proven to be challenging, and the methods relying on detecting adversarial samples are only valid when the attacker is oblivious to the detection mechanism. In this paper we first present an adversarial example detection method that provides performance guarantee to norm constrained adversaries. The method is based on the idea of training adversarial robust subspace detectors using asymmetrical adversarial training (AAT). The novel AAT objective presents a minimax problem similar to that of GANs; it has the same convergence property, and consequently supports the learning of class conditional distributions. We first demonstrate that the minimax problem could be reasonably solved by PGD attack, and then use the learned class conditional generative models to define generative detection/classification models that are both robust and more interpretable. We provide comprehensive evaluations of the above methods, and demonstrate their competitive performances and compelling properties on adversarial detection and robust classification problems.Comment: ICLR 202

Analyzing and Improving Generative Adversarial Training for Generative Modeling and Out-of-Distribution Detection

Generative adversarial training (GAT) is a recently introduced adversarial defense method. Previous works have focused on empirical evaluations of its application to training robust predictive models. In this paper we focus on theoretical understanding of the GAT method and extending its application to generative modeling and out-of-distribution detection. We analyze the optimal solutions of the maximin formulation employed by the GAT objective, and make a comparative analysis of the minimax formulation employed by GANs. We use theoretical analysis and 2D simulations to understand the convergence property of the training algorithm. Based on these results, we develop an incremental generative training algorithm, and conduct comprehensive evaluations of the algorithm's application to image generation and adversarial out-of-distribution detection. Our results suggest that generative adversarial training is a promising new direction for the above applications

Feed types driven differentiation of microbial community and functionality in marine integrated multitrophic aquaculture system

Integrated multi trophic aquaculture (IMTA) improves the production of aquatic animals by promoting nutrient utilization through different tropical levels. Microorganisms play an important role in elements cycling, energy flow and farmed-species health. The aim of this study was to evaluate how feed types, fresh frozen fish diet (FFD) or formulated diet (FD), influence the microbial community diversity and functionality in both water and sediment in a marine IMTA system. Preferable water quality, higher animal yields and higher cost efficiency were achieved in the FD pond. Feed types changed the pond bacterial community distribution, especially in the rearing water. The FFD pond was dominated with Cyanobacteria in the water, which played an important role in nitrogen fixation through photosynthesis due to the high nitrogen input of the frozen fish diet. The high carbohydrate composition in the formulated diet triggered higher metabolic pathways related to carbon and lipid metabolism in the water of the FD pond. Sediment had significantly higher microbial diversity than the rearing water. In sediment, the dominating genus, Sulfurovum and Desulfobulbus, were found to be positively correlated by network analysis, which had similar functionality in sulfur transformation. The relatively higher rates of antibiotic biosynthesis in the FFD sediment might be related to the pathogenic bacteria introduced by the trash fish diet. The difference in microbial community composition and metabolic pathways may be associated with the different pathways for nutrient cycling and animal growth performance. The formulated diet was determined to be more ecologically and economically sustainable than the frozen fish diet for marine IMTA pond systems.</p

Representation Engineering: A Top-Down Approach to AI Transparency

In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. RepE places population-level representations, rather than neurons or circuits, at the center of analysis, equipping us with novel methods for monitoring and manipulating high-level cognitive phenomena in deep neural networks (DNNs). We provide baselines and an initial analysis of RepE techniques, showing that they offer simple yet effective solutions for improving our understanding and control of large language models. We showcase how these methods can provide traction on a wide range of safety-relevant problems, including honesty, harmlessness, power-seeking, and more, demonstrating the promise of top-down transparency research. We hope that this work catalyzes further exploration of RepE and fosters advancements in the transparency and safety of AI systems.Comment: Code is available at https://github.com/andyzoujm/representation-engineerin

Effect of Different Land Use Types on the Taxonomic and Functional Diversity of Macroinvertebrates in an Urban Area of Northern China

The urbanization of riverine landscapes is an increasing threat to river ecosystems. However, it is unclear which metrics can best assess the response of macroinvertebrates to the conversion of forested lands to urban and agricultural lands. The main goal of this study is to examine whether trait-based approaches are more sensitive than taxonomic approaches in distinguishing macroinvertebrate responses to different land use types in a highly urbanized area of northern China. Results based on 14 environmental variables showed a significant difference across a human-induced environmental gradient. The results showed that no significant differences were observed in terms of taxonomic diversity indices between the different land use types. Functional evenness (FEve) and Rao’s quadratic entropy decreased with the increase in urban area caused by the intensification of human activity, demonstrating that functional diversity is more sensitive than taxonomic diversity in discriminating between different land use types. In addition, the results based on RLQ (physical–chemical variables (R), macroinvertebrate taxa (L), and species traits (Q)) and fourth-corner analyses indicated that the trait states of bi- or multivoltine, high dispersal capacity, and not-streamlined body shape were much higher in the agricultural area and positively related to farmland percentage. Taxa with large body size were dominant in urban areas and were positively correlated with EC. Overall, the observed responses of traits to environmental variables suggest that trait-based approaches should be incorporated into land use management for river restoration