Homomorphism AutoEncoder — Learning Group Structured Representations from Observed Transitions

How can agents learn internal models that veridically represent interactions with the real world is a largely open question. As machine learning is moving towards representations containing not just observational but also interventional knowledge, we study this problem using tools from representation learning and group theory. We propose methods enabling an agent acting upon the world to learn internal representations of sensory information that are consistent with actions that modify it. We use an autoencoder equipped with a group representation acting on its latent space, trained using an equivariance-derived loss in order to enforce a suitable homomorphism property on the group representation. In contrast to existing work, our approach does not require prior knowledge of the group and does not restrict the set of actions the agent can perform. We motivate our method theoretically, and show empirically that it can learn a group representation of the actions, thereby capturing the structure of the set of transformations applied to the environment. We further show that this allows agents to predict the effect of sequences of future actions with improved accuracy

Did the Paleo-Asian Ocean between North China Block and Mongolia Block exist during the Late Paleozoic? First paleomagnetic evidence from central-eastern Inner Mongolia, China

International audienceThe tectonic evolution of the Paleo-Asian Ocean between the North China Block (NCB) and the Mongolia Block (MOB) is a contentious issue, and geodynamic models remain speculative. In an effort to puzzle out this controversy, a paleomagnetic study was carried out on the Silurian to Permian formations in central- eastern Inner Mongolia (China). More than 680 sedimentary and volcanic samples were collected from 86 sites. We have established titanium-poor magnetite and hematite as the principal magnetic carriers. AMS measurements demonstrate negligible deformation of the majority of study rocks with sedimentary fabrics. From primary magnetizations, a Late Devonian and a Permian pole are calculated for IMB at: λ = 46.8°N, φ = 349.1°E, dp = 14.6°, dm = 27.3° with N = 3 and λ = 48.7°N, φ = 3.7°E, dp = 5.2°, dm = 9.1° with N = 6, respectively. Two stages of secondary magnetization are also identified probably due to Early Permian and Early Cretaceous magmatic events. As preliminary results, the comparison of our new paleomagnetic poles with available data from NCB, MOB and Siberia indicates that (1) the paleolatitude of IMB, NCB and MOB are consistent between Late Devonian and Permian, suggesting pre-Late Devonian closure of the Paleo-Asian Ocean and further evaluation of these three blocks as a single entity; (2) post-Permian intracontinental deformation was significant and characterized by block rotations, which are due to strike-slip faulting within the welded NCB-IMB-MOB block

Homomorphism Autoencoder —- Learning Group Structured Representations from Interactions

It is crucial for agents, both biological and artificial, to acquire world models that veridically represent the external world and how it is modified by the agent's own actions. We consider the case where such modifications can be modelled as transformations from a group of symmetries structuring the world state space. We use tools from representation learning and group theory to learn latent representations that account for both sensory information and the actions that alters it during interactions. We introduce the Homomorphism AutoEncoder (HAE), an autoencoder equipped with a learned group representation linearly acting on its latent space trained on 2-step transitions to implicitly enforce the group homomorphism property on the action representation. Compared to existing work, our approach makes fewer assumptions on the group representation and on which transformations the agent can sample from. We motivate our method theoretically, and demonstrate empirically that it can learn the correct representation of the groups and the topology of the environment. We also compare its performance in trajectory prediction with previous methods

Dynamic parameters of structures extracted from ambient vibration measurements: an aid for the seismic vulnerability assessment of existing buildings in moderate seismic hazard regions

During the past two decades, the use of ambient vibrations for modal analysis of structures has increased as compared to the traditional techniques (forced vibrations). The Frequency Domain Decomposition method is nowadays widely used in modal analysis because of its accuracy and simplicity. In this paper, we first present the physical meaning of the FDD method to estimate the modal parameters. We discuss then the process used for the evaluation of the building stiffness deduced from the modal shapes. The models considered here are 1D lumped-mass beams and especially the shear beam. The analytical solution of the equations of motion makes it possible to simulate the motion due to a weak to moderate earthquake and then the inter-storey drift knowing only the modal parameters (modal model). This process is finally applied to a 9-storey reinforced concrete (RC) dwelling in Grenoble (France). We successfully compared the building motion for an artificial ground motion deduced from the model estimated using ambient vibrations and recorded in the building. The stiffness of each storey and the inter-storey drift were also calculated

Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models

Large language models (LLMs) have achieved remarkable progress in various natural language processing tasks with emergent abilities. However, they face inherent limitations, such as an inability to access up-to-date information, utilize external tools, or perform precise mathematical reasoning. In this paper, we introduce Chameleon, a plug-and-play compositional reasoning framework that augments LLMs to help address these challenges. Chameleon synthesizes programs to compose various tools, including LLM models, off-the-shelf vision models, web search engines, Python functions, and rule-based modules tailored to user interests. Built on top of an LLM as a natural language planner, Chameleon infers the appropriate sequence of tools to compose and execute in order to generate a final response. We showcase the adaptability and effectiveness of Chameleon on two tasks: ScienceQA and TabMWP. Notably, Chameleon with GPT-4 achieves an 86.54% accuracy on ScienceQA, significantly improving upon the best published few-shot model by 11.37%; using GPT-4 as the underlying LLM, Chameleon achieves a 17.8% increase over the state-of-the-art model, leading to a 98.78% overall accuracy on TabMWP. Further studies suggest that using GPT-4 as a planner exhibits more consistent and rational tool selection and is able to infer potential constraints given the instructions, compared to other LLMs like ChatGPT.Comment: 25 pages, 10 figures. Project page: https://chameleon-llm.github.i

MathVista: Evaluating Math Reasoning in Visual Contexts with GPT-4V, Bard, and Other Large Multimodal Models

Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit impressive problem-solving skills in many tasks and domains, but their ability in mathematical reasoning in visual contexts has not been systematically studied. To bridge this gap, we present MathVista, a benchmark designed to combine challenges from diverse mathematical and visual tasks. It consists of 6,141 examples, derived from 28 existing multimodal datasets involving mathematics and 3 newly created datasets (i.e., IQTest, FunctionQA, and PaperQA). Completing these tasks requires fine-grained, deep visual understanding and compositional reasoning, which all state-of-the-art foundation models find challenging. With MathVista, we have conducted a comprehensive, quantitative evaluation of 12 prominent foundation models. The best-performing GPT-4V model achieves an overall accuracy of 49.9%, substantially outperforming Bard, the second-best performer, by 15.1%. Our in-depth analysis reveals that the superiority of GPT-4V is mainly attributed to its enhanced visual perception and mathematical reasoning. However, GPT-4V still falls short of human performance by 10.4%, as it often struggles to understand complex figures and perform rigorous reasoning. This significant gap underscores the critical role that MathVista will play in the development of general-purpose AI agents capable of tackling mathematically intensive and visually rich real-world tasks. We further explore the new ability of self-verification, the application of self-consistency, and the interactive chatbot capabilities of GPT-4V, highlighting its promising potential for future research. The project is available at https://mathvista.github.io/.Comment: 112 pages, 117 figures. Work in progres

Nuclear Shell Model Calculations with Fundamental Nucleon-Nucleon Interactions

Some fundamental Nucleon-Nucleon interactions and their applications to finite nuclei are reviewed. Results for the few-body systems and from Shell-Model calculations are discussed and compared to point out the advantages and disadvantages of the different Nucleon-Nucleon interactions. The recently developed Drexel University Shell Model (DUSM) code is mentioned.Comment: 16 pages, 4 figures. To appear in Phys. Rep. 199