Can We Utilize Pre-trained Language Models within Causal Discovery Algorithms?

Scaling laws have allowed Pre-trained Language Models (PLMs) into the field of causal reasoning. Causal reasoning of PLM relies solely on text-based descriptions, in contrast to causal discovery which aims to determine the causal relationships between variables utilizing data. Recently, there has been current research regarding a method that mimics causal discovery by aggregating the outcomes of repetitive causal reasoning, achieved through specifically designed prompts. It highlights the usefulness of PLMs in discovering cause and effect, which is often limited by a lack of data, especially when dealing with multiple variables. Conversely, the characteristics of PLMs which are that PLMs do not analyze data and they are highly dependent on prompt design leads to a crucial limitation for directly using PLMs in causal discovery. Accordingly, PLM-based causal reasoning deeply depends on the prompt design and carries out the risk of overconfidence and false predictions in determining causal relationships. In this paper, we empirically demonstrate the aforementioned limitations of PLM-based causal reasoning through experiments on physics-inspired synthetic data. Then, we propose a new framework that integrates prior knowledge obtained from PLM with a causal discovery algorithm. This is accomplished by initializing an adjacency matrix for causal discovery and incorporating regularization using prior knowledge. Our proposed framework not only demonstrates improved performance through the integration of PLM and causal discovery but also suggests how to leverage PLM-extracted prior knowledge with existing causal discovery algorithms

Application and comparison of Kalman filters for coastal ocean problems : an experiment with FVCOM

Author Posting. © American Geophysical Union, 2009. This article is posted here by permission of American Geophysical Union for personal use, not for redistribution. The definitive version was published in Journal of Geophysical Research 114 (2009): C05011, doi:10.1029/2007JC004548.Twin experiments were made to compare the reduced rank Kalman filter (RRKF), ensemble Kalman filter (EnKF), and ensemble square-root Kalman filter (EnSKF) for coastal ocean problems in three idealized regimes: a flat bottom circular shelf driven by tidal forcing at the open boundary; an linear slope continental shelf with river discharge; and a rectangular estuary with tidal flushing intertidal zones and freshwater discharge. The hydrodynamics model used in this study is the unstructured grid Finite-Volume Coastal Ocean Model (FVCOM). Comparison results show that the success of the data assimilation method depends on sampling location, assimilation methods (univariate or multivariate covariance approaches), and the nature of the dynamical system. In general, for these applications, EnKF and EnSKF work better than RRKF, especially for time-dependent cases with large perturbations. In EnKF and EnSKF, multivariate covariance approaches should be used in assimilation to avoid the appearance of unrealistic numerical oscillations. Because the coastal ocean features multiscale dynamics in time and space, a case-by-case approach should be used to determine the most effective and most reliable data assimilation method for different dynamical systems.P. Malanotte-Rizzoli and J. Wei were supported by the Office of Naval Research (ONR grant N00014-06-1- 0290); C. Chen and Q. Xu were supported by the U.S. GLOBEC/Georges Bank program (through NSF grants OCE-0234545, OCE-0227679, OCE- 0606928, OCE-0712903, OCE-0726851, and OCE-0814505 and NOAA grant NA-16OP2323), the NSF Arctic research grants ARC0712903, ARC0732084, and ARC0804029, and URI Sea Grant R/P-061; P. Xue was supported through the MIT Sea Grant 2006-RC-103; Z. Lai, J. Qi, and G. Cowles were supported through the Massachusetts Marine Fisheries Institute (NOAA grants NA04NMF4720332 and NA05NMF4721131); and R. Beardsley was supported through U.S. GLOBEC/Georges Bank NSF grant OCE-02227679, MIT Sea Grant NA06OAR1700019, and the WHOI Smith Chair in Coastal Oceanography

Application and comparison of Kalman filters for coastal ocean problems: An experiment with FVCOM

Twin experiments were made to compare the reduced rank Kalman filter (RRKF), ensemble Kalman filter (EnKF), and ensemble square-root Kalman filter (EnSKF) for coastal ocean problems in three idealized regimes: a flat bottom circular shelf driven by tidal forcing at the open boundary; an linear slope continental shelf with river discharge; and a rectangular estuary with tidal flushing intertidal zones and freshwater discharge. The hydrodynamics model used in this study is the unstructured grid Finite-Volume Coastal Ocean Model (FVCOM). Comparison results show that the success of the data assimilation method depends on sampling location, assimilation methods (univariate or multivariate covariance approaches), and the nature of the dynamical system. In general, for these applications, EnKF and EnSKF work better than RRKF, especially for time-dependent cases with large perturbations. In EnKF and EnSKF, multivariate covariance approaches should be used in assimilation to avoid the appearance of unrealistic numerical oscillations. Because the coastal ocean features multiscale dynamics in time and space, a case-by-case approach should be used to determine the most effective and most reliable data assimilation method for different dynamical systems.United States. Office of Naval Research (grant N00014-06-1-0290)National Science Foundation (U.S.) (grant OCE-0234545)National Science Foundation (U.S.) (grant OCE-0227679)National Science Foundation (U.S.) (grant OCE-0606928)National Science Foundation (U.S.) (grant OCE-0712903)National Science Foundation (U.S.) (grant OCE-0726851)National Science Foundation (U.S.) (grant OCE-0814505)United States. National Oceanic and Atmospheric Administration (grant NA-16OP2323)National Science Foundation (U.S.) (grant ARC0712903)National Science Foundation (U.S.) (grant ARC0732084)National Science Foundation (U.S.) (grant ARC0804029)URI Sea Grant (R/P-061)Massachusetts Institute of Technology. Sea Grant College Program (2006-RC-103)Massachusetts Marine Fisheries Institute (NOAA grant NA05NMF4721131)Massachusetts Marine Fisheries Institute (NOAA grant NA04NMF4720332)National Science Foundation (U.S.) (grant OCE-02227679)Massachusetts Institute of Technology. Sea Grant College Program (NA06OAR1700019