Remote Sensing Retrieval Study of the Surface Kinetic Parameters in the Yangtze Estuary and Its Adjacent Waters

Wind and current are significant parameters in the hydrodynamic processes, making a significant effect on the expansion of the Yangtze (Changjiang River) Diluted Water, sediment transport, resuspension and shelf circulation in the Yangtze Estuary. They are indispensable as input parameters in the numerical simulation of these phenomena. Synthetic aperture radar (SAR) can acquire data with different resolutions (down to 1 m) and coverage (up to 400 km) over a site during day or night time under all weather conditions, being capable of providing ocean surface kinetic parameters with high resolution. SAR images were collected to verify and improve the validity of wind direction retrieval by 2D fast Fourier transformation (FFT) method, wind speed by CMOD4 model and current by Doppler frequency method. These SAR-retrieved wind and current results were analyzed and assessed against in situ data and corresponding numerically simulated surface wind and current fields. Comparisons to the in situ and simulations show that 1) SAR can measure sea surface wind fields with a high resolution at sub-km scales and provide a powerful complement to conventional wind measurement techniques. 2) The Doppler shift anomaly measurements from SAR images are able to capture quantitative surface currents, thus are helpful to reveal the multi-scale upper layer dynamics around the East China Sea

Optimal Sample Complexity of Reinforcement Learning for Uniformly Ergodic Discounted Markov Decision Processes

We consider the optimal sample complexity theory of tabular reinforcement learning (RL) for controlling the infinite horizon discounted reward in a Markov decision process (MDP). Optimal min-max complexity results have been developed for tabular RL in this setting, leading to a sample complexity dependence on $\gamma$ and $\epsilon$ of the form $\tilde \Theta((1-\gamma)^{-3}\epsilon^{-2})$, where $\gamma$ is the discount factor and $\epsilon$ is the tolerance solution error. However, in many applications of interest, the optimal policy (or all policies) will induce mixing. We show that in these settings the optimal min-max complexity is $\tilde \Theta(t_{\text{minorize}}(1-\gamma)^{-2}\epsilon^{-2})$, where $t_{\text{minorize}}$ is a measure of mixing that is within an equivalent factor of the total variation mixing time. Our analysis is based on regeneration-type ideas, that, we believe are of independent interest since they can be used to study related problems for general state space MDPs

Optimal Sample Complexity for Average Reward Markov Decision Processes

We settle the sample complexity of policy learning for the maximization of the long run average reward associated with a uniformly ergodic Markov decision process (MDP), assuming a generative model. In this context, the existing literature provides a sample complexity upper bound of $\widetilde O(|S||A|t_{\text{mix}}^2 \epsilon^{-2})$ and a lower bound of $\Omega(|S||A|t_{\text{mix}} \epsilon^{-2})$. In these expressions, $|S|$ and $|A|$ denote the cardinalities of the state and action spaces respectively, $t_{\text{mix}}$ serves as a uniform upper limit for the total variation mixing times, and $\epsilon$ signifies the error tolerance. Therefore, a notable gap of $t_{\text{mix}}$ still remains to be bridged. Our primary contribution is to establish an estimator for the optimal policy of average reward MDPs with a sample complexity of $\widetilde O(|S||A|t_{\text{mix}}\epsilon^{-2})$, effectively reaching the lower bound in the literature. This is achieved by combining algorithmic ideas in Jin and Sidford (2021) with those of Li et al. (2020)

CHARACTERIZATION OF SURFACTANTS AND TRACER PROPERTIES FOR POTENTIAL EOR APPLICATIONS

There are some challenges in chemical flooding, such as, gas finger problem usually occurred in field tests, potential scale problems of chemical slug caused by precipitation due to incompatibility between chemical solution and formation brine, and drawbacks of experimental designing of chemical flooding. In this work, three challenges are mainly discussed in following chapters. Chapter one focuses on optimization of designing single well test; chapter two discusses the feasibility of foam stabilized by nanoparticles in porous media; chapter three states that coacervates problems are occurred in preparation of chemical solutions. The summary of three topics is addressed below. The first chapter, the single well chemical tracer test (SWCTT) has emerged in the past decades as a method for measuring oil saturation prior to and/or after EOR operations, to measure the recovery performance in-situ. To use this technology, the partition coefficients of the selected tracers are essential for estimating the level of residual oil at the targeted single well. Commonly, injection of short chain alcohols and ethyl acetate, a reactive tracer, is carried out for the tracer slug, mainly based on site-specific reservoir conditions, to accurately determine the level of oil saturation in-situ. However, injection of ethyl formate has been less common due to its fast hydrolysis rate under elevated temperature, which increases the challenges in data interpretation. Therefore, a systematic study for using ethyl formate under mid-range temperature (<60°C); -as commonly found in mature oil field in the U.S., show the potential to be applied for SWCTT. As part of the design effort for a series of EOR field tests to manage the project risk, we particularly assessed the relationships between the partition coefficients of reactive tracers and subsurface conditions; -such as salinity, temperatures, type of electrolytes and the equivalent alkane carbon number (EACN) of the crude oil experiments were performed under various reservoir conditions as a function of actual site characteristics at the targeted high saline formations. In brief, our data clearly show that the (oil/water) partition coefficient of ethyl formate increase steadily with increasing NaCl concentrations, ranging from 10,000mg/L (0.17M) to 250,000mg/L (4.28M). A similar upward trend was observed for increasing temperature between 25°C to 52°C; however, the partition coefficient decrease inversely with increasing the crude oil EACN over the range from 8 to 12, which are common for domestic oil samples. It was also showed that brine with high NaCl concentration yielded higher partition coefficients. In contrast, brine with high CaCl2 and BaCl2 concentration yielded lower values. And MgCl2 performed somewhat unusual trend in our tests. These results further indicate that the partition coefficient of the reactive tracer, ethyl formate, is sensitive to change in salinity, temperatures, type of electrolytes and EACN, as observed for other chemical tracers. In addition, based on the hydrolysis rate of ethyl formate under various reservoir conditions, the appropriate window of shut-in time can be pre-determined before initiating the field test. We believe that the ability of better understanding the partition coefficients and predicting the shut-in time beforehand could drastically reduce the risks of SWCTT operations. In second chapter, the application of nanoparticles dispersions in foam flooding has become an attractive chemical enhanced oil recovery (EOR) technique as compared to conventional surfactant only foaming system. This study is to expand our understanding of utilizing multi walled carbon nanotube (MWCNT) on foam stability in porous media. We developed several foaming agent formulations (surfactants and polymers) in the presence of MWNT in 3% salinity (NaCl, 2.4wt%, CaCl2, 0.6wt %). The dispersion stability of the MWCNT and the viscosity of the solutions were measured. Foam was generated in-situ, one-dimensional flow-through tests were performed by co-injecting air and foaming solution containing either the foaming agents-only or the foaming agents in the presence of MWCNT. During each experiment, the pressure drop (∆p) and the nanoparticles recovered across the sand-pack were monitored. Injection rate, gas fraction and the effect of MWCNT stabilized foams in porous media were investigated. The results reveal that foams stabilized by nanoparticles are able to generate stronger foams leading to apparent higher ∆p by introducing MWCNT that total concentration is as low as 60ppm. ∆p profile varies with gas fraction which largely affects the foam texture. Also, our data indicate the viscosity of foaming agent solutions influences ∆p values. Adding MWNT to the foaming agent solutions appears beneficial to the flooding as surfactants adsorb to nanoparticles which facilitates surfactants partitioning to the G/L interface. Thus, addition of nanoparticles in the developed surfactant-polymer foam formulations can lead to formation of stronger high-quality foams in porous media, which improves the sweep efficiency and increases the oil recovery. In third chapter, large amounts of surfactant coacervation work were focused on complex coacervation, such as mixture of surfactant and polymer, or mixture of different species of surfactants, seldom on the simple coacervation of single conventional surfactant in aqueous phase. This study aims to investigate evolution of dioctyl sulfosuccinate (AOT) /sodium chloride coavervation in aqueous solution associated with change in counterion binding degree. In this work, coacervation phase boundary of AOT in the presence of sodium chloride was obtained by spectrophotometer in terms of turbidity measurement. The activity of counterion was measured by sodium ion electrode probe. Electro kinetic parameters such as hydrodynamic aggregate size were investigated by dynamic lighting scattering (DLS). A monotonic decreasing AOT coacervate boundary was observed with increase in NaCl concentration. The degree of counterion binding, calculated by modified Corrin-Harkins equations, revealed a 3-segment behavior of AOT in salt solution. Colloid size distribution was conducted with DLS. Counterion binding degree plays an important role in the formation of surfactant aggregates. A further study of binding degree facilitates to understand coacervation

Impact of Relativistic Effects on the Primordial Non-Gaussianity Signature in the Large-Scale Clustering of Quasars

Relativistic effects in clustering observations have been shown to introduce scale-dependent corrections to the galaxy over-density field on large scales, which may hamper the detection of primordial non-Gaussianity $f_\textrm{NL}$ through the scale-dependent halo bias. The amplitude of relativistic corrections depends not only on the cosmological background expansion, but also on the redshift evolution and sensitivity to the luminosity threshold of the tracer population being examined, as parametrised by the evolution bias $b_\textrm{e}$ and magnification bias $s$. In this work, we propagate luminosity function measurements from the extended Baryon Oscillation Spectroscopic Survey (eBOSS) to $b_\textrm{e}$ and $s$ for the quasar (QSO) sample, and thereby derive constraints on relativistic corrections to its power spectrum multipoles. Although one could mitigate the impact on the $f_\textrm{NL}$ signature by adjusting the redshift range or the luminosity threshold of the tracer sample being considered, we suggest that, for future surveys probing large cosmic volumes, relativistic corrections should be forward modelled from the tracer luminosity function including its uncertainties. This will be important to quasar clustering measurements on scales $k \sim 10^{-3} h^{-1}$ Mpc in upcoming surveys such as the Dark Energy Spectroscopic Instrument (DESI), where relativistic corrections can overwhelm the expected $f_\textrm{NL}$ signature at low redshifts $z \lesssim 1$ and become comparable to $f_\textrm{NL} \simeq 1$ in the power spectrum quadrupole at redshifts $z \gtrsim 2.5$.Comment: 9 pages, 7 figures, 1 table, for submission to MNRA

Sample Complexity of Variance-reduced Distributionally Robust Q-learning

Dynamic decision making under distributional shifts is of fundamental interest in theory and applications of reinforcement learning: The distribution of the environment on which the data is collected can differ from that of the environment on which the model is deployed. This paper presents two novel model-free algorithms, namely the distributionally robust Q-learning and its variance-reduced counterpart, that can effectively learn a robust policy despite distributional shifts. These algorithms are designed to efficiently approximate the $q$-function of an infinite-horizon $\gamma$-discounted robust Markov decision process with Kullback-Leibler uncertainty set to an entry-wise $\epsilon$-degree of precision. Further, the variance-reduced distributionally robust Q-learning combines the synchronous Q-learning with variance-reduction techniques to enhance its performance. Consequently, we establish that it attains a minmax sample complexity upper bound of $\tilde O(|S||A|(1-\gamma)^{-4}\epsilon^{-2})$, where $S$ and $A$ denote the state and action spaces. This is the first complexity result that is independent of the uncertainty size $\delta$, thereby providing new complexity theoretic insights. Additionally, a series of numerical experiments confirm the theoretical findings and the efficiency of the algorithms in handling distributional shifts

Suboptimal Safety-Critical Control for Continuous Systems Using Prediction-Correction Online Optimization

This paper investigates the control barrier function (CBF) based safety-critical control for continuous nonlinear control affine systems using more efficient online algorithms by the time-varying optimization method. The idea of the algorithms is that when quadratic programming (QP) or other convex optimization algorithms needed in the CBF-based method is not computation affordable, the alternative suboptimal feasible solutions can be obtained more economically. By using the barrier-based interior point method, the constrained CBF-QP problems are transformed into unconstrained ones with suboptimal solutions tracked by two continuous descent-based algorithms. Considering the lag effect of tracking and exploiting the system information, the prediction method is added to the algorithms, which achieves exponential convergence to the time-varying suboptimal solutions. The convergence and robustness of the designed methods as well as the safety criteria of the algorithms are studied theoretically. The effectiveness is illustrated by simulations on the anti-swing and obstacle avoidance tasks