A Unified View of TD Algorithms; Introducing Full-Gradient TD and Equi-Gradient Descent TD

International audienceThis paper addresses the issue of policy evaluation in Markov Decision Processes, using linear function approximation. It provides a unified view of algorithms such as TD(lambda), LSTD(lambda), iLSTD, residual-gradient TD. It is asserted that they all consist in minimizing a gradient function and differ by the form of this function and their means of minimizing it. Two new schemes are introduced in that framework: Full-gradient TD which uses a generalization of the principle introduced in iLSTD, and EGD TD, which reduces the gradient by successive equi-gradient descents. These three algorithms form a new intermediate family with the interesting property of making much better use of the samples than TD while keeping a gradient descent scheme, which is useful for complexity issues and optimistic policy iteration

Relation between fractional flow models and fractal or long-range 2-D permeability fields

International audienceFractional flow models introduced by Barker (1988) have been increasingly popular as means of interpreting nonclassical drawdown curves obtained from well tests. Fractional flow models are intrinsically isotropic scaling models depending to first order on two exponents n and dw expressing the dimension of the structure available to flow and the flow slowdown, respectively. We study the fractional flow induced either by geometrically scaling structures such as Sierpinski- and percolation-like fractal media or by hydraulically scaling media such as long-range continuous correlated media. First, percolation and Sierpinski structures have two well-separated dw values in the range [2.6, 3] and [1.9, 2.5], respectively. The bottlenecks, characteristic of percolation, induce a more anomalous transport (larger dw values) than the impervious zones present at all scales of Sierpinskis. Second, the realization-based values of n and dw depend both on global and on local characteristics like the fractal dimension and the permeability around the well, respectively. Finally, solving the inverse problem on anomalous transient well test responses consists in comparing the (n, dw) realization-based values with field data. Indeed, well tests performed from a unique pumping well must be taken as realization-based results. For the site of Ploemeur (Brittany, France), from which n and dw have been previously determined (Le Borgne et al., 2004), the only consistent model is given by the continuous multifractals. However, the values obtained from continuous multifractals cover most of the (n, dw) plane, and realization-based results are not selective in terms of model. So this should be replaced by the comparison of (n, dw) values averaged over different pumping well locations, which however requires a significantly larger quantity of field tests

Sparse Temporal Difference Learning using LASSO

International audienceWe consider the problem of on-line value function estimation in reinforcement learning. We concentrate on the function approximator to use. To try to break the curse of dimensionality, we focus on non parametric function approximators. We propose to fit the use of kernels into the temporal difference algorithms by using regression via the LASSO. We introduce the equi-gradient descent algorithm (EGD) which is a direct adaptation of the one recently introduced in the LARS algorithm family for solving the LASSO. We advocate our choice of the EGD as a judicious algorithm for these tasks. We present the EGD algorithm in details as well as some experimental results. We insist on the qualities of the EGD for reinforcement learning

Brittle-ductile coupling : Role of ductile viscosity on brittle fracturing

International audienceLocalized or distributed deformations in continental lithosphere are supposed to be triggered by rheological contrasts, and particularly by brittle-ductile coupling. A plane-strain 2D finite-element model is used to investigate the mechanical role of a ductile layer in defining the transition from localized to distributed fracturing in a brittle layer. The coupling is performed through the shortening of a Von Mises elasto-visco-plastic layer rimed by two ductile layers. By increasing the viscosity of the ductile layers by only one order of magnitude, the fracturing mode in the brittle layer evolves from localized (few faults) to distributed (numerous faults), defining a viscosity-dependent fracturing mode. This brittle-ductile coupling can be explained by the viscous resistance of the ductile layer to fault motion, which limits the maximum displacement rate along any fault connected to the ductile interface. An increase of the viscosity will thus make necessary new faults nucleation to accommodate the boundary shortening rate

A model of fracture nucleation, growth and arrest, and consequences for fracture density and scaling

International audienceIn order to improve discrete fracture network (DFN) models, which are increasingly required into groundwater and rock mechanics applications, we propose a new DFN modeling based on the evolution of fracture network formation--nucleation, growth, and arrest--with simplified mechanical rules. The central idea of the model relies on the mechanical role played by large fractures in stopping the growth of smaller ones. The modeling framework combines, in a time-wise approach, fracture nucleation, growth, and arrest. It yields two main regimes. Below a certain critical scale, the density distribution of fracture sizes is a power law with a scaling exponent directly derived from the growth law and nuclei properties; above the critical scale, a quasi-universal self-similar regime establishes with a self-similar scaling. The density term of the dense regime is related to the details of arrest rule and to the orientation distribution of the fractures. The DFN model, so defined, is fully consistent with field cases former studied. Unlike more usual stochastic DFN models, ours is based on a simplified description of fracture interactions, which eventually reproduces the multiscale self-similar fracture size distribution often observed and reported in the literature. The model is a potential significant step forward for further applications to groundwater flow and rock mechanical issues

Right-lateral shear along the Northwest Pacific Margin and the India-Eurasia Collision

International audienceRight-lateral shear along the eastern margin of Asia, from the Eocene to the Present has led to the opening of pull-apart basins, intracontinental such as the Bohai basin, or oceanic such as the Japan Sea. We suggest in this paper that this right-lateral shear is a consequence of indentation of Asia by India. As in small-scale analog experiments, we conclude that antithetic wrench faults accommodate the counterclockwise rotation of large domino blocks between two major left-lateral shear zones (Tien Shan-Baikal-Stanovoy for the northern one, and Qin Ling for the southern one). We discuss the compatibility of this mechanism, which involves a rather small amount of extrusion, with the fast eastward expulsion described for southeast Asia. We re-emphasize the role played in the opening of marginal basins by the Pacific subduction as a free boundary to the east

Association between bacterial vaginosis and cervical intraepithelial neoplasia: systematic review and meta-analysis

Objective: Bacterial vaginosis (BV), the most common vaginal disorder among women of reproductive age, has been suggested as co-factor in the development of cervical cancer. Previous studies examining the relationship between BV and cervical intra-epithelial neoplasia (CIN) provided inconsistent and conflicting results. The aim of this study is to clarify the association between these two conditions. Methods: A systematic review and meta-analysis were conducted to summarize published literature on the association between BV and cervical pre-cancerous lesions. An extensive search of electronic databases Medline (Pubmed) and Web of Science was performed. The key words 'bacterial vaginosis' and 'bacterial infections and vaginitis' were used in combination with 'cervical intraepithelial neoplasia', 'squamous intraepithelial lesions', 'cervical lesions', 'cervical dysplasia', and 'cervical screening'. Eligible studies required a clear description of diagnostic methods used for detecting both BV and cervical precancerous lesions. Publications were included if they either reported odds ratios (OR) and corresponding 95% confidence intervals (CI) representing the magnitude of association between these two conditions, or presented data that allowed calculation of the OR. Results: Out of 329 articles, 17 cross-sectional and 2 incidence studies were selected. In addition, two studies conducted in The Netherlands, using the national KOPAC system, were retained. After testing for heterogeneity and publication bias, meta-analysis and meta-regression were performed, using a random effects model. Although heterogeneity among studies was high (chi(2) = 164.7, p < 0.01, I-2 = 88.5), a positive association between BV and cervical pre-cancerous lesions was found, with an overall estimated odds ratio of 1.51 (95% CI, 1.24-1.83). Meta-regression analysis could not detect a significant difference between studies based on BV diagnosis, CIN diagnosis or study population. Conclusions: Although most studies were cross-sectional and heterogeneity was high, this meta-analysis confirms a connection between BV and CIN

Fusion de capteurs potentiellement défaillants par filtrage particulaire

Cet article s'intéresse à l'estimation bayésienne d'un vecteur d'état à l'aide de données multicapteur obtenues séquentiellement, en considérant que les capteurs sont potentiellement défaillants. Un état augmenté avec les variables indicatrices de validité et les coefficients de fiabilité de chaque capteur est estimé par un algorithme de Monte Carlo séquentiel (aussi appelé filtre particulaire). Une attention particulière est portée au choix des fonctions d'importance. Un exemple est fourni montrant l'amélioration de l'estimation en présence de capteurs défaillants par rapport à un filtre particulaire classique

Is the Dupuit assumption suitable for predicting the groundwater seepage area in hillslopes?

International audienceMany physically based hydrological/hydrogeological models used for predicting groundwater seepage areas, including topography-based index models such as TOPMODEL, rely on the Dupuit assumption. To ensure the sound use of these simplified models, knowledge of the conditions under which they provide a reasonable approximation is critical. In this study, a Dupuit solution for the seepage length in hillslope cross sections is tested against a full-depth solution of saturated groundwater flow. In homogeneous hillslopes with horizontal impervious base and constant-slope topography, the comparison reveals that the validity of the Dupuit solution depends not only on the ratio of depth to hillslope length d/L (as might be expected), but also on the ratio of hydraulic conductivity to recharge K/R and on the topographic slope s. The validity of the Dupuit solution is shown to be in fact a unique function of another ratio, the ratio of depth to seepage length d/LS. For d/LS0.2, it increases dramatically. In practice, this criterion can be used to test the validity of Dupuit solutions. When d/LS increases beyond that cutoff, the ratio of seepage length to hillslope length LS/L given by the full-depth solution tends toward a nonzero asymptotic value. This asymptotic value is shown to be controlled by (and in many cases equal to) the parameter R/(sK). Generalization of the findings to cases featuring heterogeneity, nonhorizontal impervious base and variable-slope topography is discussed

Connectivity-consistent mapping method for 2-D discrete fracture networks

International audienceWe present a new flow computation method in 2-D discrete fracture networks (DFN) intermediary between the classical DFN flow simulation method and the projection onto continuous grids. The method divides the simulation complexity by solving for flows successively at a local mesh scale and at the global domain scale. At the local mesh scale, flows are determined by classical DFN flow simulations and approximated by an equivalent hydraulic matrix (EHM) relating heads and flow rates discretized on the mesh borders. Assembling the equivalent hydraulic matrices provides for a domain-scale discretization of the flow equation. The equivalent hydraulic matrices transfer the connectivity and flow structure complexities from the local mesh scale to the domain scale. Compared to existing geometrical mapping or equivalent tensor methods, the EHM method broadens the simulation range of flow to all types of 2-D fracture networks both below and above the representative elementary volume (REV). Additional computation linked to the derivation of the local mesh-scale equivalent hydraulic matrices increases the accuracy and reliability of the method. Compared to DFN methods, the EHM method first provides a simpler domain-scale alternative permeability model. Second, it enhances the simulation capacities to larger fracture networks where flow discretization on the DFN structure yields system sizes too large to be solved using the most advanced multigrid and multifrontal methods. We show that the EHM method continuously moves from the DFN method to the tensor representation as a function of the local mesh-scale discretization. The balance between accuracy and model simplification can be optimally controlled by adjusting the domain-scale and local mesh-scale discretizations