Crystal engineering of mixed-ligand metal-organic frameworks based on simple carboxylate and bipyridyl ligands

Over the last few decades research in supramolecular chemistry and crystal engineering have seen an exponential growth. The synthesis of metal-organic frameworks (MOFs) has attracted much interest worldwide due to the possibility of obtaining a large variety of structures with a wide range of applications in fields pertaining to storage, separation and catalysis. This work focuses on the crystal engineering of MOFs based on mixed ligands which may ultimately be used in the gas storage of pollutants, greenhouse gases or fuel. Two novel 2D mixed-ligand MOFs, both based on manganese, 4,4’-bipyridine and 1,3,5- benzenetricarboxylic acid, have been prepared and fully characterized. The employment of dimethylformamide or dimethylacetamide, as the solvent, results in two isostructural MOFs. Another novel MOF, similar in structure to the previous two, with 5-nitroisophthalic acid instead of 1,3,5-benzenetricarboxylic acid has been also prepared and characterized. This MOF has the same metal and ligand combination as, and is largely isostructural to, a literature example, but differs in method of preparation and solvent content. These Mnbased MOFs have potential voids in the structure making them candidates for gas sorption experiments. A novel 2D mixed-ligand MOF based on cobalt, 4,4’-bipyridine and 5-nitroisophthalic acid has been synthesized and fully characterized. Its structure is the same of another MOF, based on manganese, present in this work and like its manganese analogue it exhibits potential void spaces in the framework that make it a candidate for gas sorption experiments. Finally, a novel 2D MOF based on 1,3,5-benzenetricarboxylic acid and cadmium bromide has been synthesized and fully characterized. Dehydration and rehydration studies performed by combining powder X-ray diffraction with thermogravimetric analysis show that it can lose coordinated water, that comes from the reaction solvent, upon heating, and reabsorb water from the atmosphere, ultimately regaining its original structure. All MOFs were synthesized via the solvothermal method and characterized with X-ray diffraction (single crystal and powder) and thermal analyses (hot stage microscopy, differential scanning calorimetry and thermogravimetric analysis)

An Intrinsically-Motivated Approach for Learning Highly Exploring and Fast Mixing Policies

What is a good exploration strategy for an agent that interacts with an environment in the absence of external rewards? Ideally, we would like to get a policy driving towards a uniform state-action visitation (highly exploring) in a minimum number of steps (fast mixing), in order to ease efficient learning of any goal-conditioned policy later on. Unfortunately, it is remarkably arduous to directly learn an optimal policy of this nature. In this paper, we propose a novel surrogate objective for learning highly exploring and fast mixing policies, which focuses on maximizing a lower bound to the entropy of the steady-state distribution induced by the policy. In particular, we introduce three novel lower bounds, that lead to as many optimization problems, that tradeoff the theoretical guarantees with computational complexity. Then, we present a model-based reinforcement learning algorithm, IDE$^{3}$AL, to learn an optimal policy according to the introduced objective. Finally, we provide an empirical evaluation of this algorithm on a set of hard-exploration tasks.Comment: In 34th AAAI Conference on Artificial Intelligence (AAAI 2020

A Policy Gradient Method for Task-Agnostic Exploration

In a reward-free environment, what is a suitable intrinsic objective for an agent to pursue so that it can learn an optimal task-agnostic exploration policy? In this paper, we argue that the entropy of the state distribution induced by limited-horizon trajectories is a sensible target. Especially, we present a novel and practical policy-search algorithm, Maximum Entropy POLicy optimization (MEPOL), to learn a policy that maximizes a non-parametric, $k$-nearest neighbors estimate of the state distribution entropy. In contrast to known methods, MEPOL is completely model-free as it requires neither to estimate the state distribution of any policy nor to model transition dynamics. Then, we empirically show that MEPOL allows learning a maximum-entropy exploration policy in high-dimensional, continuous-control domains, and how this policy facilitates learning a variety of meaningful reward-based tasks downstream

A Tale of Sampling and Estimation in Discounted Reinforcement Learning

The most relevant problems in discounted reinforcement learning involve estimating the mean of a function under the stationary distribution of a Markov reward process, such as the expected return in policy evaluation, or the policy gradient in policy optimization. In practice, these estimates are produced through a finite-horizon episodic sampling, which neglects the mixing properties of the Markov process. It is mostly unclear how this mismatch between the practical and the ideal setting affects the estimation, and the literature lacks a formal study on the pitfalls of episodic sampling, and how to do it optimally. In this paper, we present a minimax lower bound on the discounted mean estimation problem that explicitly connects the estimation error with the mixing properties of the Markov process and the discount factor. Then, we provide a statistical analysis on a set of notable estimators and the corresponding sampling procedures, which includes the finite-horizon estimators often used in practice. Crucially, we show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties w.r.t. the alternative estimators, as it matches the lower bound without requiring a careful tuning of the episode horizon.Comment: AISTATS 202

A Tale of Sampling and Estimation in Discounted Reinforcement Learning

The most relevant problems in discounted reinforcement learning involve estimating the mean of a function under the stationary distribution of a Markov reward process, such as the expected return in policy evaluation, or the policy gradient in policy optimization. In practice, these estimates are produced through a finite-horizon episodic sampling, which neglects the mixing properties of the Markov process. It is mostly unclear how this mismatch between the practical and the ideal setting affects the estimation, and the literature lacks a formal study on the pitfalls of episodic sampling, and how to do it optimally. In this paper, we present a minimax lower bound on the discounted mean estimation problem that explicitly connects the estimation error with the mixing properties of the Markov process and the discount factor. Then, we provide a statistical analysis on a set of notable estimators and the corresponding sampling procedures, which includes the finite-horizon estimators often used in practice. Crucially, we show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties w.r.t. the alternative estimators, as it matches the lower bound without requiring a careful tuning of the episode horizon

Configurable Markov Decision Processes

In many real-world problems, there is the possibility to configure, to a limited extent, some environmental parameters to improve the performance of a learning agent. In this paper, we propose a novel framework, Configurable Markov Decision Processes (Conf-MDPs), to model this new type of interaction with the environment. Furthermore, we provide a new learning algorithm, Safe Policy-Model Iteration (SPMI), to jointly and adaptively optimize the policy and the environment configuration. After having introduced our approach and derived some theoretical results, we present the experimental evaluation in two explicative problems to show the benefits of the environment configurability on the performance of the learned policy

Synthesis and characterization of a 2-periodic cadmium-based metal-organic framework: A study on reversible water adsorption

A previously-reported cadmium-based two-periodic metal-organic framework [Cd1.5(BTC)(H2O)4.5]n⋅nH2O (CP1) has been re-synthesized, where H3BTC ¼ 1,3,5-benzenetricarboxylic acid. CP1 was characterized with single crystal X-ray diffraction (SCXRD), powder X-ray diffraction (PXRD) followed by various thermal analyses such as thermogravimetric analysis (TGA), hot stage microscopy (HSM) and differential scanning calorimetry (DSC). CP1 is composed of 2-periodic layers, which are interdigitated. Heating can effectively remove the uncoordinated and coordinated water molecules resulting in an amorphous product CP1′. The original framework can be regenerated by readsorption of water from the atmosphere, indicating that the dehydration is reversibl

Work-related allergies to storage mites in Parma (Italy) ham workers

OBJECTIVES: To investigate the role of storage mites in the development of allergic diseases among ham production workers, and to search for early alterations in lung function tests and early inflammation markers in exhaled air. Respiratory allergies due to storage mites have been reported in people with various occupations but, although such mites are unavoidable when curing ham, there are no published data concerning ham production workers. SETTING: Secondary care. DESIGN: Experimental cross-sectional study. PARTICIPANTS: 220 participants (110 ham production workers and 110 controls) were recruited. PRIMARY AND SECONDARY OUTCOME MEASURES: Workers answered a medical questionnaire, and underwent spirometry and fraction of exhaled nitric oxide at 50 mL/s (FeNO50) measurements. Those with allergic symptoms also underwent skin prick tests to determine their sensitisation to airborne allergens. A methacholine test was performed in symptomatic participants when spirometry was normal to assess airways hyper-responsiveness. RESULTS: Symptomatic storage mite sensitisation was observed in 16 workers (14.5%) (rhinoconjunctivitis in 15 (63%) and asthma in (4%)) and 2 controls (1.8%; p=0.001). Higher FeNO50 values in exposed symptomatic workers compared with healthy control participants (34.65±7.49 vs 13.29±4.29 ppb; p<0.001) suggested bronchial and nasal involvement, although their lung function parameters were normal. Regardless of exposure, a FeNO50 value of 22.5 ppb seems to be 100% sensitive and 99.4% specific in distinguishing allergic and non-allergic participants. Multivariate analysis of FeNO50 values in the symptomatic participants showed that they were positively influenced by IgE-mediated allergy (p=0.001) and reported symptom severity (p=0.041), and negatively by smoking status (p=0.049). CONCLUSIONS: Ham processing workers, as well as workers involved in any meat processing work that includes curing, should be informed about the occupational risk of sensitisation to mites

Configurable Markov Decision Processes

In many real-world problems, there is the possibility to configure, to a limited extent, some environmental parameters to improve the performance of a learning agent. In this paper, we propose a novel framework, Configurable Markov Decision Processes (Conf-MDPs), to model this new type of interaction with the environment. Furthermore, we provide a new learning algorithm, Safe Policy-Model Iteration (SPMI), to jointly and adaptively optimize the policy and the environment configuration. After having introduced our approach and derived some theoretical results, we present the experimental evaluation in two explicative problems to show the benefits of the environment configurability on the performance of the learned policy

Expression Levels of Some Antioxidant and Epidermal Growth Factor Receptor Genes in Patients with Early-Stage Non-Small Cell Lung Cancer

This study was aimed at: (i) investigating the expression profiles of some antioxidant and epidermal growth factor receptor genes in cancerous and unaffected tissues of patients undergoing lung resection for non-small cell lung cancer (NSCLC) (cross-sectional phase), (ii) evaluating if gene expression levels at the time of surgery may be associated to patients' survival (prospective phase). Antioxidant genes included heme oxygenase 1 (HO-1), superoxide dismutase-1 (SOD-1), and -2 (SOD-2), whereas epidermal growth factor receptor genes consisted of epidermal growth factor receptor (EGFR) and v-erb-b2 erythroblastic leukaemia viral oncogene homolog 2 (HER-2). Twenty-eight couples of lung biopsies were obtained and gene transcripts were quantified by Real Time RT-PCR. The average follow-up of patients lasted about 60 months. In the cancerous tissues, antioxidant genes were significantly hypo-expressed than in unaffected tissues. The HER-2 transcript levels prevailed in adenocarcinomas, whereas EGFR in squamocellular carcinomas. Patients overexpressing HER-2 in the cancerous tissues showed significantly lower 5-year survival than the others