Horizon-Free and Variance-Dependent Reinforcement Learning for Latent Markov Decision Processes

We study regret minimization for reinforcement learning (RL) in Latent Markov Decision Processes (LMDPs) with context in hindsight. We design a novel model-based algorithmic framework which can be instantiated with both a model-optimistic and a value-optimistic solver. We prove an $\widetilde{O}\left(\sqrt{M \Gamma S A K}\right)$ regret bound where $M$ is the number of contexts, $S$ is the number of states, $A$ is the number of actions, $K$ is the number of episodes, and $\Gamma \le S$ is the maximum transition degree of any state-action pair. The regret bound only scales logarithmically with the planning horizon, thus yielding the first (nearly) horizon-free regret bound for LMDP. Key in our proof is an analysis of the total variance of alpha vectors, which is carefully bounded by a recursion-based technique. We complement our positive result with a novel $\Omega\left(\sqrt{M S A K}\right)$ regret lower bound with $\Gamma = 2$, which shows our upper bound minimax optimal when $\Gamma$ is a constant. Our lower bound relies on new constructions of hard instances and an argument based on the symmetrization technique from theoretical computer science, both of which are technically different from existing lower bound proof for MDPs, and thus can be of independent interest.Comment: ICML 202

SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, and More

The emergence of large models, also known as foundation models, has brought significant advancements to AI research. One such model is Segment Anything (SAM), which is designed for image segmentation tasks. However, as with other foundation models, our experimental findings suggest that SAM may fail or perform poorly in certain segmentation tasks, such as shadow detection and camouflaged object detection (concealed object detection). This study first paves the way for applying the large pre-trained image segmentation model SAM to these downstream tasks, even in situations where SAM performs poorly. Rather than fine-tuning the SAM network, we propose \textbf{SAM-Adapter}, which incorporates domain-specific information or visual prompts into the segmentation network by using simple yet effective adapters. Our extensive experiments show that SAM-Adapter can significantly elevate the performance of SAM in challenging tasks and we can even outperform task-specific network models and achieve state-of-the-art performance in the task we tested: camouflaged object detection and shadow detection. We believe our work opens up opportunities for utilizing SAM in downstream tasks, with potential applications in various fields, including medical image processing, agriculture, remote sensing, and more

Isomer-Resolved Mobility-Mass Analysis of alpha-Pinene Ozonolysis Products

Highly oxygenated organic molecules (HOMs) are important sources of atmospheric aerosols. Resolving the molecular-level formation mechanisms of these HOMs from freshly emitted hydrocarbons improves the understanding of aerosol properties and their influence on the climate. In this study, we measure the electrical mobility and mass-to-charge ratio of alpha-pinene oxidation products using a secondary electrospray-differential mobility analyzer-mass spectrometer (SESI-DMA-MS). The mass-mobility spectrum of the oxidation products is measured with seven different reagent ions generated by the electrospray. We analyzed the mobility-mass spectra of the oxidation products C9-10H14-18O2-6. Our results show that acetate and chloride yield the highest charging efficiencies. Analysis of the mobility spectra suggests that the clusters have 1-5 isomeric structures (i.e., ion-molecule cluster structures with distinct mobilities), and the number is affected by the reagent ion. Most of the isomers are likely cluster isomers originating from binding of the reagent ion to different sites of the molecule. By comparing the number of observed isomers and measured mobilities and collision cross sections between standard pinanediol and pinonic acid to the values observed for C10H18O2 and C10H16O3 produced from oxidation of alpha-pinene, we confirm that pinanediol and pinonic acid are the only isomers for these elemental compositions in our experimental conditions. Our study shows that the SESI-DMA-MS produces new information from the first steps of oxidation of alpha-pinene.Peer reviewe

An indicator for sulfuric acid–amine nucleation in atmospheric environments

New particle formation (NPF) occurs frequently in various atmospheric environments and it is a major source of ultrafine particles. This study proposes an indicator, I, for the occurrence of NPF in the atmosphere based on the mechanism of H2SO4-amine nucleation. It characterizes the synergistic effects of the governing factors for H2SO4-amine nucleation, including H2SO4 concentration, amine concentrations, the stability of H2SO4-amine clusters, and aerosol surface area concentration. Long-term measurements in urban Beijing were used to validate this indicator. Good consistency was found between this indicator and the occurrence of NPF. NPF was usually observed with I > 1 for typical conditions in urban Beijing. The derivation and expressions of I also indicate a good positive association between the H2SO4 dimer concentration and NPF, as also verified by measurements. I was shown to be also applicable in urban Shanghai. Copyright (c) 2021 American Association for Aerosol ResearchPeer reviewe

Impacts of coagulation on the appearance time method for new particle growth rate evaluation and their corrections

The growth rate of atmospheric new particles is a key parameter that determines their survival probability of becoming cloud condensation nuclei and hence their impact on the climate. There have been several methods to estimate the new particle growth rate. However, due to the impact of coagulation and measurement uncertainties, it is still challenging to estimate the initial growth rate of new particles, especially in polluted environments with high background aerosol concentrations. In this study, we explore the influences of coagulation on the appearance time method to estimate the growth rate of sub-3 nm particles. The principle of the appearance time method and the impacts of coagulation on the retrieved growth rate are clarified via derivations. New formulae in both discrete and continuous spaces are proposed to correct for the impacts of coagulation. Aerosol dynamic models are used to test the new formulae. New particle formation in urban Beijing is used to illustrate the importance of considering the impacts of coagulation on the sub-3 nm particle growth rate and its calculation. We show that the conventional appearance time method needs to be corrected when the impacts of coagulation sink, coagulation source, and particle coagulation growth are non-negligible compared to the condensation growth. Under the simulation conditions with a constant concentration of non-volatile vapors, the corrected growth rate agrees with the theoretical growth rates. However, the uncorrected parameters, e.g., vapor evaporation and the variation in vapor concentration, may impact the growth rate obtained with the appearance time method. Under the simulation conditions with a varying vapor concentration, the average bias in the corrected 1.5-3 nm particle growth rate ranges from 6 %-44 %, and the maximum bias in the size-dependent growth rate is 150 %. During the test new particle formation event in urban Beijing, the corrected condensation growth rate of sub-3 nm particles was in accordance with the growth rate contributed by sulfuric acid condensation, whereas the conventional appearance time method overestimated the condensation growth rate of 1.5 nm particles by 80 %.Peer reviewe

Measurement report : Size distributions of urban aerosols down to 1 nm from long-term measurements

The size distributions of urban atmospheric aerosols convey important information on their origins and impacts. Their long-term characteristics, especially for sub-3 nm particles, are still limited. In this study, we examined the characteristics of atmospheric aerosol size distributions down to similar to 1 nm based on 4-year measurements in urban Beijing. Using cluster analysis, three typical types of number size distributions were identified, i.e., daytime new particle formation (NPF) type, daytime non-NPF type, and nighttime type. Combining a power law distribution and multiple lognormal distributions can well represent the sharp concentration decrease of sub3 nm particles with increasing size and the modal characteristics for those above 3 nm in the submicron size range. The daytime NPF type exhibits high concentrations of sub-3 nm aerosols together with other three modes. However, both the daytime non-NPF type and the nighttime type have a low abundance of sub-3 nm aerosol particles together with only two distinct modes. In urban Beijing, the concentration of H2SO4 monomer during the daytime with NPF is similar to that during the daytime without NPF, while significantly higher than that during the nighttime. The concentration of atmospheric sub-3 nm particles on NPF days has a strong seasonality while their seasonality on non-NPF days is less pronounced. In addition to NPF as the most important source, we show that vehicles can emit sub-3 nm particles as well, although their influence on the measured aerosol population strongly depends on the distance from the road.Peer reviewe

Assessment of particle size magnifier inversion methods to obtain the particle size distribution from atmospheric measurements

Accurate measurements of the size distribution of atmospheric aerosol nanoparticles are essential to build an understanding of new particle formation and growth. This is particularly crucial at the sub-3 nm range due to the growth of newly formed nanoparticles. The challenge in recovering the size distribution is due its complexity and the fact that not many instruments currently measure at this size range. In this study, we used the particle size magnifier (PSM) to measure atmospheric aerosols. Each day was classified into one of the following three event types: a new particle formation (NPF) event, a non-event or a haze event. We then compared four inversion methods (stepwise, kernel, Hagen-Alofs and expectation-maximization) to determine their feasibility to recover the particle size distribution. In addition, we proposed a method to pretreat the measured data, and we introduced a simple test to estimate the efficacy of the inversion itself. Results showed that all four methods inverted NPF events well; however, the stepwise and kernel methods fared poorly when inverting non-events or haze events. This was due to their algorithm and the fact that, when encountering noisy data (e.g. air mass fluctuations or low sub-3 nm particle concentrations) and under the influence of larger particles, these methods overestimated the size distribution and reported artificial particles during inversion. Therefore, using a statistical hypothesis test to discard noisy scans prior to inversion is an important first step toward achieving a good size distribution. After inversion, it is ideal to compare the integrated concentration to the raw estimate (i.e. the concentration difference at the lowest supersaturation and the highest supersaturation) to ascertain whether the inversion itself is sound. Finally, based on the analysis of the inversion methods, we provide procedures and codes related to the PSM data inversion.Peer reviewe

Responses of gaseous sulfuric acid and particulate sulfate to reduced SO2 concentration : A perspective from long-term measurements in Beijing

SO2 concentration decreased rapidly in recent years in China due to the implementation of strict control policies by the government. Particulate sulfate (pSO(4)(2-)) and gaseous H2SO4 (SA) are two major products of SO2 and they play important roles in the haze formation and new particle formation (NPF), respectively. We examined the change in pSO(4)(2-) and SA concentrations in response to reduced SO2 concentration using long-term measurement data in Beijing. Simulations from the Community Multiscale Air Quality model with a 2-D Volatility Basis Set (CMAQ/2D-VBS) were used for comparison. From 2013 to 2018, SO2 concentration in Beijing decreased by similar to 81% (from 9.1 ppb to 1.7 ppb). pSO(4)(2-) concentration in submicrometer particles decreased by similar to 60% from 2012-2013 (monthly average of similar to 10 mu g.m(-3)) to 2018-2019 (monthly average of similar to 4 mu g.m(-3)). Accordingly, the fraction of pSO(4)(2-) in these particles decreased from20-30% to b10%. Increased sulfur oxidation ratio was observed both in the measurements and the CMAQ/2D-VBS simulations. Despite the reduction in SO2 concentration, there was no obvious decrease in SA concentration based on data from several measuring periods from 2008 to 2019. This was supported by the increased SA:SO2 ratio with reduced SO2 concentration and condensation sink. NPF frequency in Beijing between 2004 and 2019 remains relatively constant. This constant NPF frequency is consistent with the relatively stable SA concentration in Beijing, while different from some other cities where NPF frequency was reported to decrease with decreased SO2 concentrations. (C) 2020 Elsevier B.V. All rights reserved.Peer reviewe