Improved Regret Bounds for Linear Adversarial MDPs via Linear Optimization

Learning Markov decision processes (MDP) in an adversarial environment has been a challenging problem. The problem becomes even more challenging with function approximation, since the underlying structure of the loss function and transition kernel are especially hard to estimate in a varying environment. In fact, the state-of-the-art results for linear adversarial MDP achieve a regret of $\tilde{O}(K^{6/7})$ ($K$ denotes the number of episodes), which admits a large room for improvement. In this paper, we investigate the problem with a new view, which reduces linear MDP into linear optimization by subtly setting the feature maps of the bandit arms of linear optimization. This new technique, under an exploratory assumption, yields an improved bound of $\tilde{O}(K^{4/5})$ for linear adversarial MDP without access to a transition simulator. The new view could be of independent interest for solving other MDP problems that possess a linear structure

Online Control with Adversarial Disturbance for Continuous-time Linear Systems

We study online control for continuous-time linear systems with finite sampling rates, where the objective is to design an online procedure that learns under non-stochastic noise and performs comparably to a fixed optimal linear controller. We present a novel two-level online algorithm, by integrating a higher-level learning strategy and a lower-level feedback control strategy. This method offers a practical and robust solution for online control, which achieves sublinear regret. Our work provides one of the first nonasymptotic results for controlling continuous-time linear systems a with finite number of interactions with the system

HOW TO UNDERSTAND POST-ACCEPTANCE INFORMATION SYSTEM USAGE BEHAVIORS: PESPECTIVE FROM IS SUCCESS MODEL

The impact of information systems (IS) on organizational performance has gained enormous attention from both academics and practitioners. However, it is the post-acceptance IS use that actually help fully realize the IS potential. We identified three types of IS usage behaviors -- routine use (RU), extended use (EU) and innovative use (IU), which can coexist in the post-acceptance stage and help with the work. Drawing on the IS success model, we proposed a research model with IS characteristics as external variables toward perceived usefulness (PU) and satisfaction to explain RU, EU and IU in respective. The relationships among three dimensions of IS characteristics -- information quality, system quality and service quality were discussed further. As RU, EU and IU reflect various extent of IS use, we suggested that they are linked. Then the model was tested by a survey of 240 ERP system users. The results provided evidence that information quality and service quality influence PU and user satisfaction via system quality, and IS success model was a good basis for understanding RU, EU and IU. We also found that RU had a positive impact directly on EU but indirectly on IU via EU. This study helps bridge the gap between IS characteristics and prediction of different types of post-acceptance IS usage behaviors

Reconstructing Three-decade Global Fine-Grained Nighttime Light Observations by a New Super-Resolution Framework

Satellite-collected nighttime light provides a unique perspective on human activities, including urbanization, population growth, and epidemics. Yet, long-term and fine-grained nighttime light observations are lacking, leaving the analysis and applications of decades of light changes in urban facilities undeveloped. To fill this gap, we developed an innovative framework and used it to design a new super-resolution model that reconstructs low-resolution nighttime light data into high resolution. The validation of one billion data points shows that the correlation coefficient of our model at the global scale reaches 0.873, which is significantly higher than that of other existing models (maximum = 0.713). Our model also outperforms existing models at the national and urban scales. Furthermore, through an inspection of airports and roads, only our model's image details can reveal the historical development of these facilities. We provide the long-term and fine-grained nighttime light observations to promote research on human activities. The dataset is available at \url{https://doi.org/10.5281/zenodo.7859205}

Learning Adversarial Low-rank Markov Decision Processes with Unknown Transition and Full-information Feedback

In this work, we study the low-rank MDPs with adversarially changed losses in the full-information feedback setting. In particular, the unknown transition probability kernel admits a low-rank matrix decomposition \citep{REPUCB22}, and the loss functions may change adversarially but are revealed to the learner at the end of each episode. We propose a policy optimization-based algorithm POLO, and we prove that it attains the $\widetilde{O}(K^{\frac{5}{6}}A^{\frac{1}{2}}d\ln(1+M)/(1-\gamma)^2)$ regret guarantee, where $d$ is rank of the transition kernel (and hence the dimension of the unknown representations), $A$ is the cardinality of the action space, $M$ is the cardinality of the model class, and $\gamma$ is the discounted factor. Notably, our algorithm is oracle-efficient and has a regret guarantee with no dependence on the size of potentially arbitrarily large state space. Furthermore, we also prove an $\Omega(\frac{\gamma^2}{1-\gamma} \sqrt{d A K})$ regret lower bound for this problem, showing that low-rank MDPs are statistically more difficult to learn than linear MDPs in the regret minimization setting. To the best of our knowledge, we present the first algorithm that interleaves representation learning, exploration, and exploitation to achieve the sublinear regret guarantee for RL with nonlinear function approximation and adversarial losses

Estimating Warehouse Rental Price using Machine Learning Techniques

Boosted by the growing logistics industry and digital transformation, the sharing warehouse market is undergoing a rapid development. Both supply and demand sides in the warehouse rental business are faced with market perturbations brought by unprecedented peer competitions and information transparency. A key question faced by the participants is how to price warehouses in the open market. To understand the pricing mechanism, we built a real world warehouse dataset using data collected from the classified advertisements websites. Based on the dataset, we applied machine learning techniques to relate warehouse price with its relevant features, such as warehouse size, location and nearby real estate price. Four candidate models are used here: Linear Regression, Regression Tree, Random Forest Regression and Gradient Boosting Regression Trees. The case study in the Beijing area shows that warehouse rent is closely related to its location and land price. Models considering multiple factors have better skill in estimating warehouse rent, compared to singlefactor estimation. Additionally, tree models have better performance than the linear model, with the best model (Random Forest) achieving correlation coefficient of 0.57 in the test set. Deeper investigation of feature importance illustrates that distance from the city center plays the most important role in determining warehouse price in Beijing, followed by nearby real estate price and warehouse size

RESEARCH OF REGION WATER SECURITY EVALUATION BY SET PAIR ANALYSIS

Abstract The set pair analysis theory was applied in the water security evaluation due to the existing uncertain relationship in the system. Connection degree is used to describe this sort of uncertain problem. In the process, the indexes information of the region water security was reflected and the difficulty of weight allocation was avoided. The water security of Shandong province was evaluated by this method. The result was compared with what is got by fuzzy comprehensive evaluation and matter-element analysis. It proves that SPA analysis is simple and practical and can be a useful evaluating tool for region water security

Global oceanic mesoscale eddies trajectories prediction with knowledge-fused neural network

Efficient eddy trajectory prediction driven by multiinformation fusion can facilitate the scientific research of oceanography, while the complicated dynamics mechanism makes this issue challenging. Benefiting from ocean observing technology, the eddy trajectory dataset can be qualified for data-intensive research paradigms. In this article, the dynamics mechanism is used to inspire the design idea of the eddy trajectory prediction neural network (termed EddyTPNet) and is also transformed into prior knowledge to guide the learning process. This study is among the first to implement eddy trajectory prediction with physics informed neural network. First, an in-depth analysis of the kinematic characteristics indicates that the longitude and latitude of the trajectory should be decoupled; second, the directional dispersion prior knowledge of global eddy propagation is embedded into the decoder of the EddyTPNet to improve the performance; finally, EddyTPNet predicts global eddy trajectories through pretraining and adapts to complex local regions via model transfer. Extensive experimental results demonstrate that EddyTPNet can reliably forecast the motion of eddies for the next seven days, ensuring a low daily mean geodetic error. This exploratory study provides valuable insights into solving the prediction problem of ocean phenomena by using knowledge-based time-series neural networks

Gene cloning, expression, and characterization of two endo-xylanases from Bacillus velezensis and Streptomyces rochei, and their application in xylooligosaccharide production

Endo-xylanase hydrolyzing xylan in cellulosic residues releasing xylobiose as the major product at neutral pH are desirable in the substitute sweeteners industry. In this study, two endo-xylanases were obtained from Streptomyces rochei and Bacillus velezensis. SrocXyn10 showed the highest identity of 77.22%, with a reported endo-xylanase. The optimum reaction temperature and pH of rSrocXyn10-Ec were pH 7.0 and 60°C, with remarkable stability at 45°C or pHs ranging from 4.5 to 11.0. rBvelXyn11-Ec was most active at pH 6.0 and 50°C, and was stable at 35°C or pH 3.5 to 10.5. Both rSrocXyn10-Ec and rBvelXyn11-Ec showed specific enzyme activities on wheat arabinoxylan (685.83 ± 13.82 and 2809.89 ± 21.26 U/mg, respectively), with no enzyme activity on non-xylan substrates. The Vmax of rSrocXyn10-Ec and rBvelXyn11-Ec were 467.86 U mg−1 and 3067.68 U mg−1, respectively. The determined Km values of rSrocXyn10-Ec and rBvelXyn11-Ec were 3.08 g L−1 and 1.45 g L−1, respectively. The predominant product of the hydrolysis of alkaline extracts from bagasse, corncob, and bamboo by rSrocXyn10-Ec and rBvelXyn11-Ec were xylooligosaccharides. Interestingly, the xylobiose content in hydrolysates by rSrocXyn10-Ec was approximately 80%, which is higher than most reported endo-xylanases. rSrocXyn10-Ec and rBvelXyn11-Ec could be excellent candidates to produce xylooligosaccharides at neutral/near-neutral pHs. rSrocXyn10-Ec also has potential value in the production of xylobiose as a substitute sweetener