Minimax Weight Learning for Absorbing MDPs

Reinforcement learning policy evaluation problems are often modeled as finite or discounted/averaged infinite-horizon MDPs. In this paper, we study undiscounted off-policy policy evaluation for absorbing MDPs. Given the dataset consisting of the i.i.d episodes with a given truncation level, we propose a so-called MWLA algorithm to directly estimate the expected return via the importance ratio of the state-action occupancy measure. The Mean Square Error (MSE) bound for the MWLA method is investigated and the dependence of statistical errors on the data size and the truncation level are analyzed. With an episodic taxi environment, computational experiments illustrate the performance of the MWLA algorithm.Comment: 36 pages, 9 figure

Evaluation of machine learning algorithms for anomaly detection

Malicious attack detection is one of the critical cyber-security challenges in the peer-to-peer smart grid platforms due to the fact that attackers' behaviours change continuously over time. In this paper, we evaluate twelve Machine Learning (ML) algorithms in terms of their ability to detect anomalous behaviours over the networking practice. The evaluation is performed on three publicly available datasets: CICIDS-2017, UNSW-NB15 and the Industrial Control System (ICS) cyber-attack datasets. The experimental work is performed through the ALICE high-performance computing facility at the University of Leicester. Based on these experiments, a comprehensive analysis of the ML algorithms is presented. The evaluation results verify that the Random Forest (RF) algorithm achieves the best performance in terms of accuracy, precision, Recall, F1-Score and Receiver Operating Characteristic (ROC) curves on all these datasets. It is worth pointing out that other algorithms perform closely to RF and that the decision regarding which ML algorithm to select depends on the data produced by the application system

Randomness invalidates criminal smart contracts

A smart contract enforces specific performance on anonymous users without centralization. It facilitates payment equity in commerce by providing irreversible transactions. Smart contracts are also used for illegal activities such as money laundering and ransomware. Such contracts include criminal smart contracts (CSCs), proposed in CCS’16, that can be efficiently implemented in existing scripting languages. This aggravates concerns about the dangers of CSCs. However, PublicLeaks, a CSC for leaking private data, is conditionally implemented as it is influenced by various factors. For example, PublicLeaks does not necessarily reach a desirable terminal state for a criminal leaking private information, and other possible terminal states may invalidate the CSC. In this study, we propose a CSC based on PublicLeaks by formulating random factors such as the donation ratio. Our contract forks into five terminal states, including a unique one in PublicLeaks due to randomness. We simulated the maximal probabilities of these terminal states and found that the desirable terminal state in PublicLeaks is reachable with low probabilities (lower than 25%). The terminal state where the criminal fails to leak private information is attained with relatively high probabilities (over 65%). Therefore, our simulations show that CSCs are not always as powerful as expected, and the risk posed by them can be mitigated

Effect of nitrogen application on enhancing high-temperature stress tolerance of tomato plants during the flowering and fruiting stage

This study was conducted to investigate the effects of nitrogen application on growth, photosynthetic performance, nitrogen metabolism activities, and fruit quality of tomato plants under high-temperature (HT) stress. Three levels of daily minimum/daily maximum temperature were adopted during the flowering and fruiting stage, namely control (CK; 18°C/28°C), sub-high temperature (SHT; 25°C/35°C), and high-temperature (HT; 30°C/40°C) stress. The levels of nitrogen (urea, 46% N) were set as 0 (N1), 125 (N2), 187.5 (N3), 250 (N4), and 312.5 (N5) kg hm2, respectively, and the duration lasted for 5 days (short-term). HT stress inhibited the growth, yield, and fruit quality of tomato plants. Interestingly, short-term SHT stress improved growth and yield via higher photosynthetic efficiency and nitrogen metabolism whereas fruit quality was reduced. Appropriate nitrogen application can enhance the high-temperature stress tolerance of tomato plants. The maximum net photosynthetic rate (PNmax), stomatal conductance (gs), stomatal limit value (LS), water-use efficiency (WUE), nitrate reductase (NR), glutamine synthetase (GS), soluble protein, and free amino acids were the highest in N3, N3, and N2, respectively, for CK, SHT, and HT stress, whereas carbon dioxide concentration (Ci), was the lowest. In addition, maximum SPAD value, plant morphology, yield, Vitamin C, soluble sugar, lycopene, and soluble solids occurred at N3-N4, N3-N4, and N2-N3, respectively, for CK, SHT, and HT stress. Based on the principal component analysis and comprehensive evaluation, we found that the optimum nitrogen application for tomato growth, yield, and fruit quality was 230.23 kg hm2 (N3-N4), 230.02 kg hm2 (N3-N4), and 115.32 kg hm2 (N2), respectively, at CK, SHT, and HT stress. Results revealed that the high yield and good fruit quality of tomato plants at high temperatures can be maintained by higher photosynthesis, nitrogen efficiency, and nutrients with moderate nitrogen

Privacy-aware secure anonymous communication protocol in CPSS cloud computing

Cloud computing has emerged as a promising paradigm for the Internet of Things (IoT) and Cyber-Physical-Social Systems (CPSS). However, the problem of how to ensure the security of data transmission and data storage in CPSS is a key issue to address. We need to protect the confidentiality and privacy of users’ data and users’ identity during the transmission and storage process in CPSS. In order to avoid users’ personal information leakage from IoT devices during the process of data processing and transmitting, we propose a certificateless encryption scheme, and conduct a security analysis under the assumption of Computational Diffie-Hellman(CDH) Problem. Furthermore, based on the proposed cryptography mechanism, we achieve a novel anonymous communication protocol to protect the identity privacy of communicating units in CPSS. In the new protocol, an anonymous communication link establishment method and an anonymous communication packet encapsulation format are proposed. The Diffie-Hellman key exchange algorithm is used to construct the anonymous keys distribution method in the new link establishment method. And in the new onion routing packet encapsulation format, the session data are firstly separated from the authentication data to decrease the number of cryptography operations. That is, by using the new onion routing packet we greatly reduces the encryption operations and promotes the forwarding efficiency of anonymous messages, implementing the privacy, security and efficiency in anonymous communication in cyber-physical-social systems

The construction of a nomogram to predict the prognosis and recurrence risks of UPJO

ObjectiveThis study was conducted to explore the risk factors for the prognosis and recurrence of ureteropelvic junction obstruction (UPJO).MethodsThe correlation of these variables with the prognosis and recurrence risks was analyzed by binary and multivariate logistic regression. Besides, a nomogram was constructed based on the multivariate logistic regression calculation. After the model was verified by the C-statistic, the ROC curve was plotted to evaluate the sensitivity of the model. Finally, the decision curve analysis (DCA) was conducted to estimate the clinical benefits and losses of intervention measures under a series of risk thresholds.ResultsPreoperative automated peritoneal dialysis (APD), preoperative urinary tract infection (UTI), preoperative renal parenchymal thickness (RPT), Mayo adhesive probability (MAP) score, and surgeon proficiency were the high-risk factors for the prognosis and recurrence of UPJO. In addition, a nomogram was constructed based on the above 5 variables. The area under the curve (AUC) was 0.8831 after self cross-validation, which validated that the specificity of the model was favorable.ConclusionThe column chart constructed by five factors has good predictive ability for the prognosis and recurrence of UPJO, which may provide more reasonable guidance for the clinical diagnosis and treatment of this disease

Preparation of Broad-Spectrum Polyclonal Antibody and Development of an Indirect Competitive-Enzyme Linked Immunosorbent Assay for Multi-Residue Detection of Biphenyl Tetrazolium Sartans in Antihypertensive Health Foods

An indirect competitive-enzyme linked immunosorbent assay (ic-ELISA) was established to detect the multi-residue of biphenyl tetrazolium sartans in antihypertensive health foods. Candesartan was coupled with bovine serum albumin to obtain immunogen. New Zealand white rabbits were immunized and a broad-spectrum antibody was obtained by an antibody screening assay. The half maximal inhibitory concentrations (IC50) for candesartan, losartan carboxylic acid, losartan potassium, olmesartan, olmesartan medoxomil, irbesartan, valsartan and valsartan methyl ester were 0.2, 0.2, 0.7, 0.04, 0.6, 0.3, 0.9 and 2.4 ng/mL, respectively. The samples were extracted with methanol and the matrix effect was eliminated by diluting the extract with standard solutions. The average recoveries of the eight target compounds were in the range from 80.6% to 120.0% with coefficients of variation equal to or below 14.0%. The results of ic-ELISA were highly correlated with those of liquid chromatography-tandem mass spectrometry (LC-MS/MS) (r > 0.97), indicating high accuracy and good reliability of ic-ELISA

Acidification oxidation reagent system optimization on coal seams and stimulation effect evaluation

China has abundant coalbed methane (CBM) resources, and most of them are low-permeability and tight reservoirs, with generally low production rate and small recovery factor. Existing technologies face great challenges to meet the demand on CBM in China. It is desirable to develop new methods to improve the production rate and enhance recovery factor. In addition to physical stimulation methods such as hydraulic fracturing and open-hole cave completion, the use of chemical methods to improve physical properties of coal reservoirs has also been a hot research topic in recent years. Coal reservoir acidification and oxidation technology can promote desorption of gas and enlarge permeability of reservoir. But for different coal rank coal reservoirs, the acidification and oxidation agents need to be optimized and their performance evaluated. Laboratory experiments are conducted to compare and analyze the physical properties coal samples from Baode, Mu’ai, and Xinjiang blocks, including coal rank, texture, macroscopic characteristics, quality, porosity, permeability, element, and mineral composition. The optimal concentration of hydrochloric acid is determined through pre-dissolution experiment of coal powder in acid solution. Then a five-factor and three-level orthogonal experiment for acid solution optimization is designed and performed by using Design-Expert software, which identifies the sensitive factors affecting the dissolution. For the coal samples in Baode, Mu’ai, and Xinjiang blocks, the oxidant types and the corresponding acidification and oxidation agent systems are optimized. Applying these acidification and oxidation agent systems to coal samples from Baode, Mu’ai, and Xinjiang blocks, the change of porosity, permeability, and wettability are compared and analyzed. Finally, through numerical simulation, the gas production is predicted for acidification and oxidation in typical well group in Block Mu’ai. Results show that the acid solution has the best dissolution at a concentration of hydrochloric acid of 3 mol/L to 4 mol/L; Top factors played in the experiment are soaking time, acid type, soaking temperature, coal sample type, and acid concentration, in descending order of importance; The optimal oxidant is a hydrogen peroxide solution with a concentration of 3%; the mixed acidification oxidant formula in Baode block is 10% HCl + 2% CH3COOH + 2% HF + 3% H2O2; The optimal mixed acidification oxidant formula in Mu’ai block is 8% HCl + 2% CH3COOH + 4% HF + 3% H2O2; the optimal mixed acidification oxidant formula in Xinjiang block is 12% HCl + 1% CH3COOH + 1% HF + 3% H2O2; The higher the coal rank, the greater the HF content in the optimal acidification oxidant system. Both acidification and oxidation improve the porosity and permeability of coal samples to some extent, and the improvement in low-rank coal is more significant than that in high-rank coal. Acidification and oxidation have different effects on the wettability of coal: Acidification increases the hydrophilicity of coal, whereas oxidation reduce the hydrophilicity of coal; and the hydrophilicity of coal samples treated by the optimized acidification and oxidation system is weakened. Reservoir simulation results show that acidification and oxidation lead to a recovery factor of 64.64% after 10 years of production, which is 19.72% higher than that without acidification and oxidation. The advantage of acidification and oxidation is 0.97% after 18 years of production. However, the acidification and oxidation saved 8 years of production time to achieve a close final recovery factor, which greatly reduces the operating costs. The optimized acidizing oxidation agent systems for CBM reservoirs with low, medium, and high ranks improved the desorption and permeability of the target reservoirs, and increase well production and recovery factor. This research provides technical support for stimulation practices of CBM reservoirs in the aforementioned blocks in China, as well as similar coal reservoirs in the world

Numerical Investigation of the Air Flow Effects on Amorphous Alloy Ribbon Formation

During the amorphous ribbon production process, the mechanisms that cause uneven thickness and herringbone on two surfaces of the ribbon are very complicated. The high velocity rotating roller causes periodic air eddies near the surface and air flow around the contact zone. The air flow might be the major reason that causes the non-uniformity thickness and the ripples. In this paper, a two-dimension model was developed to simulate the air flow eddies and a three-dimensional simulation was performed for the air flow field around the puddle zone. Simulation results show that the higher velocity of air flow around the puddle edges is closely related to thicker ribbon rim. The continuous air eddies along the roller surface and the air flow fluctuations along the roller width are the major reasons causing non-uniform convective heat transfer during ribbon solidification. The frequency of air eddy matches well with the nominal ripple frequency along the ribbon length, and the air flow fluctuation coincides with the herringbone wavelength along the ribbon width. The influences of the air flow on cooling, ribbon thickness, and herringbone phenomena are all discussed here