Minimax Weight Learning for Absorbing MDPs

Reinforcement learning policy evaluation problems are often modeled as finite or discounted/averaged infinite-horizon MDPs. In this paper, we study undiscounted off-policy policy evaluation for absorbing MDPs. Given the dataset consisting of the i.i.d episodes with a given truncation level, we propose a so-called MWLA algorithm to directly estimate the expected return via the importance ratio of the state-action occupancy measure. The Mean Square Error (MSE) bound for the MWLA method is investigated and the dependence of statistical errors on the data size and the truncation level are analyzed. With an episodic taxi environment, computational experiments illustrate the performance of the MWLA algorithm.Comment: 36 pages, 9 figure

Evaluation of machine learning algorithms for anomaly detection

Malicious attack detection is one of the critical cyber-security challenges in the peer-to-peer smart grid platforms due to the fact that attackers' behaviours change continuously over time. In this paper, we evaluate twelve Machine Learning (ML) algorithms in terms of their ability to detect anomalous behaviours over the networking practice. The evaluation is performed on three publicly available datasets: CICIDS-2017, UNSW-NB15 and the Industrial Control System (ICS) cyber-attack datasets. The experimental work is performed through the ALICE high-performance computing facility at the University of Leicester. Based on these experiments, a comprehensive analysis of the ML algorithms is presented. The evaluation results verify that the Random Forest (RF) algorithm achieves the best performance in terms of accuracy, precision, Recall, F1-Score and Receiver Operating Characteristic (ROC) curves on all these datasets. It is worth pointing out that other algorithms perform closely to RF and that the decision regarding which ML algorithm to select depends on the data produced by the application system

Randomness invalidates criminal smart contracts

A smart contract enforces specific performance on anonymous users without centralization. It facilitates payment equity in commerce by providing irreversible transactions. Smart contracts are also used for illegal activities such as money laundering and ransomware. Such contracts include criminal smart contracts (CSCs), proposed in CCS’16, that can be efficiently implemented in existing scripting languages. This aggravates concerns about the dangers of CSCs. However, PublicLeaks, a CSC for leaking private data, is conditionally implemented as it is influenced by various factors. For example, PublicLeaks does not necessarily reach a desirable terminal state for a criminal leaking private information, and other possible terminal states may invalidate the CSC. In this study, we propose a CSC based on PublicLeaks by formulating random factors such as the donation ratio. Our contract forks into five terminal states, including a unique one in PublicLeaks due to randomness. We simulated the maximal probabilities of these terminal states and found that the desirable terminal state in PublicLeaks is reachable with low probabilities (lower than 25%). The terminal state where the criminal fails to leak private information is attained with relatively high probabilities (over 65%). Therefore, our simulations show that CSCs are not always as powerful as expected, and the risk posed by them can be mitigated

Effect of nitrogen application on enhancing high-temperature stress tolerance of tomato plants during the flowering and fruiting stage

This study was conducted to investigate the effects of nitrogen application on growth, photosynthetic performance, nitrogen metabolism activities, and fruit quality of tomato plants under high-temperature (HT) stress. Three levels of daily minimum/daily maximum temperature were adopted during the flowering and fruiting stage, namely control (CK; 18°C/28°C), sub-high temperature (SHT; 25°C/35°C), and high-temperature (HT; 30°C/40°C) stress. The levels of nitrogen (urea, 46% N) were set as 0 (N1), 125 (N2), 187.5 (N3), 250 (N4), and 312.5 (N5) kg hm2, respectively, and the duration lasted for 5 days (short-term). HT stress inhibited the growth, yield, and fruit quality of tomato plants. Interestingly, short-term SHT stress improved growth and yield via higher photosynthetic efficiency and nitrogen metabolism whereas fruit quality was reduced. Appropriate nitrogen application can enhance the high-temperature stress tolerance of tomato plants. The maximum net photosynthetic rate (PNmax), stomatal conductance (gs), stomatal limit value (LS), water-use efficiency (WUE), nitrate reductase (NR), glutamine synthetase (GS), soluble protein, and free amino acids were the highest in N3, N3, and N2, respectively, for CK, SHT, and HT stress, whereas carbon dioxide concentration (Ci), was the lowest. In addition, maximum SPAD value, plant morphology, yield, Vitamin C, soluble sugar, lycopene, and soluble solids occurred at N3-N4, N3-N4, and N2-N3, respectively, for CK, SHT, and HT stress. Based on the principal component analysis and comprehensive evaluation, we found that the optimum nitrogen application for tomato growth, yield, and fruit quality was 230.23 kg hm2 (N3-N4), 230.02 kg hm2 (N3-N4), and 115.32 kg hm2 (N2), respectively, at CK, SHT, and HT stress. Results revealed that the high yield and good fruit quality of tomato plants at high temperatures can be maintained by higher photosynthesis, nitrogen efficiency, and nutrients with moderate nitrogen

Privacy-aware secure anonymous communication protocol in CPSS cloud computing

Cloud computing has emerged as a promising paradigm for the Internet of Things (IoT) and Cyber-Physical-Social Systems (CPSS). However, the problem of how to ensure the security of data transmission and data storage in CPSS is a key issue to address. We need to protect the confidentiality and privacy of users’ data and users’ identity during the transmission and storage process in CPSS. In order to avoid users’ personal information leakage from IoT devices during the process of data processing and transmitting, we propose a certificateless encryption scheme, and conduct a security analysis under the assumption of Computational Diffie-Hellman(CDH) Problem. Furthermore, based on the proposed cryptography mechanism, we achieve a novel anonymous communication protocol to protect the identity privacy of communicating units in CPSS. In the new protocol, an anonymous communication link establishment method and an anonymous communication packet encapsulation format are proposed. The Diffie-Hellman key exchange algorithm is used to construct the anonymous keys distribution method in the new link establishment method. And in the new onion routing packet encapsulation format, the session data are firstly separated from the authentication data to decrease the number of cryptography operations. That is, by using the new onion routing packet we greatly reduces the encryption operations and promotes the forwarding efficiency of anonymous messages, implementing the privacy, security and efficiency in anonymous communication in cyber-physical-social systems

Preparation of Broad-Spectrum Polyclonal Antibody and Development of an Indirect Competitive-Enzyme Linked Immunosorbent Assay for Multi-Residue Detection of Biphenyl Tetrazolium Sartans in Antihypertensive Health Foods

An indirect competitive-enzyme linked immunosorbent assay (ic-ELISA) was established to detect the multi-residue of biphenyl tetrazolium sartans in antihypertensive health foods. Candesartan was coupled with bovine serum albumin to obtain immunogen. New Zealand white rabbits were immunized and a broad-spectrum antibody was obtained by an antibody screening assay. The half maximal inhibitory concentrations (IC50) for candesartan, losartan carboxylic acid, losartan potassium, olmesartan, olmesartan medoxomil, irbesartan, valsartan and valsartan methyl ester were 0.2, 0.2, 0.7, 0.04, 0.6, 0.3, 0.9 and 2.4 ng/mL, respectively. The samples were extracted with methanol and the matrix effect was eliminated by diluting the extract with standard solutions. The average recoveries of the eight target compounds were in the range from 80.6% to 120.0% with coefficients of variation equal to or below 14.0%. The results of ic-ELISA were highly correlated with those of liquid chromatography-tandem mass spectrometry (LC-MS/MS) (r > 0.97), indicating high accuracy and good reliability of ic-ELISA

Numerical Investigation of the Air Flow Effects on Amorphous Alloy Ribbon Formation

During the amorphous ribbon production process, the mechanisms that cause uneven thickness and herringbone on two surfaces of the ribbon are very complicated. The high velocity rotating roller causes periodic air eddies near the surface and air flow around the contact zone. The air flow might be the major reason that causes the non-uniformity thickness and the ripples. In this paper, a two-dimension model was developed to simulate the air flow eddies and a three-dimensional simulation was performed for the air flow field around the puddle zone. Simulation results show that the higher velocity of air flow around the puddle edges is closely related to thicker ribbon rim. The continuous air eddies along the roller surface and the air flow fluctuations along the roller width are the major reasons causing non-uniform convective heat transfer during ribbon solidification. The frequency of air eddy matches well with the nominal ripple frequency along the ribbon length, and the air flow fluctuation coincides with the herringbone wavelength along the ribbon width. The influences of the air flow on cooling, ribbon thickness, and herringbone phenomena are all discussed here

Numerical Investigation of the Air Flow Effects on Amorphous Alloy Ribbon Formation

During the amorphous ribbon production process, the mechanisms that cause uneven thickness and herringbone on two surfaces of the ribbon are very complicated. The high velocity rotating roller causes periodic air eddies near the surface and air flow around the contact zone. The air flow might be the major reason that causes the non-uniformity thickness and the ripples. In this paper, a two-dimension model was developed to simulate the air flow eddies and a three-dimensional simulation was performed for the air flow field around the puddle zone. Simulation results show that the higher velocity of air flow around the puddle edges is closely related to thicker ribbon rim. The continuous air eddies along the roller surface and the air flow fluctuations along the roller width are the major reasons causing non-uniform convective heat transfer during ribbon solidification. The frequency of air eddy matches well with the nominal ripple frequency along the ribbon length, and the air flow fluctuation coincides with the herringbone wavelength along the ribbon width. The influences of the air flow on cooling, ribbon thickness, and herringbone phenomena are all discussed here

Slope Detection Based on Grating Sensors

Because slope detection is often subject to various electromagnetic disturbances and weather limitations, there is an urgent need to develop a monitoring system that has the advantages of strong anti-electromagnetic interference capability, long service life, and small external environmental impact.Based on this demand, a real-time slope monitoring system based on fiber Bragg grating angle, crack, and osmotic pressure sensors is designed.The design idea is to use the sensitivity of FBG to the external stress, and the angle and crack penetrating pressure information are obtained from the offset of wavelength in different sensors to determine the stability of the slope.The design and implementation of the FBG slope monitoring system can better solve the existing problems in the slope monitoring, improve the ability to predict dangers, and strengthen the safety monitoring capability.The utility value of the system is high.</p