A Semi-Markov Decision Model for Recognizing the Destination of a Maneuvering Agent in Real Time Strategy Games

Recognizing destinations of a maneuvering agent is important in real time strategy games. Because finding path in an uncertain environment is essentially a sequential decision problem, we can model the maneuvering process by the Markov decision process (MDP). However, the MDP does not define an action duration. In this paper, we propose a novel semi-Markov decision model (SMDM). In the SMDM, the destination is regarded as a hidden state, which affects selection of an action; the action is affiliated with a duration variable, which indicates whether the action is completed. We also exploit a Rao-Blackwellised particle filter (RBPF) for inference under the dynamic Bayesian network structure of the SMDM. In experiments, we simulate agents’ maneuvering in a combat field and employ agents’ traces to evaluate the performance of our method. The results show that the SMDM outperforms another extension of the MDP in terms of precision, recall, and F-measure. Destinations are recognized efficiently by our method no matter whether they are changed or not. Additionally, the RBPF infer destinations with smaller variance and less time than the SPF. The average failure rates of the RBPF are lower when the number of particles is not enough

Efficient Dynamic Deployment of Simulation Tasks in Collaborative Cloud and Edge Environments

Cloud computing has been studied and used extensively in many scenarios for its nearly unlimited resources and X as a service model. To reduce the latency for accessing the remote cloud data centers, small data centers or cloudlets are deployed near end-users, which is also called edge computing. In this paper, we mainly focus on the efficient scheduling of distributed simulation tasks in collaborative cloud and edge environments. Since simulation tasks are usually tightly coupled with each other by sending many messages and the status of tasks and hosts may also change frequently, it is essentially a dynamic bin-packing problem. Unfortunately, popular methods, such as meta-heuristics, and accurate algorithms are time-consuming and cannot deal with the dynamic changes of tasks and hosts efficiently. In this paper, we present Pool, an incremental flow-based scheduler, to minimize the overall communication cost of all tasks in a reasonable time span with the consideration of migration cost of task. After formulating such a scheduling problem as a min-cost max-flow (MCMF) problem, incremental MCMF algorithms are adopted to accelerate the procedure of calculating an optimal flow and heuristic scheduling algorithm, with the awareness of task migration cost, designed to assign tasks. Simulation experiments on Alibaba cluster trace show that Pool can schedule all of the tasks efficiently and is almost 5.8 times faster than the baseline method when few tasks and hosts change in the small problem scale

Coordinated Learning by Model Difference Identification in Multiagent Systems with Sparse Interactions

Multiagent Reinforcement Learning (MARL) is a promising technique for agents learning effective coordinated policy in Multiagent Systems (MASs). In many MASs, interactions between agents are usually sparse, and then a lot of MARL methods were devised for them. These methods divide learning process into independent learning and joint learning in coordinated states to improve traditional joint state-action space learning. However, most of those methods identify coordinated states based on assumptions about domain structure (e.g., dependencies) or agent (e.g., prior individual optimal policy and agent homogeneity). Moreover, situations that current methods cannot deal with still exist. In this paper, a modified approach is proposed to learn where and how to coordinate agents’ behaviors in more general MASs with sparse interactions. Our approach introduces sample grouping and a more accurate metric of model difference degree to identify which states of other agents should be considered in coordinated states, without strong additional assumptions. Experimental results show that the proposed approach outperforms its competitors by improving the average agent reward per step and works well in some broader scenarios

Research on transmission control method of tension tower hydraulic tension device based on PLC

Using the traditional method to control the transmission process of the tension tower hydraulic tightening device, there is a problem of slow speed step response in the range of device operating speed from 12km/h to 32km/h. For this reason, this paper designs a programmable logic controller-based transmission control method for the hydraulic tightening device of the tension tower. The two-phase static coordinate system is introduced to obtain the voltage equation, flux equation, electromagnetic torque equation and electromechanical motion equation of the hydraulic drive system, and then the mathematical model of the hydraulic drive system is constructed. The full-order observer is regarded as an adjustable model, the appropriate Lyapunov function is selected, and the function stability of the full-order observer is derived, so as to realize the identification of the speed of the tension tower hydraulic tightening device. Based on this, a transmission control model is constructed based on programmable logic controller to realize the transmission control of the tension tower hydraulic tightening device. Experimental results prove that the speed step response rate of this method is higher than that of the traditional method, and the response performance is improved

A Novel in Situ FPAR Measurement Method for Low Canopy Vegetation Based on a Digital Camera and Reference Panel

Abstract: The fraction of absorbed photosynthetically active radiation (FPAR) is a key parameter in describing the exchange of fluxes of energy, mass and momentum between the surface and atmosphere. In this study, we present a method to measure FPAR using a digital camera and a reference panel. A digital camera was used to capture color images of low canopy vegetation, which contained a reference panel in one corner of the field of view (FOV). The digital image was classified into photosynthetically active vegetation, ground litter, sunlit soil, shadow soil, and the reference panel. The relative intensity of the incident photosynthetically active radiation (PAR), scene-reflected PAR, exposed background absorbed PAR and the green vegetation-covered ground absorbed PAR were derived from the digital camera image, and then FPAR was calculated. This method was validated on eight plots with four vegetation species using FPAR measured by a SunScan instrument. A linear correlation with a coefficient of determination (R 2) of 0.942 and mean absolute error (MAE) of 0.031 was observed between FPAR values derived from the digital camera and measurement using the SunScan instrument. The result suggests that the present method can be used to accurately measure the FPAR of low canopy vegetation

Monitoring Geologic Hazards and Vegetation Recovery in the Wenchuan Earthquake Region Using Aerial Photography

On 12 May 2008, the 8.0-magnitude Wenchuan earthquake occurred in Sichuan Province, China, triggering thousands of landslides, debris flows, and barrier lakes, leading to a substantial loss of life and damage to the local environment and infrastructure. This study aimed to monitor the status of geologic hazards and vegetation recovery in a post-earthquake disaster area using high-resolution aerial photography from 2008 to 2011, acquired from the Center for Earth Observation and Digital Earth (CEODE), Chinese Academy of Sciences. The distribution and range of hazards were identified in 15 large, representative geologic hazard areas triggered by the Wenchuan earthquake. After conducting an overlay analysis, the variations of these hazards between successive years were analyzed to reflect the geologic hazard development and vegetation recovery. The results showed that in the first year after the Wenchuan earthquake, debris flows occurred frequently with high intensity. Resultantly, with the source material becoming less available and the slope structure stabilizing, the intensity and frequency of debris flows gradually decreased with time. The development rate of debris flows between 2008 and 2011 was 3% per year. The lithology played a dominant role in the formation of debris flows, and the topography and hazard size in the earthquake affected area also had an influence on the debris flow development process. Meanwhile, the overall geologic hazard area decreased at 12% per year, and the vegetation recovery on the landslide mass was 15% to 20% per year between 2008 and 2011. The outcomes of this study provide supporting data for ecological recovery as well as debris flow control and prevention projects in hazard-prone areas

A Decomposition Approach for Stochastic Shortest-Path Network Interdiction with Goal Threshold

Shortest-path network interdiction, where a defender strategically allocates interdiction resource on the arcs or nodes in a network and an attacker traverses the capacitated network along a shortest s-t path from a source to a terminus, is an important research problem with potential real-world impact. In this paper, based on game-theoretic methodologies, we consider a novel stochastic extension of the shortest-path network interdiction problem with goal threshold, abbreviated as SSPIT. The attacker attempts to minimize the length of the shortest path, while the defender attempts to force it to exceed a specific threshold with the least resource consumption. In our model, threshold constraint is introduced as a trade-off between utility maximization and resource consumption, and stochastic cases with some known probability p of successful interdiction are considered. Existing algorithms do not perform well when dealing with threshold and stochastic constraints. To address the NP-hard problem, SSPIT-D, a decomposition approach based on Benders decomposition, was adopted. To optimize the master problem and subproblem iteration, an efficient dual subgraph interdiction algorithm SSPIT-S and a local research based better-response algorithm SSPIT-DL were designed, adding to the SSPIT-D. Numerical experiments on networks of different sizes and attributes were used to illustrate and validate the decomposition approach. The results showed that the dual subgraph and better-response procedure can significantly improve the efficiency and scalability of the decomposition algorithm. In addition, the improved enhancement algorithms are less sensitive and robust to parameters. Furthermore, the application in a real-world road network demonstrates the scalability of our decomposition approach

Modified Adversarial Hierarchical Task Network Planning in Real-Time Strategy Games

The application of artificial intelligence (AI) to real-time strategy (RTS) games includes considerable challenges due to the very large state spaces and branching factors, limited decision times, and dynamic adversarial environments involved. To address these challenges, hierarchical task network (HTN) planning has been extended to develop a method denoted as adversarial HTN (AHTN), and this method has achieved favorable performance. However, the HTN description employed cannot express complex relationships among tasks and accommodate the impacts of the environment on tasks. Moreover, AHTN cannot address task failures during plan execution. Therefore, this paper proposes a modified AHTN planning algorithm with failed task repair functionality denoted as AHTN-R. The algorithm extends the HTN description by introducing three additional elements: essential task, phase, and exit condition. If any task fails during plan execution, the AHTN-R algorithm identifies and terminates all affected tasks according to the extended HTN description, and applies a novel task repair strategy based on a prioritized listing of alternative plans to maintain the validity of the previous plan. In the planning process, AHTN-R generates the priorities of alternative plans by sorting all nodes of the game search tree according to their primary features. Finally, empirical results are presented based on a µRTS game, and the performance of AHTN-R is compared to that of AHTN and to the performances of other state-of-the-art search algorithms developed for RTS games

Improving the Retrieval of Crop Canopy Chlorophyll Content Using Vegetation Index Combinations

Estimates of crop canopy chlorophyll content (CCC) can be used to monitor vegetation productivity, manage crop resources, and control disease and pests. However, making these estimates using conventional ground-based methods is time-consuming and resource-intensive when deployed over large areas. Although vegetation indices (VIs), derived from satellite sensor data, have been used to estimate CCC, they suffer from problems related to spectral saturation, soil background, and canopy structure. A new method was, therefore, proposed for combining the Medium Resolution Imaging Spectrometer (MERIS) terrestrial chlorophyll index (MTCI) and LAI-related vegetation indices (LAI-VIs) to increase the accuracy of CCC estimates for wheat and soybeans. The PROSAIL-D canopy reflectance model was used to simulate canopy spectra that were resampled to match the spectral response functions of the MERIS carried on the ENVISAT satellite. Combinations of the MTCI and LAI-VIs were then used to estimate CCC via univariate linear regression, binary linear regression and random forest regression. The accuracy using the field spectra and MERIS data was determined based on field CCC measurements. All the MTCI and LAI-VI combinations for the selected regression techniques resulted in more accurate estimates of CCC than the use of the MTCI alone (field spectra data for soybeans and wheat: R2 = 0.62 and RMSE = 77.10 μg cm−2; MERIS satellite data for soybeans: R2 = 0.24 and RMSE = 136.54 μg cm−2). The random forest regression resulted in better accuracy than the other two linear regression models. The combination resulting in the best accuracy was the MTCI and MTVI2 and random forest regression, with R2 = 0.65 and RMSE = 37.76 μg cm−2 (field spectra data) and R2 = 0.78 and RMSE = 47.96 μg cm−2 (MERIS satellite data). Combining the MTCI and a LAI-VI represents a further step towards improving the accuracy of estimation CCC based on multispectral satellite sensor data

Machine Learning Algorithms for the Retrieval of Canopy Chlorophyll Content and Leaf Area Index of Crops Using the PROSAIL-D Model with the Adjusted Average Leaf Angle

The canopy chlorophyll content (CCC) and leaf area index (LAI) are both essential indicators for crop growth monitoring and yield estimation. The PROSAIL model, which couples the properties optique spectrales des feuilles (PROSPECT) and scattering by arbitrarily inclined leaves (SAIL) radiative transfer models, is commonly used for the quantitative retrieval of crop parameters; however, its homogeneous canopy assumption limits its accuracy, especially in the case of multiple crop categories. The adjusted average leaf angle (ALAadj), which can be parameterized for a specific crop type, increases the applicability of the PROSAIL model for specific crop types with a non-uniform canopy and has the potential to enhance the performance of PROSAIL-coupled hybrid methods. In this study, the PROSAIL-D model was used to generate the ALAadj values of wheat, soybean, and maize crops based on ground-measured spectra, the LAI, and the leaf chlorophyll content (LCC). The results revealed ALAadj values of 62 degrees for wheat, 45 degrees for soybean, and 60 degrees for maize. Support vector regression (SVR), random forest regression (RFR), extremely randomized trees regression (ETR), the gradient boosting regression tree (GBRT), and stacking learning (STL) were applied to simulated data of the ALAadj in 50-band data to retrieve the CCC and LAI of the crops. The results demonstrated that the estimation accuracy of singular crop parameters, particularly the crop LAI, was greatly enhanced by the five machine learning methods on the basis of data simulated with the ALAadj. Regarding the estimation results of mixed crops, the machine learning algorithms using ALAadj datasets resulted in estimations of CCC (RMSE: RFR = 51.1 μg cm−2, ETR = 54.7 μg cm−2, GBRT = 54.9 μg cm−2, STL = 48.3 μg cm−2) and LAI (RMSE: SVR = 0.91, RFR = 1.03, ETR = 1.05, GBRT = 1.05, STL = 0.97), that outperformed the estimations without using the ALAadj (namely CCC RMSE: RFR = 93.0 μg cm−2, ETR = 60.1 μg cm−2, GBRT = 60.0 μg cm−2, STL = 68.5 μg cm−2 and LAI RMSE: SVR = 2.10, RFR = 2.28, ETR = 1.67, GBRT = 1.66, STL = 1.51). Similar findings were obtained using the suggested method in conjunction with 19-band data, demonstrating the promising potential of this method to estimate the CCC and LAI of crops at the satellite scale