Effect of turning frequency and season on composting materials from swine high-rise facilities

Composting swine slurries has several advantages, liquid slurries are converted to solids at lower moisture, the total volume and weight of material is reduced and the stabilized product is more easily transported off-site. Despite this, swine waste is generally stored, treated and applied in its liquid form. High-rise finishing facilities (HRFF) permit liquid slurries to be converted to solids which are partially decomposed underneath the HRFF and then finished in compost windrows. The purpose of this study was to evaluate the effect of turning frequency and ambient weather conditions on biological, physical and chemical properties of composted slurry-woodchip mixtures from HRFF. Compost trials were conducted in either fall (FT) or spring (ST) and piles were turned once or three times per week or upon compost temperature reaching 65 °C. Physical, chemical and microbiological characteristics were measured over the course of 112 (FT) or 143 (ST) days of composting. Total carbon, total nitrogen (N) and inorganic N decreased in all piles. Ammonium decreased while nitrate increased in all piles (including unturned), but total N losses were greatest in piles turned more frequently during the ST. Microbial populations of nitrifiers were dominated by ammonia-oxidizing archaea (3.0 X 10^3–4.2 X 10^6 cells g^-1 compost) but ammonia oxidizing bacteria (below detection to 6.0 X cells g^-1 compost) varied in response to turning and compost temperature; denitrifiers were present in high concentrations throughout the process. Swine HRFF materials composted well in windrows regardless of turning frequency and despite significant differences in starting materials and low initial C/N. Volume reduction, low moisture and low readily degradable organic matter suggest that the finished compost would have lower transportation costs and should provide value as a soil conditioner

Integral Reinforcement Learning and Experience Replay for Adaptive Optimal Control of Partially-Unknown Constrained-Input Continuous-Time Systems

In this paper, an integral reinforcement learning (IRL) algorithm on an actor-critic structure is developed to learn online the solution to the Hamilton-Jacobi-Bellman equation for partially-unknown constrained-input systems. The technique of experience replay is used to update the critic weights to solve an IRL Bellman equation. This means, unlike existing reinforcement learning algorithms, recorded past experiences are used concurrently with current data for adaptation of the critic weights. It is shown that using this technique, instead of the traditional persistence of excitation condition which is often difficult or impossible to verify online, an easy-to-check condition on the richness of the recorded data is sufficient to guarantee convergence to a near-optimal control law. Stability of the proposed feedback control law is shown and the effectiveness of the proposed method is illustrated with simulation examples

Online Solution of Nonquadratic Two-Player Zero-Sum Games Arising in the H\u3csub\u3e∞\u3c/sub\u3e Control of Constrained Input Systems

In this paper, we present an online learning algorithm to find the solution to the H∞ control problem of continuous-time systems with input constraints. A suitable nonquadratic functional is utilized to encode the input constraints into the H∞ control problem, and the related H∞ control problem is formulated as a two-player zero-sum game with a nonquadratic performance. Then, a policy iteration algorithm on an actor-critic-disturbance structure is developed to solve the Hamilton-Jacobi-Isaacs (HJI) equation associated with this nonquadratic zero-sum game. That is, three NN approximators, namely, actor, critic, and disturbance, are tuned online and simultaneously for approximating the HJI solution. The value of the actor and disturbance policies is approximated continuously by the critic NN, and then on the basis of this value estimate, the actor and disturbance NNs are updated in real time to improve their policies. The disturbance tries to make the worst possible disturbance, whereas the actor tries to make the best control input. A persistence of excitation condition is shown to guarantee convergence to the optimal saddle point solution. Stability of the closed-loop system is also guaranteed. A simulation on a nonlinear benchmark problem is performed to validate the effectiveness of the proposed approach

Adaptive Optimal Control of Unknown Constrained-Input Systems using Policy Iteration and Neural Networks

This paper presents an online policy iteration (PI) algorithm to learn the continuous-time optimal control solution for unknown constrained-input systems. The proposed PI algorithm is implemented on an actor-critic structure where two neural networks (NNs) are tuned online and simultaneously to generate the optimal bounded control policy. The requirement of complete knowledge of the system dynamics is obviated by employing a novel NN identifier in conjunction with the actor and critic NNs. It is shown how the identifier weights estimation error affects the convergence of the critic NN. A novel learning rule is developed to guarantee that the identifier weights converge to small neighborhoods of their ideal values exponentially fast. To provide an easy-to-check persistence of excitation condition, the experience replay technique is used. That is, recorded past experiences are used simultaneously with current data for the adaptation of the identifier weights. Stability of the whole system consisting of the actor, critic, system state, and system identifier is guaranteed while all three networks undergo adaptation. Convergence to a near-optimal control law is also shown. The effectiveness of the proposed method is illustrated with a simulation example

A Policy Iteration Approach to Online Optimal Control of Continuous-Time Constrained-Input Systems

This paper is an effort towards developing an online learning algorithm to find the optimal control solution for continuous-time (CT) systems subject to input constraints. The proposed method is based on the policy iteration (PI) technique which has recently evolved as a major technique for solving optimal control problems. Although a number of online PI algorithms have been developed for CT systems, none of them take into account the input constraints caused by actuator saturation. In practice, however, ignoring these constraints leads to performance degradation or even system instability. In this paper, to deal with the input constraints, a suitable nonquadratic functional is employed to encode the constraints into the optimization formulation. Then, the proposed PI algorithm is implemented on an actor-critic structure to solve the Hamilton-Jacobi-Bellman (HJB) equation associated with this nonquadratic cost functional in an online fashion. That is, two coupled neural network (NN) approximators, namely an actor and a critic are tuned online and simultaneously for approximating the associated HJB solution and computing the optimal control policy. The critic is used to evaluate the cost associated with the current policy, while the actor is used to find an improved policy based on information provided by the critic. Convergence to a close approximation of the HJB solution as well as stability of the proposed feedback control law are shown. Simulation results of the proposed method on a nonlinear CT system illustrate the effectiveness of the proposed approach

Nationally Coordinated Evaluation of Soil Nitrogen Mineralization Rate using a Standardized Aerobic Incubation Protocol

Aerobic incubation methods have been widely used to assess soil nitrogen (N) mineralization, but standardized protocols are lacking. A single silt loam soil (Catlin silt loam; fine-silty, mixed, superactive, mesic, Oxyaquic Arguidoll) was subjected to aerobic incubation at six USDA-ARS locations using a standardized protocol. Incubations were conducted at multiple temperatures, which were combined based on degree days (DD). Soil water was maintained at 60% waterfilled pore space (WFPS; constant) or allowed to fluctuate between 60 and 30% WFPS (cycle). Soil subsamples were removed periodically and extracted in 2 M potassium chloride (KCl); nitrate (NO3) and ammonium (NH4) concentrations in extracts were determined colorimetrically. For each location, the rate of soil organic-matter N (SOMN) mineralization was estimated by regressing soil inorganic N (Ni) concentration on DD, using a linear (zero-order) model. When all data were included, the mineralization rate from four datasets was not statistically different, with a rate equivalent to 0.5 mg N kg-1 soil day-1. Soil incubated at two locations exhibited significantly higher SOMN mineralization rates. To assess whether this may have been due to pre-incubation conditions, time-zero data were excluded and regression analysis was conducted again. Using this data subset, SOMN mineralization from five (of six) datasets was not significantly different. Fluctuating soil water reduced N-mineralization rate at two (of four) locations by an average of 50%; fluctuating soil water content also substantially increased variability. This composite dataset demonstrates that standardization of aerobic incubation methodology is possible

Reinforcement Q-Learning for Optimal Tracking Control of Linear Discrete-Time Systems with Unknown Dynamics

In this paper, a novel approach based on the Q-learning algorithm is proposed to solve the infinite-horizon linear quadratic tracker (LQT) for unknown discrete-time systems in a causal manner. It is assumed that the reference trajectory is generated by a linear command generator system. An augmented system composed of the original system and the command generator is constructed and it is shown that the value function for the LQT is quadratic in terms of the state of the augmented system. Using the quadratic structure of the value function, a Bellman equation and an augmented algebraic Riccati equation (ARE) for solving the LQT are derived. In contrast to the standard solution of the LQT, which requires the solution of an ARE and a noncausal difference equation simultaneously, in the proposed method the optimal control input is obtained by only solving an augmented ARE. A Q-learning algorithm is developed to solve online the augmented ARE without any knowledge about the system dynamics or the command generator. Convergence to the optimal solution is shown. A simulation example is used to verify the effectiveness of the proposed control scheme