Learning non-Markovian Decision-Making from State-only Sequences

Conventional imitation learning assumes access to the actions of demonstrators, but these motor signals are often non-observable in naturalistic settings. Additionally, sequential decision-making behaviors in these settings can deviate from the assumptions of a standard Markov Decision Process (MDP). To address these challenges, we explore deep generative modeling of state-only sequences with non-Markov Decision Process (nMDP), where the policy is an energy-based prior in the latent space of the state transition generator. We develop maximum likelihood estimation to achieve model-based imitation, which involves short-run MCMC sampling from the prior and importance sampling for the posterior. The learned model enables \textit{decision-making as inference}: model-free policy execution is equivalent to prior sampling, model-based planning is posterior sampling initialized from the policy. We demonstrate the efficacy of the proposed method in a prototypical path planning task with non-Markovian constraints and show that the learned model exhibits strong performances in challenging domains from the MuJoCo suite

Characterization of a novel secretory spherical body protein in Babesia orientalis and Babesia orientalis-infected erythrocytes

Abstract Background The spherical body, a membrane bound organelle localized in the apical organelle complex, is unique to Babesia and Theileria spp. The spherical body proteins (SBPs) secreted by spherical bodies include SBP1, SBP2, SBP3 and SBP4. Up to now, only SBP3 has been characterized in Babesia orientalis. Methods The BoSBP4 gene was amplified from cDNA and gDNA and cloned into the pGEX-6P-1 vector by homologous recombination, sequenced and analyzed by bioinformatics tools. The amino acid (aa) sequence of BoSBP4 was compared with that of Babesia bovis and Babesia bigemina as well as SBP3 of B. orientalis. The immunoreactivity was evaluated by incubating recombinant BoSBP4 (rBoSBP4) with the serum of B. orientalis-infected water buffalo. The native form of BoSBP4 was identified by incubating lysate of B. orientalis-infected water buffalo erythrocytes with the anti-rBoSBP4 mouse serum. The cellular localization of BoSBP4 was determined by indirect immunofluorescence assay. Results The full length of the BoSBP4 gene was estimated to be 945 bp without introns, encoding a 314 aa polypeptide with a predicted molecular weight of 37 kDa. The truncated recombinant protein was expressed from 70 to 945 bp as a GST fusion protein with a practical molecular weight of 70 kDa. BoSBP4 shared a 40% and 30% identity with B. bovis and B. bigemina, respectively. Furthermore, it was 31% identical to SBP3 of B. orientalis. BoSBP4 was identified in the lysate of B. orientalis-infected water buffalo erythrocytes with a molecular weight of 37 kDa, corresponding to the expected molecular mass of BoSBP4. The result of rBoSBP4 with positive serum revealed that BoSBP4 can elicit an immune response to B. orientalis-infected water buffalo. The cellular localization of BoSBP4 was detected to be adjacent to the merozoite nucleus in the intracellular phase, followed by the diffusion of the fluorescence of BoSBP4 into the cytoplasm of B. orientalis-infected erythrocytes as puncta-like specks and a gradual increase of the fluorescence. Conclusions In this study, SBP4 in B. orientalis was characterized for the first time. It may play a key role in interaction with the host cell by being secreted into the cytoplasm of the B. orientalis-infected erythrocytes to facilitate parasite growth and reproduction