Reinforce-lib: A Reinforcement Learning Library for Scientific Research

Reinforcement Learning (RL) has already achieved several breakthroughs on complex, high-dimensional, and even multi-agent tasks, gaining increasingly interest from not only the research community. Although very powerful in principle, its applicability is still limited to solving games and control problems, leaving plenty opportunities to apply and develop RL algorithms for (but not limited to) scientific domains like physics, and biology. Apart from the domain of interest, the applicability of RL is also limited by numerous difficulties encountered while training agents, like training instabilities and sensitivity to hyperparameters. For such reasons, we propose a modern, modular, simple and understandable Python RL library called reinforce-lib. Our main aim is to enable newcomers, practitioners, and researchers to easily employ RL to solve new scientific problems. Our library is available at https://github.com/Luca96/reinforce-lib

Artificial activation of ovine oocytes is required after ICSI with freeze-dried spermatozoa.

Objectives. Freeze-drying allows to store the biological samples in a dry state and represents an interesting alternative low-cost strategy of semen biobanking to save the endangered species. Here, we have established a dry sperm biobank from an endangered Italian sheep breed (Pagliarola) and tested its efficiency through ICSI. Materials and methods. The motile spermatozoa from ram ejaculates collected with artificial vagina was selected by swim-up in TRIS-based medium (2.42g TRIS, 1.36g citric acid, 1.00g fructose, 100.000 U.I. penicillin G, 100mg streptomycin, in 67.20ml bidistilled water (ddH2O); pH was adjusted to 6.7) for 20 minutes at 38.5°C. The motile spermatozoa were frozen in freeze-drying medium (10mM EGTA and 50mM NaCl in 10mM Tris–HCl buffer; pH was adjusted to 8.4) in a -80°C freezer for 75 minutes and subsequently lyophilized by the freeze-drying apparatus SP Scientific-VirTis, Freeze-dryer 2.0 BenchTop, 20 hours with a condenser temperature of -58°C and vacuum of 20 mTorr). The vials were sealed in glass vials under vacuum and stored in the dark at 4°C for 1-2 months. Just before the ICSI, the freeze-dried spermatozoa were rehydrated by adding 100µl ddH2O. To evaluate the fertilizing capability of freeze-dried spermatozoa, 108 MII sheep oocytes were subjected to ICSI and allocated to two groups: 56 oocytes were activated by incubation with 5μM ionomycin (ICSI-FDSa); 52 were left un-activated (ICSI-FDSna). Forty-four oocytes injected with frozen spermatozoa (ICSI-FS) and left un-activated, served as control. Pronuclear formation (2PN) and blastocyst development were investigated at 14-16 hours and 7-8 days after ICSI, respectively. Differences were considered statistically significant for p<0.05 (Chi-square test). Data were analyzed using PRISM, software version 5.0; GraphPad. Results. The freeze-dried spermatozoa were completely immotile after rehydration, however they maintained the capacity to fecund oocytes after ICSI. Two PN were found in 83.3% of ICSI-FDSa, 81.4% of ICSI-FS while only in 14.3% of ICSI-FDSna (p<0.05 ICSI-FDSna vs ICSI-FDSa; p<0.01 ICSI-FDSna vs ICSI-FS). Likewise, the ICSI freeze-dried spermatozoa yielded blastocysts only following artificial activation (ICSI-FDSa: 10.2%; ICSI-FS: 31%; ICSI-FDSna: 0%; p<0.05 ICSI-FDSa vs ICSI-FDSna andICSI-FS; p<0.0001 ICSI-FDSna vs ICSI-FS). Conclusions. Our finding show that freeze-dried spermatozoa have lost the capacity to trigger oocyte activation but maintained their nuclear viability, whose developmental potential was fully released following artificial activation. Our results support the evidence that freeze-drying effective approach of spermatozoa storage to save endangered species

Freeze-dried sperm: an alternative biobanking solution for endangered farm species

Introduction An ever-increasing number of domestic species are threatened with extinction. Biobanking of spermatozoa could represent a feasible and efficient way for preserving genetic heritage and to maintain biodiversity. Given the published evidence that lyophilized spermatozoa retain their fertilizing capacity, we have collected semen from an Italian endangered sheep breed (Pagliarola) and created a biobank of cryopreserved and freeze-dried spermatozoa. Material and Methods The fertilizing capacity of the all stored semen, cryopreserved and freeze-dried, was evaluated by IVF and ICSI, respectively. To evaluate the activating capability of freeze-dried spermatozoa, 108 MII sheep oocytes were subjected to ICSI, and allocated to two groups: 56 oocytes were activated by incubation with ionomycin (ICSI-FDSa) and 52 were un-activated (ICSI-FDSna). Pronuclear formation (2PN) was investigated at 14-16 hours after ICSI in fixed presumptive zygotes. Results and Discussion As expected, the fertilizing capacity of cryopreserved Pagliarola’s spermatozoa was comparable to commercial semen stocks (31.8% vs 29%, respectively). Two PN were observed in 83.3% of ICSI-FDSa while only in 14.3% of ICSI-FDSna. Likewise, only artificially activated oocytes were able to develop to blastocyst after ICSI (10.2% compare to 0% with ICSI-FDSna). Oocytes injected with frozen spermatozoa (ICSI-FS) and left un-activated, served as control (81.4% of 2PN; 31% of blastocyst). In this work, we have demonstrated for the first time that freeze-dried ram spermatozoa could drive blastocyst development following ICSI. Although the developmental potential of embryos derived from lyophilized spermatozoa was significantly lower than cryopreserved ones, sperm lyophilization may be an alternative, low cost storage option, susceptible of improvement of course, to save biodiversity in domestic species

Comparison of slow and rapid freezing for long term storage of freeze-dry ram spermatozoa

Semen lyophilization is an interesting technique that might be a cheap alternative to long-term storage under liquid nitrogen. The first significant result of this method was achieved by Wakayama and Yanagimachi in the 1998 [1] demonstrating for the first time the birth of healthy offspring from epididymal freeze-dried (mouse) spermatozoa. From this work on, the most used approach for lyophilisation is that of deep-freezing, that is directly immersing the semen sample into liquid nitrogen before vacuum drying. Recently we have shown that it is possible to establish a "dry" bank of ejaculated and epidydimal freeze-dried ram spermatozoa [2, 3]. In order to improve and make the technique more reliable, here we focused on the freezing phase, comparing two different protocols: i) Fast-freezing, where the semen is plunged directly into liquid nitrogen (LN group); ii) Slow-freezing, where the sample is progressively cooled to a final temperature of -50°C (SL group). Briefly, for the preparation of the LN group sample the protocol reported in [2] was followed, while for the SL group the semen was frozen with a freezing rate of 1°C/min until -50°C degrees, when the sample was placed inside the lyophilizer. Dry spermatozoa from both groups was used for Intracytoplasmic Sperm Injection (ICSI) and the embryo development was evaluated at 24h (2-Cells stage) and 7 days (expanded blastocyst) after fertilization. At 24h post fertilization the SL-group showed a higher number of cleaved embryos than LN-group (42/100 (42%) versus 19/75 (25.3%), P=0.0253, SL and LN respectively). At 7 days after fertilization the blastocyst rate in SL-group was higher (7/100 (7%)) than in LN-group (2/75 (2.7%)), although not statistically different. Our data shows that lyophilisation can be conveniently achieved in ram spermatozoa without previous freezing in liquid nitrogen, thus simplifying the procedure. This data supports the idea that lyophilisation might be a valuable and cheaper alternative to liquid nitrogen for long-term storage of ram semen

Short spermatozoa–oocyte co-incubation improves outcomes of IVF in sheep

The assisted reproductive technique IVF is routinely applied in humans and large animals, both to boost reproductive performance and also for basic research. Despite its value, IVF has seen very little progress in the last two decades and relies on established paradigms, such as overnight sperm–egg co-incubation. However, the long exposure of oocytes to spermatozoa in a dish increases the risk of polyspermy and could be detrimental for early stages of embryonic development. We identified a time window within which fertilization occurs, in order to reduce the length of sperm–egg co-incubation and optimize the procedure, comparing polyspermy rate and embryo development after short (shIVF) and overnight (o/nIVF) spermatozoa–oocyte co-incubation. A total of 666 in vitro–matured sheep oocytes were co-incubated with spermatozoa in IVF medium (synthetic oviductal fluid (SOF) with 20% oestrus sheep serum and 16 µM isoproterenol). First, small batches of oocytes were collected every 30 min to check for the presence of a fertilizing spermatozoon. To assess this, cumulus cells were removed and presumptive fertilized oocytes were fixed and stained with propidium iodide for nuclei and Pisum sativum agglutinin for zona pellucida (ZP) detection, respectively. Then, pronuclear formation (PN) and embryo development were evaluated after 16 h (PN), 24 h (2 cells), and 7 days of culture (blastocyst). The oocytes that were not cleaved at 24 h were stained for DNA content with Hoechst 33342. Furthermore, we evaluated embryo quality by counting cells of 8-day blastocysts after differential staining of inner cell mass (ICM) and trophectoderm (TE). We found that spermatozoa reach the ZP no earlier than 90 min from the beginning of co-incubation and achieve fertilization within 4 h. Polyspermic fertilization (>2PN) was lower in shIVF (6.5%) than in o/nIVF (17.8%; P = 0.006). This proportion of polyspermy was maintained between groups in noncleaved oocytes at 24 h from fertilization. Likewise, cleavage and blastocyst rate were higher in shIVF compared with the o/n-IVF group (2-cells: 48.3% vs. 31.6%, P = 0.001; blastocyst: 29.4% vs. 20.5%, P = 0.046, respectively). Differential staining of blastocysts revealed no significant difference in cell number between the blastocysts of the two groups. This work demonstrates that 4 h of sperm-egg interaction are sufficient to achieve fertilization, reduce polyspermy, and improve the rate of embryos reaching blastocyst stage without compromising embryo quality

Somatic cell nuclear transfer: failures, successes and the challenges ahead

Somatic cell nuclear transfer (SCNT) has a broad spectrum of potential applications, including rescue of endangered species, production of transgenic animals, drug production, and regenerative medicine. Unfortunately, the efficiency of SCNT is still disappointingly low. Many factors affecting cloning procedures have been described in several previous reviews; here we review the most effective improvements in SCNT, with a special emphasis on the effect of mitochondrial defects on SCNT embryo/ foetus development, an issue never touched upon before

Freeze-dried spermatozoa: An alternative biobanking option for endangered species

In addition to the iconic wild species, such as the pandas and Siberian tigers, an ever-increasing number of domestic species are also threatened with extinction. Biobanking of spermatozoa could preserve genetic heritages of extinct species, and maintain biodiversity of existing species. Because lyophilized spermatozoa retain fertilizing capacity, the aim was to assess whether freezedried spermatozoa are an alternative option to save endangered sheep breeds. To achieve this objective, semen was collected from an Italian endangered sheep breed (Pagliarola), and a biobank of cryopreserved and freeze-dried spermatozoa was established, and evaluated using IVF (for frozen spermatozoa) and ICSI procedures (for frozen and freeze-dried spermatozoa). As expected, the fertilizing capacity of cryopreserved Pagliarola’s spermatozoa was comparable to commercial semen stocks. To evaluate the activating capability of freeze-dried spermatozoa, 108 MII sheep oocytes were subjected to ICSI, and allocated to two groups: 56 oocytes were activated by incubation with ionomycin (ICSI-FDSa) and 52 were not activated (ICSI-FDSna). Pronuclear formation (2PN) was investigated at 14–16 h after ICSI in fixed presumptive zygotes. Only artificially activated oocytes developed into blastocysts after ICSI. In the present study, freeze-dried ram spermatozoa induced blastocyst development following ICSI at a relatively high proportion, providing evidence that sperm lyophilization is an alternative, low cost storage option for biodiversity preservation of domestic species

Controlled spermatozoa-oocyte interaction improves embryo quality in sheep

The current protocols of in vitro fertilization and culture in sheep rely on paradigms established more than 25 years ago, where Metaphase II oocytes are co-incubated with capacitated spermatozoa overnight. While this approach maximizes the number of fertilized oocytes, on the other side it exposes them to high concentration of reactive oxygen species (ROS) generated by active and degenerating spermatozoa, and positively correlates with polyspermy. Here we set up to precisely define the time frame during which spermatozoa effectively penetrates and fertilizes the oocyte, in order to drastically reduce spermatozoa-oocyte interaction. To do that, in vitro matured sheep oocytes co-incubated with spermatozoa in IVF medium were sampled every 30 min (start of incubation time 0) to verify the presence of a fertilizing spermatozoon. Having defined the fertilization time frame (4 h, data from 105 oocytes), we next compared the standard IVF procedures overnight (about 16 h spermatozoa/oocyte exposure, group o/nIVF) with a short one (4 h, group shIVF). A lower polyspermic fertilization (&gt; 2PN) was detected in shIVF (6.5%) compared to o/nIVF (17.8%), P &lt; 0.05. The o/nIVF group resulted in a significantly lower 2-cell stage embryos, than shIVF [34.6% (81/234) vs 50.6% (122/241) respectively, P &lt; 0.001]. Likewise, the development to blastocyst stage confirmed a better quality [29% (70/241) vs 23.5% (55/234), shIVF vs o/nIVF respectively] and an increased Total Cell Number (TCN) in shIVF embryos, compared with o/n ones. The data on ROS have confirmed that its generation is IVF time-dependent, with high levels in the o/nIVF group. Overall, the data suggest that a shorter oocyte-spermatozoa incubation results in an improved embryo production and a better embryo quality, very likely as a consequence of a shorter exposure to the free oxygen radicals and the ensuing oxidative stress imposed by overnight culture

The impaired development of sheep ICSI derived embryos is not related to centriole dysfunction

While intracytoplasmic sperm injection (ICSI) is an asset in human Assisted Reproduction Technologies (ART), its outcomes, in terms of blastocyst, is still unacceptably low in ruminants. The picture typically found in ICSI derived bovine and ovine embryos is an asymmetry between a high activation rate, marked by a pronuclear development, and a low first cleavage rate. Abnormal centriole function has been indicated as a possible factor which undermines embryonic development following ICSI, especially when Freeze Dried spermatozoa (FD) are used. In order to verify the hypothesis that centriole dysfunction might be responsible for low ICSI outcomes in sheep, we have investigated micro-tubular dynamics, markedly aster nucleation, in fertilized sheep zygotes by ICSI with frozen/thawed (FT) and FD spermatozoa; In Vitro Fertilized (IVF) sheep oocytes were used as control. The spermatozoa aster nucleation was assessed at different time points following ICSI and IVF by immune-detection of a-tubulin. Pronuclear stage, syngamy and embryo development were assessed. No difference was noticed in the timing of aster nucleation and microtubule elongation in ICSI-FT derived embryos with control IVF ones, while a delay was recorded in ICSI-FD ones. The proportion of 2-pronuclear stage zygotes was similar in ICSI-FT and ICSI-FD (47% and 53%, respectively), both much lower comparing the IVF ones (73%). Likewise, syngamy was observed in a minority of both ICSI groups (28.5% vs 12.5% in ICSI-FT/FD respectively) comparing to IVF controls (50%), with a high number of zygotes blocked at the 2-pronuclear stage (71.5% vs 87.5% respectively). While no significant differences were noticed in the cleavage rate between ICSI-FD, ICSI-FT and IVF groups (31%, 34% and 44%) respectively, development to blastocyst stage was markedly compromised in both ICSI groups, especially with FD spermatozoa (10% in ICIS-FD and 19% in ICSI-FT vs 33% in IVF (P &lt; 0.005, ICSI-FD vs IVF and P &lt; 0.05, IVF vs ICSI-FT, respectively). Hence, here we have demonstrated that the reduced cleavage, and the ensuing impaired development to blastocysts stage of ICSI derived sheep embryos is not related to centriole dysfunction, as suggested by other authors. The major recorded problem is the lack of syngamy in ICSI derived zygotes, an issue that should be addressed in further studies to improve ICSI procedure in sheep embryos