Understanding Policy Gradient Algorithms: A Sensitivity-Based Approach

The REINFORCE algorithm from Williams is popular in policy gradient (PG) for solving reinforcement learning (RL) problems. Meanwhile, the theoretical form of PG is from Sutton et al. Although both formulae prescribe PG, their precise connections are not yet illustrated. Recently, Nota and Thomas (2020) have found that the ambiguity causes implementation errors. Motivated by the ambiguity and implementation incorrectness, we study PG from a perturbation perspective. In particular, we derive PG in a unified framework, precisely clarify the relation between PG implementation and theory, and echo back the findings by Nota and Thomas. Diving into factors contributing to empirical successes of the existing erroneous implementations, we find that small approximation error and the experience replay mechanism play critical roles

Rare appearance of Candida tropicalis infection of the brain: Multiple micro-abscesses combined with diffuse hemorrhages

AbstractWe report a case of cerebral Candida tropicalis infection in a middle-aged patient who suffered from multiple cerebral micro-abscesses associated with diffuse hemorrhage due to perforation of esophagus. MRI revealed multiple irregular, nodular, ring-like enhancing lesions with restricted diffusion and multiple micro-hemorrhages as well as some leptomeningeal enhancements. Blood, sputum and urine cultures showed Candida tropicalis. The lesions were resolved after the patient was given early and effective treatment of anti-fungal medicine. The imaging findings provided limited differential diagnosis, leading to early diagnosis and treatment for this patient

Direct Shooting Method for Numerical Optimal Control: A Modified Transcription Approach

Direct shooting is an efficient method to solve numerical optimal control. It utilizes the Runge-Kutta scheme to discretize a continuous-time optimal control problem making the problem solvable by nonlinear programming solvers. However, conventional direct shooting raises a contradictory dynamics issue when using an augmented state to handle {high-order} systems. This paper fills the research gap by considering the direct shooting method for {high-order} systems. We derive the modified Euler and Runge-Kutta-4 methods to transcribe the system dynamics constraint directly. Additionally, we provide the global error upper bounds of our proposed methods. A set of benchmark optimal control problems shows that our methods provide more accurate solutions than existing approaches.Comment: Accepted by ECC2

1-[1-(Hydroxy­imino)eth­yl]-N-(2-methoxy­phen­yl)cyclo­propane­carboxamide

The title compound, C13H16N2O3, adopts an E configuration with respect to the C=N bond and an intra­molecular N—H⋯N hydrogen bond results in the formation of a six-membered ring. In the crystal, inter­molecular O—H⋯O hydrogen bonds link the mol­ecules into a chain propagating along the b axis. Very weak π–π stacking inter­actions [centroid–centroid distance = 4.18 (2) Å] may further consolidate the packing, forming a two-dimensional supra­molecular network

The clinicopathological factors associated with disease progression in Luminal a breast cancer and characteristics of metastasis: A retrospective study from a single center in China

Background/Aim: This study investigated the clinicopathological factors associated with outcomes in patients with Luminal A breast cancer. Patients and Methods: Retrospective analysis of the association of clinicopathological factors and breast cancer outcome in 421 patients with newly diagnosed Luminal-A breast cancer that were enrolled from January 2008 to December 2014. Clinicopathological data were analyzed to validate the relationship with disease free survival (DFS) and overall survival (OS). Kaplan-Meier curves and log-rank tests were used to analyze the value of clinicopathological factors (tumor size, node status and lymphovascular invasion), and subsequent Cox regression analysis revealed significant prognostic factors. Results: With a median of 61 months follow up, the 5-year DFS and 5-year OS rate were 98.3% and 99.3%. Cox multivariate regression analysis showed that clinical anatomic stage, tumor size, status of lymph nodes, lymphovascular invasion and systemic treatment are strong prognostic factors for clinical outcome in patients with Luminal-A breast cancer. Of all 413 patients with stage I-III breast cancer, 14 presented with metastasis (3.4%) during the follow up. Bone (6/14, 42.9%) was the most common site of metastasis followed by liver (5/14, 35.7%) and lung (4/14, 28.6%). The median survival time after metastasis was 20.4 months. Of all the sites of distant metastasis, liver metastasis was the only factor that affected survival time after metastasis (χ2=6.263, p=0.012). Conclusion: Patients with Luminal A breast cancer have excellent outcomes. Liver metastasis is an important factor compressing the survival time after distant metastasis presents

Activation of the Extracellular Signal-Regulated Kinase Signaling Is Critical for Human Umbilical Cord Mesenchymal Stem Cell Osteogenic Differentiation

Human umbilical cord mesenchymal stem cells (hUCMSCs) are recognized as candidate progenitor cells for bone regeneration. However, the mechanism of hUCMSC osteogenesis remains unclear. In this study, we revealed that mitogen-activated protein kinases (MAPKs) signaling is involved in hUCMSC osteogenic differentiation in vitro. Particularly, the activation of c-Jun N-terminal kinases (JNK) and p38 signaling pathways maintained a consistent level in hUCMSCs through the entire 21-day osteogenic differentiation period. At the same time, the activation of extracellular signal-regulated kinases (ERK) signaling significantly increased from day 5, peaked at day 9, and declined thereafter. Moreover, gene profiling of osteogenic markers, alkaline phosphatase (ALP) activity measurement, and alizarin red staining demonstrated that the application of U0126, a specific inhibitor for ERK activation, completely prohibited hUCMSC osteogenic differentiation. However, when U0126 was removed from the culture at day 9, ERK activation and osteogenic differentiation of hUCMSCs were partially recovered. Together, these findings demonstrate that the activation of ERK signaling is essential for hUCMSC osteogenic differentiation, which points out the significance of ERK signaling pathway to regulate the osteogenic differentiation of hUCMSCs as an alternative cell source for bone tissue engineering. Copyright © 2016 Chen-Shuang Li et al

Case report: Synchronous tumors of the female reproductive tract in systemic lupus erythematosus: report of two cases and review of the literature

BackgroundSystemic lupus erythematosus (SLE) is a chronic autoimmune disease that affects multiple systems. Patients with SLE are prone to a variety of malignancies, especially neoplasms of the female reproductive tract. Synchronous tumors, considered to involve multiple sites, are rare in the female reproductive tract. There are hardly any reports of SLE with synchronous reproductive tract tumors.Case presentationWe report the occurrence of two to three reproductive tract tumors in two women with SLE. A 52-year-old woman was diagnosed with vulvar cancer and cervical cancer. Another woman, aged 67, was diagnosed with concurrent vulvar cancer, vaginal cancer, and cervical cancer and also presented with a suspected lung cancer.ConclusionThe presence of synchronous tumors of the reproductive tract in patients with SLE is uncommon and can be easily disregarded. It is crucial to highlight that SLE patients with multiple primary malignancies exhibit notable late-stage presentation at the time of diagnosis, inadequate disease-free survival, poor overall survival, rapid progression rates, and mortality. Consequently, greater awareness must be raised regarding synchronous reproductive tract tumors in patients with SLE. Regular comprehensive cancer screening and management should be implemented for individuals diagnosed with SLE