Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. An open problem is the sample-inefficiency that stems from the compositionality of natural language, and from the grounding of language in sensory data and actions. We address these issues with three contributions. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions. Finally, we present a novel class of language-focused learning tasks. We show that hindsight instructions improve the learning performance, as expected. In addition, we also provide an unexpected result: We show that the learning performance of our agent can be improved by one third if, in a sense, the agent learns to talk to itself in a self-supervised manner. We achieve this by learning to generate linguistic instructions that would have been appropriate as a natural language goal for an originally unintended behavior. Our results indicate that the performance gain increases with the task-complexity.Comment: Preprint ICDL 202

Intraoperative electron radiation therapy (IOERT) in patients with locally recurrent renal cell carcinoma

Background: To analyze our experience with intraoperative electron radiation therapy (IOERT) followed by moderate doses of external beam radiation therapy (EBRT) in patients with locally recurrent renal cell carcinoma. Methods: From 1992 to 2010, 17 patients with histologically proven, locally recurrent renal cell carcinoma (median tumor size 7 cm) were treated by surgery and IOERT with a median dose of 15 Gy. All patients met the premise of curative intent including 7 patients with oligometastases at the time of recurrent surgery, which were resected and/or irradiated. The median time interval from primary surgery to local recurrence was 26 months. Eleven patients received additional 3D-conformal EBRT with a median dose of 40 Gy. Results: Surgery resulted in free but close margins in 6 patients (R0), while 9 patients suffered from microscopic (R1) and 2 patients from macroscopic (R2) residual disease. After a median follow-up of 18 months, two local recurrences were observed, resulting in an actuarial 2-year local control rate of 91%. Eight patients developed distant failures, predominantly to liver and bone, resulting in an actuarial 2-year progression free survival of 32%. An improved PFS rate was found in patients with a larger time interval between initial surgery and recurrence (> 26 months). The actuarial 2-year overall survival rate was 73%. Lower histological grading (G1/2) was the only factor associated with improved overall survival. Perioperative complications were found in 4 patients. No IOERT specific late toxicities were observed. Conclusions: Combination of surgery, IOERT and EBRT resulted in high local control rates with low toxicity in patients with locally recurrent renal cell cancer despite an unfavorable surgical outcome in the majority of patients. However, progression-free and overall survival were still limited due to a high distant failure rate, indicating the need for intensified systemic treatment especially in patients with high tumor grading and short interval to recurrence

Radiographic and safety details of vertebral body stenting : results from a multicenter chart review

Background: Up to one third of BKP treated cases shows no appreciable height restoration due to loss of both restored height and kyphotic realignment after balloon deflation. This shortcoming has called for an improved method that maintains the height and realignment reached by the fully inflated balloon until stabilization of the vertebral body by PMMA-based cementation. Restoration of the physiological vertebral body height for pain relief and for preventing further fractures of adjacent and distant vertebral bodies must be the main aim for such a method. A new vertebral body stenting system (VBS) stabilizes the vertebral body after balloon deflation until cementation. The radiographic and safety results of the first 100 cases where VBS was applied are presented. Methods: During the planning phase of an ongoing international multicenter RCT, radiographic, procedural and followup details were retrospectively transcribed from charts and xrays for developing and testing the case report forms. Radiographs were centrally assessed at the institution of the first/senior author. Results: 100 patients (62 with osteoporosis) with a total of 103 fractured vertebral bodies were treated with the VBS system. 49 were females with a mean age of 73.2 years; males were 66.7 years old. The mean preoperative anterior-middle-posterior heights were 20.3-17.6-28.0 mm, respectively. The mean local kyphotic angle was 13.1°. The mean preoperative Beck Index (anterior edge height/posterior edge height) was 0.73, the mean alternative Beck Index (middle height/posterior edge height) was 0.63. The mean postoperative heights were restored to 24.5-24.6-30.4 mm, respectively. The mean local kyphotic angle was reduced to 8.9°. The mean postoperative Beck Index was 0.81, the mean alternative one was 0.82. The overall extrusion rate was 29.1%, the symptomatic one was 1%. In the osteoporosis subgroup there were 23.8% extrusions. Within the three months followup interval there were 9% of adjacent and 4% of remote new fractures, all in the osteoporotic group. Conclusions: VBS showed its strengths especially in realignment of crush and biconcave fractures. Given that fracture mobility is present, the realignment potential is sound and increases with the severity of preoperative vertebral body deformation

Entwicklung des Datenangebots und deren Nachfrage in neun Jahren Forschungsdatenzentrum der Rentenversicherung (2004 bis 2012)

"Der vorliegende Aufsatz dokumentiert einerseits das unter Einhaltung des Datenschutzes und der Datensicherheit aufbereitete Datenangebot des Forschungsdatenzentrums der Ren-tenversicherung (FDZ-RV). Andererseits wird die Entwicklung der Nachfrage nach diesen Daten seit der Gründung des Forschungsdatenzentrums im Jahr 2004 skizziert. Das Daten-angebot wurde kontinuierlich erweitert und umfasst inzwischen Querschnitts- wie Längs-schnittsdaten der Renten-, Versicherten- sowie Rehabilitationsstatistiken. Die Entwicklung und Bereitstellung der FDZ-RV-Daten bietet den Wissenschaftlerinnen und Wissenschaftlern die Möglichkeit Analysen in den Themenfeldern Alterssicherung, Rehabilitationsleistungen, Erwerbsbiografien und weiterer Themen zu erforschen. Die Datenprodukte finden eine stei-gende Nachfrage: zunehmend mehr Wissen Schaffende verschiedener Disziplinen, die an Universitäten und Forschungsinstituten beschäftigt sind, beantragen die Mikrodaten des For-schungsdatenzentrums und gehen mit dem FDZ-RV Kooperationsprojekte ein. Die wach-sende Nachfrage nach Daten spiegelt sich auch in einem Zuwachs an Veröffentlichungen wider, der im beobachteten Zeitraum zu verzeichnen ist." [Autorenreferat] Inhalt: 1. Einleitung; 2. Datenangebot und Zugangswege; 3. Qualität der Daten; 4. Datennachfrage und auf dieser basierende Publikationen; 5. Forschungsschwerpunkte; 6. Zusammenfassun

Influence of Plasma-Isolated Anthocyanins and Their Metabolites on Cancer Cell Migration (HT-29 and Caco-2) In Vitro: Results of the ATTACH Study

Cancer mortality is mainly due to metastasis. Therefore, searching for new therapeutic agents suppressing cancer cell migration is crucial. Data from human studies regarding effects of anthocyanins on cancer progression, however, are scarce and it is unclear whether physiological concentrations of anthocyanins and their metabolites reduce cancer cell migration in vivo. In addition, interactions with chemotherapeutics like 5-fluorouracil (5-FU) are largely unknown. Thus, we combined a placebo-controlled, double-blinded, cross-over study with in vitro migration studies of colon cancer cell lines to examine the anti-migratory effects of plasma-isolated anthocyanins and their metabolites (PAM). Healthy volunteers (n = 35) daily consumed 0.33 L of an anthocyanin-rich grape/bilberry juice and an anthocyanin-depleted placebo juice for 28 days. PAM were isolated before and after intervention by solid-phase extraction. HT-29 and Caco-2 cells were incubated with PAM in a Boyden chamber. Migration of HT-29 cells was significantly inhibited by PAM from juice but not from placebo. In contrast, Caco-2 migration was not affected. Co-incubation with 5-FU and pooled PAM from volunteers (n = 10), which most effectively inhibited HT-29 migration, further reduced HT-29 migration in comparison to 5-FU alone. Therefore, PAM at physiological concentrations impairs colon cancer cell migration and may support the effectiveness of chemotherapeutics

Hole pairing and phonon dynamics in generalized 2D t−J−t-J-Holstein models

The formation of hole pairs in the planar t-J model is studied in the presence of independent {\it dynamic} vibrations of the in-plane oxygen atoms. In-plane (breathing modes) and out-of-plane (buckling modes) displacements are considered. We find strong evidences in favor of a stabilization of the two hole bound pair by out-of-plane vibrations of the in-plane oxygens. On the contrary, the breathing modes weaken the binding energy of the hole pair. These results are discussed in the context of the superconducting cuprates.Comment: 4 pages LaTeX and 6 PostScript figures in uuencoded for

Double exchange magnets: Spin-dynamics in the paramagnetic phase

The electronic structure of perovskite manganese oxides is investigated in terms of a Kondo lattice model with ferromagnetic Hund coupling and antiferromagnetic exchange between $t_{2g}$-spins using a finite temperature diagonalization technique. Results for the dynamic structure factor are consistent with recent neutron scattering experiments for the bilayer manganite La$_{1.2}$Sr$_{1.8}$Mn$_2$O$_7$ . The susceptibility shows Curie-Weiss behaviour and is used to derive a phase diagram. In the paramagnetic phase carriers are characterized as ferromagnetic polarons in an antiferromagnetic spin liquid.Comment: Revtex, 4 pages with 5 postscript figures include