Learning from Outside the Viability Kernel: Why we Should Build Robots that can Fall with Grace

Despite impressive results using reinforcement learning to solve complex problems from scratch, in robotics this has still been largely limited to model-based learning with very informative reward functions. One of the major challenges is that the reward landscape often has large patches with no gradient, making it difficult to sample gradients effectively. We show here that the robot state-initialization can have a more important effect on the reward landscape than is generally expected. In particular, we show the counter-intuitive benefit of including initializations that are unviable, in other words initializing in states that are doomed to fail.Comment: Proceedings of the 2018 IEEE International Conference on SImulation, Modeling and Programming for Autonomous Robots (SIMPAR), Brisbane, Australia, 16-19 201

Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics

Properly designing a system to exhibit favorable natural dynamics can greatly simplify designing or learning the control policy. However, it is still unclear what constitutes favorable natural dynamics and how to quantify its effect. Most studies of simple walking and running models have focused on the basins of attraction of passive limit-cycles and the notion of self-stability. We instead emphasize the importance of stepping beyond basins of attraction. We show an approach based on viability theory to quantify robust sets in state-action space. These sets are valid for the family of all robust control policies, which allows us to quantify the robustness inherent to the natural dynamics before designing the control policy or specifying a control objective. We illustrate our formulation using spring-mass models, simple low dimensional models of running systems. We then show an example application by optimizing robustness of a simulated planar monoped, using a gradient-free optimization scheme. Both case studies result in a nonlinear effective stiffness providing more robustness.Comment: 15 pages. This work has been accepted to IEEE Transactions on Robotics (2019

Records on the vanishing of Fourier coefficients of Powers Of the Dedekind Eta Function

In this paper we significantly extend Serre's table on the vanishing properties of Fourier coefficients of odd powers of the Dedekind eta function. We address several conjectures of Cohen and Str\"omberg and give a partial answer to a question of Ono. In the even-power case, we extend Lehmer's conjecture on the coefficients of the discriminant function $\Delta$ to all non-CM-forms. All our results are supported with numerical data. For example all Fourier coefficients $a_9(n)$ of the $9$-th power of the Dedekind eta function are non-vanishing for $n \leq 10^{10}$. We also relate the non-vanishing of the Fourier coefficients of $\Delta^2$ to Maeda's conjecture.Comment: 13 page

Shaping in Practice: Training Wheels to Learn Fast Hopping Directly in Hardware

Learning instead of designing robot controllers can greatly reduce engineering effort required, while also emphasizing robustness. Despite considerable progress in simulation, applying learning directly in hardware is still challenging, in part due to the necessity to explore potentially unstable parameters. We explore the concept of shaping the reward landscape with training wheels: temporary modifications of the physical hardware that facilitate learning. We demonstrate the concept with a robot leg mounted on a boom learning to hop fast. This proof of concept embodies typical challenges such as instability and contact, while being simple enough to empirically map out and visualize the reward landscape. Based on our results we propose three criteria for designing effective training wheels for learning in robotics. A video synopsis can be found at https://youtu.be/6iH5E3LrYh8.Comment: Accepted to the IEEE International Conference on Robotics and Automation (ICRA) 2018, 6 pages, 6 figure

Early stage litter decomposition rates for Swiss forests

The decomposition of belowground and aboveground tree litter was studied on five forest sites across Switzerland, ranging from 480 to 1500 m in altitude, and including calcareous and acidic soils. In addition to decomposition of local litter types (Picea abies, Fagus sylvatica, Castanea sativa), the decomposition of a standard beech litter was studied on all sites. After 2years of decomposition, mass loss ranged from 18 to 71% across the different sites and litter types. The lowest decomposition rates were observed for beech roots, while mass loss was greatest for both spruce needles and spruce roots at the low-altitude site. Mass loss during the first winter correlated best with the content of water-soluble substances. After 1year of incubation, mass loss of the standard litter varied less than did mass loss of local litter, but variance increased during the second year for aboveground litter. These observations indicate a smaller climatic influence on litter breakdown at the beginning of the decomposition process. Litter mass loss could be described using an exponential model with a decay constant depending on either lignin/N ratio or Mn content of the litter and annual soil temperature and throughfall precipitation as climatic variables. Modelling the observed mass loss indicated a strong influence of litter quality in the first 2years of decomposition, confirming the field data from the standard litter experiment. The experiment will continue for some years and is expected to yield additional data on long-term decompositio

Lignin content and chemical characteristics in maize and wheat vary between plant organs and growth stages: consequences for assessing lignin dynamics in soil

Assessing lignin turnover in soil on the basis of a 13C natural abundance labeling approach relies on the assumption that chemical characteristics of labeled and control plant inputs are similar and that the 13C content difference between labeled and control plant inputs is constant within the plant parts. We analyzed lignin in soils, roots, stems and leaves of wheat and maize at different stages of growth using the cupric oxide oxidation method. In both plants, lignin concentrations increased with growth, particularly during grain filling. Maize contained more cinnamyl moieties than wheat. Roots had higher lignin contents (especially cinnamyl moieties) than stems and leaves, and seemed to contribute more to the total soil lignin than the aboveground parts. The isotopic differences (∆ δ13C) of lignin phenols were not significantly different (p > 0.05) between plant organs, confirming assumptions underlying the natural abundance 13C labeling approach. Our data show that lignin content and phenol distribution can vary between plant organs and with the time of harvest. Consequently, the amount of annual lignin input may vary as a function of root amount and harvest date, and thus can affect the calculated apparent turnover times of lignin in natural abundance 13C labeling experiment

What about realized returns in reward-based crowdfunding?

Crowdfunding is an alternative form of financing, especially for entrepreneurs in the early-stage development phase. It is well-known that for investors, beyond altruistic motives, material returns play a crucial role, too. A previous study in this field analyzed returns offered by founders and demanded by investors, resp., by means of a data set from kickstarter.com, a leading platform for reward-based crowdfunding. The present paper additionally looks at the realized returns from an ex post view. It does not present new empirical findings, but identifies problems and limitations in this field, and presents some suggestions for future research

Fluxgate magnetorelaxometry of magnetic nanoparticles in bioanalytics

Magnetische Nanopartikel werden in vielen Bereichen der Medizin und Biotechnologie eingesetzt. Die bekanntesten Anwendungen sind der Einsatz als Kontrastmittel in der Kernspintomografie und zur Separation und Aufreinigung von Biomolekülen. Für alle Anwendungen werden spezialisierte Nanopartikel benötigt. Die Spezialisierung der Nanopartikel kann eine bestimmte Größe und Größenverteilung oder eine Funktionalisierung zur spezifischen Bindung an Zielsubstanzen sein. In der Bioanalytik können die magnetischen Relaxationseigenschaften superparamagnetischer Nanopartikel für ein Nachweisverfahren von Biomolekülen genutzt werden. In dieser Arbeit wurde ein Messgerät entwickelt und charakterisiert, mit dem die Magnetrelaxation superparamagnetischer Nanopartikel mit hochempfindlichen Magnetfeldsensoren ohne aufwendige, magnetische Abschirmung bei Raumtemperatur gemessen werden kann. Für die Messung und die Auswertung der Messdaten wurden Programme entwickelt, und die Messunsicherheit sowie die Nachweisgrenze des Systems wurden im Hinblick auf Messungen von kleinsten Konzentrationen an magnetischen Nanopartikeln und Biomolekülen bestimmt. Mit dem aufgebauten Messsystem wurden Bindungsexperimente mit superparamagnetischen Nanopartikeln an verschieden großen Zielsubstanzen durchgeführt, wobei Bindungskinetiken untersucht werden konnten. Im Rahmen des Sonderforschungsbereichs 578 (vom Gen zum Produkt) wurde das Verfahren der Magnetrelaxometrie erstmals eingesetzt, um die Vernetzungskinetik und Einbettungskapazität von Hydrogelen zu bestimmen. Nachdem die Messeinheit in einem ersten Forschungssystem getestet und eingesetzt worden war, wurde ein Demonstrator eines kompletten Messsystems inklusive Messdatenauswertung in der Größe eines 19"-Gehäuses aufgebaut. Ein solches Laborgerät ermöglicht die kostengünstige Qualitätssicherung und Anwendung von magnetischen Nanopartikeln in der Forschung und Industrie.Magnetic nanoparticles find wide application in medicine and biotechnology. Established applications include the use of magnetic nanoparticles as contrast agents for magnetic resonance imaging and for the separation and purification of biomolecules. The nanoparticles have to be tailored for a given application which, for example, requires a defined size and size distribution or a functionalization to specifically bind to targets. In bioanalytics, the magnetic properties of superparamagnetic nanoparticles can be used to detect biomolecules. A measurement system was developed and characterized that measures the magnetic relaxation with highly sensitive magnetic field sensors at room temperature without the need for costly cooling and magnetic shielding. To control the measurement cycle and to analyze the magnetorelaxation data, software applications were developed. The measurement uncertainty and the detection limits of the system were determined with respect to measuring small amounts of magnetic nanoparticles and biomolecules. With this measurement system, binding experiments with superparamagnetic nanoparticles and targets of different sizes were performed. Furthermore, binding kinetics were investigated. Within the scope of the Sonderforschungsbereich 578 (from gene to product), the method of magnetorelaxation measurement was used to characterize the load capacities and kinetics of polymerization of hydrogels. After the measurement unit had been tested successfully, a laboratory instrument was developed which integrates the measurement coil system and electronics together with a computer in a single 19" instrument. With this instrument, the characterization of magnetic nanoparticles in terms of size distribution and agglomeration as well as magnetic properties can be investigated. Such a laboratory instrument allows the cost-effective quality control of magnetic nanoparticles and their application in research and industry

Web-based system for assessment of learning achievements outcomes

The description of authoring educational web-based testing system is presented in the article. The proposed web-based system provides the process of inputting the test material and implementation in academic activity. The aim of the web-based system is to improve learning efficiency with a help of methods presented to form students' knowledge structure and estimate the students' level of learning throughout academic activity