Cooperative Reinforcement Learning Using an Expert-Measuring Weighted Strategy with WoLF

Gradient descent learning algorithms have proven effective in solving mixed strategy games. The policy hill climbing (PHC) variants of WoLF (Win or Learn Fast) and PDWoLF (Policy Dynamics based WoLF) have both shown rapid convergence to equilibrium solutions by increasing the accuracy of their gradient parameters over standard Q-learning. Likewise, cooperative learning techniques using weighted strategy sharing (WSS) and expertness measurements improve agent performance when multiple agents are solving a common goal. By combining these cooperative techniques with fast gradient descent learning, an agent’s performance converges to a solution at an even faster rate. This statement is verified in a stochastic grid world environment using a limited visibility hunter-prey model with random and intelligent prey. Among five different expertness measurements, cooperative learning using each PHC algorithm converges faster than independent learning when agents strictly learn from better performing agents

WoLF Ant

Ant colony optimization (ACO) algorithms can generate quality solutions to combinatorial optimization problems. However, like many stochastic algorithms, the quality of solutions worsen as problem sizes grow. In an effort to increase performance, we added the variable step size off-policy hill-climbing algorithm called PDWoLF (Policy Dynamics Win or Learn Fast) to several ant colony algorithms: Ant System, Ant Colony System, Elitist-Ant System, Rank-based Ant System, and Max-Min Ant System. Easily integrated into each ACO algorithm, the PDWoLF component maintains a set of policies separate from the ant colony\u27s pheromone. Similar to pheromone but with different update rules, the PDWoLF policies provide a second estimation of solution quality and guide the construction of solutions. Experiments on large traveling salesman problems (TSPs) show that incorporating PDWoLF with the aforementioned ACO algorithms that do not make use of local optimizations produces shorter tours than the ACO algorithms alone

A Temporal -omic Study of Propionibacterium freudenreichii CIRM-BIA1T Adaptation Strategies in Conditions Mimicking Cheese Ripening in the Cold

Propionibacterium freudenreichii is used as a ripening culture in Swiss cheese manufacture. It grows when cheeses are ripened in a warm room (about 24°C). Cheeses with an acceptable eye formation level are transferred to a cold room (about 4°C), inducing a marked slowdown of propionic fermentation, but P. freudenreichii remains active in the cold. To investigate the P. freudenreichii strategies of adaptation and survival in the cold, we performed the first global gene expression profile for this species. The time-course transcriptomic response of P. freudenreichii CIRM-BIA1T strain was analyzed at five times of incubation, during growth at 30°C then for 9 days at 4°C, under conditions preventing nutrient starvation. Gene expression was also confirmed by RT-qPCR for 28 genes. In addition, proteomic experiments were carried out and the main metabolites were quantified. Microarray analysis revealed that 565 genes (25% of the protein-coding sequences of P. freudenreichii genome) were differentially expressed during transition from 30°C to 4°C (P<0.05 and |fold change|>1). At 4°C, a general slowing down was observed for genes implicated in the cell machinery. On the contrary, P. freudenreichii CIRM-BIA1T strain over-expressed genes involved in lactate, alanine and serine conversion to pyruvate, in gluconeogenesis, and in glycogen synthesis. Interestingly, the expression of different genes involved in the formation of important cheese flavor compounds, remained unchanged at 4°C. This could explain the contribution of P. freudenreichii to cheese ripening even in the cold. In conclusion, P. freudenreichii remains metabolically active at 4°C and induces pathways to maintain its long-term survival

Feeding behaviour and digestion physiology in larval fish – current knowledge and gaps and bottlenecks in research

Food uptake follows rules defined by feeding behaviour that determines the kind and quantity of food ingested by fish larvae as well as how live prey and food particles are detected, captured and ingested. Feeding success depends on the progressive development of anatomical characteristics and physiological functions and on the availability of suitable food items throughout larval development. The fish larval stages present eco-morpho-physiological features very different from adults and differ from one species to another. The organoleptic properties, dimensions, detectability, movements characteristics and buoyancy of food items are all crucial features that should be considered, but is often ignored, in feeding regimes. Ontogenetic changes in digestive function lead to limitations in the ability to process certain feedstuffs. There is still a lack of knowledge about the digestion and absorption of various nutrients and about the ontogeny of basic physiological mechanisms in fish larvae, including how they are affected by genetic, dietary and environmental factors. The neural and hormonal regulation of the digestive process and of appetite is critical for optimizing digestion. These processes are still poorly described in fish larvae and attempts to develop optimal feeding regimes are often still on a ‘trial and error’ basis. A holistic understanding of feeding ecology and digestive functions is important for designing diets for fish larvae and the adaptation of rearing conditions to meet requirements for the best presentation of prey and microdiets, and their optimal ingestion, digestion and absorption. More research that targets gaps in our knowledge should advance larval rearing

Sylva narrationum Gilberti Cognati Nozereni, Diligenter purgata, et uberior reddita : adiecta sunt eiusdem argumenti & quaedam alia, lectu perquam iucunda

[s.c.