Smart Containers With Bidding Capacity: A Policy Gradient Algorithm for Semi-Cooperative Learning

Smart modular freight containers -- as propagated in the Physical Internet paradigm -- are equipped with sensors, data storage capability and intelligence that enable them to route themselves from origin to destination without manual intervention or central governance. In this self-organizing setting, containers can autonomously place bids on transport services in a spot market setting. However, for individual containers it may be difficult to learn good bidding policies due to limited observations. By sharing information and costs between one another, smart containers can jointly learn bidding policies, even though simultaneously competing for the same transport capacity. We replicate this behavior by learning stochastic bidding policies in a semi-cooperative multi agent setting. To this end, we develop a reinforcement learning algorithm based on the policy gradient framework. Numerical experiments show that sharing solely bids and acceptance decisions leads to stable bidding policies. Additional system information only marginally improves performance; individual job properties suffice to place appropriate bids. Furthermore, we find that carriers may have incentives not to share information with the smart containers. The experiments give rise to several directions for follow-up research, in particular the interaction between smart containers and transport services in self-organizing logistics.Comment: 15 page

Nanoscale Precipitation, Recovery and Grain Growth in ODS Steel Particles

International audienc

Nanoscale Precipitation, Recovery and Grain Growth in ODS Steel Particles

International audienc

On ball-milled ODS ferritic steel recrystallization: From as-milled powder particles to consolidated state

International audienceRecrystallization of a ball-milled ferritic ODS steel is studied towards its evolution from as-milled powder to consolidated state. This characterization has been made possible by using a combination of X-ray Diffraction (XRD) and an innovative method based on an Automated Crystallographic Orientation Mapping (ACOM) tool attached to a Transmission Electron Microscope (TEM). Focus Ion Beam preparation has been essential to obtain a thin section of the ODS steel powder particle and perform the ACOM-TEM study. Relevant temperatures regarding recovery and recrystallization during the heat treatment had first been identified with XRD profile analysis. Selected states were further characterized using ACOM-TEM that provides key information on microstructure, i.e. grain size and morphology, crystallite size, local texture and distortion. ACOM-TEM cartographies have revealed for the first time that the microstructure of as-milled ODS ferritic steel particles consists in very anisotropic grains containing undistorted domains and dislocation walls. This is in agreement with the nanosized crystallites measured by XRD results. The mutual benefits of XRD and ACOM-TEM methods to analyse and describe the microstructure are discussed as well as the reliability of dislocation density measurements provided by ACOM-TEM misorientation measurements. In addition, of the ACOM-TEM results, the microstructural evolution during the processing route is interpreted in terms of a competition between recovery, recrystallization, grain growth and precipitation

In situ characterization of microstructural instabilities: Recovery, recrystallization and abnormal growth in nanoreinforced steel powder

International audience: An in situ X-ray diffraction experiment was set up to study the microstructural evolution of a nanostructured oxide dispersion-strengthened ferritic steel produced by high-energy ball milling. Dislocation density and grain growth between 20 and 1100 degrees C were quantified by coupling-modified Williamson-Hall and Warren-Averbach methods. During the early stages of heating, recovery through the rearrangement of dislocations increases the coherent domain size from 23 to about 60 nm. Once the annealing temperature reaches 800 degrees C, recrystallization starts. Using a specific analysis of 2-D detector signal, it has been possible to grasp the occurrence of abnormal growth leading to bimodal grain size distribution with both ultrafine grains and coarser micronic grains. The grain growth kinetics upon heating were determined for both populations and separately quantified. Ultrafine grains exhibit a continuous moderate growth rate, leading to continuous recrystallization, whereas specific grains experience a rapid abnormal growth up to their final size after a short incubation time

Influence of oxide volume fraction on abnormal growth of nanostructured ferritic steels during non-isothermal treatments: An in situ study

International audienceNanostructured ferritic steels were mechanically alloyed with various contents of oxide-forming yttrium and titanium (0, 0.05, 0.3 and 1 wt%). The microstructure evolution of the milled powders during non-isothermal annealing treatments was studied using in situ synchrotron X-ray diffraction. Recrystallization and grain growth were quantified upon heating up to 1100 degrees C, which is the typical consolidation temperature for nanostructured ferritic steels. The temperature where abnormal grain growth occurs is observed to increase with the volume fraction of oxide nanoparticles. This demonstrates the interest of increasing the amount of alloying elements to limit the formation of the bi-modal grain microstructure. Using the nanoscale characterization of the precipitation state, the size of retained ultrafine grains (UFG) in the bimodal microstructure was found to be in agreement with the modified Zener theory demonstrating that the microstructure of ultrafine-grained steels can be tailored by the amount and size of second-phase particle