38 research outputs found
Ising models of deep neural networks
This work maps deep neural networks to classical Ising spin models, allowing
them to be described using statistical thermodynamics. The density of states
shows that structures emerge in the weights after they have been trained --
well-trained networks span a much wider range of realizable energies compared
to poorly trained ones. These structures propagate throughout the entire
network and are not observed in individual layers. The energy values correlate
to performance on tasks, making it possible to distinguish networks based on
quality without access to data. Thermodynamic properties such as specific heat
are also studied, revealing a higher critical temperature in trained networks
A new look at calendar anomalies: Multifractality and day of the week effect
Stock markets can become inefficient due to calendar anomalies known as
day-of-the-week effect. Calendar anomalies are well-known in financial
literature, but the phenomena remain to be explored in econophysics. In this
paper we use multifractal analysis to evaluate if the temporal dynamics of
market returns also exhibits calendar anomalies such as day-of-the-week
effects. We apply the multifractal detrended fluctuation analysis (MF-DFA) to
daily returns of market indices around the world for each day of the week. Our
results indicate that individual days of the week are characterized by distinct
multifractal properties. Monday returns tend to exhibit more persistent
behavior and richer multifractal structures than other day-resolved returns.
Shuffling the series reveals that multifractality arises both from a broad
probability density function and from long-term correlations. From the
time-dependent multifractal analysis we find that multifractal spectra for
Monday returns are much wider than for other days during periods of financial
crises. The presence of day-of-the-week effects in multifractal dynamics of
market returns motivates further research on calendar anomalies from an
econophysics perspective
Paths to collapse for isolated skyrmions in few-monolayer ferromagnetic films
Magnetic skyrmions are topological spin configurations in materials with
chiral Dzyaloshinskii-Moriya interaction (DMI), that are potentially useful for
storing or processing information. To date, DMI has been found in few bulk
materials, but can also be induced in atomically thin magnetic films in contact
with surfaces with large spin-orbit interactions. Recent experiments have
reported that isolated magnetic skyrmions can be stabilized even near room
temperature in few-atom thick magnetic layers sandwiched between materials that
provide asymmetric spin-orbit coupling. Here we present the minimum-energy path
analysis of three distinct mechanisms for the skyrmion collapse, based on ab
initio input and the performed atomic-spin simulations. We focus on the
stability of a skyrmion in three atomic layers of Co, either epitaxial on the
Pt(111) surface, or within a hybrid multilayer where DMI nontrivially varies
per monolayer due to competition between different symmetry-breaking from two
sides of the Co film. In laterally finite systems, their constrained geometry
causes poor thermal stability of the skyrmion toward collapse at the boundary,
which we show to be resolved by designing the high-DMI structure within an
extended film with lower or no DMI
ASSESSMENT OF MASS TRANSPORT PROPERTIES IN BIMODAL MICROPOROUS NETWORKS BY COMBINING GRAVIMETRIC AND INFRARED SPECTROSCOPY
International audienceInterest of this study is to develop classical macroscopic diffusion measurements to probe and allow distinguishing a true hierarchical material from the other material. To this aim, the AGIR methodology (Analysis by Gravimetry and IR spectroscopy) will be used Gravimetric measurements provide more classical approach to molecular uptake measurements, whereas infrared (IR) spectroscopy can provide additional data on the interaction, orientation and environment of the molecules on the solid surfaces. In this study we evaluated the mass transport properties in a mechanical mixture of hierarchical H-MFI and H-FAU zeolites. Using AGIR technique, we have been able to distinguish the diffusivity of each component present in the mechanical mixture of following pure zeolites: H-MFI and H-FAU respectively. The potential and limitations of this new methodology will be discussed
Anomalous optical response of graphene on hexagonal boron nitride substrates
Graphene/hBN heterostructures can be considered as one of the basic building
blocks for the next-generation optoelectronics mostly owing to the record-high
electron mobilities. However, currently, the studies of the intrinsic optical
properties of graphene are limited to the standard substrates (SiO2/Si, glass,
quartz) despite the growing interest in graphene/hBN heterostructures. This can
be attributed to a challenging task of the determination of hBN's strongly
anisotropic dielectric tensor in the total optical response. In this study, we
overcome this issue through imaging spectroscopic ellipsometry utilizing
simultaneous analysis of hBN's optical response with and without graphene
monolayers. Our technique allowed us to retrieve the optical constants of
graphene from graphene/hBN heterostructures in a broad spectral range of
250-950 nm. Our results suggest that graphene's absorption on hBN may exceed
the one of graphene on SiO2/Si by about 60 %
Multiscale Mechanistic Insights of Shaped Catalyst Body Formulations and Their Impact on Catalytic Properties
International audienceZeolite-based catalysts are globally employed in many industrial processes, such as in crude-oil refining and in the production of bulk chemicals. However, to be implemented in industrial reactors efficiently, zeolite powders are required to be shaped in catalyst bodies. Scale-up of zeolite catalysts into such forms comes with side effects to its overall physicochem-ical properties and to those of its constituting components. Although fundamental research into "technical" solid catalysts is scarce, binder effects have been reported to significantly impact their catalytic properties and lifetime. Given the large number of additional (in)organic components added in the formulation, it is somehow surprising to see that there is a distinct lack of research into the unintentional impact organic additives can have on the properties of the zeolite and the catalyst bodies in general. Here, we systematically prepared a series of alumina-bound zeolite ZSM-5-based catalyst bodies, with organic additives such as peptizing, plasticizing, and lubricating agents, to rationalize their impacts on the physicochemical properties of the shaped catalyst bodies. By utilizing a carefully selected arsenal of bulk and high-spatial resolution multiscale characterization techniques, as well as specifically sized bioinspired fluorescent nanoprobes to study pore accessibility, we clearly show that, although the organic additives achieve their primary function of a mechanically robust material, uncontrolled processes are taking place in parallel. We reveal that the extrusion process can lead to zeolite dealumination (from acid peptizing treatment, and localized steaming upon calcination); meso-and macropore structural rearrangement (via burning-out of organic plasticizing and lubricating agents upon calcination); and abating of known alumina binder effects (via scavenging of Al species via chelating lubricating agents), which significantly impact catalytic performance. Understanding the mechanisms behind such effects in industrial-grade catalyst formulations can lead to enhanced design of these important materials, which can improve process efficiency in a vast range of industrial catalytic reactions
FP8 Formats for Deep Learning
FP8 is a natural progression for accelerating deep learning training
inference beyond the 16-bit formats common in modern processors. In this paper
we propose an 8-bit floating point (FP8) binary interchange format consisting
of two encodings - E4M3 (4-bit exponent and 3-bit mantissa) and E5M2 (5-bit
exponent and 2-bit mantissa). While E5M2 follows IEEE 754 conventions for
representatio of special values, E4M3's dynamic range is extended by not
representing infinities and having only one mantissa bit-pattern for NaNs. We
demonstrate the efficacy of the FP8 format on a variety of image and language
tasks, effectively matching the result quality achieved by 16-bit training
sessions. Our study covers the main modern neural network architectures - CNNs,
RNNs, and Transformer-based models, leaving all the hyperparameters unchanged
from the 16-bit baseline training sessions. Our training experiments include
large, up to 175B parameter, language models. We also examine FP8
post-training-quantization of language models trained using 16-bit formats that
resisted fixed point int8 quantization
Microscaling Data Formats for Deep Learning
Narrow bit-width data formats are key to reducing the computational and
storage costs of modern deep learning applications. This paper evaluates
Microscaling (MX) data formats that combine a per-block scaling factor with
narrow floating-point and integer types for individual elements. MX formats
balance the competing needs of hardware efficiency, model accuracy, and user
friction. Empirical results on over two dozen benchmarks demonstrate
practicality of MX data formats as a drop-in replacement for baseline FP32 for
AI inference and training with low user friction. We also show the first
instance of training generative language models at sub-8-bit weights,
activations, and gradients with minimal accuracy loss and no modifications to
the training recipe
Propriétés acido-basiques de catalyseurs pour la conversion de la biomasse
Glycerol and fructose are molecules that are readily available in substantial quantities fromthe biomass. In this work dehydration routes for valorization of these compounds wereinvestigated. Therefore, zirconia and titania based catalysts, and calcium phosphate materialswere prepared and evaluated in the glycerol dehydration in gas phase. Niobia-ceria mixedoxides and mesoporous Nb2O5-MeO2 (M = Ce, Zr, Ti) mixed oxides were prepared andtested in fructose dehydration reaction in aqueous phase. The surface acid-base properties ofthe studied catalysts were correlated to their catalytic performance.Le glycérol et le fructose sont des molécules qui peuvent être extraites facilement de labiomasse et en des quantités substantielles. Ce travail de recherche porte sur la déshydratationcomme moyen de valoriser ces composés. C’est dans ce but que des catalyseurs supportés suroxydes de zirconium et de titane, ainsi que des matériaux de type phosphate de calcium, ontété préparés et testés pour la réaction de déshydratation du glycérol en phase gazeuse. Desoxydes mixtes de niobium et cerium ainsi que des oxydes mixtes mésoporeux de Nb2O5-MeO2 (M = Ce, Zr, Ti) ont été également préparés et cette fois-ci testés pour la réaction dedéshydratation du fructose en milieu aqueux. Dans les deux cas, les propriétés acido-basiquesde surface des catalyseurs étudiés ont été corrélées à leur efficacité catalytique