1,670 research outputs found
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning
The recent surge of generative AI has been fueled by the generative power of
diffusion probabilistic models and the scalable capabilities of large language
models. Despite their potential, it remains elusive whether diffusion language
models can solve general language tasks comparable to their autoregressive
counterparts. This paper demonstrates that scaling diffusion models w.r.t.
data, sizes, and tasks can effectively make them strong language learners. We
build competent diffusion language models at scale by first acquiring knowledge
from massive data via masked language modeling pretraining thanks to their
intrinsic connections. We then reprogram pretrained masked language models into
diffusion language models via diffusive adaptation, wherein task-specific
finetuning and instruction finetuning are explored to unlock their versatility
in solving general language tasks. Experiments show that scaling diffusion
language models consistently improves performance across downstream language
tasks. We further discover that instruction finetuning can elicit zero-shot and
few-shot in-context learning abilities that help tackle many unseen tasks by
following natural language instructions, and show promise in advanced and
challenging abilities such as reasoning.Comment: added reference
An Experimental Study of the Water Transfer Through Confined Compacted GMZ Bentonite
International audienceGMZ bentonite has been considered as a possible material for engineered barrier in the Chinese program of nuclear waste disposal at great depth. In the present work, the hydraulic conductivity of this bentonite was determined by simultaneous profile method. A specific infiltration cell equipped with five resistive relative humidity probes was designed for this purpose. The water retention properties were studied under both confined and unconfined conditions; the results shows that at high suctions (> 4 MPa) the water retention capacity is independent of the confining condition, and by contrast, at low suctions (< 4MPa) the confined condition resulted in significant low water retention. Furthermore, the microstructure was investigated at Mercury Intrusion Porosimetry (MIP) and Environmental Scanning Electron Microscope (ESEM) in different states: on oven-dried powder, bentonite slurry, as-compacted and wetted samples. It has been observed that the soil powder is constituted of aggregates of various sizes; this aggregates are destroyed by fully saturation at a water content equal to the liquid limit; compaction at the initial water content of 11-12% and a dry density of 1.7 – 1.75 Mg/m3 led to a microstructure characterized by an dense assembly of relatively well preserved aggregates; saturation of the compacted sample under constant volume condition defined a non-homogeneous microstructure with the presence of well preserved aggregates. This non-homogeneous microstructure would be due to the non uniform distribution of the generated swelling pressure within the soil sample upon wetting. The hydraulic conductivity determined has been found decreasing firstly and then increasing with suction decrease from the initial value of about 80 MPa to zero; the decrease can be attributed to the large pore clogging due to soft gel creation by exfoliation process, as observed at ESEM
TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments
Although the estimation of 3D human pose and shape (HPS) is rapidly
progressing, current methods still cannot reliably estimate moving humans in
global coordinates, which is critical for many applications. This is
particularly challenging when the camera is also moving, entangling human and
camera motion. To address these issues, we adopt a novel 5D representation
(space, time, and identity) that enables end-to-end reasoning about people in
scenes. Our method, called TRACE, introduces several novel architectural
components. Most importantly, it uses two new "maps" to reason about the 3D
trajectory of people over time in camera, and world, coordinates. An additional
memory unit enables persistent tracking of people even during long occlusions.
TRACE is the first one-stage method to jointly recover and track 3D humans in
global coordinates from dynamic cameras. By training it end-to-end, and using
full image information, TRACE achieves state-of-the-art performance on tracking
and HPS benchmarks. The code and dataset are released for research purposes.Comment: Project page: https://www.yusun.work/TRACE/TRACE.htm
Thermal-mechanical behavior of compacted GMZ Bentonite
International audienceThe THM behavior of compacted GMZ bentonite has been investigated using a suction-temperature controlled isotropic cell. The results obtained were compared with the existing results on other reference bentonites (MX80, FEBEX, FoCa, and Kunigel-V1). It has been observed that the coefficient of thermal expansion of the compacted GMZ bentonite is 2 x 10-4°C-1, similar to the values of compacted MX80 and FEBEX bentonites. The heating tests of the GMZ bentonite also show that the suction is an important parameter that governs the thermal volumetric behavior of unsaturated soils. Unlike temperature, suction has a significant effect on the compressibility parameters. Examination of the mineralogy of various bentonites showed that a good correlation can be generally established between the montmorillonite content and the cations exchange capacity (CEC) or the specific surface area (S). Nevertheless, both the basic geotechnical properties and the swelling potential seem to depend not only on the montmorillonite content but also on other factors such as the nature of base exchangeable cations. The quartz content of the GMZ bentonite is relatively high (11.7%). This could explain its relatively large values of thermal conductivity
A volume-preserving sharpening approach for the propagation of sharp phase boundaries in multiphase lattice Boltzmann simulations
Lattice Boltzmann models that recover a macroscopic description of multiphase flow of immiscible liquids typically represent the boundaries between phases using a scalar function, the phase field, that varies smoothly over several grid points. Attempts to tune the model parameters to minimise the thicknesses of these interfaces typically lead to the interfaces becoming fixed to the underlying grid instead of advecting with the fluid velocity. This phenomenon, known as lattice pinning, is strikingly similar to that associated with the numerical simulation of conservation laws coupled to stiff algebraic source terms. We present a lattice Boltzmann formulation of the model problem proposed by LeVeque and Yee [J. Comput. Phys. 86, 187] to study the latter phenomenon in the context of computational combustion, and offer a volume-conserving extension in multiple space dimensions. Inspired by the random projection method of Bao and Jin [J. Comput. Phys. 163, 216] we further generalise this formulation by introducing a uniformly distributed quasi-random variable into the term responsible for the sharpening of phase boundaries. This method is mass conserving and the statistical average of this method is shown to significantly delay the onset of pinning
Scalable wavelength-multiplexing photonic reservoir computing
Photonic reservoir computing (PRC) is a special hardware recurrent neural
network, which is featured with fast training speed and low training cost. This
work shows a wavelength-multiplexing PRC architecture, taking advantage of the
numerous longitudinal modes in a Fabry-Perot semiconductor laser. These modes
construct connected physical neurons in parallel, while an optical feedback
loop provides interactive virtual neurons in series. We experimentally
demonstrate a four-channel wavelength-multiplexing PRC, which runs four times
faster than the single-channel case. It is proved that the multiplexing PRC
exhibits superior performance on the task of signal equalization in an optical
fiber communication link. Particularly, this scheme is highly scalable owing to
the rich mode resources in Fabry-Perot lasers
- …