Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning

The recent surge of generative AI has been fueled by the generative power of diffusion probabilistic models and the scalable capabilities of large language models. Despite their potential, it remains elusive whether diffusion language models can solve general language tasks comparable to their autoregressive counterparts. This paper demonstrates that scaling diffusion models w.r.t. data, sizes, and tasks can effectively make them strong language learners. We build competent diffusion language models at scale by first acquiring knowledge from massive data via masked language modeling pretraining thanks to their intrinsic connections. We then reprogram pretrained masked language models into diffusion language models via diffusive adaptation, wherein task-specific finetuning and instruction finetuning are explored to unlock their versatility in solving general language tasks. Experiments show that scaling diffusion language models consistently improves performance across downstream language tasks. We further discover that instruction finetuning can elicit zero-shot and few-shot in-context learning abilities that help tackle many unseen tasks by following natural language instructions, and show promise in advanced and challenging abilities such as reasoning.Comment: added reference

An Experimental Study of the Water Transfer Through Confined Compacted GMZ Bentonite

International audienceGMZ bentonite has been considered as a possible material for engineered barrier in the Chinese program of nuclear waste disposal at great depth. In the present work, the hydraulic conductivity of this bentonite was determined by simultaneous profile method. A specific infiltration cell equipped with five resistive relative humidity probes was designed for this purpose. The water retention properties were studied under both confined and unconfined conditions; the results shows that at high suctions (> 4 MPa) the water retention capacity is independent of the confining condition, and by contrast, at low suctions (< 4MPa) the confined condition resulted in significant low water retention. Furthermore, the microstructure was investigated at Mercury Intrusion Porosimetry (MIP) and Environmental Scanning Electron Microscope (ESEM) in different states: on oven-dried powder, bentonite slurry, as-compacted and wetted samples. It has been observed that the soil powder is constituted of aggregates of various sizes; this aggregates are destroyed by fully saturation at a water content equal to the liquid limit; compaction at the initial water content of 11-12% and a dry density of 1.7 – 1.75 Mg/m3 led to a microstructure characterized by an dense assembly of relatively well preserved aggregates; saturation of the compacted sample under constant volume condition defined a non-homogeneous microstructure with the presence of well preserved aggregates. This non-homogeneous microstructure would be due to the non uniform distribution of the generated swelling pressure within the soil sample upon wetting. The hydraulic conductivity determined has been found decreasing firstly and then increasing with suction decrease from the initial value of about 80 MPa to zero; the decrease can be attributed to the large pore clogging due to soft gel creation by exfoliation process, as observed at ESEM

TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments

Although the estimation of 3D human pose and shape (HPS) is rapidly progressing, current methods still cannot reliably estimate moving humans in global coordinates, which is critical for many applications. This is particularly challenging when the camera is also moving, entangling human and camera motion. To address these issues, we adopt a novel 5D representation (space, time, and identity) that enables end-to-end reasoning about people in scenes. Our method, called TRACE, introduces several novel architectural components. Most importantly, it uses two new "maps" to reason about the 3D trajectory of people over time in camera, and world, coordinates. An additional memory unit enables persistent tracking of people even during long occlusions. TRACE is the first one-stage method to jointly recover and track 3D humans in global coordinates from dynamic cameras. By training it end-to-end, and using full image information, TRACE achieves state-of-the-art performance on tracking and HPS benchmarks. The code and dataset are released for research purposes.Comment: Project page: https://www.yusun.work/TRACE/TRACE.htm

Thermal-mechanical behavior of compacted GMZ Bentonite

International audienceThe THM behavior of compacted GMZ bentonite has been investigated using a suction-temperature controlled isotropic cell. The results obtained were compared with the existing results on other reference bentonites (MX80, FEBEX, FoCa, and Kunigel-V1). It has been observed that the coefficient of thermal expansion of the compacted GMZ bentonite is 2 x 10-4°C-1, similar to the values of compacted MX80 and FEBEX bentonites. The heating tests of the GMZ bentonite also show that the suction is an important parameter that governs the thermal volumetric behavior of unsaturated soils. Unlike temperature, suction has a significant effect on the compressibility parameters. Examination of the mineralogy of various bentonites showed that a good correlation can be generally established between the montmorillonite content and the cations exchange capacity (CEC) or the specific surface area (S). Nevertheless, both the basic geotechnical properties and the swelling potential seem to depend not only on the montmorillonite content but also on other factors such as the nature of base exchangeable cations. The quartz content of the GMZ bentonite is relatively high (11.7%). This could explain its relatively large values of thermal conductivity

A volume-preserving sharpening approach for the propagation of sharp phase boundaries in multiphase lattice Boltzmann simulations

Lattice Boltzmann models that recover a macroscopic description of multiphase flow of immiscible liquids typically represent the boundaries between phases using a scalar function, the phase field, that varies smoothly over several grid points. Attempts to tune the model parameters to minimise the thicknesses of these interfaces typically lead to the interfaces becoming fixed to the underlying grid instead of advecting with the fluid velocity. This phenomenon, known as lattice pinning, is strikingly similar to that associated with the numerical simulation of conservation laws coupled to stiff algebraic source terms. We present a lattice Boltzmann formulation of the model problem proposed by LeVeque and Yee [J. Comput. Phys. 86, 187] to study the latter phenomenon in the context of computational combustion, and offer a volume-conserving extension in multiple space dimensions. Inspired by the random projection method of Bao and Jin [J. Comput. Phys. 163, 216] we further generalise this formulation by introducing a uniformly distributed quasi-random variable into the term responsible for the sharpening of phase boundaries. This method is mass conserving and the statistical average of this method is shown to significantly delay the onset of pinning

Scalable wavelength-multiplexing photonic reservoir computing

Photonic reservoir computing (PRC) is a special hardware recurrent neural network, which is featured with fast training speed and low training cost. This work shows a wavelength-multiplexing PRC architecture, taking advantage of the numerous longitudinal modes in a Fabry-Perot semiconductor laser. These modes construct connected physical neurons in parallel, while an optical feedback loop provides interactive virtual neurons in series. We experimentally demonstrate a four-channel wavelength-multiplexing PRC, which runs four times faster than the single-channel case. It is proved that the multiplexing PRC exhibits superior performance on the task of signal equalization in an optical fiber communication link. Particularly, this scheme is highly scalable owing to the rich mode resources in Fabry-Perot lasers