How Generative Spoken Language Modeling Encodes Noisy Speech: Investigation from Phonetics to Syntactics

We examine the speech modeling potential of generative spoken language modeling (GSLM), which involves using learned symbols derived from data rather than phonemes for speech analysis and synthesis. Since GSLM facilitates textless spoken language processing, exploring its effectiveness is critical for paving the way for novel paradigms in spoken-language processing. This paper presents the findings of GSLM's encoding and decoding effectiveness at the spoken-language and speech levels. Through speech resynthesis experiments, we revealed that resynthesis errors occur at the levels ranging from phonology to syntactics and GSLM frequently resynthesizes natural but content-altered speech.Comment: Accepted to INTERSPEECH 202

On the origin of discrepancies between observed and simulated memory of Arctic Sea ice

To investigate the inherent predictability of sea ice and its representation in climate models, we compare the seasonal-to-interannual memory of Arctic sea ice as given by lagged correlations of sea-ice area anomalies in large model ensembles (Max Planck Institute Grand Ensemble and Coupled Model Intercomparison Project phase 6) and multiple observational products. We find that state-of-the-art climate models significantly overestimate the memory of pan-Arctic sea-ice area from the summer months into the following year. This cannot be explained by internal variability. We further show that the observed summer memory can be disentangled regionally into a reemergence of positive correlations in the perennial ice zone and negative correlations in the seasonal ice zone; the latter giving rise to the discrepancy between observations and model simulations. These findings could explain some of the predictability gap between potential and operational forecast skill of Arctic sea-ice area identified in previous studies

Frequency and power of human alpha oscillations drift systematically with time-on-task

Oscillatory neural activity is a fundamental characteristic of the mammalian brain spanning multiple levels of spatial and temporal scale. Current theories of neural oscillations and analysis techniques employed to investigate their functional significance are based on an often implicit assumption: In the absence of experimental manipulation, the spectral content of any given EEG- or MEG-recorded neural oscillator remains approximately stationary over the course of a typical experimental session (∼1 h), spontaneously fluctuating only around its dominant frequency. Here, we examined this assumption for ongoing neural oscillations in the alpha-band (8–13 Hz). We found that alpha peak frequency systematically decreased over time, while alpha-power increased. Intriguingly, these systematic changes showed partial independence of each other: Statistical source separation (independent component analysis) revealed that while some alpha components displayed concomitant power increases and peak frequency decreases, other components showed either unique power increases or frequency decreases. Interestingly, we also found these components to differ in frequency. Components that showed mixed frequency/power changes oscillated primarily in the lower alpha-band (∼8–10 Hz), while components with unique changes oscillated primarily in the higher alpha-band (∼9–13 Hz). Our findings provide novel clues on the time-varying intrinsic properties of large-scale neural networks as measured by M/EEG, with implications for the analysis and interpretation of studies that aim at identifying functionally relevant oscillatory networks or at driving them through external stimulation

Detection and prediction of urban archetypes at the pedestrian scale: computational toolsets, morphological metrics, and machine learning methods

Granular, dense, and mixed-use urban morphologies are hallmarks of walkable and vibrant streets. However, urban systems are notoriously complex and planned urban development, which grapples with varied interdependent and oft conflicting criteria, may — despite best intentions — yield aberrant morphologies fundamentally at odds with the needs of pedestrians and the resiliency of neighbourhoods. This work addresses the measurement, detection, and prediction of pedestrian-friendly urban archetypes by developing techniques for high-resolution urban analytics at the pedestrian scale. A spatial-analytic computational toolset, the cityseer-api Python package, is created to assess localised centrality, land-use, and statistical metrics using contextually sensitive workflows applied directly over the street network. cityseer-api subsequently facilitates a review of mixed-use and street network centrality methods to improve their utility concerning granular urban analysis. Unsupervised machine learning methods are applied to recover ‘signatures’ — urban archetypes — using Principal Component Analysis, Variational Autoencoders, and clustering methods from a high-resolution multi-variable and multi-scalar dataset consisting of centralities, land-uses, and population densities for Greater London. Supervised deep-learning methods applied to a similar dataset developed for 931 towns and cities in Great Britain demonstrate how, with the aid of domain knowledge, machine-learning classifiers can learn to discriminate between ‘artificial’ and ‘historical’ urban archetypes. These methods use complex systems thinking as a departure point and illustrate how high-resolution spatial-analytic quantitative methods can be combined with machine learning to extrapolate benchmarks in keeping with more qualitatively framed urban morphological conceptions. Such tools may aid urban design professionals in better anticipating the outcomes of varied design scenarios as part of iterative and scalable workflows. These techniques may likewise provide robust and demonstrable feedback as part of planning review and approvals processes

A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VISis needed. In this paper, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: "what visualization processes can be assisted by ML?" and "how ML techniques can be used to solve visualization problems?" This survey reveals seven main processes where the employment of ML techniques can benefit visualizations:Data Processing4VIS, Data-VIS Mapping, InsightCommunication, Style Imitation, VIS Interaction, VIS Reading, and User Profiling. The seven processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations.Meanwhile, the seven processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this paper can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at https://ml4vis.github.ioComment: 19 pages, 12 figures, 4 table