29 research outputs found

    Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

    Full text link
    Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications.Comment: 28 pages, 14 figure

    Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

    Get PDF
    Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, thereby providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications

    The Dispersion of the Gauss-Markov Source

    Get PDF
    The Gauss-Markov source produces U_i = aU_(i–1) + Z_i for i ≥ 1, where U_0 = 0, |a| 0, and we show that the dispersion has a reverse waterfilling representation. This is the first finite blocklength result for lossy compression of sources with memory. We prove that the finite blocklength rate-distortion function R(n; d; ε) approaches the rate-distortion function R(d) as R(n; d; ε) = R(d)+ √ V(d)/n Q–1(ε)+o(1√n), where V (d) is the dispersion, ε ε 2 (0; 1) is the excess-distortion probability, and Q^(-1) is the inverse Q-function. We give a reverse waterfilling integral representation for the dispersion V (d), which parallels that of the rate-distortion functions for Gaussian processes. Remarkably, for all 0 < d ≥ σ^2 (1+|σ|)^2, R(n; d; ε) of the Gauss-Markov source coincides with that of Z_i, the i.i.d. Gaussian noise driving the process, up to the second-order term. Among novel technical tools developed in this paper is a sharp approximation of the eigenvalues of the covariance matrix of n samples of the Gauss-Markov source, and a construction of a typical set using the maximum likelihood estimate of the parameter a based on n observations

    The Dispersion of the Gauss-Markov Source

    Get PDF
    The Gauss-Markov source produces U_i = aU_(i–1) + Z_i for i ≥ 1, where U_0 = 0, |a| 0, and we show that the dispersion has a reverse waterfilling representation. This is the first finite blocklength result for lossy compression of sources with memory. We prove that the finite blocklength rate-distortion function R(n; d; ε) approaches the rate-distortion function R(d) as R(n; d; ε) = R(d)+ √ V(d)/n Q–1(ε)+o(1√n), where V (d) is the dispersion, ε ε 2 (0; 1) is the excess-distortion probability, and Q^(-1) is the inverse Q-function. We give a reverse waterfilling integral representation for the dispersion V (d), which parallels that of the rate-distortion functions for Gaussian processes. Remarkably, for all 0 < d ≥ σ^2 (1+|σ|)^2, R(n; d; ε) of the Gauss-Markov source coincides with that of Z_i, the i.i.d. Gaussian noise driving the process, up to the second-order term. Among novel technical tools developed in this paper is a sharp approximation of the eigenvalues of the covariance matrix of n samples of the Gauss-Markov source, and a construction of a typical set using the maximum likelihood estimate of the parameter a based on n observations

    Zero-delay source-channel coding

    Get PDF
    In this thesis, we investigate the zero-delay transmission of source samples over three different types of communication channel models. First, we consider the zero-delay transmission of a Gaussian source sample over an additive white Gaussian noise (AWGN) channel in the presence of an additive white Gaussian (AWG) interference, which is fully known by the transmitter. We propose three parameterized linear and non-linear transmission schemes for this scenario, and compare the corresponding mean square error (MSE) performances with that of a numerically optimized encoder, obtained using the necessary optimality conditions. Next, we consider the zero-delay transmission of a Gaussian source sample over an AWGN channel with a one-bit analog-to-digital (ADC) front end. We study this problem under two different performance criteria, namely the MSE distortion and the distortion outage probability (DOP), and obtain the optimal encoder and the decoder for both criteria. As generalizations of this scenario, we consider the performance with a K-level ADC front end as well as with multiple one-bit ADC front ends. We derive necessary conditions for the optimal encoder and decoder, which are then used to obtain numerically optimized encoder and decoder mappings. Finally, we consider the transmission of a Gaussian source sample over an AWGN channel with a one-bit ADC front end in the presence of correlated side information at the receiver. Again, we derive the necessary optimality conditions, and using these conditions obtain numerically optimized encoder and decoder mappings. We also consider the scenario in which the side information is available also at the encoder, and obtain the optimal encoder and decoder mappings. The performance of the latter scenario serves as a lower bound on the performance of the case in which the side information is available only at the decoder.Open Acces

    Information Theory and Machine Learning

    Get PDF
    The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems

    Cancelamento de interferência em sistemas celulares distribuídos

    Get PDF
    Doutoramento em Engenharia ElectrotécnicaO tema principal desta tese é o problema de cancelamento de interferência para sistemas multi-utilizador, com antenas distribuídas. Como tal, ao iniciar, uma visão geral das principais propriedades de um sistema de antenas distribuídas é apresentada. Esta descrição inclui o estudo analítico do impacto da ligação, dos utilizadores do sistema, a mais antenas distribuídas. Durante essa análise é demonstrado que a propriedade mais importante do sistema para obtenção do ganho máximo, através da ligação de mais antenas de transmissão, é a simetria espacial e que os utilizadores nas fronteiras das células são os mais bene ciados. Tais resultados são comprovados através de simulação. O problema de cancelamento de interferência multi-utilizador é considerado tanto para o caso unidimensional (i.e. sem codi cação) como para o multidimensional (i.e. com codi cação). Para o caso unidimensional um algoritmo de pré-codi cação não-linear é proposto e avaliado, tendo como objectivo a minimização da taxa de erro de bit. Tanto o caso de portadora única como o de multipla-portadora são abordados, bem como o cenário de antenas colocadas e distribuidas. É demonstrado que o esquema proposto pode ser visto como uma extensão do bem conhecido esquema de zeros forçados, cuja desempenho é provado ser um limite inferior para o esquema generalizado. O algoritmo é avaliado, para diferentes cenários, através de simulação, a qual indica desempenho perto do óptimo, com baixa complexidade. Para o caso multi-dimensional um esquema para efectuar "dirty paper coding" binário, tendo como base códigos de dupla camada é proposto. No desenvolvimento deste esquema, a compressão com perdas de informação, é considerada como um subproblema. Resultados de simulação indicam transmissão dedigna proxima do limite de Shannon.This thesis focus on the interference cancellation problem for multiuser distributed antenna systems. As such it starts by giving an overview of the main properties of a distributed antenna system. This overview includes, an analytical investigation of the impact of the connection of additional distributed antennas, to the system users. That analysis shows that the most important system property to reach the maximum gain, with the connection of additional transmit antennas, is spatial symmetry and that the users at the cell borders are the most bene ted. The multiuser interference problem has been considered for both the one dimensional (i.e. without coding) and multidimensional (i.e. with coding) cases. In the unidimensional case, we propose and evaluate a nonlinear precoding algorithm for the minimization of the bit-error-rate, of a multiuser MIMO system. Both the single-carrier and multi-carrier cases are tackled as well as the co-located and distributed scenarios. It is demonstrated that the proposed scheme can be viewed as an extension of the well-known zero-forcing, whose performance is proven to be a lower bound for the generalized scheme. The algorithm was validated extensively through numerical simulations, which indicate a performance close to the optimal, with reduced complexity. For the multi-dimensional case, a binary dirty paper coding scheme, base on bilayer codes, is proposed. In the development of this scheme, we consider the lossy compression of a binary source as a sub-problem. Simulation results indicate reliable transmission close to the Shannon limit

    Metodi Matriciali per l'Acquisizione Efficiente e la Crittografia di Segnali in Forma Compressa

    Get PDF
    The idea of balancing the resources spent in the acquisition and encoding of natural signals strictly to their intrinsic information content has interested nearly a decade of research under the name of compressed sensing. In this doctoral dissertation we develop some extensions and improvements upon this technique's foundations, by modifying the random sensing matrices on which the signals of interest are projected to achieve different objectives. Firstly, we propose two methods for the adaptation of sensing matrix ensembles to the second-order moments of natural signals. These techniques leverage the maximisation of different proxies for the quantity of information acquired by compressed sensing, and are efficiently applied in the encoding of electrocardiographic tracks with minimum-complexity digital hardware. Secondly, we focus on the possibility of using compressed sensing as a method to provide a partial, yet cryptanalysis-resistant form of encryption; in this context, we show how a random matrix generation strategy with a controlled amount of perturbations can be used to distinguish between multiple user classes with different quality of access to the encrypted information content. Finally, we explore the application of compressed sensing in the design of a multispectral imager, by implementing an optical scheme that entails a coded aperture array and Fabry-Pérot spectral filters. The signal recoveries obtained by processing real-world measurements show promising results, that leave room for an improvement of the sensing matrix calibration problem in the devised imager

    Narrow-band few photon filter and phase lock control for EIT with Cs in a nanofiber dipole trap

    Get PDF
    Nicht angegeben.This Master thesis was performed around an experiment aiming at the investigation of the optical properties of laser cooled Cesium (Cs) atoms dipole trapped in the evanescent field of an optical nanofiber. Two parts of the total experiment are covered in this thesis. The focus of the first part is the beam preparation of the EIT control and probe lasers which ensures a phase stable joint performance of both beams necessary for the implementation of EIT. This is achieved by an optical phase-locked loop (OPLL) locking the probe to the control laser. The performance of this OPLL is examined with an out-of-loop phase noise measurement. The second part of this thesis concerns the efficient detection of the prospective few-photon EIT probe signal, which will be immersed in a broadband noise background. While the weak EIT probe signal (<pW power) is expected to be extremely narrow-band (<kHz) the noise has a power of ~5pW within a wavelength window of 10nm around the probe wavelength. Conventional optical filters fail in efficiently separating the signal from the fluorescence. Here, two strategies are elaborated theoretically aiming at a reasonable solution for narrow-band few-photon filtering: one employing a diffraction grating and another based on homodyne detection. Experimental proposals for filters based on both options are made, followed by an experimental realization and analysis of a compact test setup, a homodyne saturation spectroscopy
    corecore