29 research outputs found
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Communication systems to date primarily aim at reliably communicating bit
sequences. Such an approach provides efficient engineering designs that are
agnostic to the meanings of the messages or to the goal that the message
exchange aims to achieve. Next generation systems, however, can be potentially
enriched by folding message semantics and goals of communication into their
design. Further, these systems can be made cognizant of the context in which
communication exchange takes place, providing avenues for novel design
insights. This tutorial summarizes the efforts to date, starting from its early
adaptations, semantic-aware and task-oriented communications, covering the
foundations, algorithms and potential implementations. The focus is on
approaches that utilize information theory to provide the foundations, as well
as the significant role of learning in semantics and task-aware communications.Comment: 28 pages, 14 figure
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, thereby providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications
The Dispersion of the Gauss-Markov Source
The Gauss-Markov source produces U_i = aU_(i–1) + Z_i for i ≥ 1, where U_0 = 0, |a| 0, and we show that the dispersion has a reverse waterfilling representation. This is the first finite blocklength result for lossy compression of sources with memory. We prove that the finite blocklength rate-distortion function R(n; d; ε) approaches the rate-distortion function R(d) as R(n; d; ε) = R(d)+ √ V(d)/n Q–1(ε)+o(1√n), where V (d) is the dispersion, ε ε 2 (0; 1) is the excess-distortion probability, and Q^(-1) is the inverse Q-function. We give a reverse waterfilling integral representation for the dispersion V (d), which parallels that of the rate-distortion functions for Gaussian processes. Remarkably, for all 0 < d ≥ σ^2 (1+|σ|)^2, R(n; d; ε) of the Gauss-Markov source coincides with that of Z_i, the i.i.d. Gaussian noise driving the process, up to the second-order term. Among novel technical tools developed in this paper is a sharp approximation of the eigenvalues of the covariance matrix of n samples of the Gauss-Markov source, and a construction of a typical set using the maximum likelihood estimate of the parameter a based on n observations
The Dispersion of the Gauss-Markov Source
The Gauss-Markov source produces U_i = aU_(i–1) + Z_i for i ≥ 1, where U_0 = 0, |a| 0, and we show that the dispersion has a reverse waterfilling representation. This is the first finite blocklength result for lossy compression of sources with memory. We prove that the finite blocklength rate-distortion function R(n; d; ε) approaches the rate-distortion function R(d) as R(n; d; ε) = R(d)+ √ V(d)/n Q–1(ε)+o(1√n), where V (d) is the dispersion, ε ε 2 (0; 1) is the excess-distortion probability, and Q^(-1) is the inverse Q-function. We give a reverse waterfilling integral representation for the dispersion V (d), which parallels that of the rate-distortion functions for Gaussian processes. Remarkably, for all 0 < d ≥ σ^2 (1+|σ|)^2, R(n; d; ε) of the Gauss-Markov source coincides with that of Z_i, the i.i.d. Gaussian noise driving the process, up to the second-order term. Among novel technical tools developed in this paper is a sharp approximation of the eigenvalues of the covariance matrix of n samples of the Gauss-Markov source, and a construction of a typical set using the maximum likelihood estimate of the parameter a based on n observations
Zero-delay source-channel coding
In this thesis, we investigate the zero-delay transmission of source samples over three
different types of communication channel models. First, we consider the zero-delay
transmission of a Gaussian source sample over an additive white Gaussian noise (AWGN)
channel in the presence of an additive white Gaussian (AWG) interference, which is
fully known by the transmitter. We propose three parameterized linear and non-linear
transmission schemes for this scenario, and compare the corresponding mean square
error (MSE) performances with that of a numerically optimized encoder, obtained using
the necessary optimality conditions. Next, we consider the zero-delay transmission of a
Gaussian source sample over an AWGN channel with a one-bit analog-to-digital (ADC)
front end. We study this problem under two different performance criteria, namely the
MSE distortion and the distortion outage probability (DOP), and obtain the optimal
encoder and the decoder for both criteria. As generalizations of this scenario, we consider
the performance with a K-level ADC front end as well as with multiple one-bit ADC
front ends. We derive necessary conditions for the optimal encoder and decoder, which
are then used to obtain numerically optimized encoder and decoder mappings. Finally,
we consider the transmission of a Gaussian source sample over an AWGN channel with
a one-bit ADC front end in the presence of correlated side information at the receiver.
Again, we derive the necessary optimality conditions, and using these conditions obtain
numerically optimized encoder and decoder mappings. We also consider the scenario
in which the side information is available also at the encoder, and obtain the optimal
encoder and decoder mappings. The performance of the latter scenario serves as a lower
bound on the performance of the case in which the side information is available only at
the decoder.Open Acces
Information Theory and Machine Learning
The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems
Cancelamento de interferência em sistemas celulares distribuídos
Doutoramento em Engenharia ElectrotécnicaO tema principal desta tese é o problema de cancelamento de interferência
para sistemas multi-utilizador, com antenas distribuídas. Como tal, ao iniciar,
uma visão geral das principais propriedades de um sistema de antenas
distribuídas é apresentada. Esta descrição inclui o estudo analítico do impacto
da ligação, dos utilizadores do sistema, a mais antenas distribuídas.
Durante essa análise é demonstrado que a propriedade mais importante do
sistema para obtenção do ganho máximo, através da ligação de mais antenas
de transmissão, é a simetria espacial e que os utilizadores nas fronteiras das
células são os mais bene ciados. Tais resultados são comprovados através
de simulação. O problema de cancelamento de interferência multi-utilizador
é considerado tanto para o caso unidimensional (i.e. sem codi cação) como
para o multidimensional (i.e. com codi cação). Para o caso unidimensional
um algoritmo de pré-codi cação não-linear é proposto e avaliado, tendo
como objectivo a minimização da taxa de erro de bit. Tanto o caso de
portadora única como o de multipla-portadora são abordados, bem como o
cenário de antenas colocadas e distribuidas. É demonstrado que o esquema
proposto pode ser visto como uma extensão do bem conhecido esquema
de zeros forçados, cuja desempenho é provado ser um limite inferior para
o esquema generalizado. O algoritmo é avaliado, para diferentes cenários,
através de simulação, a qual indica desempenho perto do óptimo, com baixa
complexidade. Para o caso multi-dimensional um esquema para efectuar
"dirty paper coding" binário, tendo como base códigos de dupla camada é
proposto. No desenvolvimento deste esquema, a compressão com perdas de
informação, é considerada como um subproblema. Resultados de simulação
indicam transmissão dedigna proxima do limite de Shannon.This thesis focus on the interference cancellation problem for multiuser distributed
antenna systems. As such it starts by giving an overview of the
main properties of a distributed antenna system. This overview includes, an
analytical investigation of the impact of the connection of additional distributed
antennas, to the system users. That analysis shows that the most
important system property to reach the maximum gain, with the connection
of additional transmit antennas, is spatial symmetry and that the users at
the cell borders are the most bene ted. The multiuser interference problem
has been considered for both the one dimensional (i.e. without coding) and
multidimensional (i.e. with coding) cases. In the unidimensional case, we
propose and evaluate a nonlinear precoding algorithm for the minimization
of the bit-error-rate, of a multiuser MIMO system. Both the single-carrier
and multi-carrier cases are tackled as well as the co-located and distributed
scenarios. It is demonstrated that the proposed scheme can be viewed as an
extension of the well-known zero-forcing, whose performance is proven to be
a lower bound for the generalized scheme. The algorithm was validated extensively
through numerical simulations, which indicate a performance close
to the optimal, with reduced complexity. For the multi-dimensional case, a
binary dirty paper coding scheme, base on bilayer codes, is proposed. In the
development of this scheme, we consider the lossy compression of a binary
source as a sub-problem. Simulation results indicate reliable transmission
close to the Shannon limit
Metodi Matriciali per l'Acquisizione Efficiente e la Crittografia di Segnali in Forma Compressa
The idea of balancing the resources spent in the acquisition and encoding of natural signals strictly to their intrinsic information content has interested nearly a decade of research under the name of compressed sensing. In this doctoral dissertation we develop some extensions and improvements upon this technique's foundations, by modifying the random sensing matrices on which the signals of interest are projected to achieve different objectives.
Firstly, we propose two methods for the adaptation of sensing matrix ensembles to the second-order moments of natural signals. These techniques leverage the maximisation of different proxies for the quantity of information acquired by compressed sensing, and are efficiently applied in the encoding of electrocardiographic tracks with minimum-complexity digital hardware.
Secondly, we focus on the possibility of using compressed sensing as a method to provide a partial, yet cryptanalysis-resistant form of encryption; in this context, we show how a random matrix generation strategy with a controlled amount of perturbations can be used to distinguish between multiple user classes with different quality of access to the encrypted information content.
Finally, we explore the application of compressed sensing in the design of a multispectral imager, by implementing an optical scheme that entails a coded aperture array and Fabry-Pérot spectral filters. The signal recoveries obtained by processing real-world measurements show promising results, that leave room for an improvement of the sensing matrix calibration problem in the devised imager
Narrow-band few photon filter and phase lock control for EIT with Cs in a nanofiber dipole trap
Nicht angegeben.This Master thesis was performed around an experiment aiming at the investigation of the optical properties of laser cooled Cesium (Cs) atoms dipole trapped in the evanescent field of an optical nanofiber. Two parts of the total experiment are covered in this thesis. The focus of the first part is the beam preparation of the EIT control and probe lasers which ensures a phase stable joint performance of both beams necessary for the implementation of EIT. This is achieved by an optical phase-locked loop (OPLL) locking the probe to the control laser. The performance of this OPLL is examined with an out-of-loop phase noise measurement.
The second part of this thesis concerns the efficient detection of the prospective few-photon EIT probe signal, which will be immersed in a broadband noise background. While the weak EIT probe signal (<pW power) is expected to be extremely narrow-band (<kHz) the noise has a power of ~5pW within a wavelength window of 10nm around the probe wavelength. Conventional optical filters fail in efficiently separating the signal from the fluorescence. Here, two strategies are elaborated theoretically aiming at a reasonable solution for narrow-band few-photon filtering: one employing a diffraction grating and another based on homodyne detection. Experimental proposals for filters based on both options are made, followed by an experimental realization and analysis of a compact test setup, a homodyne saturation spectroscopy