An efficient algorithm for data parallelism based on stochastic optimization

Deep neural network models can achieve greater performance in numerous machine learning tasks by raising the depth of the model and the amount of training data samples. However, these essential procedures will proportionally raise the cost of training deep neural network models. Accelerating the training process of deep neural network models in a distributed computing environment has become the most often utilized strategy for developers in order to better cope with a huge quantity of training overhead. The current deep neural network model is the stochastic gradient descent (SGD) technique. It is one of the most widely used training techniques in network models, although it is prone to gradient obsolescence during parallelization, which impacts the overall convergence. The majority of present solutions are geared at high-performance nodes with minor performance changes. Few studies have taken into account the cluster environment in high-performance computing (HPC), where the performance of each node varies substantially. A dynamic batch size stochastic gradient descent approach based on performance-aware technology is suggested to address the aforesaid difficulties (DBS-SGD). By assessing the processing capacity of each node, this method dynamically allocates the minibatch of each node, guaranteeing that the update time of each iteration between nodes is essentially the same, lowering the average gradient of the node. The suggested approach may successfully solve the asynchronous update strategy’s gradient outdated problem. The Mnist and cifar10 are two widely used image classification benchmarks, that are employed as training data sets, and the approach is compared with the asynchronous stochastic gradient descent (ASGD) technique. The experimental findings demonstrate that the proposed algorithm has better performance as compared with existing algorithms

Energy Efficient Secure Communication Model against Cooperative Eavesdropper

In a wiretap channel system model, the jammer node adopts the energy-harvesting signal as artificial noise (jamming signal) against the cooperative eavesdroppers. There are two eavesdroppers in the wiretap channel: eavesdropper E1 is located near the transmitter and eavesdropper E2 is located near the jammer. The eavesdroppers are equipped with multiple antennas and employ the iterative block decision feedback equalization decoder to estimate the received signal, i.e., information signal at E1 and jamming signal at E2. It is assumed that E1 has the channel state information (CSI) of the channel between transmitter and E1, and similarly, E2 has the CSI of channel between jammer and E2. The eavesdroppers establish communication link between them and cooperate with each other to reduce the information signal interference at E2 and jamming signal interference at E1. The performance of decoders depends on the signal to interference plus noise ratio (SINR) of the received signal. The power of information signal is fixed and the power of the jamming signal is adjusted to improve the SINR of the received signal. This research work is solely focused on optimizing the jamming signal power to degrade the performance of cooperative eavesdroppers. The jamming signal power is optimized for the given operating SINR with the support of simulated results. The jamming signal power optimization leads to better energy conservation and degrades the performance of eavesdroppers

Hybrid multi-user equalizer for massive MIMO millimeter-wave dynamic subconnected architecture

This paper proposes a hybrid multi-user equalizer for the uplink of broadband millimeterwave massive multiple input/multiple output (MIMO) systems with dynamic subarray antennas. Hybrid subconnected architectures are more suitable for practical applications since the number of required phase shifters is lower than in fully connected architectures. We consider a set of only analog precoded users transmitting to a base station and sharing the same radio resources. At the receiver end, the hybrid multi-user equalizer is designed by minimizing the sum of the mean square error (MSE) of all subcarriers, considering a two-step approach. In the first step, the digital part is iteratively computed as a function of the analog part. It is considered that the digital equalizers are computed on a per subcarrier basis, while the analog equalizer is constant over the subcarriers and the digital iterations due to hardware constraints. In the second step, the analog equalizer with dynamic antenna mapping is derived to connect the best set of antennas to each radio frequency (RF) chain. For each subset of antennas, one antenna and a quantized phase shifter are selected at a time, taking into account all previously selected antennas. The results show that the proposed hybrid dynamic two-step equalizer achieves a performance close to the fully connected counterpart, although it is less complex in terms of hardware and signal processing requirements.publishe

Iterative frequency-domain detection for IA-precoded MC-CDMA systems

Interference alignment (IA) is a promising technique that allows high capacity gains in interfering channels. On the other hand, iterative frequency-domain detection receivers based on the IB-DFE concept (Iterative Block Decision Feedback Equalization) can efficiently exploit the inherent space-frequency diversity of the MIMO MC-CDMA systems. In this paper we combine iterative IA precoding at the transmitter with IB-DFE based processing at the receiver for MC-CDMA systems. The receiver is designed in two steps: first a linear filter is used to mitigate the inter-user aligned interference, and then an iterative frequency-domain receiver is designed to efficiently separate the spatial streams in the presence of residual inter-user aligned interference at the output of the filter. The matrices for this non- linear space-frequency equalizer are obtained by minimizing the overall mean square error (MSE) of all data streams at each subcarrier. Our receiver structure is explicitly designed taking into account the residual inter-user interference, allowing both an efficient separation of the spatial streams and a reduction in the number of iterations of the IA procedure. We also propose a simple, yet accurate analytical approach for obtaining the performance of the proposed receiver structure. Our scheme achieves the maximum degrees of freedom provided by the IA precoding, while allowing an almost optimum space-diversity gain, with performance close to the matched filter bound (MFB)

Técnicas de equalização iterativa para arquiteturas híbridas sub-conectadas na banda de ondas milimétricas

Mestrado em Engenharia Eletrónica e TelecomunicaçõesThe millimeter wave communications and the use of a massive number of antennas are two promising technologies that being combined allow to achieve the multi Gb/s required by future 5G wireless systems. As this type of systems has a high number of antennas it is impossible to use a fully digital architecture, due to hardware limitations. Therefore, the design of signal processing techniques for hybrid analog-digital architectures is a requirement. Depending on the structure of the analog part the hybrid analog-digital architectures may be fully connected or sub-connected. Although the fully connected hybrid architectures allow to connect all RF chains to any antenna element, they involve a high cost due to its structural and computational complexity. As such, the sub-connected hybrid architectures become more attractive, since either at the hardware level or from the computational point of view they are less demanding. In this dissertation, we propose a hybrid iterative block multiuser equalizer for sub-connected millimeter wave massive MIMO systems. The user terminal transceiver has low-complexity and as such employ a pure analog random precoder, with a single RF chain. For the base station, a sub-connected hybrid analog-digital equalizer is designed to remove the multiuser interference. The hybrid equalizer is optimized using the average bit-error-rate as a metric. Due to the coupling between the RF chains in the optimization problem the computation of the optimal solution is way too complex. To address this problem, we compute the analog part of the equalizer sequentially over the RF chains using a dictionary built from the array response vectors. The proposed sub-connected hybrid iterative multiuser equalizer is compared with a recently proposed fully connected hybrid analog-digital approach and with the fully digital architecture. The results show that the performance of the proposed scheme is close to the fully connected hybrid approach after just a few iterations.As comunicações na banda das ondas milimétricas e o uso massivo de antenas são duas tecnologias promissoras que, sendo combinadas permitem alcançar elevadas taxas de transmissão, na ordem dos multi Gb/s, exigidas pelos futuros sistemas sem fios da 5G. Como estes sistemas possuem um número elevado de antenas, torna-se impossível o uso de uma arquitetura totalmente digital devido às limitações de hardware. Desta forma, é necessário projetar técnicas de processamento de sinal para arquiteturas híbridas analógico-digitais. Dentro das arquiteturas híbridas, foram propostas duas formas de lidar com a parte analógica, que são, a forma totalmente conectada e a forma sub-conectada. Embora as arquiteturas híbridas totalmente conectadas permitam interligar todas as cadeias RF a qualquer elemento de antena, estas envolvem um elevado custo devido à sua complexidade estrutural e computacional. Assim sendo, as arquiteturas híbridas sub-conectadas tornam-se mais atraentes pois são menos exigentes do ponto de vista computacional, bem como ao nível do hardware. Nesta dissertação, é proposto um equalizador iterativo para um sistema com uma arquitetura hibrida sub-conectada, com múltiplos utilizadores e um número massivo de antenas a operar na banda das ondas milimétricas. Os terminais dos utilizadores têm baixa complexidade e utilizam pré-codificadores aleatórios analógicos puros, cada um com uma única cadeia RF. Para a estação base, projetou-se um equalizador híbrido analógico-digital de arquitectura sub-conectada, para remover a interferência multiutilizador. O equalizador híbrido é otimizado usando a taxa média de erro de bit como métrica. Devido ao acoplamento entre as cadeias de RF no problema de otimização, o cálculo das soluções ótimas possui elevada complexidade. Para ultrapassar este problema, calculou-se a parte analógica de cada cadeia de RF do equalizador de forma sequencial, usando um dicionário construído a partir da resposta do agregado de antenas. Compara-se o equalizador iterativo híbrido para sistemas multiutilizador de arquitectura sub-conectada proposto com uma abordagem híbrida analógica/digital totalmente conectada, recentemente proposta na literatura e com uma arquitetura totalmente digital. Os resultados mostram que o desempenho do esquema proposto aproximasse da abordagem híbrida totalmente conectada após apenas algumas iterações

Técnicas de equalização e pré-codificação para sistemas MC-CDMA

Mestrado em Engenharia Eletrónica e TelecomunicaçõesO número de dispositivos com ligações e aplicações sem fios está a aumentar exponencialmente, causando problemas de interferência e diminuindo a capacidade do sistema. Isto desencadeou uma procura por uma eficiência espectral superior e, consequentemente, tornou-se necessário desenvolver novas arquitecturas celulares que suportem estas novas exigências. Coordenação ou cooperação multicelular é uma arquitectura promissora para sistemas celulares sem fios. Esta ajuda a mitigar a interferência entre células, melhorando a equidade e a capacidade do sistema. É, portanto, uma arquitectura já em estudo ao abrigo da tecnologia LTE-Advanced sob o conceito de coordenação multiponto (CoMP). Nesta dissertação, considerámos um sistema coordenado MC-CDMA com pré-codificação e equalização iterativas. Uma das técnicas mais eficientes de pré-codificação é o alinhamento de interferências (IA). Este é um conceito relativamente novo que permite aumentar a capacidade do sistema em canais de elevada interferência. Sabe-se que, para os sistemas MC-CDMA, os equalizadores lineares convencionais não são os mais eficientes, devido à interferência residual entre portadoras (ICI). No entanto, a equalização iterativa no domínio da frequência (FDE) foi identificada como sendo uma das técnicas mais eficientes para lidar com ICI e explorar a diversidade oferecida pelos sistemas MIMO MC-CDMA. Esta técnica é baseada no conceito Iterative Block Decision Feedback Equalization (IB-DFE). Nesta dissertação, é proposto um sistema MC-CDMA que une a pré-codificação iterativa do alinhamento de interferências no transmissor ao equalizador baseado no IB-DFE, com cancelamento sucessivo de interferências (SIC) no receptor. Este é construído por dois blocos: um filtro linear, que mitiga a interferência inter-utilizador, seguido por um bloco iterativo no domínio da frequência, que separa eficientemente os fluxos de dados espaciais na presença de interferência residual inter-utilizador alinhada. Este esquema permite atingir o número máximo de graus de liberdade e permite simultaneamente um ganho óptimo de diversidade espacial. O desempenho deste esquema está perto do filtro adaptado- Matched Filter Bound (MFB).The number of devices with wireless connections and applications is increasing exponentially, causing interference problems and reducing the system’s capacity gain. This initiated a search for a higher spectral efficiency and therefore it became necessary to develop new cellular architectures that support these new requirements. Multicell cooperation or coordination is a promising architecture for cellular wireless systems to mitigate intercell interference, improving system fairness and increasing capacity, and thus is already under study in LTE-Advanced under the coordinated multipoint (CoMP) concept. In this thesis, efficient iterative precoding and equalization is considered for coordinated MC-CDMA based systems. One of the most efficient precoding techniques is interference alignment (IA), which is a relatively new concept that allows high capacity gains in interfering channels. It is well known that for MC-CDMA systems standard linear equalizers are not the most efficient due to residual inter carrier interference (ICI). However, iterative frequency-domain equalization (FDE) has been identified as one of the most efficient technique to deal with ICI and exploit the inherent space-frequency diversity of the MIMO MC-CDMA systems, namely the one based on Iterative Block Decision Feedback Equalization (IB-DFE) concept. In this thesis, it is proposed a MC-CDMA system that joins iterative IA precoding at the transmitter with IB-DFE successive interference cancellation (SIC) based receiver structure. The receiver is implemented in two steps: a linear filter, which mitigates the inter-user aligned interference, followed by an iterative frequency-domain receiver, which efficiently separates the spatial streams in the presence of residual inter-user aligned interference. This scheme provides the maximum degrees of freedom (DoF) and allows almost the optimum space-diversity gain. The scheme performance is close to the matched filter bound (MFB)