Sky-wave over-the-horizon radar simulation tool

This work deals with the entire process of target detection and ranging by a sky-wave over-the-horizon radar (OTHR) computational model simulation. The different processing stages of the transmitted signal along its space-time trajectory from transmission to digital signal processing are modelled. With this simulation tool a moving target present over the sea can be detected according to a set of given initial conditions together with the ionosphere model inputs and the target electromagnetic model. Initial conditions as well as the modulation and filtering options among other parameters of the model can be set easily. The present work is intended to be a further contribution to OTHR studies, providing a user-friendly tool of easy application in order to improve a radar design, facilitate its implementation, as well as for debugging algorithms and signal processing techniques.Fil: Saavedra, Zenon. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tucumán; Argentina. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología. Departamento de Electricidad, Electrónica y Computación. Laboratorio de Telecomunicaciones; ArgentinaFil: Zimmerman, Diego. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tucumán; Argentina. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología. Departamento de Electricidad, Electrónica y Computación. Laboratorio de Telecomunicaciones; ArgentinaFil: Cabrera, Miguel Angel. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología. Departamento de Electricidad, Electrónica y Computación. Laboratorio de Telecomunicaciones; ArgentinaFil: Elias, Ana Georgina. Universidad Nacional de Tucumán. Instituto de Física del Noroeste Argentino. - Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet Noa Sur. Instituto de Física del Noroeste Argentino; Argentina. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología; Argentin

Developing a framework for horizontally scalable network flow analytics on the Hadoop ecosytem

This study proposes an improved way of analyzing raw network data on Hadoop called hcap. This new framework is evaluated against three common methods currently used for this type of analytics; conversion to text, conversion to Parquet, and direct parsing of PCAP binaries with the hadoop-pcap library both withand without logs. The comparison was conducted with four key performance indicators: preprocessing, storage efficiency, data retention, and query response time. Because the original hadoop-pcap framework failed to process larger datasets, its version with logs suppressed was instead used for the evaluation. Results show that Parquet outperforms hcap by 90% and hadoop-pcap with its logs suppressed by 96% in terms of query response time while text also runs 80% faster than hcap and 92% faster than hadoop-pcap with its logs suppressed, however, it also runs 30% slower in scan and aggregate queries and 70% and 40% slower in joins and aggregate-joins respectively when compared to Parquet. The framework created in this study not only provided an improved method for parsing PCAP binaries on Hadoop, outperforming hadoop-pcap by at least 20%, it also provided analternative technique for conversion to Parquet, reducing preprocessing time by a factor of 5

Towards Large Scale Packet Capture and Network Flow Analysis on Hadoop

Network traffic continues to grow yearly at a compounded rate. However, network traffic is still being analyzed on vertically scaled machines that do not scale as well as distributed computing platforms. Hadoop\u27s horizontally scalable ecosystem provides a better environment for processing these network captures stored in packet capture (PCAP) files. This paper proposes a framework called hcap for analyzing PCAPs on Hadoop inspired by the Rseaux IP Europens\u27 (RIPE\u27s) existing hadoop-pcap library but built completely from the ground up. The hcap framework improves several aspects of the hadoop-pcap library, namely protocol, error, and log handling. Results show that, while other methods still outperform hcap, it not only performs better than hadoop-pcap by 15% in scan queries and 18% in join queries, but it\u27s more tolerant to broken PCAP entries which reduces preprocessing time and data loss, while also speeding up the conversion process used in other methods by 85%

Earth's magnetic field effect on MUF calculation and consequences for hmF2 trend estimates

Knowledge of the state of the upper atmosphere, and in particular of the ionosphere, is essential in several applications such as systems used in radio frequency communications, satellite positioning and navigation. In general, these systems depend on the state and evolution of the ionosphere. In all applications involving the ionosphere an essential task is to determine the path and modifications of ray propagation through the ionospheric plasma. The ionospheric refractive index and the maximum usable frequency (MUF) that can be received over a given distance are some key parameters that are crucial for such technological applications. However, currently the representation of these parameters are in general simplified, neglecting the effects of Earth's magnetic field. The value of M(3000)F2, related to the MUF that can be received over 3000 km is routinely scaled from ionograms using a technique which also neglects the geomagnetic field effects assuming a standard simplified propagation model. M(3000)F2 is expected to be affected by a systematic trend linked to the secular variations of Earth's magnetic field. On the other hand, among the upper atmospheric effects expected from increasing greenhouse gases concentration is the lowering of the F2-layer peak density height, hmF2. This ionospheric parameter is usually estimated using the M(3000)F2 factor, so it would also carry this “systematic trend”. In this study, the geomagnetic field effect on MUF estimations is analyzed as well as its impact on hmF2 long-term trend estimations. We find that M(3000)F2 increases when the geomagnetic field is included in its calculation, and hence hmF2, estimated using existing methods involving no magnetic field for M(3000)F2 scaling, would present a weak but steady trend linked to these variations which would increase or compensate the few kilometers decrease (~2 km per decade) expected from greenhouse gases effect.Fil: Elias, Ana Georgina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tucumán; Argentina. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología; ArgentinaFil: Zossi, Bruno Santiago. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tucumán; Argentina. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología; ArgentinaFil: Yigit, Erdal. George Mason University; Estados UnidosFil: Saavedra, Zenon. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tucumán; Argentina. Universidad Nacional de Tucuman. Facultad de Ciencias Exactas y Tecnología. Departamento de Electricidad, Electrónica y Computación; ArgentinaFil: de Haro Barbás, Blas Federico. Universidad Nacional de Tucumán. Facultad de Ciencias Exactas y Tecnología; Argentin

Adapting Block-Sized Captures for Faster Network Flow Analysis on the Hadoop Ecosystem

With the rapid and continuous growth of annual network traffic comes the need to develop systems that can efficiently scale to meet the demands of analyzing all this traffic data. The Hadoop ecosystem provides an environment that is capable of addressing this need, because of its horizontal scalability and its data locality optimization feature. The latter feature improves parallel analysis of data by placing computing tasks within the same node that contains the block of data to be analyzed. However, this feature cannot be taken advantage of by those input formats that are not splittable within the Hadoop Distributed File System. The PCAP format used for capturing network data is one such file format. To address this issue, this paper proposes the inclusion of a minimal preprocessing step before PCAP files are fed into Hadoop and analyzed using the hcap framework, which is currently the fastest framework for analyzing PCAP data in Hadoop. This preprocessing step is designed to adapt the PCAP files into properly split blocks in order to take advantage of Hadoop\u27s data locality optimization feature. Results show a significant improvement in query response time with a performance gain of 92%, 89%, 91%, and, 87% for scan, aggregate, join, and aggregate-join queries respectively when compared to the original hcap framework