Experimental Evaluation Platform for Voice Transmission Over Internet of Things (VoIoTs)

Internet of Things (IoTs) is an example of the last advances in Information and Communication Technologies. In particular, with the revolutionary development of Wireless Sensor Network (WSN) technologies, researchers largely focused on take benefits of integration embedded low-cost, low-power WSN technology in a various IoTs applications. Real-time voice transmission over IoTs is one interesting application that began to be explored by many researchers. Thus, this paper presents a performance study for transmission of voice over WSN (VoWSN) with and without presence of Internet. A framework using a Raspberry Pi3 (RPi3) and open source FFmpeg technology for processing, compressing and streaming voice to a remote computer is proposed, implemented and evaluated. The performance of the proposed framework is evaluated by studying its behavior utilizing three audio encoding algorithms: AC3, MP3 and OPUS with different sampling rates and a set of evaluation metrics such as :One-way delay, jitter, Bandwidth (B.W), CPU usage and packet losses

Opus audiokoodekki matkapuhelinverkoissa

The latest generations in mobile networks have enabled a possibility to include high quality audio coding in data transmission. On the other hand, an on-going effort to move the audio signal processing from dedicated hardware to data centers with generalized hardware introduces a challenge of providing enough computational power needed by the virtualized network elements. This thesis evaluates the usage of a modern hybrid audio codec called Opus in a virtualized network element. It is performed by integrating the codec, testing it for functionality and performance on a general purpose processor, as well as evaluating the performance in comparison to the digital signal processor's performance. Functional testing showed that the codec was integrated successfully and bit compliance with the Opus standard was met. The performance results showed that although the digital signal processor computes the encoder's algorithms with less clock cycles, related to the processor's whole capacity the general purpose processor performs more efficiently due to higher clock frequency. For the decoder this was even clearer, when the generic hardware spends on average less clock cycles for performing the algorithms.Uusimmat sukupolvet matkapuhelinverkoissa mahdollistavat korkealaatuisen audiokoodauksen tiedonsiirrossa. Toisaalta audiosignaalinkäsittelyn siirtäminen sovelluskohtaisesta laitteistosta keskitettyjen palvelinkeskusten yleiskäyttöiseen laitteistoon on käynnissä, mikä aiheuttaa haasteita tarjota riittävästi laskennallista tehoa virtualisoituja verkkoelementtejä varten. Tämä diplomityö arvioi modernin hybridikoodekin, Opuksen, käyttöä virtualisoidussa verkkoelementissä. Se on toteutettu integroimalla koodekki, testaamalla funktionaalisuutta ja suorituskykyä yleiskäyttöisellä prosessorilla sekä arvioimalla suorituskykyä verrattuna digitaalisen signaaliprosessorin suorituskykyyn. Funktionaalinen testaus osoitti että koodekki oli integroitu onnistuneesti ja että bittitason yhdenmukaisuus Opuksen standardin kanssa saavutettiin. Suorituskyvyn testitulokset osoittivat, että vaikka enkoodaus tuotti vähemmän kellojaksoja digitaalisella signaaliprosessorilla, yleiskäyttöinen prosessori suoriutuu tehokkaammin suhteutettuna prosessorin kokonaiskapasiteettiin korkeamman kellotaajuuden ansiosta. Dekooderilla tämä näkyi vielä selkeämmin, sillä yleiskäyttöinen prosessori kulutti keskimäärin vähemmän kellojaksoja algoritmien suorittamiseen

Videopuhelun tallentaminen ja toisto

Recording a video call recording is beneficial in cases like job interviews and business meetings. Skype or Google Hangouts for example have no recording implemented without additional plugins. MP4 file container is capable of containing different types of media tracks, is widely used and is supported by media players and has a feature called RTP/RTCP Reception Hint Tracks. Hint tracks contain media transmission instructions, which can be used, e.g., to record RTP stream into a file. Without this information the video call session cannot be replayed afterwards. The purpose of this Thesis is to implement and verify the usage of RTP and RTCP Reception Hint Tracks in video call recording. No open-source MP4 multiplexing library or a media player with support for RTP/RTCP Reception Hint Tracks was found, so the support had to be implemented in both the library and the media player. The setup includes Linphone and L-SMASH for recording and VLC media player for playback. The created MP4 file has two RTP Reception Hint Tracks, two RTCP Reception Hint Tracks, and two media tracks. The GSM audio is chosen because it is supported by Linphone, L-SMASH, and VLC media player. H.264/AVC is chosen for video, because it is the best available codec supported by the three software. Tests were carried out using two laptops with both having recording enabled. From the tests it is concluded that using the RTP Reception Hint Track increases the total CPU usage by less than 1% and the size of the recorded video call by 4% over the conventional media tracks. The implementation shows that RTP Reception Hint Tracks meet well the needs of implementations with choice of different codecs

uvgRTP 2.0: Open-Source RTP Library For Real-Time VVC/HEVC Streaming

Real-time video transport plays a central role in various interactive and streaming media applications. This paper presents a new release of our open-source Real-time Transport Protocol (RTP) library called uvgRTP (github.com/ultravideo/uvgRTP) that is designed for economic video and audio transmission in real time. It is the first public library that comes with built-in support for modern VVC, HEVC, and AVC video codecs and Opus audio codec. It can also be tailored to diversified media formats with an easy-to-use generic API. According to our experiments, uvgRTP can stream 8K VVC video at 300 fps with an average round-trip latency of 4.9 ms over a 10 Gbit link. This cross-platform library can be run on Windows and Linux operating systems and the permissive BSD 2-Clause license makes it accessible to a broad range of commercial and academic streaming media applications.acceptedVersionPeer reviewe

Design and Implement a Hybrid WebRTC SignallingMechanism for Unidirectional & Bi-directional VideoConferencing

WebRTC (Web Real-Time Communication) is a technology that enables browser-to-browser communication. Therefore, a signalling mechanism must be negotiated to create a connection between peers. The main aim of this paper is to create and implement a WebRTC hybrid signalling mechanism named (WebNSM) for video conferencing based on the Socket.io (API) mechanism and Firefox. WebNSM was designed over a combination of different topologies, such as simplex, star and mesh. Therefore it offers several communications at the same time as one-to-one (unidirectional/bidirectional), one-to-many (unidirectional) and many-to-many (bi-directional) without any downloading or installation. In this paper, WebRTC video conferencing was accomplished via LAN and WAN networks, including the evaluation of resources in WebRTC like bandwidth consumption, CPU performance, memory usage, Quality of Experience (QoE) and maximum links and RTPs calculation. This paper presents a novel signalling mechanism among different users, devices and networks to offer multi-party video conferencing using various topologies at the same time, as well as other typical features such as using the same server, determining room initiator, keeping the communication active even if the initiator or another peer leaves, etc. This scenario highlights the limitations of resources and the use of different topologies for WebRTC video conferencing

A comparative study of RTC applications

Real-Time Communication (RTC) applications have become ubiquitous and are nowadays fundamental for people to communicate with friends and relatives, as well as for enterprises to allow remote working and save travel costs. Countless competing platforms differ in the ease of use, features they implement, supported user equipment and targeted audience (consumer of business). However, there is no standard protocol or interoperability mechanism. This picture complicates the traffic management, making it hard to isolate RTC traffic for prioritization or obstruction. Moreover, undocumented operation could result in the traffic being blocked at firewalls or middleboxes. In this paper, we analyze 13 popular RTC applications, from widespread consumer apps, like Skype and Whatsapp, to business platforms dedicated to enterprises - Microsoft Teams and Webex Teams. We collect packet traces under different conditions and illustrate similarities and differences in their use of the network. We find that most applications employ the well-known RTP protocol, but we observe a few cases of different (and even undocumented) approaches. The majority of applications allow peer-to-peer communication during calls with only two participants. Six of them send redundant data for Forward Error Correction or encode the user video at different bitrates. In addition, we notice that many of them are easy to identify by looking at the destination servers or the domain names resolved via DNS. The packet traces we collected, along with the metadata we extract, are made available to the community

Automatic real-time transcription of multimedia conference

Cílem práce je řešení pro přepis multimediální konference založené na protokolu WebRTC v reálném čase za pomoci kombinace existujících technologií a řešení v oblasti konferencí, přenosu médií a rozpoznávání řeči. Aplikace je naprogramována v Javě. Pro signalizaci se používá protokol WebSocket a pro přenos audio dat protokol RTP. Součástí řešení je modulární transkripční back-end využívající rozhraní Google Cloud Speech-to-text API a řešení pro rozpoznávání řeči vyvinuté v Laboratoři počítačového zpracování řeči (SpeechLab) na Technické univerzitě v Liberci. Přepisy jsou zobrazeny v prohlížečích účastníků v reálném čase a zároveň jsou zapisovány do souboru. Práce obsahuje příklady přepisovaných konverzací.This work focuses on performing real-time transcription of a multimedia conference based on WebRTC protocol by combining existing technologies and solutions in conferencing, media transmission and speech recognition in one application. The result application is written in Java. It uses WebSocket to communicate with a conferencing application, RTP for receiving audio data and suggests modular transcription back-ends with Google Cloud Speech-to-text API and speech recognition engine developed by the Laboratory of Computer Speech Processing (SpeechLab) in Technical University of Liberec already successfully integrated. Transcripts are stored in files and also can be displayed in browsers in real-time. Examples of transcribed conversations are provided

Synchronization of streamed audio between multiple playback devices over an unmanaged IP network

When designing and implementing a prototype supporting inter-destination media synchronization – synchronized playback between multiple devices receiving the same stream – there are a lot of aspects that need to be considered, especially when working with unmanaged networks. Not only is a proper streaming protocol essential, but also a way to obtain and maintain the synchronization of the clocks of the devices. The thesis had a few constraints, namely that the server producing the stream should be written for the .NET-platform and that the clients receiving it should be using the media framework GStreamer. This framework provides methods for both achieving synchronization as well as resynchronization. As the provided resynchro- nization methods introduced distortions in the audio, an alternative method was implemented. This method focused on minimizing the distortions, thus maintain- ing a smooth playback. After the prototype had been implemented, it was tested to see how well it performed under the influence of packet loss and delay. The accuracy of the synchronization was also tested under optimal conditions using two different time synchronization protocols. What could be concluded from this was that a good synchronization could be maintained on unloaded networks using the proposed method, but when introducing delay the prototype struggled more. This was mainly due to the usage of the Network Time Protocol (NTP), which is known to perform badly on networks with asymmetric paths.When working with synchronized playback it is not enough just obtain- ing it – it also needs to be maintained. Implementing a prototype thus involves many parts ranging from choosing a proper streaming protocol, to handling glitch free resynchronization of audio. Synchronization between multiple speakers has a wide area of application, ranging from home entertainment solutions to big malls where announcements should appear synchronized over the entire perimeter. In order to achieve this, two main parts are involved: the streaming of the audio, and the actual synchronization. The streaming itself poses problems mostly since the prototype should not only work on dedicated networks, but rather on all kinds, such as the Internet. As the information over these networks are transmitted in packets, and the path from source to destination crosses many sub networks, the packets may be delayed or even lost. This may create an audible distortion in the playback. The next part is the synchronization. This is most easily achieved by putting a time on each packet stating when in the future it should be played out. If then all receivers play it back at the specified time, synchronization is achieved. This however requires that all the receivers share the idea of when a specific time is – the clocks at all the receivers must be synchronized. By using existing software and hardware solutions, such as the Network Time Protocol (NTP) or the Precision Time Protocol (PTP), this can be accomplished. The accuracy of the synchronization is therefore partly dependent on how well these solutions work. Another valid aspect is how accurate the synchronization must be for the sound to be perceived as synchronized by humans. This is usually in the range of a few tens of milliseconds to five milliseconds depending on the sound. When a global time has been distributed to all receivers, matters get more complicated as there is more than one clock to consider at each receiver. Apart from the previously mentioned clock, now called the ’system clock’, there is also an audio clock, which is a hardware clock positioned on the sound card. This audio clock decides the rate at which media is played out. Altering the system clock to synchronize it to a common time is one thing, but altering the audio clock while media is being played will inevitably mean a jump in the playback, and thus a distortion. Although an initial synchronization can be achieved, the two clocks will over time tick in slightly different pace, thus drifting away from each other. This creates a need for the audio clock to continuously correct itself to follow the system clock. In the media framework GStreamer, used for handling the media at the re- ceivers, two alternatives to solve the correction problem were available. Quick evaluations of these two methods however showed that either audible glitches or ’oscillations’ occurred in the sound, when the clocks were corrected. A new method, which basically combines the two existing, was therefore implemented. With this method the audio clock is continuously corrected, but in a smaller and less aggressive way. Listening tests revealed much smaller, often not audible, distortions, while the synchronization performance was at par with the existing methods. More thorough testing showed that the synchronization over networks with light traffic was in the microsecond-range, thus far below the threshold of what will appear as synchronized. During worse conditions – simulated hostile environments – the synchronization quickly reached unacceptable levels though. This was due to the previously mentioned NTP, and not the implemented method on the other hand

Building and Using WebRTC Scenarios.

Import 22/07/2015V diplomové práci je pojednáváno o implementaci WebRTC pomocí pobočkové ústředny Asterisk a pomocí SipML5 klienta, který byl vytvořen společností Doubango Telecom. V praktické části je realizováno zprovoznění WebRTC a jeho následné otestování pomocí SipML5 klienta ovládaného přes webové rozhraní. Dále bylo provedeno měření přenosových parametrů technologie WebRTC pomocí paketového analyzátoru Wireshark. Tyto parametry jsou porovnávány s konvenčními SIP relacemi, které využívají pro přenos dat protokol RTP a SRTP. Zpracování a porovnání dat probíhá pomocí software Matlab, ve kterém byly vytvořeny skripty pro analýzu dat. Výsledné naměřené hodnoty jsou následně využity pro nástroj na výpočet celkových nároků na realizaci daného počtu audio a video hovorů.This work is about the implementation of WebRTC using a PBX Asterisk and SipML5 client, what was created by Doubango Telecom. In the practical part is realized putting into operation of WebRTC and this is subsequent testing using SipML5 client controlled via a web interface. Subsequently, was made measurement of transmission parameters technology WebRTC using a packet analyzer Wireshark. These parameters are then compared with conventional SIP relationships, which use for the data transfer protocol RTP and SRTP. Processing and comparison of data is done by using Matlab software, in which were created simple scripts for data analysis. The results of measurements are then used to calculate a tool for total demands the implementation of a number of audio and video calls.440 - Katedra telekomunikační technikydobř