Service Abstractions for Scalable Deep Learning Inference at the Edge

Deep learning driven intelligent edge has already become a reality, where millions of mobile, wearable, and IoT devices analyze real-time data and transform those into actionable insights on-device. Typical approaches for optimizing deep learning inference mostly focus on accelerating the execution of individual inference tasks, without considering the contextual correlation unique to edge environments and the statistical nature of learning-based computation. Specifically, they treat inference workloads as individual black boxes and apply canonical system optimization techniques, developed over the last few decades, to handle them as yet another type of computation-intensive applications. As a result, deep learning inference on edge devices still face the ever increasing challenges of customization to edge device heterogeneity, fuzzy computation redundancy between inference tasks, and end-to-end deployment at scale. In this thesis, we propose the first framework that automates and scales the end-to-end process of deploying efficient deep learning inference from the cloud to heterogeneous edge devices. The framework consists of a series of service abstractions that handle DNN model tailoring, model indexing and query, and computation reuse for runtime inference respectively. Together, these services bridge the gap between deep learning training and inference, eliminate computation redundancy during inference execution, and further lower the barrier for deep learning algorithm and system co-optimization. To build efficient and scalable services, we take a unique algorithmic approach of harnessing the semantic correlation between the learning-based computation. Rather than viewing individual tasks as isolated black boxes, we optimize them collectively in a white box approach, proposing primitives to formulate the semantics of the deep learning workloads, algorithms to assess their hidden correlation (in terms of the input data, the neural network models, and the deployment trials) and merge common processing steps to minimize redundancy

Diseño centrado en calidad para la difusión Peer-to-Peer de video en vivo

El uso de redes Peer-to-Peer (P2P) es una forma escalable para ofrecer servicios de video sobre Internet. Este documento hace foco en la definición, desarrollo y evaluación de una arquitectura P2P para distribuir video en vivo. El diseño global de la red es guiado por la calidad de experiencia (Quality of Experience - QoE), cuyo principal componente en este caso es la calidad del video percibida por los usuarios finales, en lugar del tradicional diseño basado en la calidad de servicio (Quality of Service - QoE) de la mayoría de los sistemas. Para medir la calidad percibida del video, en tiempo real y automáticamente, extendimos la recientemente propuesta metodología Pseudo-Subjective Quality Assessment (PSQA). Dos grandes líneas de investigación son desarrolladas. Primero, proponemos una técnica de distribución de video desde múltiples fuentes con las características de poder ser optimizada para maximizar la calidad percibida en contextos de muchas fallas y de poseer muy baja señalización (a diferencia de los sistemas existentes). Desarrollamos una metodología, basada en PSQA, que nos permite un control fino sobre la forma en que la señal de video es dividida en partes y la cantidad de redundancia agregada, como una función de la dinámica de los usuarios de la red. De esta forma es posible mejorar la robustez del sistema tanto como sea deseado, contemplando el límite de capacidad en la comunicación. En segundo lugar, presentamos un mecanismo estructurado para controlar la topología de la red. La selección de que usuarios servirán a que otros es importante para la robustez de la red, especialmente cuando los usuarios son heterogéneos en sus capacidades y en sus tiempos de conexión.Nuestro diseño maximiza la calidad global esperada (evaluada usando PSQA), seleccionado una topología que mejora la robustez del sistema. Además estudiamos como extender la red con dos servicios complementarios: el video bajo demanda (Video on Demand - VoD) y el servicio MyTV. El desafío en estos servicios es como realizar búsquedas eficientes sobre la librería de videos, dado al alto dinamismo del contenido. Presentamos una estrategia de "caching" para las búsquedas en estos servicios, que maximiza el número total de respuestas correctas a las consultas, considerando una dinámica particular en los contenidos y restricciones de ancho de banda. Nuestro diseño global considera escenarios reales, donde los casos de prueba y los parámetros de configuración surgen de datos reales de un servicio de referencia en producción. Nuestro prototipo es completamente funcional, de uso gratuito, y basado en tecnologías bien probadas de código abierto

Evaluation of unidirectional background push content download services for the delivery of television programs

Este trabajo de tesis presenta los servicios de descarga de contenido en modo push como un mecanismo eficiente para el envío de contenido de televisión pre-producido sobre redes de difusión. Hoy en día, los operadores de red dedican una cantidad considerable de recursos de red a la entrega en vivo de contenido televisivo, tanto sobre redes de difusión como sobre conexiones unidireccionales. Esta oferta de servicios responde únicamente a requisitos comerciales: disponer de los contenidos televisivos en cualquier momento y lugar. Sin embargo, desde un punto de vista estrictamente académico, el envío en vivo es únicamente un requerimiento para el contenido en vivo, no para contenidos que ya han sido producidos con anterioridad a su emisión. Más aún, la difusión es solo eficiente cuando el contenido es suficientemente popular. Los servicios bajo estudio en esta tesis utilizan capacidad residual en redes de difusión para enviar contenido pre-producido para que se almacene en los equipos de usuario. La propuesta se justifica únicamente por su eficiencia. Por un lado, genera valor de recursos de red que no se aprovecharían de otra manera. Por otro lado, realiza la entrega de contenidos pre-producidos y populares de la manera más eficiente: sobre servicios de descarga de contenidos en difusión. Los resultados incluyen modelos para la popularidad y la duración de contenidos, valiosos para cualquier trabajo de investigación basados en la entrega de contenidos televisivos. Además, la tesis evalúa la capacidad residual disponible en redes de difusión, por medio de estudios empíricos. Después, estos resultados son utilizados en simulaciones que evalúan las prestaciones de los servicios propuestos en escenarios diferentes y para aplicaciones diferentes. La evaluación demuestra que este tipo de servicios son un recurso muy útil para la entrega de contenido televisivo.This thesis dissertation presents background push Content Download Services as an efficient mechanism to deliver pre-produced television content through existing broadcast networks. Nowadays, network operators dedicate a considerable amount of network resources to live streaming live, through both broadcast and unicast connections. This service offering responds solely to commercial requirements: Content must be available anytime and anywhere. However, from a strictly academic point of view, live streaming is only a requirement for live content and not for pre-produced content. Moreover, broadcasting is only efficient when the content is sufficiently popular. The services under study in this thesis use residual capacity in broadcast networks to push popular, pre-produced content to storage capacity in customer premises equipment. The proposal responds only to efficiency requirements. On one hand, it creates value from network resources otherwise unused. On the other hand, it delivers popular pre-produced content in the most efficient way: through broadcast download services. The results include models for the popularity and the duration of television content, valuable for any research work dealing with file-based delivery of television content. Later, the thesis evaluates the residual capacity available in broadcast networks through empirical studies. These results are used in simulations to evaluate the performance of background push content download services in different scenarios and for different applications. The evaluation proves that this kind of services can become a great asset for the delivery of television contentFraile Gil, F. (2013). Evaluation of unidirectional background push content download services for the delivery of television programs [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/31656TESI

Content-prioritised video coding for British Sign Language communication.

Video communication of British Sign Language (BSL) is important for remote interpersonal communication and for the equal provision of services for deaf people. However, the use of video telephony and video conferencing applications for BSL communication is limited by inadequate video quality. BSL is a highly structured, linguistically complete, natural language system that expresses vocabulary and grammar visually and spatially using a complex combination of facial expressions (such as eyebrow movements, eye blinks and mouth/lip shapes), hand gestures, body movements and finger-spelling that change in space and time. Accurate natural BSL communication places specific demands on visual media applications which must compress video image data for efficient transmission. Current video compression schemes apply methods to reduce statistical redundancy and perceptual irrelevance in video image data based on a general model of Human Visual System (HVS) sensitivities. This thesis presents novel video image coding methods developed to achieve the conflicting requirements for high image quality and efficient coding. Novel methods of prioritising visually important video image content for optimised video coding are developed to exploit the HVS spatial and temporal response mechanisms of BSL users (determined by Eye Movement Tracking) and the characteristics of BSL video image content. The methods implement an accurate model of HVS foveation, applied in the spatial and temporal domains, at the pre-processing stage of a current standard-based system (H.264). Comparison of the performance of the developed and standard coding systems, using methods of video quality evaluation developed for this thesis, demonstrates improved perceived quality at low bit rates. BSL users, broadcasters and service providers benefit from the perception of high quality video over a range of available transmission bandwidths. The research community benefits from a new approach to video coding optimisation and better understanding of the communication needs of deaf people

DINÂMICA ESPAÇO TEMPO DA DEMANDA POR INTERNET: AS CIDADES DO FUTURO NO CENÁRIO BRASILEIRO DE TELECOMUNICAÇÕES

As necessidades da sociedade digital, como conectividade e acessibilidade, são alteradas de acordo com o tempo e o espaço. Logo, a dinâmica espaço-temporal da demanda por internet transforma o planejamento de investimentos em infraestrutura de telecomunicações em grande desafio. Antever as necessidades de uma sociedade digital em um futuro próximo é uma tarefa complexa, pois o desempenho do mercado é repleto de inovações e incertezas. Então, como convergir a oferta e a demanda das necessidades da sociedade digital com as cidades do futuro? Este trabalho apresenta pesquisa experimental em que uma metodologia é desenvolvida. Para isso, busca compreender os fenômenos que constituem o comportamento da demanda por internet e avaliar os efeitos espaço-temporais em sua dinâmica. Os resultados mostram que os efeitos ao longo do tempo têm impactos distintos sobre a demanda por internet em cada localidade. Tais observações mostram, sobretudo, que a demanda por internet é uma função dependente do espaço-tempo, capaz de influenciar decisões de investimento em infraestrutura de telecomunicações, maximizando o lucro e gerando externalidades positivas, como, por exemplo, na universalização do acesso à internet, a difusão da sociedade digital e a edificação das cidades do futuro

Popularity-based video caching techniques for cache-enabled networks: a survey

The proliferation of the mobile Internet and connected devices, which offer a variety of services at different levels of performance is a major challenge for the fifth generation of wireless networks and beyond. Innovative solutions are needed to leverage recent advances in machine storage/memory, context awareness, and edge computing. Cache-enabled networks and techniques such as edge caching are envisioned to reduce content delivery times and traffic congestion in wireless networks. Only a few contents are popular, accounting for the majority of viewers, so caching them reduces the latency and download time. However, given the dynamic nature of user behavior, the integration of popularity prediction into caching is of paramount importance to better network utilization and user satisfaction. In this paper, we first present an overview of caching in wireless networks and then provide a detailed comparison of traditional and popularity-based caching. We discuss the attributes of videos and the evaluation criteria of caching policies. We summarize some of the recent work on proactive caching, focusing on prediction strategies. Finally, we provide insight into the potential opportunities and challenges as well as some open research problems enable the realization of efficient deployment of popularity-based caching as part of the next-generation mobile networks

Invest to Save: Report and Recommendations of the NSF-DELOS Working Group on Digital Archiving and Preservation

Digital archiving and preservation are important areas for research and development, but there is no agreed upon set of priorities or coherent plan for research in this area. Research projects in this area tend to be small and driven by particular institutional problems or concerns. As a consequence, proposed solutions from experimental projects and prototypes tend not to scale to millions of digital objects, nor do the results from disparate projects readily build on each other. It is also unclear whether it is worthwhile to seek general solutions or whether different strategies are needed for different types of digital objects and collections. The lack of coordination in both research and development means that there are some areas where researchers are reinventing the wheel while other areas are neglected. Digital archiving and preservation is an area that will benefit from an exercise in analysis, priority setting, and planning for future research. The WG aims to survey current research activities, identify gaps, and develop a white paper proposing future research directions in the area of digital preservation. Some of the potential areas for research include repository architectures and inter-operability among digital archives; automated tools for capture, ingest, and normalization of digital objects; and harmonization of preservation formats and metadata. There can also be opportunities for development of commercial products in the areas of mass storage systems, repositories and repository management systems, and data management software and tools.

Avaliação da qualidade de experiência de vídeo em várias tecnologias

Mestrado em Engenharia Eletrónica e TelecomunicaçõesNowadays the internet is associated with many services. Combined with this fact, there is a marked increase of the users joining this service. In this perspective, it is required that the service providers guarantee a minimum quality to the network services. The Quality of Experience of services is quite crucial in the development of services in networks. Also noteworthy, the tra c increase in multimedia services, including video streaming, increases the probability of congesting the networks. In the perspective of the service provider, the monitoring is a solution to avoid saturation in network. This way, this dissertation proposes to develop a platform that allows a multimedia tra c monitoring in the Meo Go service provided by the operator Portugal Telecom Communications. The architecture of the adaptive streaming over HTTP has been studied and tested to obtain the quality of experience metrics. This adaptive streaming technique presents the smooth streaming, an architecture made by Microsoft company, and it is used in the Meo Go service. Then, it is monitored the metrics obtained with the video player. This analysis is done objectively and subjectively. In this phase, the objective implementation of the method allows to obtain the prediction value of the Quality of Experience by consumers. The selected metrics were derived from the state / performance of network and terminal device. The obtained metrics aim to simulate human action in video score quality. Otherwise, subjectively, it is conducted a survey based in a questionnaire to compare methods. In this phase it was created an on-line platform to allow the obtain a greater number of rankings and data processing. In the obtained results, rstly in the smooth streaming player, it is shown the adaptive streaming implementation technique. On the next phase, test scenarios were created to demonstrate the functioning of the method in many cases, with greater relevance for those ones with higher dynamic complexity. From the perspective of subjective and objective methods, these have values that con rm the architecture of the implemented module. Over time, the performance of the scoring the quality of video streaming services approaches the one in a human mental action.Nos dias de hoje a Internet é um dos meios com mais serviços associados. Conjugado a este facto, existe um acentuado aumento de utilizadores a aderir a este serviço. Nesta perspectiva existe a necessidade de garantir uma qualidade mínima por parte dos prestadores de serviços. A Qualidade de Experiência que os consumidores têm dos serviços é bastante crucial no desenvolvimento e optimização dos serviços nas redes. É ainda de salientar que o aumento do tráfego multimédia, nomeadamente os streamings de vídeo, apresenta incrementos na probabilidade de as redes se congestionarem. Na perspectiva do prestador de serviços a monitorização é a solução para evitar a saturação total. Neste sentido, esta dissertação pretende desenvolver uma plataforma que permite a monitorização do tráfego de multimédia do serviço do Meo Go, fornecido pela operadora Portugal Telecom Comunicações. Neste trabalho foi necessário investigar e testar a arquitectura do streaming adaptativo sobre HTTP para ser possível obter métricas de qualidade de experiência. Este streaming adaptativo apresenta a técnica de smooth streaming, sendo esta arquitectura projectada pela empresa Microsoft e utilizada no serviço Meo Go. Posteriormente foram monitorizadas as métricas que se obtiveram no player de vídeo. Esta análise foi realizada de forma objectiva e subjectiva. Nesta fase da implementação objectiva do método em que se pretende obter uma predição do valor de Qualidade de Experiência por parte do consumidor, foram seleccionadas as métricas oriundas do estado/desempenho da rede e do dispositivo terminal. As métricas obtidas entram num processo de tratamento que pretende simular a ação humana nas classificações da qualidade dos vídeos. De outra forma, subjectivamente, foi realizada uma pesquisa, com base num questionário, de modo a comparar os métodos. Nesta etapa foi gerada uma plataforma online que possibilitou obter um maior número de classificações dos vídeos para posteriormente se proceder ao tratamento de dados. Nos resultados obtidos, primeiramente ao nível do player de smooth streaming, estes permitem analisar a técnica de implementação de streaming adaptativo. Numa fase seguinte foram criados cenários de teste para comprovar o funcionamento do método em diversas situações, tendo com maior relevância aqueles que contêm dinâmicas mais complexas. Na perspectiva dos métodos subjectivo e objectivo, estes apresentam valores que confirmam a arquitectura do módulo implementado. Adicionalmente, o desempenho do método em classificar a qualidade de serviço de vídeo streaming, ao longo do tempo, apresentou valores que se aproximam da dinâmica esperada numa ação mental humana