MagicVideo: Efficient Video Generation With Latent Diffusion Models

We present an efficient text-to-video generation framework based on latent diffusion models, termed MagicVideo. Given a text description, MagicVideo can generate photo-realistic video clips with high relevance to the text content. With the proposed efficient latent 3D U-Net design, MagicVideo can generate video clips with 256x256 spatial resolution on a single GPU card, which is 64x faster than the recent video diffusion model (VDM). Unlike previous works that train video generation from scratch in the RGB space, we propose to generate video clips in a low-dimensional latent space. We further utilize all the convolution operator weights of pre-trained text-to-image generative U-Net models for faster training. To achieve this, we introduce two new designs to adapt the U-Net decoder to video data: a framewise lightweight adaptor for the image-to-video distribution adjustment and a directed temporal attention module to capture frame temporal dependencies. The whole generation process is within the low-dimension latent space of a pre-trained variation auto-encoder. We demonstrate that MagicVideo can generate both realistic video content and imaginary content in a photo-realistic style with a trade-off in terms of quality and computational cost. Refer to https://magicvideo.github.io/# for more examples

A service oriented broker-based approach for dynamic resource discovery in virtual networks

© 2015, Rabah et al.; licensee Springer. In the past few years, the concept of network virtualization has received significant attention from industry and research fora. This concept applies virtualization to networking infrastructures by enabling the dynamic creation of several co-existing logical network instances (or virtual networks) over a shared physical network infrastructure (or substrate network). Due to the potential it offers in terms of diversifying existing networks and ensuring the co-existence of heterogeneous network architectures on top of shared substrates, network virtualization is often considered as an enabler of a polymorphic Internet and a cornerstone of the future Internet architecture. One of the challenges associated with the network virtualization concept is the description, publication, and discovery of virtual resources that can be composed to form virtual networks. To achieve those tasks, there is a need for an expressive information model facilitating information representation and sharing, as well as an efficient resource publication and discovery framework. In this paper, we propose a service oriented, broker-based framework for virtual resource description, publication, and discovery. This framework relies on a novel service-oriented hierarchical business model and an expressive information model for resources/services description. The detailed framework’s architecture is presented, and its operation is illustrated using a REST-based content distribution scenario. Furthermore, a proof-of-concept prototype implementation realized using various technologies/tools (e.g. Jersey, JAXB, PostgreSQL, and Xen cloud platform) is presented along with a detailed performance analysis of the system. When compared to existing virtual resource discovery frameworks, our broker-based virtual resource discovery framework offers signification performance improvements of the virtual resources’ discovery operation, in terms of response time (92.8% improvement) and incurred network load (77.3% improvement), when dealing with multiple resource providers. Furthermore, relying on a broker as intermediary role simplifies the resources’ discovery and selection operations, and improves the overall efficiency of the virtual network embedding process

Modeling toolkit for comparing AC vs. DC electrical distribution efficiency in buildings, A

2021 Summer.Includes bibliographical references.An increasing proportion of electrical devices in residential and commercial buildings operate from direct current (DC) power sources. In addition, distributed power generation systems such as solar photovoltaic (PV) and energy storage natively produce DC power. However, traditional power distribution is based on an alternating current (AC) model. Performing the necessary conversions between AC and DC power to make DC devices compatible with AC distribution results in energy losses. For these reasons, DC distribution may offer energy efficiency advantages in comparison to AC distribution. However, reasonably fast computation and comparison of electrical efficiencies of AC-only, DC-only, and hybrid AC/DC distributions systems is challenging because DC devices are typically (nonlinear) power-electronic converters that produce harmonic content. While detailed time-domain modeling can be used to simulate these harmonics, it is not computationally efficient or practical for many building designers. To address this need, this research describes a toolkit for computation of harmonic spectra and energy efficiency in mixed AC and DC electrical distribution systems, using a Harmonic Power Flow (HPF) methodology. The toolkit includes a library of two-port linear and nonlinear device models which can be used to construct and simulate an electrical distribution system. This dissertation includes a description of the mathematical theory and framework underlying the toolkit, development and fitting of linear and nonlinear device models, software implementation in Modelica, verification of the toolkit with laboratory measurements, and discussion of ongoing and future work to employ the toolkit to a variety of building designs

Patch-based semantic labelling of images.

PhDThe work presented in this thesis is focused at associating a semantics to the content of an image, linking the content to high level semantic categories. The process can take place at two levels: either at image level, towards image categorisation, or at pixel level, in se- mantic segmentation or semantic labelling. To this end, an analysis framework is proposed, and the different steps of part (or patch) extraction, description and probabilistic modelling are detailed. Parts of different nature are used, and one of the contributions is a method to complement information associated to them. Context for parts has to be considered at different scales. Short range pixel dependences are accounted by associating pixels to larger patches. A Conditional Random Field, that is, a probabilistic discriminative graphical model, is used to model medium range dependences between neighbouring patches. Another contribution is an efficient method to consider rich neighbourhoods without having loops in the inference graph. To this end, weak neighbours are introduced, that is, neighbours whose label probability distribution is pre-estimated rather than mutable during the inference. Longer range dependences, that tend to make the inference problem intractable, are addressed as well. A novel descriptor based on local histograms of visual words has been proposed, meant to both complement the feature descriptor of the patches and augment the context awareness in the patch labelling process. Finally, an alternative approach to consider multiple scales in a hierarchical framework based on image pyramids is proposed. An image pyramid is a compositional representation of the image based on hierarchical clustering. All the presented contributions are extensively detailed throughout the thesis, and experimental results performed on publicly available datasets are reported to assess their validity. A critical comparison with the state of the art in this research area is also presented, and the advantage in adopting the proposed improvements are clearly highlighted

Multimedia delivery in the future internet

The term “Networked Media” implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on-line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Internet has evolved and changed the way we work and live. End users of the Internet have been confronted with a bewildering range of media, services and applications and of technological innovations concerning media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens’ quality of life, working conditions, edutainment and safety. In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and innovative applications “on the move”, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6) and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way ahead in the area of Content Aware media delivery platforms

A schema-based P2P network to enable publish-subscribe for multimedia content in open hypermedia systems

Open Hypermedia Systems (OHS) aim to provide efficient dissemination, adaptation and integration of hyperlinked multimedia resources. Content available in Peer-to-Peer (P2P) networks could add significant value to OHS provided that challenges for efficient discovery and prompt delivery of rich and up-to-date content are successfully addressed. This paper proposes an architecture that enables the operation of OHS over a P2P overlay network of OHS servers based on semantic annotation of (a) peer OHS servers and of (b) multimedia resources that can be obtained through the link services of the OHS. The architecture provides efficient resource discovery. Semantic query-based subscriptions over this P2P network can enable access to up-to-date content, while caching at certain peers enables prompt delivery of multimedia content. Advanced query resolution techniques are employed to match different parts of subscription queries (subqueries). These subscriptions can be shared among different interested peers, thus increasing the efficiency of multimedia content dissemination