3,320 research outputs found
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
Many recent works on knowledge distillation have provided ways to transfer
the knowledge of a trained network for improving the learning process of a new
one, but finding a good technique for knowledge distillation is still an open
problem. In this paper, we provide a new perspective based on a decision
boundary, which is one of the most important component of a classifier. The
generalization performance of a classifier is closely related to the adequacy
of its decision boundary, so a good classifier bears a good decision boundary.
Therefore, transferring information closely related to the decision boundary
can be a good attempt for knowledge distillation. To realize this goal, we
utilize an adversarial attack to discover samples supporting a decision
boundary. Based on this idea, to transfer more accurate information about the
decision boundary, the proposed algorithm trains a student classifier based on
the adversarial samples supporting the decision boundary. Experiments show that
the proposed method indeed improves knowledge distillation and achieves the
state-of-the-arts performance.Comment: Accepted to AAAI 201
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
An activation boundary for a neuron refers to a separating hyperplane that
determines whether the neuron is activated or deactivated. It has been long
considered in neural networks that the activations of neurons, rather than
their exact output values, play the most important role in forming
classification friendly partitions of the hidden feature space. However, as far
as we know, this aspect of neural networks has not been considered in the
literature of knowledge transfer. In this paper, we propose a knowledge
transfer method via distillation of activation boundaries formed by hidden
neurons. For the distillation, we propose an activation transfer loss that has
the minimum value when the boundaries generated by the student coincide with
those by the teacher. Since the activation transfer loss is not differentiable,
we design a piecewise differentiable loss approximating the activation transfer
loss. By the proposed method, the student learns a separating boundary between
activation region and deactivation region formed by each neuron in the teacher.
Through the experiments in various aspects of knowledge transfer, it is
verified that the proposed method outperforms the current state-of-the-art.Comment: Accepted to AAAI 201
FleXR: A System Enabling Flexibly Distributed Extended Reality
Extended reality (XR) applications require computationally demanding
functionalities with low end-to-end latency and high throughput. To enable XR
on commodity devices, a number of distributed systems solutions enable
offloading of XR workloads on remote servers. However, they make a priori
decisions regarding the offloaded functionalities based on assumptions about
operating factors, and their benefits are restricted to specific deployment
contexts. To realize the benefits of offloading in various distributed
environments, we present a distributed stream processing system, FleXR, which
is specialized for real-time and interactive workloads and enables flexible
distributions of XR functionalities. In building FleXR, we identified and
resolved several issues of presenting XR functionalities as distributed
pipelines. FleXR provides a framework for flexible distribution of XR pipelines
while streamlining development and deployment phases. We evaluate FleXR with
three XR use cases in four different distribution scenarios. In the results,
the best-case distribution scenario shows up to 50% less end-to-end latency and
3.9x pipeline throughput compared to alternatives.Comment: 11 pages, 11 figures, conference pape
Poster: Enabling Flexible Edge-assisted XR
Extended reality (XR) is touted as the next frontier of the digital future.
XR includes all immersive technologies of augmented reality (AR), virtual
reality (VR), and mixed reality (MR). XR applications obtain the real-world
context of the user from an underlying system, and provide rich, immersive, and
interactive virtual experiences based on the user's context in real-time. XR
systems process streams of data from device sensors, and provide
functionalities including perceptions and graphics required by the
applications. These processing steps are computationally intensive, and the
challenge is that they must be performed within the strict latency requirements
of XR. This poses limitations on the possible XR experiences that can be
supported on mobile devices with limited computing resources.
In this XR context, edge computing is an effective approach to address this
problem for mobile users. The edge is located closer to the end users and
enables processing and storing data near them. In addition, the development of
high bandwidth and low latency network technologies such as 5G facilitates the
application of edge computing for latency-critical use cases [4, 11]. This work
presents an XR system for enabling flexible edge-assisted XR.Comment: extended abstract of 2 pages, 1 figure, 2 table
- β¦