2,745 research outputs found
Deep generative models for network data synthesis and monitoring
Measurement and monitoring are fundamental tasks in all networks, enabling the down-stream management and optimization of the network.
Although networks inherently
have abundant amounts of monitoring data, its access and effective measurement is
another story. The challenges exist in many aspects. First, the inaccessibility of network monitoring data for external users, and it is hard to provide a high-fidelity dataset
without leaking commercial sensitive information. Second, it could be very expensive
to carry out effective data collection to cover a large-scale network system, considering the size of network growing, i.e., cell number of radio network and the number of
flows in the Internet Service Provider (ISP) network. Third, it is difficult to ensure fidelity and efficiency simultaneously in network monitoring, as the available resources
in the network element that can be applied to support the measurement function are
too limited to implement sophisticated mechanisms. Finally, understanding and explaining the behavior of the network becomes challenging due to its size and complex
structure. Various emerging optimization-based solutions (e.g., compressive sensing)
or data-driven solutions (e.g. deep learning) have been proposed for the aforementioned challenges. However, the fidelity and efficiency of existing methods cannot yet
meet the current network requirements.
The contributions made in this thesis significantly advance the state of the art in
the domain of network measurement and monitoring techniques. Overall, we leverage
cutting-edge machine learning technology, deep generative modeling, throughout the
entire thesis. First, we design and realize APPSHOT , an efficient city-scale network
traffic sharing with a conditional generative model, which only requires open-source
contextual data during inference (e.g., land use information and population distribution). Second, we develop an efficient drive testing system — GENDT, based on generative model, which combines graph neural networks, conditional generation, and quantified model uncertainty to enhance the efficiency of mobile drive testing. Third, we
design and implement DISTILGAN, a high-fidelity, efficient, versatile, and real-time
network telemetry system with latent GANs and spectral-temporal networks. Finally,
we propose SPOTLIGHT , an accurate, explainable, and efficient anomaly detection system of the Open RAN (Radio Access Network) system. The lessons learned through
this research are summarized, and interesting topics are discussed for future work in
this domain. All proposed solutions have been evaluated with real-world datasets and
applied to support different applications in real systems
On the Generation of Realistic and Robust Counterfactual Explanations for Algorithmic Recourse
This recent widespread deployment of machine learning algorithms presents many new challenges. Machine learning algorithms are usually opaque and can be particularly difficult to interpret. When humans are involved, algorithmic and automated decisions can negatively impact people’s lives. Therefore, end users would like to be insured against potential harm. One popular way to achieve this is to provide end users access to algorithmic recourse, which gives end users negatively affected by algorithmic decisions the opportunity to reverse unfavorable decisions, e.g., from a loan denial to a loan acceptance. In this thesis, we design recourse algorithms to meet various end user needs. First, we propose methods for the generation of realistic recourses. We use generative models to suggest recourses likely to occur under the data distribution. To this end, we shift the recourse action from the input space to the generative model’s latent space, allowing to generate counterfactuals that lie in regions with data support. Second, we observe that small changes applied to the recourses prescribed to end users likely invalidate the suggested recourse after being nosily implemented in practice. Motivated by this observation, we design methods for the generation of robust recourses and for assessing the robustness of recourse algorithms to data deletion requests. Third, the lack of a commonly used code-base for counterfactual explanation and algorithmic recourse algorithms and the vast array of evaluation measures in literature make it difficult to compare the per formance of different algorithms. To solve this problem, we provide an open source benchmarking library that streamlines the evaluation process and can be used for benchmarking, rapidly developing new methods, and setting up new
experiments. In summary, our work contributes to a more reliable interaction of end users and machine learned models by covering fundamental aspects of the recourse process and suggests new solutions towards generating realistic and robust counterfactual explanations for algorithmic recourse
Natural and Technological Hazards in Urban Areas
Natural hazard events and technological accidents are separate causes of environmental impacts. Natural hazards are physical phenomena active in geological times, whereas technological hazards result from actions or facilities created by humans. In our time, combined natural and man-made hazards have been induced. Overpopulation and urban development in areas prone to natural hazards increase the impact of natural disasters worldwide. Additionally, urban areas are frequently characterized by intense industrial activity and rapid, poorly planned growth that threatens the environment and degrades the quality of life. Therefore, proper urban planning is crucial to minimize fatalities and reduce the environmental and economic impacts that accompany both natural and technological hazardous events
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
Exploration autonome et efficiente de chantiers miniers souterrains inconnus avec un drone filaire
Abstract: Underground mining stopes are often mapped using a sensor located at the end of a pole that the operator introduces into the stope from a secure area. The sensor emits laser beams that provide the distance to a detected wall, thus creating a 3D map. This produces shadow zones and a low point density on the distant walls. To address these challenges, a research team from the Université de Sherbrooke is designing a tethered drone equipped with a rotating LiDAR for this mission, thus benefiting from several points of view. The wired transmission allows for unlimited flight time, shared computing, and real-time communication. For compatibility with the movement of the drone after tether entanglements, the excess length is integrated into an onboard spool, contributing to the drone payload. During manual piloting, the human factor causes problems in the perception and comprehension of a virtual 3D environment, as well as the execution of an optimal mission. This thesis focuses on autonomous navigation in two aspects: path planning and exploration. The system must compute a trajectory that maps the entire environment, minimizing the mission time and respecting the maximum onboard tether length. Path planning using a Rapidly-exploring Random Tree (RRT) quickly finds a feasible path, but the optimization is computationally expensive and the performance is variable and unpredictable. Exploration by the frontier method is representative of the space to be explored and the path can be optimized by solving a Traveling Salesman Problem (TSP) but existing techniques for a tethered drone only consider the 2D case and do not optimize the global path. To meet these challenges, this thesis presents two new algorithms. The first one, RRT-Rope, produces an equal or shorter path than existing algorithms in a significantly shorter computation time, up to 70% faster than the next best algorithm in a representative environment. A modified version of RRT-connect computes a feasible path, shortened with a deterministic technique that takes advantage of previously added intermediate nodes. The second algorithm, TAPE, is the first 3D cavity exploration method that focuses on minimizing mission time and unwound tether length. On average, the overall path is 4% longer than the method that solves the TSP, but the tether remains under the allowed length in 100% of the simulated cases, compared to 53% with the initial method. The approach uses a 2-level hierarchical architecture: global planning solves a TSP after frontier extraction, and local planning minimizes the path cost and tether length via a decision function. The integration of these two tools in the NetherDrone produces an intelligent system for autonomous exploration, with semi-autonomous features for operator interaction. This work opens the door to new navigation approaches in the field of inspection, mapping, and Search and Rescue missions.La cartographie des chantiers miniers souterrains est souvent réalisée à l’aide d’un capteur situé au bout d’une perche que l’opérateur introduit dans le chantier, depuis une zone sécurisée. Le capteur émet des faisceaux laser qui fournissent la distance à un mur détecté, créant ainsi une carte en 3D. Ceci produit des zones d’ombres et une faible densité de points sur les parois éloignées. Pour relever ces défis, une équipe de recherche de l’Université de Sherbrooke conçoit un drone filaire équipé d’un LiDAR rotatif pour cette mission, bénéficiant ainsi de plusieurs points de vue. La transmission filaire permet un temps de vol illimité, un partage de calcul et une communication en temps réel. Pour une compatibilité avec le mouvement du drone lors des coincements du fil, la longueur excédante est intégrée dans une bobine embarquée, qui contribue à la charge utile du drone. Lors d’un pilotage manuel, le facteur humain entraîne des problèmes de perception et compréhension d’un environnement 3D virtuel, et d’exécution d’une mission optimale. Cette thèse se concentre sur la navigation autonome sous deux aspects : la planification de trajectoire et l’exploration. Le système doit calculer une trajectoire qui cartographie l’environnement complet, en minimisant le temps de mission et en respectant la longueur maximale de fil embarquée. La planification de trajectoire à l’aide d’un Rapidly-exploring Random Tree (RRT) trouve rapidement un chemin réalisable, mais l’optimisation est coûteuse en calcul et la performance est variable et imprévisible. L’exploration par la méthode des frontières est représentative de l’espace à explorer et le chemin peut être optimisé en résolvant un Traveling Salesman Problem (TSP), mais les techniques existantes pour un drone filaire ne considèrent que le cas 2D et n’optimisent pas le chemin global. Pour relever ces défis, cette thèse présente deux nouveaux algorithmes. Le premier, RRT-Rope, produit un chemin égal ou plus court que les algorithmes existants en un temps de calcul jusqu’à 70% plus court que le deuxième meilleur algorithme dans un environnement représentatif. Une version modifiée de RRT-connect calcule un chemin réalisable, raccourci avec une technique déterministe qui tire profit des noeuds intermédiaires préalablement ajoutés. Le deuxième algorithme, TAPE, est la première méthode d’exploration de cavités en 3D qui minimise le temps de mission et la longueur du fil déroulé. En moyenne, le trajet global est 4% plus long que la méthode qui résout le TSP, mais le fil reste sous la longueur autorisée dans 100% des cas simulés, contre 53% avec la méthode initiale. L’approche utilise une architecture hiérarchique à 2 niveaux : la planification globale résout un TSP après extraction des frontières, et la planification locale minimise le coût du chemin et la longueur de fil via une fonction de décision. L’intégration de ces deux outils dans le NetherDrone produit un système intelligent pour l’exploration autonome, doté de fonctionnalités semi-autonomes pour une interaction avec l’opérateur. Les travaux réalisés ouvrent la porte à de nouvelles approches de navigation dans le domaine des missions d’inspection, de cartographie et de recherche et sauvetage
Borel versions of the Local Lemma and LOCAL algorithms for graphs of finite asymptotic separation index
Asymptotic separation index is a parameter that measures how easily a Borel
graph can be approximated by its subgraphs with finite components. In contrast
to the more classical notion of hyperfiniteness, asymptotic separation index is
well-suited for combinatorial applications in the Borel setting. The main
result of this paper is a Borel version of the Lov\'asz Local Lemma -- a
powerful general-purpose tool in probabilistic combinatorics -- under a finite
asymptotic separation index assumption. As a consequence, we show that locally
checkable labeling problems that are solvable by efficient randomized
distributed algorithms admit Borel solutions on bounded degree Borel graphs
with finite asymptotic separation index. From this we derive a number of
corollaries, for example a Borel version of Brooks's theorem for graphs with
finite asymptotic separation index
Finding Small Complete Subgraphs Efficiently
(I) We revisit the algorithmic problem of finding all triangles in a graph
with vertices and edges. According to a result of Chiba and
Nishizeki (1985), this task can be achieved by a combinatorial algorithm
running in time, where is the
graph arboricity. We provide a new very simple combinatorial algorithm for
finding all triangles in a graph and show that is amenable to the same running
time analysis. We derive these worst-case bounds from first principles and with
very simple proofs that do not rely on classic results due to Nash-Williams
from the 1960s.
(II) We extend our arguments to the problem of finding all small complete
subgraphs of a given fixed size. We show that the dependency on and
in the running time of the algorithm of
Chiba and Nishizeki for listing all copies of , where , is
asymptotically tight.
(III) We give improved arboricity-sensitive running times for counting and/or
detection of copies of , for small . A key ingredient in
our algorithms is, once again, the algorithm of Chiba and Nishizeki. Our new
algorithms are faster than all previous algorithms in certain high-range
arboricity intervals for every .Comment: 14 pages, 1 figure. arXiv admin note: substantial text overlap with
arXiv:2105.0126
- …