Fast and Lean Immutable Multi-Maps on the JVM based on Heterogeneous Hash-Array Mapped Tries

An immutable multi-map is a many-to-many thread-friendly map data structure with expected fast insert and lookup operations. This data structure is used for applications processing graphs or many-to-many relations as applied in static analysis of object-oriented systems. When processing such big data sets the memory overhead of the data structure encoding itself is a memory usage bottleneck. Motivated by reuse and type-safety, libraries for Java, Scala and Clojure typically implement immutable multi-maps by nesting sets as the values with the keys of a trie map. Like this, based on our measurements the expected byte overhead for a sparse multi-map per stored entry adds up to around 65B, which renders it unfeasible to compute with effectively on the JVM. In this paper we propose a general framework for Hash-Array Mapped Tries on the JVM which can store type-heterogeneous keys and values: a Heterogeneous Hash-Array Mapped Trie (HHAMT). Among other applications, this allows for a highly efficient multi-map encoding by (a) not reserving space for empty value sets and (b) inlining the values of singleton sets while maintaining a (c) type-safe API. We detail the necessary encoding and optimizations to mitigate the overhead of storing and retrieving heterogeneous data in a hash-trie. Furthermore, we evaluate HHAMT specifically for the application to multi-maps, comparing them to state-of-the-art encodings of multi-maps in Java, Scala and Clojure. We isolate key differences using microbenchmarks and validate the resulting conclusions on a real world case in static analysis. The new encoding brings the per key-value storage overhead down to 30B: a 2x improvement. With additional inlining of primitive values it reaches a 4x improvement

Use of Information Visualization Techniques for Collection Management in Libraries: A Conceptual Review

This paper presents a conceptual review exploring the application of information visualization techniques in the context of collection management in libraries. Collection management plays a crucial role in ensuring libraries offer relevant and diverse resources to meet the information needs of users. Information visualization, with its ability to visually represent complex data, has emerged as a powerful tool for enhancing collection management practices. Drawing upon a comprehensive literature review, this paper examines the theoretical foundations, benefits, challenges, and practical applications of information visualization techniques in library collection management. It discusses various visualization methods, such as charts, graphs, and maps, and explores their potential in assessing collection composition, analyzing usage patterns, and supporting decision-making processes. The paper highlights the benefits of information visualization in improving user engagement, optimizing resource allocation, and facilitating data-driven decision making. It also addresses challenges related to data integration, technology infrastructure, and ethical considerations. Through real-world case studies and examples, this conceptual review provides insights into successful implementations of information visualization in collection management. The paper concludes by emphasizing the potential of information visualization techniques to transform collection management practices in libraries, enhancing the accessibility, relevance, and impact of library resources

GiViP: A Visual Profiler for Distributed Graph Processing Systems

Analyzing large-scale graphs provides valuable insights in different application scenarios. While many graph processing systems working on top of distributed infrastructures have been proposed to deal with big graphs, the tasks of profiling and debugging their massive computations remain time consuming and error-prone. This paper presents GiViP, a visual profiler for distributed graph processing systems based on a Pregel-like computation model. GiViP captures the huge amount of messages exchanged throughout a computation and provides an interactive user interface for the visual analysis of the collected data. We show how to take advantage of GiViP to detect anomalies related to the computation and to the infrastructure, such as slow computing units and anomalous message patterns.Comment: Appears in the Proceedings of the 25th International Symposium on Graph Drawing and Network Visualization (GD 2017

Goal-oriented Dialogue Policy Learning from Failures

Reinforcement learning methods have been used for learning dialogue policies. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the very few successful dialogues in early learning phase. Hindsight experience replay (HER) enables learning from failures, but the vanilla HER is inapplicable to dialogue learning due to the implicit goals. In this work, we develop two complex HER methods providing different trade-offs between complexity and performance, and, for the first time, enabled HER-based dialogue policy learning. Experiments using a realistic user simulator show that our HER methods perform better than existing experience replay methods (as applied to deep Q-networks) in learning rate