582 research outputs found
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
(b2023 to 2014) The UNBELIEVABLE similarities between the ideas of some people (2006-2016) and my ideas (2002-2008) in physics (quantum mechanics, cosmology), cognitive neuroscience, philosophy of mind, and philosophy (this manuscript would require a REVOLUTION in international academy environment!)
(b2023 to 2014) The UNBELIEVABLE similarities between the ideas of some people (2006-2016) and my ideas (2002-2008) in physics (quantum mechanics, cosmology), cognitive neuroscience, philosophy of mind, and philosophy (this manuscript would require a REVOLUTION in international academy environment!
Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming
Sketching algorithms have recently proven to be a powerful approach both for
designing low-space streaming algorithms as well as fast polynomial time
approximation schemes (PTAS). In this work, we develop new techniques to extend
the applicability of sketching-based approaches to the sparse dictionary
learning and the Euclidean -means clustering problems. In particular, we
initiate the study of the challenging setting where the dictionary/clustering
assignment for each of the input points must be output, which has
surprisingly received little attention in prior work. On the fast algorithms
front, we obtain a new approach for designing PTAS's for the -means
clustering problem, which generalizes to the first PTAS for the sparse
dictionary learning problem. On the streaming algorithms front, we obtain new
upper bounds and lower bounds for dictionary learning and -means clustering.
In particular, given a design matrix in a
turnstile stream, we show an space
upper bound for -sparse dictionary learning of size , an space upper bound for -means clustering, as
well as an space upper bound for -means clustering on random
order row insertion streams with a natural "bounded sensitivity" assumption. On
the lower bounds side, we obtain a general lower bound for -means clustering, as well as an
lower bound for algorithms which can estimate the
cost of a single fixed set of candidate centers.Comment: To appear in NeurIPS 202
Solving Tensor Low Cycle Rank Approximation
Large language models have become ubiquitous in modern life, finding
applications in various domains such as natural language processing, language
translation, and speech recognition. Recently, a breakthrough work [Zhao,
Panigrahi, Ge, and Arora Arxiv 2023] explains the attention model from
probabilistic context-free grammar (PCFG). One of the central computation task
for computing probability in PCFG is formulating a particular tensor low rank
approximation problem, we can call it tensor cycle rank. Given an third order tensor , we say that has cycle rank- if there
exists three size matrices , and such that for each
entry in each \begin{align*} A_{a,b,c} = \sum_{i=1}^k \sum_{j=1}^k \sum_{l=1}^k
U_{a,i+k(j-1)} \otimes V_{b, j + k(l-1)} \otimes W_{c, l + k(i-1) }
\end{align*} for all . For the tensor
classical rank, tucker rank and train rank, it has been well studied in [Song,
Woodruff, Zhong SODA 2019]. In this paper, we generalize the previous
``rotation and sketch'' technique in page 186 of [Song, Woodruff, Zhong SODA
2019] and show an input sparsity time algorithm for cycle rank
Algorithms for sparse convolution and sublinear edit distance
In this PhD thesis on fine-grained algorithm design and complexity, we investigate output-sensitive and sublinear-time algorithms for two important problems. (1) Sparse Convolution: Computing the convolution of two vectors is a basic algorithmic primitive with applications across all of Computer Science and Engineering. In the sparse convolution problem we assume that the input and output vectors have at most t nonzero entries, and the goal is to design algorithms with running times dependent on t. For the special case where all entries are nonnegative, which is particularly important for algorithm design, it is known since twenty years that sparse convolutions can be computed in near-linear randomized time O(t log^2 n). In this thesis we develop a randomized algorithm with running time O(t \log t) which is optimal (under some mild assumptions), and the first near-linear deterministic algorithm for sparse nonnegative convolution. We also present an application of these results, leading to seemingly unrelated fine-grained lower bounds against distance oracles in graphs. (2) Sublinear Edit Distance: The edit distance of two strings is a well-studied similarity measure with numerous applications in computational biology. While computing the edit distance exactly provably requires quadratic time, a long line of research has lead to a constant-factor approximation algorithm in almost-linear time. Perhaps surprisingly, it is also possible to approximate the edit distance k within a large factor O(k) in sublinear time O~(n/k + poly(k)). We drastically improve the approximation factor of the known sublinear algorithms from O(k) to k^{o(1)} while preserving the O(n/k + poly(k)) running time.In dieser Doktorarbeit über feinkörnige Algorithmen und Komplexität untersuchen wir ausgabesensitive Algorithmen und Algorithmen mit sublinearer Lauf-zeit für zwei wichtige Probleme. (1) Dünne Faltungen: Die Berechnung der Faltung zweier Vektoren ist ein grundlegendes algorithmisches Primitiv, das in allen Bereichen der Informatik und des Ingenieurwesens Anwendung findet. Für das dünne Faltungsproblem nehmen wir an, dass die Eingabe- und Ausgabevektoren höchstens t Einträge ungleich Null haben, und das Ziel ist, Algorithmen mit Laufzeiten in Abhängigkeit von t zu entwickeln. Für den speziellen Fall, dass alle Einträge nicht-negativ sind, was insbesondere für den Entwurf von Algorithmen relevant ist, ist seit zwanzig Jahren bekannt, dass dünn besetzte Faltungen in nahezu linearer randomisierter Zeit O(t \log^2 n) berechnet werden können. In dieser Arbeit entwickeln wir einen randomisierten Algorithmus mit Laufzeit O(t \log t), der (unter milden Annahmen) optimal ist, und den ersten nahezu linearen deterministischen Algorithmus für dünne nichtnegative Faltungen. Wir stellen auch eine Anwendung dieser Ergebnisse vor, die zu scheinbar unverwandten feinkörnigen unteren Schranken gegen Distanzorakel in Graphen führt. (2) Sublineare Editierdistanz: Die Editierdistanz zweier Zeichenketten ist ein gut untersuchtes Ähnlichkeitsmaß mit zahlreichen Anwendungen in der Computerbiologie. Während die exakte Berechnung der Editierdistanz nachweislich quadratische Zeit erfordert, hat eine lange Reihe von Forschungsarbeiten zu einem Approximationsalgorithmus mit konstantem Faktor in fast-linearer Zeit geführt. Überraschenderweise ist es auch möglich, die Editierdistanz k innerhalb eines großen Faktors O(k) in sublinearer Zeit O~(n/k + poly(k)) zu approximieren. Wir verbessern drastisch den Approximationsfaktor der bekannten sublinearen Algorithmen von O(k) auf k^{o(1)} unter Beibehaltung der O(n/k + poly(k))-Laufzeit
LIPIcs, Volume 261, ICALP 2023, Complete Volume
LIPIcs, Volume 261, ICALP 2023, Complete Volum
Compiling higher-order specifications to SMT solvers : how to deal with rejection constructively
Modern verification tools frequently rely on compiling high-level specifications to SMT queries. However, the high-level specification language is usually more expressive than the available solvers and therefore some syntactically valid specifications must be rejected by the tool. In such cases, the challenge is to provide a comprehensible error message to the user that relates the original syntactic form of the specification to the semantic reason it has been rejected. In this paper we demonstrate how this analysis may be performed by combining a standard unification-based type-checker with type classes and automatic generalisation. Concretely, type-checking is used as a constructive procedure for under-approximating whether a given specification lies in the subset of problems supported by the solver. Any resulting proof of rejection can be transformed into a detailed explanation to the user. The approach is compositional and does not require the user to add extra typing annotations to their program. We subsequently describe how the type system may be leveraged to provide a sound and complete compilation procedure from suitably typed expressions to SMT queries, which we have verified in Agda
Gabriel Vacariu (c2023 to 2014) The UNBELIEVABLE similarities between the ideas of some people (2006-2016) and my ideas (2002-2008) in physics (quantum mechanics, cosmology), cognitive neuroscience, philosophy of mind, and philosophy
Unbelievable similar ideas to my ideas published long before..
Domain Theory in Constructive and Predicative Univalent Foundations
We develop domain theory in constructive and predicative univalent
foundations (also known as homotopy type theory). That we work predicatively
means that we do not assume Voevodsky's propositional resizing axioms. Our work
is constructive in the sense that we do not rely on excluded middle or the
axiom of (countable) choice. Domain theory studies so-called directed complete
posets (dcpos) and Scott continuous maps between them and has applications in
programming language semantics, higher-type computability and topology. A
common approach to deal with size issues in a predicative foundation is to work
with information systems, abstract bases or formal topologies rather than
dcpos, and approximable relations rather than Scott continuous functions. In
our type-theoretic approach, we instead accept that dcpos may be large and work
with type universes to account for this. A priori one might expect that complex
constructions of dcpos result in a need for ever-increasing universes and are
predicatively impossible. We show that such constructions can be carried out in
a predicative setting. We illustrate the development with applications in the
semantics of programming languages: the soundness and computational adequacy of
the Scott model of PCF and Scott's model of the untyped
-calculus. We also give a predicative account of continuous and
algebraic dcpos, and of the related notions of a small basis and its rounded
ideal completion. The fact that nontrivial dcpos have large carriers is in fact
unavoidable and characteristic of our predicative setting, as we explain in a
complementary chapter on the constructive and predicative limitations of
univalent foundations. Our account of domain theory in univalent foundations is
fully formalised with only a few minor exceptions. The ability of the proof
assistant Agda to infer universe levels has been invaluable for our purposes.Comment: PhD thesis, extended abstract in the pdf. v5: Fixed minor typos in
6.2.18, 6.2.19 and 6.4.
- …