139 research outputs found
Extractor-Based Time-Space Lower Bounds for Learning
A matrix corresponds to the following
learning problem: An unknown element is chosen uniformly at random. A
learner tries to learn from a stream of samples, , where for every , is chosen uniformly at random and
.
Assume that are such that any submatrix of of at least
rows and at least columns, has a bias
of at most . We show that any learning algorithm for the learning
problem corresponding to requires either a memory of size at least
, or at least samples. The
result holds even if the learner has an exponentially small success probability
(of ).
In particular, this shows that for a large class of learning problems, any
learning algorithm requires either a memory of size at least or an exponential number of samples, achieving a
tight lower bound on the size
of the memory, rather than a bound of obtained in previous works [R17,MM17b].
Moreover, our result implies all previous memory-samples lower bounds, as
well as a number of new applications.
Our proof builds on [R17] that gave a general technique for proving
memory-samples lower bounds
Recommended from our members
Time-Space Lower Bounds for Two-Pass Learning
A line of recent works showed that for a large class of learning problems, any learning algorithm requires either super-linear memory size or a super-polynomial number of samples [Raz, 2016; Kol et al., 2017; Raz, 2017; Moshkovitz and Moshkovitz, 2018; Beame et al., 2018; Garg et al., 2018]. For example, any algorithm for learning parities of size n requires either a memory of size Omega(n^{2}) or an exponential number of samples [Raz, 2016].
All these works modeled the learner as a one-pass branching program, allowing only one pass over the stream of samples. In this work, we prove the first memory-samples lower bounds (with a super-linear lower bound on the memory size and super-polynomial lower bound on the number of samples) when the learner is allowed two passes over the stream of samples. For example, we prove that any two-pass algorithm for learning parities of size n requires either a memory of size Omega(n^{1.5}) or at least 2^{Omega(sqrt{n})} samples.
More generally, a matrix M: A x X - > {-1,1} corresponds to the following learning problem: An unknown element x in X is chosen uniformly at random. A learner tries to learn x from a stream of samples, (a_1, b_1), (a_2, b_2) ..., where for every i, a_i in A is chosen uniformly at random and b_i = M(a_i,x).
Assume that k,l, r are such that any submatrix of M of at least 2^{-k} * |A| rows and at least 2^{-l} * |X| columns, has a bias of at most 2^{-r}. We show that any two-pass learning algorithm for the learning problem corresponding to M requires either a memory of size at least Omega (k * min{k,sqrt{l}}), or at least 2^{Omega(min{k,sqrt{l},r})} samples
Quantum Versus Randomized Communication Complexity, with Efficient Players
We study a new type of separations between quantum and classical communication complexity, separations that are obtained using quantum protocols where all parties are efficient, in the sense that they can be implemented by small quantum circuits, with oracle access to their inputs. Our main result qualitatively matches the strongest known separation between quantum and classical communication complexity [Dmitry Gavinsky, 2016] and is obtained using a quantum protocol where all parties are efficient. More precisely, we give an explicit partial Boolean function f over inputs of length N, such that:
(1) f can be computed by a simultaneous-message quantum protocol with communication complexity polylog(N) (where at the beginning of the protocol Alice and Bob also have polylog(N) entangled EPR pairs).
(2) Any classical randomized protocol for f, with any number of rounds, has communication complexity at least ??(N^{1/4}).
(3) All parties in the quantum protocol of Item (1) (Alice, Bob and the referee) can be implemented by quantum circuits of size polylog(N) (where Alice and Bob have oracle access to their inputs).
Items (1), (2) qualitatively match the strongest known separation between quantum and classical communication complexity, proved by Gavinsky [Dmitry Gavinsky, 2016]. Item (3) is new. (Our result is incomparable to the one of Gavinsky. While he obtained a quantitatively better lower bound of ?(N^{1/2}) in the classical case, the referee in his quantum protocol is inefficient).
Exponential separations of quantum and classical communication complexity have been studied in numerous previous works, but to the best of our knowledge the efficiency of the parties in the quantum protocol has not been addressed, and in most previous separations the quantum parties seem to be inefficient. The only separations that we know of that have efficient quantum parties are the recent separations that are based on lifting [Arkadev Chattopadhyay et al., 2019; Arkadev Chattopadhyay et al., 2019]. However, these separations seem to require quantum protocols with at least two rounds of communication, so they imply a separation of two-way quantum and classical communication complexity but they do not give the stronger separations of simultaneous-message quantum communication complexity vs. two-way classical communication complexity (or even one-way quantum communication complexity vs. two-way classical communication complexity).
Our proof technique is completely new, in the context of communication complexity, and is based on techniques from [Ran Raz and Avishay Tal, 2019]. Our function f is based on a lift of the forrelation problem, using xor as a gadget
ENRICHING APPLICATION PROGRAMMING INTERFACE (API) DOCUMENTATION USING A LARGE LANGUAGE MODEL (LLM) ARCHITECTURE TO IMPROVE LLM API INTERPRETABILITY
Proposed herein is an innovative system designed to address the critical challenge of maintaining accurate and detailed Application Programming Interface (API) documentation in the fast-past environment of software development. Broadly, the system streamlines the creation and updating an of API documentation by leveraging a Large Language Model (LLM) that intelligently analyzes code changes and generates documentation patches/updates based on any detected code changes
Neuronal CTCF Is Necessary for Basal and Experience-Dependent Gene Regulation, Memory Formation, and Genomic Structure of BDNF and Arc
SummaryCCCTC-binding factor (CTCF) is an organizer of higher-order chromatin structure and regulates gene expression. Genetic studies have implicated mutations in CTCF in intellectual disabilities. However, the role of CTCF-mediated chromatin structure in learning and memory is unclear. We show that depletion of CTCF in postmitotic neurons, or depletion in the hippocampus of adult mice through viral-mediated knockout, induces deficits in learning and memory. These deficits in learning and memory at the beginning of adulthood are correlated with impaired long-term potentiation and reduced spine density, with no changes in basal synaptic transmission and dendritic morphogenesis and arborization. Cognitive disabilities are associated with downregulation of cadherin and learning-related genes. In addition, CTCF knockdown attenuates fear-conditioning-induced hippocampal gene expression of key learning genes and loss of long-range interactions at the BDNF and Arc loci. This study thus suggests that CTCF-dependent gene expression regulation and genomic organization are regulators of learning and memory
Limited flexibility in departure timing of migratory passerines at the East-Mediterranean flyway
The rapid pace of current global warming lead to the advancement of spring migration in the majority of long-distance migratory bird species. While data on arrival timing to breeding grounds in Europe is plentiful, information from the African departure sites are scarce. Here we analysed changes in arrival timing at a stopover site in Israel and any links to Enhanced Vegetation Index (EVI) on the species-specific African non-breeding range in three migratory passerines between 2000–2017. Differences in wing length between early and late arriving individuals were also examined as a proxy for migration distance. We found that male redstart, but not females, advanced arrival to stopover site, but interestingly, not as a response to EVI phenology. Blackcap and barred warbler did not shift arrival timing significantly, although the arrival of blackcap was dependent on EVI. Barred warbler from the early arrival phase had longer wings, suggesting different populations. Our study further supports the existence species-specific migration decisions and inter-sexual differences, which may be triggered by both exogenous (local vegetation condition) and endogenous cues. Given rapid rate of changes in environmental conditions at higher latitudes, some migrants may experience difficulty in the race to match global changes to ensure their survival
- …