415 research outputs found
About Adaptive Coding on Countable Alphabets: Max-Stable Envelope Classes
In this paper, we study the problem of lossless universal source coding for
stationary memoryless sources on countably infinite alphabets. This task is
generally not achievable without restricting the class of sources over which
universality is desired. Building on our prior work, we propose natural
families of sources characterized by a common dominating envelope. We
particularly emphasize the notion of adaptivity, which is the ability to
perform as well as an oracle knowing the envelope, without actually knowing it.
This is closely related to the notion of hierarchical universal source coding,
but with the important difference that families of envelope classes are not
discretely indexed and not necessarily nested.
Our contribution is to extend the classes of envelopes over which adaptive
universal source coding is possible, namely by including max-stable
(heavy-tailed) envelopes which are excellent models in many applications, such
as natural language modeling. We derive a minimax lower bound on the redundancy
of any code on such envelope classes, including an oracle that knows the
envelope. We then propose a constructive code that does not use knowledge of
the envelope. The code is computationally efficient and is structured to use an
{E}xpanding {T}hreshold for {A}uto-{C}ensoring, and we therefore dub it the
\textsc{ETAC}-code. We prove that the \textsc{ETAC}-code achieves the lower
bound on the minimax redundancy within a factor logarithmic in the sequence
length, and can be therefore qualified as a near-adaptive code over families of
heavy-tailed envelopes. For finite and light-tailed envelopes the penalty is
even less, and the same code follows closely previous results that explicitly
made the light-tailed assumption. Our technical results are founded on methods
from regular variation theory and concentration of measure
Recruitment of focal adhesion kinase and paxillin to β1 integrin promotes cancer cell migration via mitogen activated protein kinase activation
BACKGROUND: Integrin-extracellular matrix interactions activate signaling cascades such as mitogen activated protein kinases (MAPK). Integrin binding to extracellular matrix increases tyrosine phosphorylation of focal adhesion kinase (FAK). Inhibition of FAK activity by expression of its carboxyl terminus decreases cell motility, and cells from FAK deficient mice also show reduced migration. Paxillin is a focal adhesion protein which is also phosphorylated on tyrosine. FAK recruitment of paxillin to the cell membrane correlates with Shc phosphorylation and activation of MAPK. Decreased FAK expression inhibits papilloma formation in a mouse skin carcinogenesis model. We previously demonstrated that MAPK activation was required for growth factor induced in vitro migration and invasion by human squamous cell carcinoma (SCC) lines. METHODS: Adapter protein recruitment to integrin subunits was examined by co-immunoprecipitation in SCC cells attached to type IV collagen or plastic. Stable clones overexpressing FAK or paxillin were created using the lipofection technique. Modified Boyden chambers were used for invasion assays. RESULTS: In the present study, we showed that FAK and paxillin but not Shc are recruited to the β1 integrin cytoplasmic domain following attachment of SCC cells to type IV collagen. Overexpression of either FAK or paxillin stimulated cancer cell migration on type IV collagen and invasion through reconstituted basement membrane which was dependent on MAPK activity. CONCLUSIONS: We concluded that recruitment of focal adhesion kinase and paxillin to β1 integrin promoted cancer cell migration via the mitogen activated protein kinase pathway
Induced Model Matching: How Restricted Models Can Help Larger Ones
We consider scenarios where a very accurate predictive model using restricted
features is available at the time of training of a larger, full-featured,
model. This restricted model may be thought of as "side-information", derived
either from an auxiliary exhaustive dataset or on the same dataset, by forcing
the restriction. How can the restricted model be useful to the full model? We
propose an approach for transferring the knowledge of the restricted model to
the full model, by aligning the full model's context-restricted performance
with that of the restricted model's. We call this methodology Induced Model
Matching (IMM) and first illustrate its general applicability by using logistic
regression as a toy example. We then explore IMM's use in language modeling,
the application that initially inspired it, and where it offers an explicit
foundation in contrast to the implicit use of restricted models in techniques
such as noising. We demonstrate the methodology on both LSTM and transformer
full models, using -grams as restricted models. To further illustrate the
potential of the principle whenever it is much cheaper to collect restricted
rather than full information, we conclude with a simple RL example where POMDP
policies can improve learned MDP policies via IMM
Large alphabets: Finite, infinite, and scaling models
How can we effectively model situations with large alphabets? On a pragmatic level, any engineered system, be it for inference, communication, or encryption, requires working with a finite number of symbols. Therefore, the most straight-forward model is a finite alphabet. However, to emphasize the disproportionate size of the alphabet, one may want to compare its finite size with the length of data at hand. More generally, this gives rise to scaling models that strive to capture regimes of operation where one anticipates such imbalance. Large alphabets may also be idealized as infinite. The caveat then is that such generality strips away many of the convenient machinery of finite settings. However, some of it may be salvaged by refocusing the tasks of interest, such as by moving from sequence to pattern compression, or by minimally restricting the classes of infinite models, such as via tail properties. In this paper we present an overview of models for large alphabets, some recent results, and possible directions in this area
Rare Probability Estimation under Regularly Varying Heavy Tails
This paper studies the problem of estimating the probability of symbols that have occurred very rarely, in samples drawn independently from an unknown, possibly infinite, discrete distribution. In particular, we study the multiplicative consistency of estimators, defined as the ratio of the estimate to the true quantity converging to one. We first show that the classical Good-Turing estimator is not universally consistent in this sense, despite enjoying favorable additive properties. We then use Karamata's theory of regular variation to prove that regularly varying heavy tails are sufficient for consistency. At the core of this result is a multiplicative concentration that we establish both by extending the McAllester-Ortiz additive concentration for the missing mass to all rare probabilities and by exploiting regular variation. We also derive a family of estimators which, in addition to being consistent, address some of the shortcomings of the Good-Turing estimator. For example, they perform smoothing implicitly and have the absolute discounting structure of many heuristic algorithms. This also establishes a discrete parallel to extreme value theory, and many of the techniques therein can be adapted to the framework that we set forth.National Science Foundation (U.S.) (Grant 6922470)United States. Office of Naval Research (Grant 6918937
Modeling Access Differences to Reduce Disparity in Resource Allocation
Motivated by COVID-19 vaccine allocation, where vulnerable subpopulations are
simultaneously more impacted in terms of health and more disadvantaged in terms
of access to the vaccine, we formalize and study the problem of resource
allocation when there are inherent access differences that correlate with
advantage and disadvantage. We identify reducing resource disparity as a key
goal in this context and show its role as a proxy to more nuanced downstream
impacts. We develop a concrete access model that helps quantify how a given
allocation translates to resource flow for the advantaged vs. the
disadvantaged, based on the access gap between them. We then provide a
methodology for access-aware allocation. Intuitively, the resulting allocation
leverages more vaccines in locations with higher vulnerable populations to
mitigate the access gap and reduce overall disparity. Surprisingly, knowledge
of the access gap is often not needed to perform access-aware allocation. To
support this formalism, we provide empirical evidence for our access model and
show that access-aware allocation can significantly reduce resource disparity
and thus improve downstream outcomes. We demonstrate this at various scales,
including at county, state, national, and global levels.Comment: Association for Computing Machinery (2022
- …