Beyond Binomial and Negative Binomial: Adaptation in Bernoulli Parameter Estimation

Estimating the parameter of a Bernoulli process arises in many applications, including photon-efficient active imaging where each illumination period is regarded as a single Bernoulli trial. Motivated by acquisition efficiency when multiple Bernoulli processes are of interest, we formulate the allocation of trials under a constraint on the mean as an optimal resource allocation problem. An oracle-aided trial allocation demonstrates that there can be a significant advantage from varying the allocation for different processes and inspires a simple trial allocation gain quantity. Motivated by realizing this gain without an oracle, we present a trellis-based framework for representing and optimizing stopping rules. Considering the convenient case of Beta priors, three implementable stopping rules with similar performances are explored, and the simplest of these is shown to asymptotically achieve the oracle-aided trial allocation. These approaches are further extended to estimating functions of a Bernoulli parameter. In simulations inspired by realistic active imaging scenarios, we demonstrate significant mean-squared error improvements: up to 4.36 dB for the estimation of p and up to 1.80 dB for the estimation of log p.Comment: 13 pages, 16 figure

Beyond binomial and negative binomial: adaptation in Bernoulli parameter estimation

Estimating the parameter of a Bernoulli process arises in many applications, including photon-efficient active imaging where each illumination period is regarded as a single Bernoulli trial. Motivated by acquisition efficiency when multiple Bernoulli processes (e.g., multiple pixels) are of interest, we formulate the allocation of trials under a constraint on the mean as an optimal resource allocation problem. An oracle-aided trial allocation demonstrates that there can be a significant advantage from varying the allocation for different processes and inspires the introduction of a simple trial allocation gain quantity. Motivated by achieving this gain without an oracle, we present a trellis-based framework for representing and optimizing stopping rules. Considering the convenient case of Beta priors, three implementable stopping rules with similar performances are explored, and the simplest of these is shown to asymptotically achieve the oracle-aided trial allocation. These approaches are further extended to estimating functions of a Bernoulli parameter. In simulations inspired by realistic active imaging scenarios, we demonstrate significant mean-squared error improvements up to 4.36 dB for the estimation of p and up to 1.86 dB for the estimation of log p.https://arxiv.org/abs/1809.08801https://arxiv.org/abs/1809.08801First author draf

Finite Simulation Budget Allocation for Ranking and Selection

We consider a simulation-based ranking and selection (R&S) problem under a fixed budget setting. Existing budget allocation procedures focus either on asymptotic optimality or on one-step-ahead allocation efficiency. Neither of them depends on the fixed simulation budget, the ignorance of which could lead to an inefficient allocation, especially when the simulation budget is finite. In light of this, we develop a finite-budget allocation rule that is adaptive to the simulation budget. Theoretical results show that the budget allocation strategies are distinctively different between a finite budget and a sufficiently large budget. Our proposed allocation rule can dynamically determine the ratio of budget allocated to designs according to different simulation budget and is optimal when the simulation budget goes to infinity, indicating it not only possesses desirable finite-budget properties but also achieves asymptotic optimality. Based on the proposed allocation rule, two efficient finite simulation budget allocation algorithms are developed. In the numerical experiments, we use both synthetic examples and a case study to show the superior efficiency of our proposed allocation rule

Bandit-Based Task Assignment for Heterogeneous Crowdsourcing

We consider a task assignment problem in crowdsourcing, which is aimed at collecting as many reliable labels as possible within a limited budget. A challenge in this scenario is how to cope with the diversity of tasks and the task-dependent reliability of workers, e.g., a worker may be good at recognizing the name of sports teams, but not be familiar with cosmetics brands. We refer to this practical setting as heterogeneous crowdsourcing. In this paper, we propose a contextual bandit formulation for task assignment in heterogeneous crowdsourcing, which is able to deal with the exploration-exploitation trade-off in worker selection. We also theoretically investigate the regret bounds for the proposed method, and demonstrate its practical usefulness experimentally

A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning

Recently using machine learning (ML) based techniques to optimize modern database management systems has attracted intensive interest from both industry and academia. With an objective to tune a specific component of a DBMS (e.g., index selection, knobs tuning), the ML-based tuning agents have shown to be able to find better configurations than experienced database administrators. However, one critical yet challenging question remains unexplored -- how to make those ML-based tuning agents work collaboratively. Existing methods do not consider the dependencies among the multiple agents, and the model used by each agent only studies the effect of changing the configurations in a single component. To tune different components for DBMS, a coordinating mechanism is needed to make the multiple agents cognizant of each other. Also, we need to decide how to allocate the limited tuning budget among the agents to maximize the performance. Such a decision is difficult to make since the distribution of the reward for each agent is unknown and non-stationary. In this paper, we study the above question and present a unified coordinating framework to efficiently utilize existing ML-based agents. First, we propose a message propagation protocol that specifies the collaboration behaviors for agents and encapsulates the global tuning messages in each agent's model. Second, we combine Thompson Sampling, a well-studied reinforcement learning algorithm with a memory buffer so that our framework can allocate budget judiciously in a non-stationary environment. Our framework defines the interfaces adapted to a broad class of ML-based tuning agents, yet simple enough for integration with existing implementations and future extensions. We show that it can effectively utilize different ML-based agents and find better configurations with 1.4~14.1X speedups on the workload execution time compared with baselines.Comment: Accepted at 2023 International Conference on Management of Data (SIGMOD '23

Budgeted Reinforcement Learning in Continuous State Space

A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints. It relies on a notion of risk implemented in the shape of a cost signal constrained to lie below an - adjustable - threshold. So far, BMDPs could only be solved in the case of finite state spaces with known dynamics. This work extends the state-of-the-art to continuous spaces environments and unknown dynamics. We show that the solution to a BMDP is a fixed point of a novel Budgeted Bellman Optimality operator. This observation allows us to introduce natural extensions of Deep Reinforcement Learning algorithms to address large-scale BMDPs. We validate our approach on two simulated applications: spoken dialogue and autonomous driving.Comment: N. Carrara and E. Leurent have equally contribute

Rates of sustainable forest harvest depend on rotation length and weathering of soil minerals

Abstract Removals of forest biomass in the northeastern US may intensify over the coming decades due to increased demand for renewable energy. For forests to regenerate successfully following intensified harvests, the nutrients removed from the ecosystem in the harvested biomass (including N, P, Ca, Mg, and K) must be replenished through a combination of plant-available nutrients in the soil rooting zone, atmospheric inputs, weathering of primary minerals, biological N fixation, and fertilizer additions. Few previous studies (especially in North America) have measured soil nutrient pools beyond exchangeable cations, but over the long rotations common in this region, other pools which turn over more slowly are important. We constructed nutrient budgets at the rotation time scale for three harvest intensities and compared these with detailed soil data of exchangeable, organic, and primary mineral stocks of in soils sampled in 15 northern hardwood stands developed on granitic till soils in the White Mountain region of New Hampshire, USA. This comparison can be used to estimate how many times each stand might be harvested without diminishing productivity or requiring fertilization. Under 1990s rates of N deposition, N inputs exceeded removals except in the most intensive management scenario considered. Net losses of Ca, K, Mg, and P per rotation were potentially quite severe, depending on the assumptions used.Biologically accelerated soil weathering may explain the lack of observed deficiencies in regenerating forests of the region. Sites differed widely in the long-term nutrient capital available to support additional removals before encountering limitations (e.g., a fourfold difference in available Ca, and a tenfold difference in weatherable Ca). Intensive short-rotation biomass removal could rapidly deplete soil nutrient capital, but traditional long rotations, even under intensive harvesting, are unlikely to induce nutrient depletion in the 21st century. Weatherable P may ultimately limit biomass production on granitic bedrock (in as few as 6 rotations). Understanding whether and how soil weathering rates respond to nutrient demand will be critical to determining long-term sustainability of repeated intensive harvesting over centuries