8 research outputs found
DISeR: Designing Imaging Systems with Reinforcement Learning
Imaging systems consist of cameras to encode visual information about the
world and perception models to interpret this encoding. Cameras contain (1)
illumination sources, (2) optical elements, and (3) sensors, while perception
models use (4) algorithms. Directly searching over all combinations of these
four building blocks to design an imaging system is challenging due to the size
of the search space. Moreover, cameras and perception models are often designed
independently, leading to sub-optimal task performance. In this paper, we
formulate these four building blocks of imaging systems as a context-free
grammar (CFG), which can be automatically searched over with a learned camera
designer to jointly optimize the imaging system with task-specific perception
models. By transforming the CFG to a state-action space, we then show how the
camera designer can be implemented with reinforcement learning to intelligently
search over the combinatorial space of possible imaging system configurations.
We demonstrate our approach on two tasks, depth estimation and camera rig
design for autonomous vehicles, showing that our method yields rigs that
outperform industry-wide standards. We believe that our proposed approach is an
important step towards automating imaging system design.Comment: ICCV 2023. Project Page: https://tzofi.github.io/dise
ORCa: Glossy Objects as Radiance Field Cameras
Reflections on glossy objects contain valuable and hidden information about
the surrounding environment. By converting these objects into cameras, we can
unlock exciting applications, including imaging beyond the camera's
field-of-view and from seemingly impossible vantage points, e.g. from
reflections on the human eye. However, this task is challenging because
reflections depend jointly on object geometry, material properties, the 3D
environment, and the observer viewing direction. Our approach converts glossy
objects with unknown geometry into radiance-field cameras to image the world
from the object's perspective. Our key insight is to convert the object surface
into a virtual sensor that captures cast reflections as a 2D projection of the
5D environment radiance field visible to the object. We show that recovering
the environment radiance fields enables depth and radiance estimation from the
object to its surroundings in addition to beyond field-of-view novel-view
synthesis, i.e. rendering of novel views that are only directly-visible to the
glossy object present in the scene, but not the observer. Moreover, using the
radiance field we can image around occluders caused by close-by objects in the
scene. Our method is trained end-to-end on multi-view images of the object and
jointly estimates object geometry, diffuse radiance, and the 5D environment
radiance field.Comment: for more information, see https://ktiwary2.github.io/objectsascam
Towards Zero Shot Learning in Restless Multi-armed Bandits
Restless multi-arm bandits (RMABs), a class of resource allocation problems
with broad application in areas such as healthcare, online advertising, and
anti-poaching, have recently been studied from a multi-agent reinforcement
learning perspective. Prior RMAB research suffers from several limitations,
e.g., it fails to adequately address continuous states, and requires retraining
from scratch when arms opt-in and opt-out over time, a common challenge in many
real world applications. We address these limitations by developing a neural
network-based pre-trained model (PreFeRMAB) that has general zero-shot ability
on a wide range of previously unseen RMABs, and which can be fine-tuned on
specific instances in a more sample-efficient way than retraining from scratch.
Our model also accommodates general multi-action settings and discrete or
continuous state spaces. To enable fast generalization, we learn a novel single
policy network model that utilizes feature information and employs a training
procedure in which arms opt-in and out over time. We derive a new update rule
for a crucial -network with theoretical convergence guarantees and
empirically demonstrate the advantages of our approach on several challenging,
real-world inspired problems
Micronutrient Deficiency Prediction via Publicly Available Satellite Data
Micronutrient deficiency (MND), which is a form of malnutrition that can have serious health consequences, is difficult to diagnose in early stages without blood draws, which are expensive and time-consuming to collect and process. It is even more difficult at a public health scale seeking to identify regions at higher risk of MND. To provide data more widely and frequently, we propose an accurate, scalable, low-cost, and interpretable regional-level MND prediction system. Specifically, our work is the first to use satellite data, such as forest cover, weather, and presence of water, to predict deficiency of micronutrients such as iron, Vitamin B12, and Vitamin A, directly from their biomarkers. We use real-world, ground truth biomarker data collected from four different regions across Madagascar for training, and demonstrate that satellite data are viable for predicting regional-level MND, surprisingly exceeding the performance of baseline predictions based only on survey responses. Our method could be broadly applied to other countries where satellite data are available, and potentially create high societal impact if these predictions are used by policy makers, public health officials, or healthcare providers
Predicting micronutrient deficiency with publicly available satellite data
Micronutrient deficiency (MND), which is a form of malnutrition that can have serious health consequences, is difficult to diagnose in early stages without blood draws, which are expensive and time-consuming to collect and process. It is even more difficult at a public health scale seeking to identify regions at higher risk of MND. To provide data more widely and frequently, we propose an accurate, scalable, low-cost, and interpretable regional-level MND prediction system. Specifically, our work is the first to use satellite data, such as forest cover, weather, and presence of water, to predict deficiency of micronutrients such as iron, Vitamin B12, and Vitamin A, directly from their biomarkers. We use real-world, ground truth biomarker data collected from four different regions across Madagascar for training, and demonstrate that satellite data are viable for predicting regional-level MND, surprisingly exceeding the performance of baseline predictions based only on survey responses. Our method could be broadly applied to other countries where satellite data are available, and potentially create high societal impact if these predictions are used by policy makers, public health officials, or healthcare providers.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/176274/1/aaai12080.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/176274/2/aaai12080_am.pd
Using Public Data to Predict Demand for Mobile Health Clinics
Improving health equity is an urgent task for our society. The advent of mobile clinics plays an important role in enhancing health equity, as they can provide easier access to preventive healthcare for patients from disadvantaged populations. For effective functioning of mobile clinics, accurate prediction of demand (expected number of individuals visiting mobile clinic) is the key to their daily operations and staff/resource allocation. Despite its importance, there have been very limited studies on predicting demand of mobile clinics. To the best of our knowledge, we are among the first to explore this area, using AI-based techniques. A crucial challenge in this task is that there are no known existing data sources from which we can extract useful information to account for the exogenous factors that may affect the demand, while considering protection of client privacy. We propose a novel methodology that completely uses public data sources to extract the features, with several new components that are designed to improve the prediction. Empirical evaluation on a real-world dataset from the mobile clinic The Family Van shows that, by leveraging publicly available data (which introduces no extra monetary cost to the mobile clinics), our AI-based method achieves 26.4% - 51.8% lower Root Mean Squared Error (RMSE) than the historical average-based estimation (which is presently employed by mobile clinics like The Family Van). Our algorithm makes it possible for mobile clinics to plan proactively, rather than reactively, as what has been doing