On the adequacy of untuned warmup for adaptive optimization

Adaptive optimization algorithms such as Adam are widely used in deep learning. The stability of such algorithms is often improved with a warmup schedule for the learning rate. Motivated by the difficulty of choosing and tuning warmup schedules, recent work proposes automatic variance rectification of Adam's adaptive learning rate, claiming that this rectified approach ("RAdam") surpasses the vanilla Adam algorithm and reduces the need for expensive tuning of Adam with warmup. In this work, we refute this analysis and provide an alternative explanation for the necessity of warmup based on the magnitude of the update term, which is of greater relevance to training stability. We then provide some "rule-of-thumb" warmup schedules, and we demonstrate that simple untuned warmup of Adam performs more-or-less identically to RAdam in typical practical settings. We conclude by suggesting that practitioners stick to linear warmup with Adam, with a sensible default being linear warmup over $2 / (1 - \beta_2)$ training iterations.Comment: AAAI 202

Recent Developments in AI and USPTO Open Data

The USPTO disseminates one of the largest publicly accessible repositories of scientific, technical, and commercial data worldwide. USPTO data has historically seen frequent use in fields such as patent analytics, economics, and prosecution & litigation tools. This article highlights an emerging class of usecases directed to the research, development, and application of artificial intelligence technology. Such usecases contemplate both the delivery of artificial intelligence capabilities for practical IP applications and the enablement of future state-of-the-art artificial intelligence research via USPTO data products. Examples from both within and beyond the USPTO are offered as case studies.Comment: SIGIR 2022 / PatentSemTec

When can we kick (some) humans “out of the loop”? An examination of the use of ai in medical imaging for lumbar spinal stenosis

Artificial intelligence (AI) has attracted an increasing amount of attention, both positive and negative. Its potential applications in healthcare are indeed manifold and revolutionary, and within the realm of medical imaging and radiology (which will be the focus of this paper), significant increases in accuracy and speed, as well as significant savings in cost, stand to be gained through the adoption of this technology. Because of its novelty, a norm of keeping humans “in the loop” wherever AI mechanisms are deployed has become synonymous with good ethical practice in some circles. It has been argued that keeping humans “in the loop” is important for reasons of safety, accountability, and the maintenance of institutional trust. However, as the application of machine learning for the detection of lumbar spinal stenosis (LSS) in this paper’s case study reveals, there are some scenarios where an insistence on keeping humans in the loop (or in other words, the resistance to automation) seems unwarranted and could possibly lead us to miss out on very real and important opportunities in healthcare—particularly in low-resource settings. It is important to acknowledge these opportunity costs of resisting automation in such contexts, where better options may be unavailable. Using an AI model based on convolutional neural networks developed by a team of researchers at NUH/NUS medical school in Singapore for automated detection and classification of the lumbar spinal canal, lateral recess, and neural foraminal narrowing in an MRI scan of the spine to diagnose LSS, we will aim to demonstrate that where certain criteria hold (e.g., the AI is as accurate or better than human experts, risks are low in the event of an error, the gain in wellbeing is significant, and the task being automated is not essentially or importantly human), it is both morally permissible and even desirable to kick the humans out of the loop

Using RZWQM to Predict Herbicide Leaching Losses in Subsurface Drainage Water

If you are not an ASABE member or if your employer has not arranged for access to the full-text, Click here for options. USING RZWQM TO PREDICT HERBICIDE LEACHING LOSSES IN SUBSURFACE DRAINAGE WATER Published by the American Society of Agricultural and Biological Engineers, St. Joseph, Michigan www.asabe.org Citation: Transactions of the ASAE. Vol. 47(5): 1415-1426 . (doi: 10.13031/2013.17621) @2004 Authors: A. Bakhsh, L. Ma, L. R. Ahuja, J. L. Hatfield, R. S. Kanwar Keywords: Herbicides, RZWQM, Subsurface drainage, Water quality Improvements have been made in the pesticide component of the Root Zone Water Quality Model (RZWQM) since its release in 1999 for the Management System Evaluation Areas (MSEA) project. This study was designed to evaluate the herbicide leaching component of the model using data on subsurface drainage flow and herbicide leaching losses for a 6-year (1992 to 1997) period. A sensitivity analysis was conducted for the key parameters important in the pesticide calibration process. The model was calibrated using 1992 data and validated using 1993 to 1997 data collected from a tile-drained field within the Walnut Creek watershed in central Iowa. The model evaluation criterion was based on percent difference between the predicted and measured data (%D), root mean square error (RMSE), and model efficiency (EF). Atrazine and metolachlor were applied to corn in 1993, 1995, and 1997, and metribuzin was used during the soybean growing seasons in 1992, 1994, and 1996 at the standard application rates used in Iowa. The predicted subsurface drainage volumes were in close agreement with the measured data showing %D = 1, RMSE = 8, and EF = 0.99, when averaged over the validation years. Herbicide half-life (t1/2) and soil organic based partitioning coefficient (Koc) were found to be the most sensitive parameters for simulating herbicide leaching losses in subsurface drainage water. Both t1/2 and Koc affected the mass and temporal distribution of the herbicide leaching losses in subsurface drainage flows. The predicted herbicide leaching losses in subsurface drainage water were the same order of magnitude as the measured data, when averaged across the validation years. The study also revealed that herbicide leaching losses were significantly (P \u3c 0.05) controlled by the drainage volume (R2 = 0.97). The model, however, underpredicted herbicide leaching losses after crop harvest and during early spring, possibly because of preferential flow paths developed during these periods. More improvements may be needed in the RZWQM to consider the dynamics of the preferential flow paths development in cultivated soils similar to that of the study area

The Nashua agronomic, water quality, and economic dataset

This paper describes a dataset relating management to nitrogen (N) loading and crop yields from 1990 to 2003 on 36, 0.4 ha (1 ac) individually tile-drained plots on the Northeast Research and Demonstration Farm near Nashua, Iowa, United States. The field-measured data were used to calibrate the Root Zone Water Quality Model (RZWQM), and the results were summarized in a special issue ofGeoderma (Ahuja and Hatfield 2007). With a comprehensive, long-term measured dataset and a model that simulates many of the components of the agricultural system, one can begin to understand the effects of management practices on N loading, crop yields, and net income to the farmers. Other researchers can use this dataset to assess the effects of management on similar tile-drained systems occurring some distance from Nashua, under alternative climates and soils, with other management systems, or with simulation models using different process representations. By integrating the understanding developed at Nashua with datasets from other highly monitored sites and other sources, progress can be made in addressing problems related to excessive N fluxes in the Mississippi Basin. An example 30-year RZWQM simulation of 18 management systems implies that significant management changes are needed to meet the goal of reducing N loads to the Gulf of Mexico by 45%. This paper and the associated datasets are intended to be used in conjunction with the analyses and process descriptions presented in the Geoderma special issue. The datasets and additional explanatory materials are available for download at http://apps.tucson.ars.ag.gov/nashua

Simulated N management effects on corn yield and tile-drainage nitrate loss

Thoroughly tested simulation models are needed to help quantify the long-term effects of agriculture. We evaluated the Root Zone Water Quality Model (RZWQM) response to different N management strategies and then used the tested model with observed weather data from 1961–2003 to quantify long-term effects on corn (Zea mays L.) yield and flow weighted nitrate-N concentration in subsurface “tile” drainage water (Nconc). Fourteen years (1990–2003) of field data from 30, 0.4 ha plots in northeast Iowa were available for model testing. Annual crop yield, nitrate-N loss to subsurface “tile” drainage water (Nloss), Nconc, and subsurface “tile” drainage amount (drain) for various management scenarios were averaged over plots and years to create five chemical fertilizer and five swine manure treatments. Predicted corn yield and Nconc for the 10 treatments were significantly correlated with observed data (R2 \u3e 0.83). The Root Mean Square Errors (RMSE) were 15% and 18% of its observed average Nconc for chemical fertilizer and manure treatments, respectively. Corresponding RMSEs for corn yields were 8% and 10% of its observed average corn yields for chemical fertilizer and manure treatments. The long-term simulations indicate that average corn yield plateaus and Nloss accelerates as quadratic functions of increasing spring UAN-N rates from 100 to 200 kg N/ha. Winter wheat (Triticum aestivum L.) sowed after corn and soybean [Glycine max (L.) Merr.] harvest was predicted to reduce long-term Nloss by 5 to 6 kg N/ha, which appears consistent with published field studies and may be a treatment to ameliorate agricultural management with potential for elevated Nloss such as swine manure application to soybean. The results suggest that after calibration and thorough testing, RZWQM can be used to quantify the relative effects of corn production and Nconc under several alternative management practices

The ICO Approach to Quantifying and Restoring Forest Spatial Pattern: Implementation Guide

This document is intended as a “How To” guide for managers and stakeholders wishing to implement the Individual, Clumps, and Openings (ICO) method for silvicultural prescriptions and/or monitoring. This guide is organized into stand-alone chapters. Managers should read and use chapters as they find useful to their own needs.https://scholarworks.umt.edu/ico/1002/thumbnail.jp