A Machine Learning Outlook: Post-processing of Global Medium-range Forecasts

Post-processing typically takes the outputs of a Numerical Weather Prediction (NWP) model and applies linear statistical techniques to produce improve localized forecasts, by including additional observations, or determining systematic errors at a finer scale. In this pilot study, we investigate the benefits and challenges of using non-linear neural network (NN) based methods to post-process multiple weather features -- temperature, moisture, wind, geopotential height, precipitable water -- at 30 vertical levels, globally and at lead times up to 7 days. We show that we can achieve accuracy improvements of up to 12% (RMSE) in a field such as temperature at 850hPa for a 7 day forecast. However, we recognize the need to strengthen foundational work on objectively measuring a sharp and correct forecast. We discuss the challenges of using standard metrics such as root mean squared error (RMSE) or anomaly correlation coefficient (ACC) as we move from linear statistical models to more complex non-linear machine learning approaches for post-processing global weather forecasts.Comment: 9 pages, 4 figures, 1 tabl

Quantifying Uncertainties in Land Surface Microwave Emissivity Retrievals

Uncertainties in the retrievals of microwave land surface emissivities were quantified over two types of land surfaces: desert and tropical rainforest. Retrievals from satellite-based microwave imagers, including SSM/I, TMI and AMSR-E, were studied. Our results show that there are considerable differences between the retrievals from different sensors and from different groups over these two land surface types. In addition, the mean emissivity values show different spectral behavior across the frequencies. With the true emissivity assumed largely constant over both of the two sites throughout the study period, the differences are largely attributed to the systematic and random errors in the retrievals. Generally these retrievals tend to agree better at lower frequencies than at higher ones, with systematic differences ranging 1~4% (3~12 K) over desert and 1~7% (3~20 K) over rainforest. The random errors within each retrieval dataset are in the range of 0.5~2% (2~6 K). In particular, at 85.0/89.0 GHz, there are very large differences between the different retrieval datasets, and within each retrieval dataset itself. Further investigation reveals that these differences are mostly likely caused by rain/cloud contamination, which can lead to random errors up to 10~17 K under the most severe conditions

SPARE-ICE: synergistic Ice Water Path from passive operational sensors

This article presents SPARE-ICE, the Synergistic Passive Atmospheric Retrieval Experiment-ICE. SPARE-ICE is the first Ice Water Path (IWP) product combining infrared and microwave radiances. By using only passive operational sensors, the SPARE-ICE retrieval can be used to process data from at least the NOAA 15 to 19 and MetOp satellites, obtaining time series from 1998 onward. The retrieval is developed using collocations between passive operational sensors (solar, terrestrial infrared, microwave), the CloudSat radar, and the CALIPSO lidar. The collocations form a retrieval database matching measurements from passive sensors against the existing active combined radar-lidar product 2C-ICE. With this retrieval database, we train a pair of artificial neural networks to detect clouds and retrieve IWP. When considering solar, terrestrial infrared, and microwave-based measurements, we show that any combination of two techniques performs better than either single-technique retrieval. We choose not to include solar reflectances in SPARE-ICE, because the improvement is small, and so that SPARE-ICE can be retrieved both daytime and nighttime. The median fractional error between SPARE-ICE and 2C-ICE is around a factor 2, a figure similar to the random error between 2C-ICE ice water content (IWC) and in situ measurements. A comparison of SPARE-ICE with Moderate Resolution Imaging Spectroradiometer (MODIS), Pathfinder Atmospheric Extended (PATMOS-X), and Microwave Surface and Precipitation Products System (MSPPS) indicates that SPARE-ICE appears to perform well even in difficult conditions. SPARE-ICE is available for public use

Dynamic Inversion of Global Surface Microwave Emissivity Using a 1DVAR Approach

A variational inversion scheme is used to extract microwave emissivity spectra from brightness temperatures over a multitude of surface types. The scheme is called the Microwave Integrated Retrieval System and has been implemented operationally since 2007 at NOAA. This study focuses on the Advance Microwave Sounding Unit (AMSU)/MHS pair onboard the NOAA-18 platform, but the algorithm is applied routinely to multiple microwave sensors, including the Advanced Technology Microwave Sounder (ATMS) on Suomi-National Polar-orbiting Partnership (SNPP), Special Sensor Microwave Imager/Sounder (SSMI/S) on the Defense Meteorological Satellite Program (DMSP) flight units, as well as to the Global Precipitation Mission (GPM) Microwave Imager (GMI), to name a few. The emissivity spectrum retrieval is entirely based on a physical approach. To optimize the use of information content from the measurements, the emissivity is extracted simultaneously with other parameters impacting the measurements, namely, the vertical profiles of temperature, moisture and cloud, as well as the skin temperature and hydrometeor parameters when rain or ice are present. The final solution is therefore a consistent set of parameters that fit the measured brightness temperatures within the instrument noise level. No ancillary data are needed to perform this dynamic emissivity inversion. By allowing the emissivity to be part of the retrieved state vector, it becomes easy to handle the pixel-to-pixel variation in the emissivity over non-oceanic surfaces. This is particularly important in highly variable surface backgrounds. The retrieved emissivity spectrum by itself is of value (as a wetness index for instance), but it is also post-processed to determine surface geophysical parameters. Among the parameters retrieved from the emissivity using this approach are snow cover, snow water equivalent and effective grain size over snow-covered surfaces, sea-ice concentration and age from ice-covered ocean surfaces and wind speed over ocean surfaces. It could also be used to retrieve soil moisture and vegetation information from land surfaces. Accounting for the surface emissivity in the state vector has the added advantage of allowing an extension of the retrieval of some parameters over non-ocean surfaces. An example shown here relates to extending the total precipitable water over non-ocean surfaces and to a certain extent, the amount of suspended cloud. The study presents the methodology and performance of the emissivity retrieval and highlights a few examples of some of the emissivity-based products

Using microwave brightness temperature diurnal cycle to improve emissivity retrievals over land

To retrieve microwave land emissivity, infrared surface skin temperatures have been used as surface physical temperature since there is no global information on physical vegetation/soil temperature profiles. However, passive microwave emissions originate fromdeeper layerswith respect to the skin temperature. So, this inconsistency in sensitivity depths between skin temperatures and microwave temperaturesmay introduce a discrepancy in the determined emissivity. Previous studies showed that this inconsistency can lead to significant differences between day and night retrievals of land emissivity which can exceed 10%. This study proposes an approach to address this inconsistency and improve the retrieval of land emissivity using microwave observations from AdvancedMicrowave Scanning Radiometer–Earth Observing System(AMSR-E). The diurnal cycle of the microwave brightness temperature (Tb) was constructed over the globe for different frequencies/polarizations using a constellation of satellites. Principal component analysis (PCA) was conducted to evaluate the spatial variation of the Tb diurnal cycle. The diurnal amplitudes of microwave temperatures observed in desert areas were not consistent with the larger amplitudes of the diurnal cycle of skin temperature. Densely vegetated areas with more moisture have shown smaller amplitudes. A lookup table of effective temperature (Teff) anomalies is constructed based on the Tb diurnal cycle to resolve the inconsistencies between infrared and Tb diurnal variation. This lookup table of Teff anomalies is a weighted average over the layers contributing to the microwave signal, for each channel and month. The integration of this Teff in the retrieval of land emissivity reduced the differences between day and night retrieved emissivities to less than 0.01 for AMSR-E observations