FP8 Formats for Deep Learning

FP8 is a natural progression for accelerating deep learning training inference beyond the 16-bit formats common in modern processors. In this paper we propose an 8-bit floating point (FP8) binary interchange format consisting of two encodings - E4M3 (4-bit exponent and 3-bit mantissa) and E5M2 (5-bit exponent and 2-bit mantissa). While E5M2 follows IEEE 754 conventions for representatio of special values, E4M3's dynamic range is extended by not representing infinities and having only one mantissa bit-pattern for NaNs. We demonstrate the efficacy of the FP8 format on a variety of image and language tasks, effectively matching the result quality achieved by 16-bit training sessions. Our study covers the main modern neural network architectures - CNNs, RNNs, and Transformer-based models, leaving all the hyperparameters unchanged from the 16-bit baseline training sessions. Our training experiments include large, up to 175B parameter, language models. We also examine FP8 post-training-quantization of language models trained using 16-bit formats that resisted fixed point int8 quantization

Analysis of action oriented effects on perceptual process of object recognition using physiological responses

Action on any objects provides perceptual information about the environment. There is a significant evidence that human visual system responds to action possibilities in an image as perceiving any ones action stimulates human motor system. However very limited studies have been done to analyze the effect of object affordance during action perception and execution. To study the effect of object affordance on human perception, in this paper we have analyzed the human brain signals using EEG based oscillatory activity of brain. EEG responses corresponding to images of objects shown with correct, incorrect and without grips are examined. Exploration of different gripping effects has been done by extracting Alpha and Beta frequency bands using Discrete Wavelet Transform based band extraction method, then baseline normalized power of Alpha and Beta frequency bands at 24 positions of motor area of left and right side of brain are examined. The result shows that 12 pooled electrodes at central and central parietal region provides a clear discrimination among the three gripping cases in terms of calculated power. The presented research explores new applicabilities of object affordance to develop a variety of Brain Computer Interface (BCI) based devices and to improve motor imagery ability among motor disorder related patients

Attention modulates motor system activation during action observation: evidence for inhibitory rebound

Perceiving another individual’s actions activates the human motor system. We investigated whether this effect is stronger when the observed action is relevant to the observer’s task. The mu rhythm (oscillatory activity in the 8- to 13-Hz band over sensorimotor cortex) was measured while participants watched videos of grasping movements. In one of two conditions, the participants had to later report how many times they had seen a certain kind of grasp. In the other condition, they viewed the identical videos but had to later report how many times they had seen a certain colour change. The colour change and the grasp always occurred simultaneously. Results show mu rhythm attenuation when watching the videos relative to baseline. This attenuation was stronger when participants later reported the grasp rather than the colour, suggesting that the motor system is more strongly activated when the observed grasping actions were relevant to the observer’s task. Moreover, when the graspable object disappeared after the offset of the video, there was subsequent mu rhythm enhancement, reflecting a post-stimulus inhibitory rebound. This enhancement was again stronger when making judgments about the grasp than the colour, suggesting that the stronger activation is followed by a stronger inhibitory rebound

Metaplastic breast carcinomas exhibit EGFR, but not HER2, gene amplification and overexpression: immunohistochemical and chromogenic in situ hybridization analysis

INTRODUCTION: Metaplastic breast carcinomas constitute a heterogeneous group of neoplasms, accounting for less than 1% of all invasive mammary carcinomas. Approximately 70–80% of metaplastic breast carcinomas overexpress the epidermal growth factor receptor (EGFR). Human epidermal growth factor receptor (HER)2 and EGFR have attracted much attention in the medical literature over the past few years owing to the fact that humanized monoclonal antibodies against HER2 and therapies directed against the extracellular ligand-binding domain or the intracellular tyrosine kinase domain of EGFR have proven successful in treating certain types of human cancer. We investigated whether HER2 and EGFR overexpression was present and evaluated gene amplification in a series of metaplastic breast carcinomas. METHOD: Twenty-five metaplastic breast carcinomas were immunohistochemically analyzed using a monoclonal antibody (31G7) for EGFR and two antibodies for HER2 (Herceptest and CB11) and scored using the Herceptest scoring system. Gene amplification was evaluated by chromogenic in situ hybridization using Zymed Spot-Light EGFR and HER2 amplification probe. The results were evaluated by bright field microscopy under 40× and 63× objective lenses. RESULTS: Nineteen (76%) metaplastic breast carcinomas exhibited EGFR ovexpression, and among these EGFR amplification (defined either by large gene clusters or >5 signals/nucleus in >50% of neoplastic cells) was detected in seven cases (37%): three carcinomas with squamous differentiation and four spindle cell carcinomas. One case exhibited HER2 overexpression of grade 2+ (>10% of cells with weak to moderate complete membrane staining), but HER2 gene amplification was not detected. CONCLUSION: Metaplastic breast carcinomas frequently overexpressed EGFR, which was associated with EGFR gene amplification in one-third of cases. Our findings suggest that some patients with metaplastic breast carcinomas might benefit from novel therapies targeting EGFR. Because most metaplastic breast carcinomas overexpress EGFR without gene amplification, further studies to evaluate EGFR activating mutations are warranted

Aspects of molecular phenotype and its correlations with breast cancer behaviour and taxonomy

The assessment of breast cancer morphology remains an important element in the evaluation of prognosis and therapeutic planning for this disease. The tumour morphology reflects the molecular profile that produced it and consequently each can be predictive of the other

State of the climate in 2013

In 2013, the vast majority of the monitored climate variables reported here maintained trends established in recent decades. ENSO was in a neutral state during the entire year, remaining mostly on the cool side of neutral with modest impacts on regional weather patterns around the world. This follows several years dominated by the effects of either La Niña or El Niño events. According to several independent analyses, 2013 was again among the 10 warmest years on record at the global scale, both at the Earths surface and through the troposphere. Some regions in the Southern Hemisphere had record or near-record high temperatures for the year. Australia observed its hottest year on record, while Argentina and New Zealand reported their second and third hottest years, respectively. In Antarctica, Amundsen-Scott South Pole Station reported its highest annual temperature since records began in 1957. At the opposite pole, the Arctic observed its seventh warmest year since records began in the early 20th century. At 20-m depth, record high temperatures were measured at some permafrost stations on the North Slope of Alaska and in the Brooks Range. In the Northern Hemisphere extratropics, anomalous meridional atmospheric circulation occurred throughout much of the year, leading to marked regional extremes of both temperature and precipitation. Cold temperature anomalies during winter across Eurasia were followed by warm spring temperature anomalies, which were linked to a new record low Eurasian snow cover extent in May. Minimum sea ice extent in the Arctic was the sixth lowest since satellite observations began in 1979. Including 2013, all seven lowest extents on record have occurred in the past seven years. Antarctica, on the other hand, had above-average sea ice extent throughout 2013, with 116 days of new daily high extent records, including a new daily maximum sea ice area of 19.57 million km2 reached on 1 October. ENSO-neutral conditions in the eastern central Pacific Ocean and a negative Pacific decadal oscillation pattern in the North Pacific had the largest impacts on the global sea surface temperature in 2013. The North Pacific reached a historic high temperature in 2013 and on balance the globally-averaged sea surface temperature was among the 10 highest on record. Overall, the salt content in nearsurface ocean waters increased while in intermediate waters it decreased. Global mean sea level continued to rise during 2013, on pace with a trend of 3.2 mm yr-1 over the past two decades. A portion of this trend (0.5 mm yr-1) has been attributed to natural variability associated with the Pacific decadal oscillation as well as to ongoing contributions from the melting of glaciers and ice sheets and ocean warming. Global tropical cyclone frequency during 2013 was slightly above average with a total of 94 storms, although the North Atlantic Basin had its quietest hurricane season since 1994. In the Western North Pacific Basin, Super Typhoon Haiyan, the deadliest tropical cyclone of 2013, had 1-minute sustained winds estimated to be 170 kt (87.5 m s-1) on 7 November, the highest wind speed ever assigned to a tropical cyclone. High storm surge was also associated with Haiyan as it made landfall over the central Philippines, an area where sea level is currently at historic highs, increasing by 200 mm since 1970. In the atmosphere, carbon dioxide, methane, and nitrous oxide all continued to increase in 2013. As in previous years, each of these major greenhouse gases once again reached historic high concentrations. In the Arctic, carbon dioxide and methane increased at the same rate as the global increase. These increases are likely due to export from lower latitudes rather than a consequence of increases in Arctic sources, such as thawing permafrost. At Mauna Loa, Hawaii, for the first time since measurements began in 1958, the daily average mixing ratio of carbon dioxide exceeded 400 ppm on 9 May. The state of these variables, along with dozens of others, and the 2013 climate conditions of regions around the world are discussed in further detail in this 24th edition of the State of the Climate series. © 2014, American Meteorological Society. All rights reserved