Diagnosing Model Performance Under Distribution Shift

Prediction models can perform poorly when deployed to target distributions different from the training distribution. To understand these operational failure modes, we develop a method, called DIstribution Shift DEcomposition (DISDE), to attribute a drop in performance to different types of distribution shifts. Our approach decomposes the performance drop into terms for 1) an increase in harder but frequently seen examples from training, 2) changes in the relationship between features and outcomes, and 3) poor performance on examples infrequent or unseen during training. These terms are defined by fixing a distribution on $X$ while varying the conditional distribution of $Y \mid X$ between training and target, or by fixing the conditional distribution of $Y \mid X$ while varying the distribution on $X$. In order to do this, we define a hypothetical distribution on $X$ consisting of values common in both training and target, over which it is easy to compare $Y \mid X$ and thus predictive performance. We estimate performance on this hypothetical distribution via reweighting methods. Empirically, we show how our method can 1) inform potential modeling improvements across distribution shifts for employment prediction on tabular census data, and 2) help to explain why certain domain adaptation methods fail to improve model performance for satellite image classification

Disaster Image Classification by Fusing Multimodal Social Media Data

Social media datasets have been widely used in disaster assessment and management. When a disaster occurs, many users post messages in a variety of formats, e.g., image and text, on social media platforms. Useful information could be mined from these multimodal data to enable situational awareness and to support decision making during disasters. However, the multimodal data collected from social media contain a lot of irrelevant and misleading content that needs to be filtered out. Existing work has mostly used unimodal methods to classify disaster messages. In other words, these methods treated the image and textual features separately. While a few methods adopted multimodality to deal with the data, their accuracy cannot be guaranteed. This research seamlessly integrates image and text information by developing a multimodal fusion approach to identify useful disaster images collected from social media platforms. In particular, a deep learning method is used to extract the visual features from social media, and a FastText framework is then used to extract the textual features. Next, a novel data fusion model is developed to combine both visual and textual features to classify relevant disaster images. Experiments on a real-world disaster dataset, CrisisMMD, are performed, and the validation results demonstrate that the method consistently and significantly outperforms the previously published state-of-the-art work by over 3%, with a performance improvement from 84.4% to 87.6%

Land use change on the surface area and the influence on carbon

China has diversified landforms, the three-dimensional space area check is more accurate to help determine China’s land use change and the caused carbon variations. This study explored a new method to check China’s surface area and examine the terrestrial carbon changes for the period of 2000–2020. The results show that China’s surface area increased by 13.9% compared with the planar area, with the increased area measuring 133 × 104 km2. The south and the west, especially the southwest, usually have a high area increasing rate. Woodland has the highest area increasing rate for all the provinces. 10% of the land had its land use type changed. Cropland, grassland and unused show total land area decrease, woodland, water, and impervious all increased. The mean increasing rate of land transfer on surface area varied between 1.39% and 38.84%. The total amount of land use-type change caused carbon loss reached −5907.44 × 104 t, of −3168.97 × 104 t from vegetation storage loss, −2738.77 × 104 t from NPP and water. There were only seven provinces show carbon increase, which were more located in the west. Per unit of woodland loss will cause higher carbon release than other land use types. Land use control need to be further strengthened, especially for the protection of woodland at mountain regions

Interim Futility Analysis for Longitudinal Data With Adaptive Timing and Error Rate Preservation

<p>There are many clinical trials where longitudinal endpoints are used and the primary endpoint is quite often either based on the rate of change or change from baseline at a specific long-term follow-up time point. When such trials are monitored, it is possible that interim futility analyses will be planned such that the trials can be terminated early if the treatment does not induce any benefit to the patients. For such trials, subjects with incomplete follow-up pose challenges in the timing, analysis, and decision making at the interim futility look. We propose an efficient interim futility analysis based on the slope of a linear regression, which incorporates all the data available at the interim analysis. Our approach has the added advantage of providing a data-driven decision on triggering the interim analysis when sufficient information has been collected such that the desired properties for the established futility rule are guaranteed. The construction of interim futility rules and the timing of the interim analysis are discussed and the method is illustrated with an example involving a placebo-controlled comparison of longitudinal proteinuria measurements. Supplementary materials for this article are available online.</p

A noninvasive model discriminating significant histological changes in treatment-naive chronic hepatitis B patients with normal ALT

Abstract Background Traditionally part of chronic hepatitis B (CHB) patients with normal alanine aminotransferase (ALT) are recommended to antiviral therapy referring to liver biopsy. However, liver biopsy is an invasive method with various potential complications. A noninvasive model was established in the study to evaluate liver histology and to identify the need of antiviral therapy. Methods A total of 614 liver biopsied CHB patients with ALT less than upper limit of normal from 2 centers were retrospectively analyzed. They were divided into a training cohort and a validation cohort. A noninvasive model to predict the significant liver histological changes was established and validated. Results The results of analysis showed that ALT, Age, platelet (PLT) and liver stiffness (LS) were independent risk factors for significant liver injury. The model was established based on the 4 indexes, with the area under the curve of 0.85 and 0.87 in training cohort and validation cohort. Meanwhile, 2 cut-off scores were selected. By applying the low cut-off score (− 0.207), patients without significant liver injury could be identified with high accuracy, with negative predictive value of 72.7% and 73.7% in training and validation cohorts. By applying the high cut-off score (0.537), the presence of significant liver injury could be diagnosed with high accuracy, with positive predictive value of 90.3% and 88.8% in the training and validation cohorts. By applying the model, liver biopsy would have been avoided in 87.6% (538/614) patients, with correct prediction in 87.9% (473/538). Conclusion The novel noninvasive model composed of ALT, Age, PLT, LS can correctly assess liver histology in CHB patient with normal ALT, which helps to determine the need of antiviral therapy without liver biopsy

Red Light Combined with Blue Light Irradiation Regulates Proliferation and Apoptosis in Skin Keratinocytes in Combination with Low Concentrations of Curcumin

<div><p>Curcumin is a widely known natural phytochemical from plant <i>Curcuma longa</i>. In recent years, curcumin has received increasing attention because of its capability to induce apoptosis and inhibit cell proliferation as well as its anti-inflammatory properties in different cancer cells. However, the therapeutic benefits of curcumin are severely hampered due to its particularly low absorption via trans-dermal or oral bioavailability. Phototherapy with visible light is gaining more and more support in dermatological therapy. Red light is part of the visible light spectrum, which is able to deeply penetrate the skin to about 6 mm, and directly affect the fibroblast of the skin dermis. Blue light is UV-free irradiation which is fit for treating chronic inflammation diseases. In this study, we show that curcumin at low concentrations (1.25–3.12 μM) has a strong anti-proliferative effect on TNF-α-induced psoriasis-like inflammation when applied in combination with light-emitting-diode devices. The treatment was especially effective when LED blue light at 405 nm was combined with red light at 630 or 660 nm, which markedly amplified the anti-proliferative and apoptosis-inducing effects of curcumin. The experimental results demonstrated that this treatment reduced the viability of human skin keratinocytes, decreased cell proliferation, induced apoptosis, inhibited NF-κB activity and activated caspase-8 and caspase-9 while preserving the cell membrane integrity. Moreover, the combined treatment also down-regulated the phosphorylation level of Akt and ERK. Taken together, our results indicated that the combination of curcumin with LED blue light united red light irradiation can attain a higher efficiency of regulating proliferation and apoptosis in skin keratinocytes.</p></div

Curcumin combined with red united blue light inhibited cell proliferation.

<p>(A) Flow cytometric analysis of HaCaT cells without any treatment. (B)-(E) Flow cytometric analysis of HaCaT cells which were pre-incubated with curcumin (2.5 μM) for 2 h and then protected from light or separately irradiated with blue light and two combinations of blue and red light. (F) Quantification of cell cycle distribution (G1, S and G2/M). Each bar represents the mean of the three independent experiments, and the differences between cells treated with light irradiation or not are particularly evident at <i>p</i><0.05(*) or <i>p</i><0.01(**) level.</p

Curcumin combined with red united blue light inhibited TNF-α-induced NF-κB activation.

<p>HaCaT cells were pre-incubated with curcumin (3.12 μM) for 2 h, and then separately irradiated with blue light and two combinations of blue and red light, or protected from light. Subsequently, the cells were treated with TNF-α (20 ng/ml) for 1 h, and the nuclear extracts were prepared and analysed. (A) The expression level of phospho NF-κB p65 (pNF-κB p65) was detected by western blot, with NF-κB p65 as a loading control. (B) Densitometry analysis of phosphorylated p65. Bars with different characters are statistically different at <i>p</i><0.05(*) level. The results shown are representative of three independent experiments.</p