Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning

As the size of the pre-trained language model (PLM) continues to increase, numerous parameter-efficient transfer learning methods have been proposed recently to compensate for the tremendous cost of fine-tuning. Despite the impressive results achieved by large pre-trained language models (PLMs) and various parameter-efficient transfer learning (PETL) methods on sundry benchmarks, it remains unclear if they can handle inputs that have been distributionally shifted effectively. In this study, we systematically explore how the ability to detect out-of-distribution (OOD) changes as the size of the PLM grows or the transfer methods are altered. Specifically, we evaluated various PETL techniques, including fine-tuning, Adapter, LoRA, and prefix-tuning, on three different intention classification tasks, each utilizing various language models with different scales.Comment: *SEM 202

Universal Domain Adaptation for Robust Handling of Distributional Shifts in NLP

When deploying machine learning systems to the wild, it is highly desirable for them to effectively leverage prior knowledge to the unfamiliar domain while also firing alarms to anomalous inputs. In order to address these requirements, Universal Domain Adaptation (UniDA) has emerged as a novel research area in computer vision, focusing on achieving both adaptation ability and robustness (i.e., the ability to detect out-of-distribution samples). While UniDA has led significant progress in computer vision, its application on language input still needs to be explored despite its feasibility. In this paper, we propose a comprehensive benchmark for natural language that offers thorough viewpoints of the model's generalizability and robustness. Our benchmark encompasses multiple datasets with varying difficulty levels and characteristics, including temporal shifts and diverse domains. On top of our testbed, we validate existing UniDA methods from computer vision and state-of-the-art domain adaptation techniques from NLP literature, yielding valuable findings: We observe that UniDA methods originally designed for image input can be effectively transferred to the natural language domain while also underscoring the effect of adaptation difficulty in determining the model's performance.Comment: Findings of EMNLP 202

The Effect of Breastfeeding Duration and Parity on the Risk of Epithelial Ovarian Cancer: A Systematic Review and Meta-analysis

Review Objectives: We conducted a systematic review and meta-analysis to summarize current evidence regarding the association of parity and duration of breastfeeding with the risk of epithelial ovarian cancer (EOC). Methods: A systematic search of relevant studies published by December 31, 2015 was performed in PubMed and EMBASE. A random-effect model was used to obtain the summary relative risks (RRs) and 95% confidence intervals (CIs). Results: Thirty-two studies had parity categories of 1, 2, and ≥3. The summary RRs for EOC were 0.72 (95% CI, 0.65 to 0.79), 0.57 (95% CI, 0.49 to 0.65), and 0.46 (95% CI, 0.41 to 0.52), respectively. Small to moderate heterogeneity was observed for one birth (p<0.01; Q=59.46; I 2 =47.9%). Fifteen studies had breastfeeding categories of <6 months, 6-12 months, and >13 months. The summary RRs were 0.79 (95% CI, 0.72 to 0.87), 0.72 (95% CI, 0.64 to 0.81), and 0.67 (95% CI, 0.56 to 0.79), respectively. Only small heterogeneity was observed for <6 months of breastfeeding (p=0.17; Q=18.79, I 2 =25.5%). Compared to nulliparous women with no history of breastfeeding, the joint effects of two births and <6 months of breastfeeding resulted in a 0.5-fold reduced risk for EOC. Conclusions: The first birth and breastfeeding for <6 months were associated with significant reductions in EOC risk. Key words: Ovarian neoplasms, Parity, Breast feeding, Reproduction, Risk factors, Meta-analysis Received: June 29, 2016 Accepted: September 8, 2016 Corresponding author: Suekyung Park, MD, PhD 103 Daehak-ro, Jongno-gu, Seoul 03080, Korea Tel: +82-2-740-8338, Fax: +82-2-747-4830 E-mail: [email protected] This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/bync/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. INTRODUCTION Worldwide, ovarian cancer is the seventh most common cancer in women. Furthermore, it is the sixth leading cause of cancer deaths in women and the second most common cause of death among those with gynecologic cancers 350 to 8%), germ cell tumors (3% to 5%), and other rare types of ovarian cancer Most ovarian cancers are life-threatening and are notorious for having a poor prognosis, as they are usually diagnosed at an advanced stage. Moreover, screening results based on pelvic imaging or tumor markers for early detection remain unsatisfactory Reproductive risk factors for epithelial ovarian cancer (EOC) have been extensively explored in epidemiologic studies. For instance, a pooled analysis of 12 US case-control studies in 1992 showed that parous women and those who had breastfed had a lower risk of EOC Since 1992, many studies from around the world have reported associations of parity and breastfeeding with ovarian cancer. However, findings concerning the protective role of increasing parity and duration of breastfeeding remain inconsistent. For parity, some studies have indicated that the first birth reduces ovarian cancer risk more than subsequent births Therefore, we conducted a systematic review and metaanalysis to summarize the current evidence regarding the association of parity and duration of breastfeeding with EOC risk. The aim of this study was to clarify the threshold for risk reduction among the studies without heterogeneity across the results. An additional aim was to perform a meta-analysis to estimate the joint risk reductions associated with parity and breastfeeding. METHODS Search Strategy We performed a literature search including studies published through December 2015 using the following search terms in the PubMed and EMBASE databases (1) (parity or "number of live births") and (ovary or ovarian) and (cancer or tumor or neoplasm or malignancy) or (2) (breastfeeding or lactation) and (ovary or ovarian) and (cancer or tumor or neoplasm or malignancy). Furthermore, to find any additional published studies, a manual search was performed by checking all references of prior meta-analyses [5,6.8,20-23] and of all the original studies. This systematic review was planned, conducted, and reported in adherence to the standards of quality for reporting meta-analyses Study Selection To be included, studies had to meet the following criteria: (1) the studies were observational (case-control or cohort studies), (2) the exposures of interest were the number of live births and the total duration of breastfeeding, (3) the outcome of interest was EOC, (4) odds ratios (ORs) or relative risk (RR) estimates with 95% confidence intervals (CIs) were reported or sufficient data were present to allow the calculation of these effect measures, and (5) articles were published in the English language. In the case of overlapping data, the study with the largest number of cases was included. As fertility treatments and BRCA mutation effects on EOC may alter the association between parity/breastfeeding and EOC [26], we excluded studies conducted on specific populations, such as BRCA-1 or BRCA-2 mutation carriers or infertile women treated with fertility drugs. The detailed steps of our literature search are shown in Data Extraction Data extraction was conducted independently by two authors. Disagreements were discussed and resolved by consensus. The following data were collected from each study: the first author's last name, publication year, study region and design, study period, participant age, sample size (cases and 351 Parity and Breastfeeding Effects on Ovarian Cancer Risk controls or cohort size), exposure variables (parity or total breastfeeding duration), study-specific adjusted RR or OR with 95% CIs for each exposure category, and factors matched or adjusted for in the design or data analysis. If no adjusted RR or OR was presented, we included crude estimates. If no RRs or ORs were presented in a given study, we calculated them and the 95% CIs according to the raw frequencies presented in the article. The quality of the study was assessed independently by two authors using the 9-star Newcastle-Ottawa Scale (range, 0 to 9 stars) Statistical Analysis The study-specific RRs or ORs with 95% CIs were used to determine the principal outcome. Because the OR closely approximates the RR for rare diseases, the RR can be estimated from a case-control study using the OR as an approximation One study did not provide the required risk estimates for analysis or separate the risk estimates for different categories of parity or breastfeeding duration. For this study, we used the method proposed by Fleiss and Gross [30]. This method allows adjusted effect estimates and CIs to be calculated for any alternative comparison of levels and can help in a dose-response meta-analysis. Briefly, we combined risk estimates obtained through a simple fixed-effects meta-analysis wherein the subjects were divided into unexposed groups (i=0) and exposed groups (i=1, …, n), and estimates (Ri) with lower and upper 95% CIs were available. To obtain the R1+, we meta-analyzed R1, R2, R3, …, Rn using a fixed-effect model. The categories of parity or breastfeeding duration varied across studies; accordingly, the number of studies included in each metaanalysis and the summary RRs in each meta-analysis were different depending upon the number of categories. Statistical heterogeneity among studies was evaluated with the Cochran Q and I-squared statistics 352 with ≤7 stars considered low-quality as per the 9-star Newcastle-Ottawa Scale; and (3) year of publication (<2000, ≥ 2000), respectively. Publication bias was evaluated using the Begg rank correlation and the Egger linear regression test, in which p-vlaue <0.05 were considered representative of statistically significant publication bias From the meta-analyzed result, to calculate the RR for the joint effect of parity and breastfeeding, we applied the log-linear dose-response model proposed by Berlin et al. We configured the following formula for the multivariate linear logit regression of two factors: Logit P=α + β1χ1 + β2χ2; where P is the probability of a particular outcome (EOC risk), α is the intercept from the linear regression equation, β is the regression coefficient multiplied by some value of the predictor, and χ is the risk factor (parity and breastfeeding). Using this equation yields the value of the RR for the joint effects of parity and breastfeeding duration. For example, in the case of a subject who has no risk factors, logit(P) is α. In this case, the probability of EOC is exp(α)=1.0. In the case of a subject with only χ1, logit(P) is α+β1. In the case of a subject with both χ1 and χ2, logit(P) is α+β1+β2. Accordingly, the probability of EOC is exp(β1+β2)=OR1×OR2. Since the category of parity and breastfeeding duration varied across studies, to calculate the RR for the joint effect of parity and breastfeeding, we used the summary RR for parity and breastfeeding duration that contained the largest number of studies. All statistical analyses were performed with Stata version 12.0 (StataCorp., College Station, TX, USA). RESULTS Study Characteristics The characteristics of the 32 studies included with data regarding parity and the 15 studies included with data regarding breastfeeding are shown in Supplemental 353 Parity and Breastfeeding Effects on Ovarian Cancer Risk Africa. For breastfeeding, two cohort studies and 13 case-control studies were included. The included studies were conducted between 1978 and 2008. Of the 15 studies, seven were performed in North America, six in Europe, one in Asia, and one in Australia. Parity and Epithelial Ovarian Cancer Risk Thirty-two studies had parity categories of 1, 2, and ≥3. The summary RRs for the first, second, and third births were 0.72 (95% CI, 0.65 to 0.79), 0.57 (95% CI, 0.49 to 0.65), and 0.46 (95% CI, 0.41 to 0.52), respectively Duration of Breastfeeding and Epithelial Ovarian Cancer Risk Fifteen studies had breastfeeding categories of <6 months, 6-12 months, and ≥13 months. The summary RRs for these categories were 0.79 (95% CI, 0.72 to 0.87), 0.72 (95% CI, 0.64 to 0.81) and 0.67 (95% CI, 0.56 to 0.79), respectively Subgroup Analysis According to Study Design, Study Quality, and Publication Year The results from the subgroup analysis according to study design, study quality, and publication year are shown in Relative Risk for the Joint Effect of Parity and Breastfeeding The RR for the joint effect of parity and breastfeeding, obtained using the summary RR from the analysis of 32 studies with parity categories of 1, 2, and ≥3 and 15 studies with breastfeeding categories of <6 months, 6-12 months, and ≥ 13 months, is shown in DISCUSSION The findings of this meta-analysis indicate that parity and breastfeeding experiences in women can help prevent EOC, which is typically life-threatening and has a poor prognosis. In particular, the first birth and the first six months of breastfeeding had a greater protective effect than did subsequent births and/or additional breastfeeding, although multiparity and additional breastfeeding did provide some additional protection. The risk reduction effect of the first birth on EOC risk was almost 30%, and the combined effect of the first birth and <6 months of breastfeeding was 40%; thus, breastfeeding provided a nearly 10% greater risk reduction. In regards to parity, the EOC risk reduction was highest for the first birth, with some additional protection from the second birth. However, slightly less risk reduction was observed for the third birth Pregnancy and breastfeeding are thought to reduce EOC risk Ho Kyung Sung, et al. 354 by decreasing pituitary gonadotropin levels and inducing anovulation [7,35]. Pregnancy and breastfeeding are expected to decrease the likelihood of spontaneous genetic mutation under the incessant ovulation hypothesis and of the hyperproliferation of inclusion cysts under the gonadotropin hypothesis. However, the observation that multiparity and additional breastfeeding did not provide an equal amount of protection does not provide evidence for either of these hypotheses. Nev- The summary RRs (95% CIs) in each meta-analysis were estimated using a random effect model. 3 Studies with ≥8 stars were considered high-quality as per the 9-star Newcastle-Ottawa Scale. 4 Studies with ≤7 stars were considered low-quality as per the 9-star Newcastle-Ottawa Scale. 355 Parity and Breastfeeding Effects on Ovarian Cancer Risk ertheless, the results of two experimental studies provide biological evidence for the relatively weaker protective effect of additional parity and breastfeeding [36,37]. For instance, high progesterone levels during pregnancy can increase apoptosis, which may clear transformed cells from the ovarian epithelium, meaning that all the accumulated transformed cells are washed fully out by the first pregnancy. Therefore, the first pregnancy provides a stronger protective effect than subsequent pregnancies [36]. In regards to breastfeeding, breastfeeding in the first few months completely inhibits the pulsatile secretion of gonadotropin-releasing hormone and luteinizing hormone, leading to suppression of ovulation [37]. After a couple of months, ovulatory activity may return, even though breastfeeding continues [37]; thus, a longer duration of breastfeeding does not provide an additional protective effect. Our finding of decreased EOC risk with longer breastfeeding is similar to that reported by prior meta-analyses in 2013 and 2014 [22,23], but differs from that of a meta-analysis of nine case-control studies conducted in developed countries in 2001, in which breastfeeding for ≥12 months was associated with a significant 0.72-fold reduced risk of EOC compared to never having breastfed, while breastfeeding <12 months did not show such an association (OR, 0.95; 95% CI, 0.80 to 1.12) The strength of this meta-analysis is that it included all available studies, and the large number of EOC cases allowed for the investigation of the risk associated with different categories of parity and breastfeeding duration. However, the current study also has several limitations. First, our meta-analysis wa

The Effect of Network Structure on Performance in South Korea SMEs: The Moderating Effects of Absorptive Capacity

This study has analyzed how the network structure (density, centrality) affects the performance (convergence, overall) in Small and Medium-Sized Enterprises (SMEs) with the different business exchange activities. In addition, we conducted a moderating effect of absorptive capacity (ability, motivation). Based on previous literature about the network and absorptive capacity, research hypotheses were developed and tested using a sample of 226 SMEs in South Korea. The results show that network density and centrality have a positive effect on perceived convergence and overall performance. These results are consistent with previous network studies relating to structural characteristics. To raise total performances in firms with different business exchange activities, they need to enhance density and centrality. For example, firms with different business exchange activities and high levels of density and centrality are able to easily obtain various types of information from other members of 7000 firms in Korea. Through this, they can improve performance. We also conducted the moderated regression analysis. The results show that employee ability has a positive moderating effect on the relationship between centrality and the overall performance while employee motivation has a negative moderating effect. The implications and directions for future study along with limitations are presented

Curable Area Substantiation of Self-Healing in Concrete Using Neutral Axis

The self-healing nature of concrete has been proved in many studies using various methods. However, the underlying mechanisms and the distinct area of self-healing have not been identified in detail. This study focuses on the limits of the area of self-healing. A bending specimen with a notch is used herein, and its flexural strength and stiffness before and after healing are compared and used for self-healing assessment. In addition, the neutral axis of the specimen was measured using successive strain gauges attached to the crack propagation part. Although the strength and stiffness of the concrete recovered after self-healing, the change in the location of the neutral axis before and after healing was insignificant, which indicates that physical recovery did not occur for once-opened crack areas

The Effect of Network Structure on Performance in South Korea SMEs: The Moderating Effects of Absorptive Capacity

This study has analyzed how the network structure (density, centrality) affects the performance (convergence, overall) in Small and Medium-Sized Enterprises (SMEs) with the different business exchange activities. In addition, we conducted a moderating effect of absorptive capacity (ability, motivation). Based on previous literature about the network and absorptive capacity, research hypotheses were developed and tested using a sample of 226 SMEs in South Korea. The results show that network density and centrality have a positive effect on perceived convergence and overall performance. These results are consistent with previous network studies relating to structural characteristics. To raise total performances in firms with different business exchange activities, they need to enhance density and centrality. For example, firms with different business exchange activities and high levels of density and centrality are able to easily obtain various types of information from other members of 7000 firms in Korea. Through this, they can improve performance. We also conducted the moderated regression analysis. The results show that employee ability has a positive moderating effect on the relationship between centrality and the overall performance while employee motivation has a negative moderating effect. The implications and directions for future study along with limitations are presented

Economic feasibility studies of high pressure PEM water electrolysis for distributed H-2 refueling stations

In this paper, we report economic feasibility studies focusing on profitability analysis of high pressure polymer electrolyte membrane (PEM) water electrolysis for distributed H-2 refueling stations in Korea. From capital and operating costs, a unit H-2 production cost of 6.24 $kgH(2)(-1) was obtained for a H-2 capacity of 700 m(3) h(-1), which is equivalent to handling about 300 fuel cell electric vehicles. Based on cost estimations, profitability analysis using cash flow diagrams was performed to assess the economic feasibility of high pressure PEM water electrolysis and various key economic indicators like net present value (NPV), discounted payback period (DPBP), and present value ratio were obtained for both different discount rates and capacity factors. In conclusion, employment of high pressure PEM water electrolysis was found to be economically profitable showing reasonably high NPVs (16-52 MM$) and short DPBPs (4-7 years)