Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning

Learning from demonstration (LfD) is a popular technique that uses expert demonstrations to learn robot control policies. However, the difficulty in acquiring expert-quality demonstrations limits the applicability of LfD methods: real-world data collection is often costly, and the quality of the demonstrations depends greatly on the demonstrator's abilities and safety concerns. A number of works have leveraged data augmentation (DA) to inexpensively generate additional demonstration data, but most DA works generate augmented data in a random fashion and ultimately produce highly suboptimal data. In this work, we propose Guided Data Augmentation (GuDA), a human-guided DA framework that generates expert-quality augmented data. The key insight of GuDA is that while it may be difficult to demonstrate the sequence of actions required to produce expert data, a user can often easily identify when an augmented trajectory segment represents task progress. Thus, the user can impose a series of simple rules on the DA process to automatically generate augmented samples that approximate expert behavior. To extract a policy from GuDA, we use off-the-shelf offline reinforcement learning and behavior cloning algorithms. We evaluate GuDA on a physical robot soccer task as well as simulated D4RL navigation tasks, a simulated autonomous driving task, and a simulated soccer task. Empirically, we find that GuDA enables learning from a small set of potentially suboptimal demonstrations and substantially outperforms a DA strategy that samples augmented data randomly

The role of online interaction in user stickiness of social commerce: The shopping value perspective

Recent studies indicate that online interactions generate significant effects on consumer behavior, such as customer loyalty and satisfaction, through shopping values, while there are only very few studies devoted to user stickiness. Therefore, the purpose of this paper is to contribute to the existing literature by analyzing the effects of online interactions on user stickiness in a social commerce environment through the perspective of shopping values. This study measures online interactions through perceived ease of use, perceived usefulness and social interactivity. Shopping values are evaluated by both utilitarian and hedonic values. Then a Stimulus (S)-Organism (O)-Response (R) model that relates online interaction, shopping values and user stickiness, is constructed to derive several research hypotheses. Based on 183 valid questionnaires the results reveal that perceived ease of use and social interactivity can positively affect utilitarian shopping values and user stickiness, while perceived usefulness and social interactivity positively affect hedonic shopping values and then influence user stickiness. Hence, online interactions play a crucial role in strengthening user stickiness in social commerce. Therefore, for social commerce platforms, managers need to continue improving their perceived ease of use, perceived usefulness and social interactivity to attract and retain their users

Separation of Antioxidant Peptides from Pepsin Hydrolysate of Whey Protein Isolate by ATPS of EOPO Co-polymer (UCON)/Phosphate

Abstract An aqueous two-phase system (ATPS) consisting of poly(ethylene glycol-ran-propylene glycol) monobutyl ether (UCON)/phosphate was developed for the separation of the antioxidant peptides from pepsin hydrolysate of Whey Protein Isolate (WPI). The efficiency of the separation was evaluated based on the DPPH radical scavenging activity, ABTS radical scavenging activity and ferric reducing antioxidant power (FRAP) of the separated peptides. The effects of some parameters on the partition of antioxidant peptides were investigated. An efficient separation of antioxidant peptides was achieved using ATPS with pH of 4.0, 4 mL of UCON solution (40%, w/w), 4 mL of KH2PO4 solution (15.5%, w/w), 2 mL of WPI hydrolysate and 0.40 g/10 mL of NaCl. Reversed-phase high-performance liquid chromatography (RP-HPLC), amino acid analyzer and matrix-assisted laser desorption ionization time-of-flight mass spectrometry (MALDI-TOF MS) were used to characterize the purified peptides separated by the ATPS. The peptides in top phase were less polar than those in bottom phase. More antioxidative and hydrophobic amino acids were extracted to the top phase of ATPS, and the peptides with the amino acid sequences with antioxidant activities moved to the top phase as well. In conclusion, antioxidant peptides were successfully separated from the WPI hydrolysate by UCON/phosphate ATPS

The Evolution Mode and Driving Mechanisms of the Relationship between Construction Land Use and Permanent Population in Urban and Rural Contexts: Evidence from China’s Land Survey

(1) Background: Achieving harmonious human–land relations is one of the key objectives of sustainable urban–rural development, and the degree of decoupling of permanent population levels from changes in construction land use is an important factor in related analyses. Due to the existence of huge urban–rural differences, rethinking China’s human–land relations from the perspective of integrating urban and rural areas is of great value for the advancement of high-quality urban–rural development. (2) Methods: By studying the lower reaches of the Yangtze and Yellow Rivers of China, and based on data from the second and third national land surveys of China, this paper analyzes the spatio-temporal evolution of urban and rural population, construction land use, and human–land relations from 2009 to 2019 using exploratory spatial data analysis (ESDA) and a decoupling model; in addition, this paper proposes a differentiated zoning management strategy and establishes a new framework that integrates evolutionary patterns, human–land relations, spatial effects, and policy design. (3) Results: The geographic distribution patterns of urban and rural population and construction land use remained stable over time, with high levels of spatial heterogeneity, agglomeration, and correlation. Changes in urban and rural population levels and construction land use are becoming increasingly diversified and complex, with both increases and reductions existing side by side. Based on a Boston Consulting Group matrix, the evolution patterns of urban and rural population and construction land use are divided into four types, referred to as star-cities, cow-cities, question-cities, and dog-cities. Over the time period examined in this paper, the spatial autocorrelation of urban land evolution patterns turned from negative to positive; however, that of rural land, as well as those of urban and rural population evolution patterns, were statistically insignificant. Urban human–land relations are coordinated, in general, and are mostly in a state of either weak decoupling or expansive coupling. In contrast, rural human–land relations are seriously imbalanced, and most of them are in a state of strong negative decoupling. Human–land relations are dominated by regressive changes in urban areas but remain unchanged in rural areas. Cold- and hot-spot cities are concentrated in clusters or in bands, forming a core-periphery structure. The formation and evolution of the decoupling relationship between construction land use and permanent population are the results of multiple factors, including urbanization, industrialization, globalization, and government demand and policy intervention. The interaction effects between different factors show bifactor enhancement and nonlinear enhancement, with complex driving mechanisms and large urban–rural differences. It should be highlighted that the influence intensity, operation mechanism, and changes in the trends for different factors vary greatly. Urbanization rate, gross domestic product, and government revenue are key factors that exert a strong direct driving force; international trade, foreign direct investment, and per capita GDP are important factors, while the remaining factors are auxiliary factors that remain heavily dependent on interaction effects. (4) Conclusions: To further transform human–land relations from imbalanced to coordinated, we divide the study area into four area types based on the concept of urban–rural community: urban and rural intensive policy areas, urban intensive policy areas, rural intensive policy areas, and urban and rural controlled policy areas. Furthermore, we put forward suggestions on the differentiated management of land use for the four types of policy areas

The Evolution Mode and Driving Mechanisms of the Relationship between Construction Land Use and Permanent Population in Urban and Rural Contexts: Evidence from China’s Land Survey

(1) Background: Achieving harmonious human–land relations is one of the key objectives of sustainable urban–rural development, and the degree of decoupling of permanent population levels from changes in construction land use is an important factor in related analyses. Due to the existence of huge urban–rural differences, rethinking China’s human–land relations from the perspective of integrating urban and rural areas is of great value for the advancement of high-quality urban–rural development. (2) Methods: By studying the lower reaches of the Yangtze and Yellow Rivers of China, and based on data from the second and third national land surveys of China, this paper analyzes the spatio-temporal evolution of urban and rural population, construction land use, and human–land relations from 2009 to 2019 using exploratory spatial data analysis (ESDA) and a decoupling model; in addition, this paper proposes a differentiated zoning management strategy and establishes a new framework that integrates evolutionary patterns, human–land relations, spatial effects, and policy design. (3) Results: The geographic distribution patterns of urban and rural population and construction land use remained stable over time, with high levels of spatial heterogeneity, agglomeration, and correlation. Changes in urban and rural population levels and construction land use are becoming increasingly diversified and complex, with both increases and reductions existing side by side. Based on a Boston Consulting Group matrix, the evolution patterns of urban and rural population and construction land use are divided into four types, referred to as star-cities, cow-cities, question-cities, and dog-cities. Over the time period examined in this paper, the spatial autocorrelation of urban land evolution patterns turned from negative to positive; however, that of rural land, as well as those of urban and rural population evolution patterns, were statistically insignificant. Urban human–land relations are coordinated, in general, and are mostly in a state of either weak decoupling or expansive coupling. In contrast, rural human–land relations are seriously imbalanced, and most of them are in a state of strong negative decoupling. Human–land relations are dominated by regressive changes in urban areas but remain unchanged in rural areas. Cold- and hot-spot cities are concentrated in clusters or in bands, forming a core-periphery structure. The formation and evolution of the decoupling relationship between construction land use and permanent population are the results of multiple factors, including urbanization, industrialization, globalization, and government demand and policy intervention. The interaction effects between different factors show bifactor enhancement and nonlinear enhancement, with complex driving mechanisms and large urban–rural differences. It should be highlighted that the influence intensity, operation mechanism, and changes in the trends for different factors vary greatly. Urbanization rate, gross domestic product, and government revenue are key factors that exert a strong direct driving force; international trade, foreign direct investment, and per capita GDP are important factors, while the remaining factors are auxiliary factors that remain heavily dependent on interaction effects. (4) Conclusions: To further transform human–land relations from imbalanced to coordinated, we divide the study area into four area types based on the concept of urban–rural community: urban and rural intensive policy areas, urban intensive policy areas, rural intensive policy areas, and urban and rural controlled policy areas. Furthermore, we put forward suggestions on the differentiated management of land use for the four types of policy areas

<i>ins-7</i> Gene Expression Is Partially Regulated by the DAF-16/IIS Signaling Pathway in <i>Caenorhabditis elegans</i> under Celecoxib Intervention

<div><p>DAF-16 target genes are employed as reporters of the insulin/IGF-1 like signal pathway (IIS), and this is notably true when <i>Caenorhabditis elegans</i> (<i>C. elegans</i>) is used to study the action of anti-aging compounds on IIS activity. However, some of these genes may not be specific to DAF-16, even if their expression levels are altered when DAF-16 is activated. Celecoxib was reported to extend the lifespan of <i>C. elegans</i> through activation of DAF-16. Our results confirmed the function of celecoxib on aging; however, we found that the expression of <i>ins-7</i>, a DAF-16 target gene, was abnormally regulated by celecoxib. <i>ins-7</i> plays an important role in regulating aging, and its expression is suppressed in <i>C. elegans</i> when DAF-16 is activated. However, we found that celecoxib upregulated the expression of <i>ins-7</i> in contrast to its role in DAF-16 activation. Our subsequent analysis indicated that the expression level of <i>ins-7</i> in <i>C. elegans</i> was negatively regulated by DAF-16 activity. Additionally, its expression was also positively regulated by DAF-16-independent mechanisms, at least following external pharmacological intervention. Our study suggests that <i>ins-7</i> is not a specific target gene of DAF-16, and should not be chosen as a reporter for IIS activity. This conclusion is important in the study of INSs on aging in <i>C. elegans</i>, especially under the circumstance of drug intervention.</p></div

The effects of celecoxib on DAF-16::GFP and PQM-1::GFP nuclear localization.

<p>Celecoxib enhanced DAF-16 nuclear localization (<b>A, B</b>) and PQM-1::GFP nuclear localization in <i>daf-16 (RNAi)</i> worms (<b>A, C</b>). However, PQM-1::GFP nuclear localization was not significantly changed following celecoxib treatment in N2 worms (<b>A, D</b>). <i>ins-7</i> was upregulated following celecoxib treatment in <i>pqm-1</i> mutants (<b>E</b>). Error bars represent SD. P values were calculated using a T-test, *P<0.05; **P<0.01; ***P<0.001. Also see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0100320#pone.0100320.s002" target="_blank">Table S2</a> and <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0100320#pone.0100320.s004" target="_blank">S4</a>.</p

Celecoxib regulates DAF-16 target gene expression.

<p><i>scl-20, K09F6.6, ins-7</i>, and <i>sod-3</i> expression profiles in <i>daf-2 (e1370) III</i> mutants compared to N2 worms (<b>A</b>). <i>ins-7</i> was significantly upregulated in N2 worms treated with 10 µM celecoxib (<b>B</b>). Celecoxib was previously reported to regulate DAF-16 target genes by decreasing IIS signal transduction activity. <i>ins-7</i> was reported to be a target gene of DAF-16 and downregulated when IIS signal activity was decreased. These results indicated <i>ins-7</i> was uncharacteristically upregulated in celecoxib-treated N2 worms. The results of at least three independent experiments are presented. Data are averages of real-time PCR results ± Standard Deviation (SD). Error bars represent SD. P values were calculated by using a T-test, ***P<0.001. Also see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0100320#pone.0100320.s002" target="_blank">Table S2</a>.</p

The effects of celecoxib on lifespan.

<p>P values were calculated for individual experiments, each including control and experimental animals and performed at the same time. The table shows the number of dead animals. Animals that crawled off the plate, bagged, or burst were censored and were therefore excluded from all analysis. All statistics were calculated using SPSS software (SPSS, Chicago, IL, USA). The log rank (Mantel-Cox) test was used for statistical analysis. #no significant different (P>0.05).</p