Speeding up Context-based Sentence Representation Learning with Non-autoregressive Convolutional Decoding

Context plays an important role in human language understanding, thus it may also be useful for machines learning vector representations of language. In this paper, we explore an asymmetric encoder-decoder structure for unsupervised context-based sentence representation learning. We carefully designed experiments to show that neither an autoregressive decoder nor an RNN decoder is required. After that, we designed a model which still keeps an RNN as the encoder, while using a non-autoregressive convolutional decoder. We further combine a suite of effective designs to significantly improve model efficiency while also achieving better performance. Our model is trained on two different large unlabelled corpora, and in both cases the transferability is evaluated on a set of downstream NLP tasks. We empirically show that our model is simple and fast while producing rich sentence representations that excel in downstream tasks

Rethinking Skip-thought: A Neighborhood based Approach

We study the skip-thought model with neighborhood information as weak supervision. More specifically, we propose a skip-thought neighbor model to consider the adjacent sentences as a neighborhood. We train our skip-thought neighbor model on a large corpus with continuous sentences, and then evaluate the trained model on 7 tasks, which include semantic relatedness, paraphrase detection, and classification benchmarks. Both quantitative comparison and qualitative investigation are conducted. We empirically show that, our skip-thought neighbor model performs as well as the skip-thought model on evaluation tasks. In addition, we found that, incorporating an autoencoder path in our model didn't aid our model to perform better, while it hurts the performance of the skip-thought model

Improved Roundtrip Spanners, Emulators, and Directed Girth Approximation

Roundtrip spanners are the analog of spanners in directed graphs, where the roundtrip metric is used as a notion of distance. Recent works have shown existential results of roundtrip spanners nearly matching the undirected case, but the time complexity for constructing roundtrip spanners is still widely open. This paper focuses on developing fast algorithms for roundtrip spanners and related problems. For any $n$-vertex directed graph $G$ with $m$ edges (with non-negative edge weights), our results are as follows: - 3-roundtrip spanner faster than APSP: We give an $\tilde{O}(m\sqrt{n})$-time algorithm that constructs a roundtrip spanner of stretch $3$ and optimal size $O(n^{3/2})$. Previous constructions of roundtrip spanners of the same size either required $\Omega(nm)$ time [Roditty, Thorup, Zwick SODA'02; Cen, Duan, Gu ICALP'20], or had worse stretch $4$ [Chechik and Lifshitz SODA'21]. - Optimal roundtrip emulator in dense graphs: For integer $k\ge 3$, we give an $O(kn^2\log n)$-time algorithm that constructs a roundtrip \emph{emulator} of stretch $(2k-1)$ and size $O(kn^{1+1/k})$, which is optimal for constant $k$ under Erd\H{o}s' girth conjecture. Previous work of [Thorup and Zwick STOC'01] implied a roundtrip emulator of the same size and stretch, but it required $\Omega(nm)$ construction time. Our improved running time is near-optimal for dense graphs. - Faster girth approximation in sparse graphs: We give an $\tilde{O}(mn^{1/3})$-time algorithm that $4$-approximates the girth of a directed graph. This can be compared with the previous $2$-approximation algorithm in $\tilde{O}(n^2, m\sqrt{n})$ time by [Chechik and Lifshitz SODA'21]. In sparse graphs, our algorithm achieves better running time at the cost of a larger approximation ratio.Comment: To appear in SODA 202

Approximation Algorithms and Hardness for nn-Pairs Shortest Paths and All-Nodes Shortest Cycles

We study the approximability of two related problems on graphs with $n$ nodes and $m$ edges: $n$-Pairs Shortest Paths ($n$-PSP), where the goal is to find a shortest path between $O(n)$ prespecified pairs, and All Node Shortest Cycles (ANSC), where the goal is to find the shortest cycle passing through each node. Approximate $n$-PSP has been previously studied, mostly in the context of distance oracles. We ask the question of whether approximate $n$-PSP can be solved faster than by using distance oracles or All Pair Shortest Paths (APSP). ANSC has also been studied previously, but only in terms of exact algorithms, rather than approximation. We provide a thorough study of the approximability of $n$-PSP and ANSC, providing a wide array of algorithms and conditional lower bounds that trade off between running time and approximation ratio. A highlight of our conditional lower bounds results is that for any integer $k\ge 1$, under the combinatorial $4k$-clique hypothesis, there is no combinatorial algorithm for unweighted undirected $n$-PSP with approximation ratio better than $1+1/k$ that runs in $O(m^{2-2/(k+1)}n^{1/(k+1)-\epsilon})$ time. This nearly matches an upper bound implied by the result of Agarwal (2014). A highlight of our algorithmic results is that one can solve both $n$-PSP and ANSC in $\tilde O(m+ n^{3/2+\epsilon})$ time with approximation factor $2+\epsilon$ (and additive error that is function of $\epsilon$), for any constant $\epsilon>0$. For $n$-PSP, our conditional lower bounds imply that this approximation ratio is nearly optimal for any subquadratic-time combinatorial algorithm. We further extend these algorithms for $n$-PSP and ANSC to obtain a time/accuracy trade-off that includes near-linear time algorithms.Comment: Abstract truncated to meet arXiv requirement. To appear in FOCS 202

Conservation management improves agroecosystem function and resilience of soil nitrogen cycling in response to seasonal changes in climate

Understanding how conservation agricultural management improves soil nitrogen (N) stability in the face of climate change can help increase agroecosystem productivity and mitigate runoff, leaching and downstream water quality issues. We conducted a 2-year field study in a 36-year-old rain-fed cotton production system to evaluate the impacts of changing climatic factors (temperature and precipitation) on soil N under conservation management, including moderate inorganic N fertilizer application (0 and 67 kg N ha−1 ), winter cover crops (fallow; winter wheat, Triticum aestivum L.; hairy vetch, Vicia villosa Roth), and reduced tillage (no-till; disk tillage). Structural equation modeling (SEM) was used to quantify and compare the effects of conservation management and climatic factors on soil N concentrations. Fertilizer and vetch cover crops increased soil total N concentration by 16% and 18%, respectively, and also increased microbial N transformation rate by 41% and 168%. In addition, vetch cover crops also increased soil labile N concentrations by 57%, 21%, and 79%, i.e., extractable organic N, ammonium, and nitrate, respectively. The highest soil δ15N value (6.4 ± 0.3‰) was observed under the 67 kg N ha−1 fertilizer-wheat-disk tillage treatment, and the lowest value (4.8 ± 0.3‰) under the zero-fertilizer-wheat-no-till treatment, indicating fertilizer and tillage might accelerate microbial N transformation. The SEM showed positive effects of temperature and precipitation on labile N concentrations, suggesting destabilization of soil N and the potential for soil N loss under increased temperature and intensified precipitation. Fertilizer and vetch use might mitigate some of the effects of temperature by accelerating microbial N transformations, with vetch having a larger effect than fertilizer (0.35 vs. 0.15, Table 1). No-till can reduce some of the effects of precipitation on soil labile N by maintaining soil structure. Our study suggests that fertilizer, vetch cover crop, and no-till might help improve function and resilience of agroecosystems in relation to soil N cycling. Soil N stabilization in cropping systems can be enhanced by adjusting agricultural management

Cover Crops and Corn Residue Removal: Impacts on Soil Hydraulic Properties and Their Relationships with Carbon

Large-scale crop residue removal may negatively affect soil water dynamics. Integrating cover crop (CC) with crop residue management can be a strategy to offset potential adverse effects of residue removal. We studied: (i) the impact of corn (Zea mays L.) residue removal (56%) with and without the use of winter rye (Secale cereale L.) CC on soil hydraulic properties, (ii) whether CC would ameliorate residue removal effects on hydraulic properties, and (iii) relationships of hydraulic properties with soil organic C (SOC) and other properties under irrigated no-till continuous corn on a silt loam in south central Nebraska after 5 and 6 yr of management. Cover crops did not affect soil hydraulic properties. However, residue removal reduced cumulative water infiltration by about 45% in one year. Across years, residue removal reduced plant available water (PAW) by 32% and mean weight diameter of water-stable aggregates (MWD) by 23% for the upper 5-cm soil depth. Under no CC, residue removal reduced SOC concentration by 25% in the 0- to 5-cm and by 11% in the 5- to 10-cm depths. Under residue removal, CC increased SOC concentration by 18% in the 0- to 5-cm and by 8% in the 5 to 10-cm depths. Cover crop did not completely offset the residue removal-induced decrease in SOC concentration in the upper 5-cm depth. Plant available water decreased as SOC concentration and MWD decreased. After 6 yr, corn residue removal adversely affected soil hydraulic properties and SOC concentration, but CC was unable to fully offset such adverse impacts

Electrospun 1D and 2D Carbon and Polyvinylidene Fluoride (PVDF) Piezoelectric Nanocomposites

Piezoelectric nanocomposite fibrous membranes consisting of polymer polyvinylidene fluoride (PVDF) as matrix and incorporating 1D carbon nanotubes (CNTs) and 2D graphene oxide (GO) were prepared using an electrospinning process. The influence of the filler type, loading, and dispersion status on the total PVDF crystallinity (X_{c}); Piezoelectric nanocomposite fibrous membranes consisting of polymer polyvinylidene fluoride (PVDF) as matrix and incorporating 1D carbon nanotubes (CNTs) and 2D graphene oxide (GO) were prepared using an electrospinning process. The influence of the filler type, loading, and dispersion status on the total PVDF crystallinity (F_{β}); the volume fraction of β phase in the samples (v_{β}); and the piezoelectric coefficient d_{33} were investigated. The V_{β} is used to assess the formation of β phase for the first time, which considered the combined influence of fillers on X_{c} and F_{β}, and is more practical than other investigations using only F_{β} for the assessment. The inclusion of all types of carbon fillers had resulted in a considerable reduction in the X_{c} compared with the neat PVDF, and the X_{c} decreased with the CNT loading while increased with the GO loading. The addition of CNT and GO had also reduced the F_{β} compared with the neat PVDF, and F_{β} increased with CNT loading while decreased as GO loading increased. The v_{β} is significantly reduced by the addition of CNT and GO, while v_{β} decreases with CNT and GO loading increases. Since the calculation of V_{β} has considered the combined influence of fillers on X_{c} and F_{β}, both of which were reduced by incorporating CNT and GO, the reduction of v_{β} was expected. The v_{β} of the PVDF/CNT composites were higher than that of the PVDF/GO composites. Although it is generally anticipated that d_{33} increases with v_{β}, it is observed that in the presence of CNT, d_{33} is dominated by the increase in electric conductivity of the composites during and after the electrospinning process, giving rise to transport of charges, produced by β crystals within the fiber to the surface of the sample. In addition, the 1D CNTs may have promoted the orientation of β crystals in the d_{33} direction, therefore, enhancing the d_{33} of the composites despite the hindrance of the β-phase formation (i.e., the reduction of v_{β}). Adding CNTs can also improve piezoelectricity through interfacial polarization, which increases the dielectric constant of composite (mobile charges within CNTs facilitate composite polarization). CNT loadings higher than 0.01 wt.% are sufficient to outperform the neat PVDF, and d_{33} becomes 59.7% higher than the neat PVDF at 0.03 wt.% loading, but only GO loadings of 0.5 wt.% achieved comparable d_{33} to the neat PVDF; further increase in GO loading had resulted in a decline in d_{33}. The low conductivity of GO, the influence of flocculation, and the lower aspect ratio compared with CNT may result in lower electron transfer and less orientation of the β-phase polycrystalline. The d_{33} of the PVDF/CNT composites is higher than that of the PVDF/GO composites despite much higher loading of GO. This study aims to contribute to the development of PVDF nanocomposites in piezoelectric energy harvesting applications (e.g., self-powered biosensors and wireless sensor networks)