Data Pipeline Management in Practice: Challenges and Opportunities

Data pipelines involve a complex chain of interconnected activities that starts with a data source and ends in a data sink. Data pipelines are important for data-driven organizations since a data pipeline can process data in multiple formats from distributed data sources with minimal human intervention, accelerate data life cycle activities, and enhance productivity in data-driven enterprises. However, there are challenges and opportunities in implementing data pipelines but practical industry experiences are seldom reported. The findings of this study are derived by conducting a qualitative multiple-case study and interviews with the representatives of three companies. The challenges include data quality issues, infrastructure maintenance problems, and organizational barriers. On the other hand, data pipelines are implemented to enable traceability, fault-tolerance, and reduce human errors through maximizing automation thereby producing high-quality data. Based on multiple-case study research with five use cases from three case companies, this paper identifies the key challenges and benefits associated with the implementation and use of data pipelines

Relationships of low serum vitamin D3 with anthropometry and markers of the metabolic syndrome and diabetes in overweight and obesity

Low serum 25 hydroxyvitamin D3 (vitamin D3) is known to perturb cellular function in many tissues, including the endocrine pancreas, which are involved in obesity and type II diabetes mellitus (TIIDM). Vitamin D3 insufficiency has been linked to obesity, whether obesity is assessed by body mass index (BMI) or waist circumference (waist). Central obesity, using waist as the surrogate, is associated with the metabolic syndrome (MetSyn), insulin resistance, TIIDM and atherosclerotic cardiovascular disease (CVD). We tested how vitamin D3 was related to measures of fat mass, MetSyn markers, haemoglobin A1c (HbA1c) and MetSyn in a cross-sectional sample of 250 overweight and obese adults of different ethnicities. There were modest inverse associations of vitamin D3 with body weight (weight) (r = -0.21, p = 0.0009), BMI (r = -0.18, p = 0.005), waist (r = -0.14, p = 0.03), [but not body fat % (r = -0.08, p = 0.24)], and HbA1c (r = -0.16, p = 0.01). Multivariable regression carried out separately for BMI and waist showed a decrease of 0.74 nmol/L (p = 0.002) in vitamin D3 per 1 kg/m2 increase in BMI and a decrease of 0.29 nmol/L (p = 0.01) per 1 cm increase in waist, with each explaining approximately 3% of the variation in vitamin D3 over and above gender, age, ethnicity and season

Phenology of Scramble Polygyny in a Wild Population of Chrysolemid Beetles: The Opportunity for and the Strength of Sexual Selection

Recent debate has highlighted the importance of estimating both the strength of sexual selection on phenotypic traits, and the opportunity for sexual selection. We describe seasonal fluctuations in mating dynamics of Leptinotarsa undecimlineata (Coleoptera: Chrysomelidae). We compared several estimates of the opportunity for, and the strength of, sexual selection and male precopulatory competition over the reproductive season. First, using a null model, we suggest that the ratio between observed values of the opportunity for sexual selections and their expected value under random mating results in unbiased estimates of the actual nonrandom mating behavior of the population. Second, we found that estimates for the whole reproductive season often misrepresent the actual value at any given time period. Third, mating differentials on male size and mobility, frequency of male fighting and three estimates of the opportunity for sexual selection provide contrasting but complementary information. More intense sexual selection associated to male mobility, but not to male size, was observed in periods with high opportunity for sexual selection and high frequency of male fights. Fourth, based on parameters of spatial and temporal aggregation of female receptivity, we describe the mating system of L. undecimlineata as a scramble mating polygyny in which the opportunity for sexual selection varies widely throughout the season, but the strength of sexual selection on male size remains fairly weak, while male mobility inversely covaries with mating success. We suggest that different estimates for the opportunity for, and intensity of, sexual selection should be applied in order to discriminate how different behavioral and demographic factors shape the reproductive dynamic of populations

Genetic polymorphisms of RANTES, IL1-A, MCP-1 and TNF-A genes in patients with prostate cancer

<p>Abstract</p> <p>Background</p> <p>Inflammation has been implicated as an etiological factor in several human cancers, including prostate cancer. Allelic variants of the genes involved in inflammatory pathways are logical candidates as genetic determinants of prostate cancer risk. The purpose of this study was to investigate whether single nucleotide polymorphisms of genes that lead to increased levels of pro-inflammatory cytokines and chemokines are associated with an increased prostate cancer risk.</p> <p>Methods</p> <p>A case-control study design was used to test the association between prostate cancer risk and the polymorphisms <it>TNF-A</it>-308 A/G (rs 1800629), <it>RANTES</it>-403 G/A (rs 2107538), <it>IL1-A</it>-889 C/T (rs 1800587) and <it>MCP-1 </it>2518 G/A (rs 1024611) in 296 patients diagnosed with prostate cancer and in 311 healthy controls from the same area.</p> <p>Results</p> <p>Diagnosis of prostate cancer was significantly associated with <it>TNF-A </it>GA + AA genotype (OR, 1.61; 95% CI, 1.09–2.64) and <it>RANTES </it>GA + AA genotype (OR, 1.44; 95% CI, 1.09–2.38). A alleles in <it>TNF-A </it>and <it>RANTES </it>influenced prostate cancer susceptibility and acted independently of each other in these subjects. No epistatic effect was found for the combination of different polymorphisms studied. Finally, no overall association was found between prostate cancer risk and <it>IL1-A </it>or <it>MCP-1 </it>polymorphisms.</p> <p>Conclusion</p> <p>Our results and previously published findings on genes associated with innate immunity support the hypothesis that polymorphisms in proinflammatory genes may be important in prostate cancer development.</p

DeepZipper: A Novel Deep-learning Architecture for Lensed Supernovae Identification

Large-scale astronomical surveys have the potential to capture data on large numbers of strongly gravitationally lensed supernovae (LSNe). To facilitate timely analysis and spectroscopic follow-up before the supernova fades, an LSN needs to be identified soon after it begins. To quickly identify LSNe in optical survey data sets, we designed ZipperNet, a multibranch deep neural network that combines convolutional layers (traditionally used for images) with long short-term memory layers (traditionally used for time series). We tested ZipperNet on the task of classifying objects from four categories—no lens, galaxy-galaxy lens, lensed Type-Ia supernova, lensed core-collapse supernova—within high-fidelity simulations of three cosmic survey data sets: the Dark Energy Survey, Rubin Observatory’s Legacy Survey of Space and Time (LSST), and a Dark Energy Spectroscopic Instrument (DESI) imaging survey. Among our results, we find that for the LSST-like data set, ZipperNet classifies LSNe with a receiver operating characteristic area under the curve of 0.97, predicts the spectroscopic type of the lensed supernovae with 79% accuracy, and demonstrates similarly high performance for LSNe 1–2 epochs after first detection. We anticipate that a model like ZipperNet, which simultaneously incorporates spatial and temporal information, can play a significant role in the rapid identification of lensed transient systems in cosmic survey experiments

Objectively measured physical activity in European adults: cross-sectional findings from the Food4Me study

Background Comparisons of objectively measured physical activity (PA) between residents of European countries measured concurrently with the same protocol are lacking. We aimed to compare PA between the seven European countries involved in the Food4Me Study, using accelerometer data collected remotely via the Internet. Methods Of the 1607 participants recruited, 1287 (539 men and 748 women) provided at least 3 weekdays and 2 weekend days of valid accelerometer data (TracmorD) at baseline and were included in the present analyses. Results Men were significantly more active than women (physical activity level = 1.74 vs. 1.70, p < 0.001). Time spent in light PA and moderate PA differed significantly between countries but only for women. Adherence to the World Health Organization recommendation to accumulate at least 150 min of moderate-equivalent PA weekly was similar between countries for men (range: 54–65%) but differed significantly between countries for women (range: 26–49%). Prevalence estimates decreased substantially for men and women in all seven countries when PA guidelines were defined as achieving 30 min of moderate and vigorous PA per day. Conclusions We were able to obtain valid accelerometer data in real time via the Internet from 80% of participants. Although our estimates are higher compared with data from Sweden, Norway, Portugal and the US, there is room for improvement in PA for all countries involved in the Food4Me Study

Candidate periodically variable quasars from the Dark Energy Survey and the Sloan Digital Sky Survey

Periodically variable quasars have been suggested as close binary supermassive black holes. We present a systematic search for periodic light curves in 625 spectroscopically confirmed quasars with a median redshift of 1.8 in a 4.6 deg2 overlapping region of the Dark Energy Survey Supernova (DES-SN) fields and the Sloan Digital Sky Survey Stripe 82 (SDSS-S82). Our sample has a unique 20-yr long multicolour (griz) light curve enabled by combining DES-SN Y6 observations with archival SDSS-S82 data. The deep imaging allows us to search for periodic light curves in less luminous quasars (down to r ∼23.5 mag) powered by less massive black holes (with masses ≳ 108.5M⊙) at high redshift for the first time. We find five candidates with significant (at >99.74 per cent single-frequency significance in at least two bands with a global p-value of ∼7 × 10−4–3 × 10−3 accounting for the look-elsewhere effect) periodicity with observed periods of ∼3–5 yr (i.e. 1–2 yr in rest frame) having ∼4–6 cycles spanned by the observations. If all five candidates are periodically variable quasars, this translates into a detection rate of ∼0.8+0.5−0.3 per cent or ∼1.1+0.7−0.5 quasar per deg2. Our detection rate is 4–80 times larger than those found by previous searches using shallower surveys over larger areas. This discrepancy is likely caused by differences in the quasar populations probed and the survey data qualities. We discuss implications on the future direct detection of low-frequency gravitational waves. Continued photometric monitoring will further assess the robustness and characteristics of these candidate periodic quasars to determine their physical origins