How to Host a Data Competition: Statistical Advice for Design and Analysis of a Data Competition

Data competitions rely on real-time leaderboards to rank competitor entries and stimulate algorithm improvement. While such competitions have become quite popular and prevalent, particularly in supervised learning formats, their implementations by the host are highly variable. Without careful planning, a supervised learning competition is vulnerable to overfitting, where the winning solutions are so closely tuned to the particular set of provided data that they cannot generalize to the underlying problem of interest to the host. This paper outlines some important considerations for strategically designing relevant and informative data sets to maximize the learning outcome from hosting a competition based on our experience. It also describes a post-competition analysis that enables robust and efficient assessment of the strengths and weaknesses of solutions from different competitors, as well as greater understanding of the regions of the input space that are well-solved. The post-competition analysis, which complements the leaderboard, uses exploratory data analysis and generalized linear models (GLMs). The GLMs not only expand the range of results we can explore, they also provide more detailed analysis of individual sub-questions including similarities and differences between algorithms across different types of scenarios, universally easy or hard regions of the input space, and different learning objectives. When coupled with a strategically planned data generation approach, the methods provide richer and more informative summaries to enhance the interpretation of results beyond just the rankings on the leaderboard. The methods are illustrated with a recently completed competition to evaluate algorithms capable of detecting, identifying, and locating radioactive materials in an urban environment.Comment: 36 page

Wave spectra of 2D dusty plasma solids and liquids

Brownian dynamics simulations were carried out to study wave spectra of two-dimensional dusty plasma liquids and solids for a wide range of wavelengths. The existence of a longitudinal dust thermal mode was confirmed in simulations, and a cutoff wavenumber in the transverse mode was measured. Dispersion relations, resulting from simulations, were compared with those from analytical theories, such as the random-phase approximation (RPA), quasi-localized charged approximation (QLCA), and harmonic approximation (HA). An overall good agreement between the QLCA and simulations was found for wide ranges of states and wavelengths after taking into account the direct thermal effect in the QLCA, while for the RPA and HA good agreement with simulations were found in the high and low temperature limits, respectively.Comment: 26 pages, 9 figure

CFHT Legacy Ultraviolet Extension (CLUE): witnessing galaxy transformations up to 7 Mpc from rich cluster cores

Using the optical data from the Wide component of the Canada-France-Hawaii Telescope (CFHT) Legacy Survey, and new ultraviolet (UV) data from GALEX, we study the colours and specific star formation rates (SSFRs) of ∼ 100 galaxy clusters at 0.16 < z < 0.36, over areas extending out to radii of r∼ 7 Mpc. We use a multicolour, statistical background subtraction method to study the galaxy population at this radius; thus our results pertain to those galaxies which constitute an excess over the average field density. We find that the average SSFR and its distribution of the star-forming galaxies (with at z∼ 0.2 and at z∼ 0.3) have no measurable dependence on the clustercentric radius and are consistent with the field values. However, the fraction of galaxies with SFR above these thresholds, and the fraction of optically blue galaxies, are lower for the overdense galaxy population in the cluster outskirts compared with the average field value, at all stellar masses and at all radii out to at least 7 Mpc. Most interestingly, the fraction of blue galaxies that are forming stars at a rate below our UV detection limit is much higher in all radial bins around our cluster sample compared with the general field value. This is most noticeable for massive galaxies ; while almost all blue field galaxies of this mass have detectable star formation, this is true for less than 20 per cent of the blue cluster galaxies, even at 7 Mpc from the cluster centre. Our results support a scenario where galaxies are pre-processed in locally overdense regions in a way that reduces their SFR below our UV detection limit, but not to zer

Can health economics aid decision making in healthcare innovation in academia?

Health economics is extensively used by reimbursement agencies to make decisions on whether to adopt new medical technologies. It is also used by the healthcare industry as an aid to decision making during product development. This paper proposes that academic healthcare technology transfer could also benefit from adopting health economics. The study investigated the knowledge and attitudes of academic staff involved in developing and transferring healthcare technologies. The participants had little or no prior knowledge of health economics; however, the majority reported that this method had the potential to aid development and commercialisation. The innovations developed by the workshop attendees and their commercialisation plans were of a type that could potentially benefit from cost-effectiveness calculations. Health economics has the potential to make valuable contribution to academic healthcare innovation. Research is required to develop this method further and ensure that it can be successfully applied in academia

Live Imaging of Type I Collagen Assembly Dynamics in Osteoblasts Stably Expressing GFP and mCherry-Tagged Collagen Constructs

Type I collagen is the most abundant extracellular matrix protein in bone and other connective tissues and plays key roles in normal and pathological bone formation as well as in connective tissue disorders and fibrosis. Although much is known about the collagen biosynthetic pathway and its regulatory steps, the mechanisms by which it is assembled extracellularly are less clear. We have generated GFPtpz and mCherry-tagged collagen fusion constructs for live imaging of type I collagen assembly by replacing the α2(I)-procollagen N-terminal propeptide with GFPtpz or mCherry. These novel imaging probes were stably transfected into MLO-A5 osteoblast-like cells and fibronectin-null mouse embryonic fibroblasts (FN-null-MEFs) and used for imaging type I collagen assembly dynamics and its dependence on fibronectin. Both fusion proteins co-precipitated with α1(I)-collagen and remained intracellular without ascorbate but were assembled into α1(I) collagen-containing extracellular fibrils in the presence of ascorbate. Immunogold-EM confirmed their ultrastuctural localization in banded collagen fibrils. Live cell imaging in stably transfected MLO-A5 cells revealed the highly dynamic nature of collagen assembly and showed that during assembly the fibril networks are continually stretched and contracted due to the underlying cell motion. We also observed that cell-generated forces can physically reshape the collagen fibrils. Using co-cultures of mCherry- and GFPtpz-collagen expressing cells, we show that multiple cells contribute collagen to form collagen fiber bundles. Immuno-EM further showed that individual collagen fibrils can receive contributions of collagen from more than one cell. Live cell imaging in FN-null-MEFs expressing GFPtpz-collagen showed that collagen assembly was both dependent upon and dynamically integrated with fibronectin assembly. These GFP-collagen fusion constructs provide a powerful tool for imaging collagen in living cells and have revealed novel and fundamental insights into the dynamic mechanisms for the extracellular assembly of collagen

Prediction and Simulator Verification of Roll/Lateral Adverse Aeroservoelastic Rotorcraft–Pilot Couplings

The involuntary interaction of a pilot with an aircraft can be described as pilot-assisted oscillations. Such phenomena are usually only addressed late in the design process when they manifest themselves during ground/flight testing. Methods to be able to predict such phenomena as early as possible are therefore useful. This work describes a technique to predict the adverse aeroservoelastic rotorcraft–pilot couplings, specifically between a rotorcraft’s roll motion and the resultant involuntary pilot lateral cyclic motion. By coupling linear vehicle aeroservoelastic models and experimentally identified pilot biodynamic models, pilot-assisted oscillations and no-pilot-assisted oscillation conditions have been numerically predicted for a soft-in-plane hingeless helicopter with a lightly damped regressive lead–lag mode that strongly interacts with the roll modeat a frequency within the biodynamic band of the pilots. These predictions have then been verified using real-time flight-simulation experiments. The absence of any similar adverse couplings experienced while using only rigid-body models in the flight simulator verified that the observed phenomena were indeed aeroelastic in nature. The excellent agreement between the numerical predictions and the observed experimental results indicates that the techniques developed in this paper can be used to highlight the proneness of new or existing designs to pilot-assisted oscillation