How to Host a Data Competition: Statistical Advice for Design and Analysis of a Data Competition

Data competitions rely on real-time leaderboards to rank competitor entries and stimulate algorithm improvement. While such competitions have become quite popular and prevalent, particularly in supervised learning formats, their implementations by the host are highly variable. Without careful planning, a supervised learning competition is vulnerable to overfitting, where the winning solutions are so closely tuned to the particular set of provided data that they cannot generalize to the underlying problem of interest to the host. This paper outlines some important considerations for strategically designing relevant and informative data sets to maximize the learning outcome from hosting a competition based on our experience. It also describes a post-competition analysis that enables robust and efficient assessment of the strengths and weaknesses of solutions from different competitors, as well as greater understanding of the regions of the input space that are well-solved. The post-competition analysis, which complements the leaderboard, uses exploratory data analysis and generalized linear models (GLMs). The GLMs not only expand the range of results we can explore, they also provide more detailed analysis of individual sub-questions including similarities and differences between algorithms across different types of scenarios, universally easy or hard regions of the input space, and different learning objectives. When coupled with a strategically planned data generation approach, the methods provide richer and more informative summaries to enhance the interpretation of results beyond just the rankings on the leaderboard. The methods are illustrated with a recently completed competition to evaluate algorithms capable of detecting, identifying, and locating radioactive materials in an urban environment.Comment: 36 page

Double-Axis Rotary Shadowing for High-Resolution Scanning Electron Microscopy

Thin continuous metal coatings and a scanning electron microscope-generated spot size in the range of the visualized particles, are necessary prerequisites if one hopes to extract high-resolution topographic information in the scanning electron microscope. Chemical fixation and dehydration in organic solvents at room temperature lead to severe ultrastructural artifacts which can be avoided by cryofixation and freeze-drying of the specimen. 0.9 to 2.7 nm thick homogeneous layers of chromium and germanium can be deposited onto the surface of cryofixed and freeze-dried specimens at high sub-zero temperatures by electron beam evaporation using double-axis rotary shadowing . Theoretical calculations of the layer geometry of a double-axis rotary shadowed hemisphere and practical experiments on periodical test specimens demonstrate the usefulness of this technique. The resolution obtainable in an in-lens field emission scanning electron microscope is close to transmission electron microscope studies and image reconstructions of the same specimens. Double-axis rotary metal shadowed immunolabelled specimens allow the detection of small colloidal gold markers in the backscattered electron-image. High topographic resolution is obtained in the secondary electron-image

Helium irradiation effects in polycrystalline Si, silica, and single crystal Si

Transmission electron microscopy (TEM) has been used to investigate the effects of room temperature 6 keV helium ion irradiation of a thin (≈55 nm thick) tri-layer consisting of polycrystalline Si, silica, and single-crystal Si. The ion irradiation was carried out in situ within the TEM under conditions where approximately 24% of the incident ions came to rest in the specimen. This paper reports on the comparative development of irradiation-induced defects (primarily helium bubbles) in the polycrystalline Si and single-crystal Si under ion irradiation and provides direct measurement of a radiation-induced increase in the width of the polycrystalline layer and shrinkage of the silica layer. Analysis using TEM and electron energy-loss spectroscopy has led to the hypothesis that these result from helium-bubble-induced swelling of the silicon and radiation-induced viscoelastic flow processes in the silica under the influence of stresses applied by the swollen Si layers. The silicon and silica layers are sputtered as a result of the helium ion irradiation; however, this is estimated to be a relatively minor effect with swelling and stress-related viscoelastic flow being the dominant mechanisms of dimensional change

Workshop - Feminist Engineering Education: Building a Community of Practice

As a result of a series of papers and special sessions held at FIE between 2004 and 2008, a community of CSET educators interested in exploring feminist pedagogies has formed at FIE. Past participants in these sessions have expressed a desire to learn more about what makes a set of pedagogies feminist. At the same time, there has been an increase in the number of research papers at FIE that draw on feminist research methods or topics. This workshop is designed to meet the needs of members of the FIE community who have some experience with feminist pedagogies or research methods, and who are looking to develop intellectual relationships with others also working in engineering educational research. We will discuss the history of feminist education and feminist research methods in the US, including a connection to science and engineering education; participants will then work in small groups focusing on a sub-topic (feminist pedagogies, feminist research methods, and feminist research topics/content). We invite participants to bring part of a project (such as a syllabus or course plan, assignment, class or research project, research question or protocol) to this workshop; some time will be dedicated to the guided design/redesign of this project

Low temperature expansion for the 3-d Ising Model

We compute the weak coupling expansion for the energy of the three dimensional Ising model through 48 excited bonds. We also compute the magnetization through 40 excited bonds. This was achieved via a recursive enumeration of states of fixed energy on a set of finite lattices. We use a linear combination of lattices with a generalization of helical boundary conditions to eliminate finite volume effects.Comment: 10 pages, IASSNS-HEP-92/42, BNL-4767

Finite-Size Effects in the Interface of 3D Ising Model

The interface between domains of opposite magnetization in the 3D Ising model near the critical temperature displays universal finite-size effects which can be described in terms of a gaussian model of capillary waves. It turns out that these finite-size corrections depend rather strongly on the shape of the lattice. This prediction, which has no adjustable parameters, is tested and accurately verified for various lattice shapes by means of numerical simulations with a cluster algorithm. This supports also a long-standing conjecture on the finite-size effects in Wilson loops of Lattice Gauge Theories.Comment: 13 pages, plain latex, two figures not included, sorry DFTT 68/9

Observations of Colloidal Gold Labelled Platelet Microtubules: High Voltage Electron Microscopy and Low Voltage-High Resolution Scanning Electron Microscopy

18 nm colloidal gold-antitubulin and 4 nm colloidal gold-antitubulin were used to label microtubules in adherent, fully spread platelets. Both sizes of marker effectively labelled microtubules in the partially extracted platelets. However only the 4 nm gold penetrated the dense microfilament matrix of the inner filamentous zone so that portions of microtubules within this cytoskeletal zone could be tracked. The gold marker could be visualized well with 1 MeV high voltage transmission EM and with 5 kV or greater secondary imaging or 20 kV backscattered imaging of carbon only coated samples. 1 kV secondary imaging permitted high resolution imaging of the surface of tubules and the microfilaments with their respective associated material. Individual gold-antibody complexes were difficult to identify by shape alone due to the tendency of the antibody coats to blend together when in very close approximation and due to the presence of other molecules or molecular aggregates similar in size to the gold-antibody labels. Microtubules were seen to wind in and out of the inner and outer filamentous zones as they encircled the granulomere. Some tubules were seen to dead end at the peripheral web. Numerous smaller microtubule loops were present principally in the outer filamentous zone and tubules could be followed as they went from the outer filamentous zone through the inner filamentous zone and into the granulomere

Towards the development of a hybrid-integrated chip interferometer for online surface profile measurements

Non-destructive testing and online measurement of surface features are pressing demands in manufacturing. Thus optical techniques are gaining importance for characterization of complex engineering surfaces. Harnessing integrated optics for miniaturization of interferometry systems onto a silicon wafer and incorporating a compact optical probe would enable the development of a handheld sensor for embedded metrology applications. In this work, we present the progress in the development of a hybrid photonics based metrology sensor device for online surface profile measurements. The measurement principle along with test and measurement results of individual components has been presented. For non-contact measurement, a spectrally encoded lateral scanning probe based on the laser scanning microscopy has been developed to provide fast measurement with lateral resolution limited to the diffraction limit. The probe demonstrates a lateral resolution of ∼3.6 μm while high axial resolution (sub-nanometre) is inherently achieved by interferometry. Further the performance of the hybrid tuneable laser and the scanning probe was evaluated by measuring a standard step height sample of 100 nm

Overlap Distribution of the Three-Dimensional Ising Model

We study the Parisi overlap probability density P_L(q) for the three-dimensional Ising ferromagnet by means of Monte Carlo (MC) simulations. At the critical point P_L(q) is peaked around q=0 in contrast with the double peaked magnetic probability density. We give particular attention to the tails of the overlap distribution at the critical point, which we control over up to 500 orders of magnitude by using the multi-overlap MC algorithm. Below the critical temperature interface tension estimates from the overlap probability density are given and their approach to the infinite volume limit appears to be smoother than for estimates from the magnetization.Comment: 7 pages, RevTex, 9 Postscript figure