HARD: Hard Augmentations for Robust Distillation

Knowledge distillation (KD) is a simple and successful method to transfer knowledge from a teacher to a student model solely based on functional activity. However, current KD has a few shortcomings: it has recently been shown that this method is unsuitable to transfer simple inductive biases like shift equivariance, struggles to transfer out of domain generalization, and optimization time is magnitudes longer compared to default non-KD model training. To improve these aspects of KD, we propose Hard Augmentations for Robust Distillation (HARD), a generally applicable data augmentation framework, that generates synthetic data points for which the teacher and the student disagree. We show in a simple toy example that our augmentation framework solves the problem of transferring simple equivariances with KD. We then apply our framework in real-world tasks for a variety of augmentation models, ranging from simple spatial transformations to unconstrained image manipulations with a pretrained variational autoencoder. We find that our learned augmentations significantly improve KD performance on in-domain and out-of-domain evaluation. Moreover, our method outperforms even state-of-the-art data augmentations and since the augmented training inputs can be visualized, they offer a qualitative insight into the properties that are transferred from the teacher to the student. Thus HARD represents a generally applicable, dynamically optimized data augmentation technique tailored to improve the generalization and convergence speed of models trained with KD

Pedicle Screw-Associated Violation of the Adjacent Unfused Facet Joint: Clinical Outcomes and Fusion Rates

STUDY DESIGN: Retrospective review of a prospective randomized trial. OBJECTIVES: To compare outcome scores and fusion rates in patients with and without pedicle screw-associated facet joint violation (FJV) after a single-level lumbar fusion. METHODS: Clinical outcomes data and computed tomography (CT) imaging were reviewed for 157 patients participating in a multicenter prospective trial. Post-operative CT scans at 12-months follow-up were examined for fusion status and FJV. Patient-reported outcomes (PROs) included Oswestry Disability Index (ODI) and Visual Analog Scale (VAS) for leg and low back pain. Chi-square test of independence was used to compare proportions between groups on categorical measures. Two-sample t-test was used to identify differences in mean patient outcome scores. Logistic regression models were performed to determine association between FJV and fusion rates. RESULTS: Of the 157 patients included, there were 18 (11.5%) with FJV (Group A) and 139 (88.5%) without FJV (Group B). Patients with FJV experienced less improvement in ODI (P = .004) and VAS back pain scores (P = .04) vs patients without FJV. There was no difference in mean VAS leg pain (P = .4997). The rate of fusion at 12-months for patients with FJV (27.8%) was lower compared to those without FJV (71.2%) (P = .0002). Patients with FJV were 76% less likely to have a successful fusion at 12-months. CONCLUSION: Pedicle screw-associated violation of the adjacent unfused facet joint during single-level lumbar fusion is associated with less improvement in back pain, back pain-associated disability, and a lower fusion rate at 1-year after surgery

Comprehensive vascular imaging using optical coherence tomography-based angiography and photoacoustic tomography

Studies have proven the relationship between cutaneous vasculature abnormalities and dermatological disorders, but to image vasculature noninvasively in vivo, advanced optical imaging techniques are required. In this study, we imaged a palm of a healthy volunteer and three subjects with cutaneous abnormalities with photoacoustic tomography (PAT) and optical coherence tomography with angiography extension (OCTA). Capillaries in the papillary dermis that are too small to be discerned with PAT are visualized with OCTA. From our results, we speculate that the PA signal from the palm is mostly from hemoglobin in capillaries rather than melanin, knowing that melanin concentration in volar skin is significantly smaller than that in other areas of the skin. We present for the first time OCTA images of capillaries along with the PAT images of the deeper vessels, demonstrating the complementary effective imaging depth range and the visualization capabilities of PAT and OCTA for imaging human skin in vivo. The proposed imaging system in this study could significantly improve treatment monitoring of dermatological diseases associated with cutaneous vasculature abnormalities

Molecular Details of Retinal Guanylyl Cyclase 1/GCAP-2 Interaction

The rod outer segment guanylyl cyclase 1 (ROS-GC1) is an essential component of photo-transduction in the retina. In the light-induced signal cascade, membrane-bound ROS-GC1 restores cGMP levels in the dark in a calcium-dependent manner. With decreasing calcium concentration in the intracellular compartment, ROS-GC1 is activated via the intracellular site by guanylyl cyclase-activating proteins (GCAP-1/-2). Presently, the exact activation mechanism is elusive. To obtain structural insights into the ROS-GC1 regulation by GCAP-2, chemical cross-linking/mass spectrometry studies using GCAP-2 and three ROS-GC1 peptides were performed in the presence and absence of calcium. The majority of cross-links were identified with the C-terminal lobe of GCAP-2 and a peptide comprising parts of ROS-GC1's catalytic domain and C-terminal extension. Consistently with the cross-linking results, surface plasmon resonance and fluorescence measurements confirmed specific binding of this ROS-GC peptide to GCAP-2 with a dissociation constant in the low micromolar range. These results imply that a region of the catalytic domain of ROS-GC1 can participate in the interaction with GCAP-2. Additional binding surfaces upstream of the catalytic domain, in particular the juxtamembrane domain, can currently not be excluded

ICC-CLASS: isotopically-coded cleavable crosslinking analysis software suite

<p>Abstract</p> <p>Background</p> <p>Successful application of crosslinking combined with mass spectrometry for studying proteins and protein complexes requires specifically-designed crosslinking reagents, experimental techniques, and data analysis software. Using isotopically-coded ("heavy and light") versions of the crosslinker and cleavable crosslinking reagents is analytically advantageous for mass spectrometric applications and provides a "handle" that can be used to distinguish crosslinked peptides of different types, and to increase the confidence of the identification of the crosslinks.</p> <p>Results</p> <p>Here, we describe a program suite designed for the analysis of mass spectrometric data obtained with isotopically-coded <it>cleavable </it>crosslinkers. The suite contains three programs called: DX, DXDX, and DXMSMS. DX searches the mass spectra for the presence of ion signal doublets resulting from the light and heavy isotopic forms of the isotopically-coded crosslinking reagent used. DXDX searches for possible mass matches between cleaved and uncleaved isotopically-coded crosslinks based on the established chemistry of the cleavage reaction for a given crosslinking reagent. DXMSMS assigns the crosslinks to the known protein sequences, based on the isotopically-coded and un-coded MS/MS fragmentation data of uncleaved and cleaved peptide crosslinks.</p> <p>Conclusion</p> <p>The combination of these three programs, which are tailored to the analytical features of the specific isotopically-coded cleavable crosslinking reagents used, represents a powerful software tool for automated high-accuracy peptide crosslink identification. See: <url>http://www.creativemolecules.com/CM_Software.htm</url></p

Natural Image Coding in V1: How Much Use is Orientation Selectivity?

Orientation selectivity is the most striking feature of simple cell coding in V1 which has been shown to emerge from the reduction of higher-order correlations in natural images in a large variety of statistical image models. The most parsimonious one among these models is linear Independent Component Analysis (ICA), whereas second-order decorrelation transformations such as Principal Component Analysis (PCA) do not yield oriented filters. Because of this finding it has been suggested that the emergence of orientation selectivity may be explained by higher-order redundancy reduction. In order to assess the tenability of this hypothesis, it is an important empirical question how much more redundancies can be removed with ICA in comparison to PCA, or other second-order decorrelation methods. This question has not yet been settled, as over the last ten years contradicting results have been reported ranging from less than five to more than hundred percent extra gain for ICA. Here, we aim at resolving this conflict by presenting a very careful and comprehensive analysis using three evaluation criteria related to redundancy reduction: In addition to the multi-information and the average log-loss we compute, for the first time, complete rate-distortion curves for ICA in comparison with PCA. Without exception, we find that the advantage of the ICA filters is surprisingly small. Furthermore, we show that a simple spherically symmetric distribution with only two parameters can fit the data even better than the probabilistic model underlying ICA. Since spherically symmetric models are agnostic with respect to the specific filter shapes, we conlude that orientation selectivity is unlikely to play a critical role for redundancy reduction