QSGD: communication-efficient SGD via gradient quantization and encoding

Parallel implementations of stochastic gradient descent (SGD) have received significant research attention, thanks to its excellent scalability properties. A fundamental barrier when parallelizing SGD is the high bandwidth cost of communicating gradient updates between nodes; consequently, several lossy compresion heuristics have been proposed, by which nodes only communicate quantized gradients. Although effective in practice, these heuristics do not always converge. In this paper, we propose Quantized SGD (QSGD), a family of compression schemes with convergence guarantees and good practical performance. QSGD allows the user to smoothly trade off communication bandwidth and convergence time: nodes can adjust the number of bits sent per iteration, at the cost of possibly higher variance. We show that this trade-off is inherent, in the sense that improving it past some threshold would violate information-theoretic lower bounds. QSGD guarantees convergence for convex and non-convex objectives, under asynchrony, and can be extended to stochastic variance-reduced techniques. When applied to training deep neural networks for image classification and automated speech recognition, QSGD leads to significant reductions in end-to-end training time. For instance, on 16GPUs, we can train the ResNet-152 network to full accuracy on ImageNet 1.8× faster than the full-precision variant

In situ Biological Dose Mapping Estimates the Radiation Burden Delivered to ‘Spared’ Tissue between Synchrotron X-Ray Microbeam Radiotherapy Tracks

Microbeam radiation therapy (MRT) using high doses of synchrotron X-rays can destroy tumours in animal models whilst causing little damage to normal tissues. Determining the spatial distribution of radiation doses delivered during MRT at a microscopic scale is a major challenge. Film and semiconductor dosimetry as well as Monte Carlo methods struggle to provide accurate estimates of dose profiles and peak-to-valley dose ratios at the position of the targeted and traversed tissues whose biological responses determine treatment outcome. The purpose of this study was to utilise γ-H2AX immunostaining as a biodosimetric tool that enables in situ biological dose mapping within an irradiated tissue to provide direct biological evidence for the scale of the radiation burden to ‘spared’ tissue regions between MRT tracks. Γ-H2AX analysis allowed microbeams to be traced and DNA damage foci to be quantified in valleys between beams following MRT treatment of fibroblast cultures and murine skin where foci yields per unit dose were approximately five-fold lower than in fibroblast cultures. Foci levels in cells located in valleys were compared with calibration curves using known broadbeam synchrotron X-ray doses to generate spatial dose profiles and calculate peak-to-valley dose ratios of 30–40 for cell cultures and approximately 60 for murine skin, consistent with the range obtained with conventional dosimetry methods. This biological dose mapping approach could find several applications both in optimising MRT or other radiotherapeutic treatments and in estimating localised doses following accidental radiation exposure using skin punch biopsies

Detecting intratumoral heterogeneity of EGFR activity by liposome-based in vivo transfection of a fluorescent biosensor

Despite decades of research in the epidermal growth factor receptor (EGFR) signalling field, and many targeted anti-cancer drugs that have been tested clinically, the success rate for these agents in the clinic is low, particularly in terms of the improvement of overall survival. Intratumoral heterogeneity is proposed as a major mechanism underlying treatment failure of these molecule-targeted agents. Here we highlight the application of fluorescence lifetime microscopy (FLIM)-based biosensing to demonstrate intratumoral heterogeneity of EGFR activity. For sensing EGFR activity in cells, we used a genetically encoded CrkII-based biosensor which undergoes conformational changes upon tyrosine-221 phosphorylation by EGFR. We transfected this biosensor into EGFR-positive tumour cells using targeted lipopolyplexes bearing EGFR-binding peptides at their surfaces. In a murine model of basal-like breast cancer, we demonstrated a significant degree of intratumoral heterogeneity in EGFR activity, as well as the pharmacodynamic effect of a radionuclide-labeled EGFR inhibitor in situ. Furthermore, a significant correlation between high EGFR activity in tumour cells and macrophage-tumour cell proximity was found to in part account for the intratumoral heterogeneity in EGFR activity observed. The same effect of macrophage infiltrate on EGFR activation was also seen in a colorectal cancer xenograft. In contrast, a non-small cell lung cancer xenograft expressing a constitutively active EGFR conformational mutant exhibited macrophage proximity-independent EGFR activity. Our study validates the use of this methodology to monitor therapeutic response in terms of EGFR activity. In addition, we found iNOS gene induction in macrophages that are cultured in tumour cell-conditioned media as well as an iNOS activity-dependent increase in EGFR activity in tumour cells. These findings point towards an immune microenvironment-mediated regulation that gives rise to the observed intratumoral heterogeneity of EGFR signalling activity in tumour cells in vivo

The potential of optical proteomic technologies to individualize prognosis and guide rational treatment for cancer patients

Genomics and proteomics will improve outcome prediction in cancer and have great potential to help in the discovery of unknown mechanisms of metastasis, ripe for therapeutic exploitation. Current methods of prognosis estimation rely on clinical data, anatomical staging and histopathological features. It is hoped that translational genomic and proteomic research will discriminate more accurately than is possible at present between patients with a good prognosis and those who carry a high risk of recurrence. Rational treatments, targeted to the specific molecular pathways of an individual’s high-risk tumor, are at the core of tailored therapy. The aim of targeted oncology is to select the right patient for the right drug at precisely the right point in their cancer journey. Optical proteomics uses advanced optical imaging technologies to quantify the activity states of and associations between signaling proteins by measuring energy transfer between fluorophores attached to specific proteins. Förster resonance energy transfer (FRET) and fluorescence lifetime imaging microscopy (FLIM) assays are suitable for use in cell line models of cancer, fresh human tissues and formalin-fixed paraffin-embedded tissue (FFPE). In animal models, dynamic deep tissue FLIM/FRET imaging of cancer cells in vivo is now also feasible. Analysis of protein expression and post-translational modifications such as phosphorylation and ubiquitination can be performed in cell lines and are remarkably efficiently in cancer tissue samples using tissue microarrays (TMAs). FRET assays can be performed to quantify protein-protein interactions within FFPE tissue, far beyond the spatial resolution conventionally associated with light or confocal laser microscopy. Multivariate optical parameters can be correlated with disease relapse for individual patients. FRET-FLIM assays allow rapid screening of target modifiers using high content drug screens. Specific protein-protein interactions conferring a poor prognosis identified by high content tissue screening will be perturbed with targeted therapeutics. Future targeted drugs will be identified using high content/throughput drug screens that are based on multivariate proteomic assays. Response to therapy at a molecular level can be monitored using these assays while the patient receives treatment: utilizing re-biopsy tumor tissue samples in the neoadjuvant setting or by examining surrogate tissues. These technologies will prove to be both prognostic of risk for individuals when applied to tumor tissue at first diagnosis and predictive of response to specifically selected targeted anticancer drugs. Advanced optical assays have great potential to be translated into real-life benefit for cancer patients

Communication complexity of approximate matching in distributed graphs

In this paper we consider the communication complexity of approximation algorithms for maximum matching in a graph in the message-passing model of distributed computation. The input graph consists of n vertices and edges partitioned over a set of k sites. The output is an alpha-approximate maximum matching in the input graph which has to be reported by one of the sites. We show a lower bound on the communication complexity of Omega(alpha^2 k n) and show that it is tight up to poly-logarithmic factors. This lower bound also applies to other combinatorial problems on graphs in the message-passing computation model, including max-flow and graph sparsification

A novel optical bioimaging method for direct assessment of ovarian cancer chemotherapy response at laparoscopy

In patients with advanced ovarian cancer (AOC), additional imaging of disseminated disease at laparoscopy could complement conventional imaging for estimation of chemotherapy response. We developed an image segmentation method and evaluated its use in making accurate and objective measurements of peritoneal metastases in comparison to Response Evaluation Criteria In Solid Tumors (RECIST) criteria. A software tool using a custom ImageJ macro-based approach was employed to estimate lesion size by converting image pixels into unit length. The software tool was tested as a proof-of-principle in an AOC patient with two isolated peritoneal deposits. Image analysis of representative laparoscopic snapshots before and after three cycles of neoadjuvant chemotherapy (NACT) revealed an average tumor nodule response ratio (TNRR) of 40% (partial response), which was in concordance with RECIST evaluation by computed tomography (CT). We demonstrated the feasibility of using this novel anatomical analysis for direct assessment of chemotherapy response in an AOC patient as an adjunct to RECIST criteria

A novel optical bioimaging method for direct assessment of ovarian cancer chemotherapy response at laparoscopy

In patients with advanced ovarian cancer (AOC), additional imaging of disseminated disease at laparoscopy could complement conventional imaging for estimation of chemotherapy response. We developed an image segmentation method and evaluated its use in making accurate and objective measurements of peritoneal metastases in comparison to Response Evaluation Criteria In Solid Tumors (RECIST) criteria. A software tool using a custom ImageJ macro-based approach was employed to estimate lesion size by converting image pixels into unit length. The software tool was tested as a proof-of-principle in an AOC patient with two isolated peritoneal deposits. Image analysis of representative laparoscopic snapshots before and after three cycles of neoadjuvant chemotherapy (NACT) revealed an average tumor nodule response ratio (TNRR) of 40% (partial response), which was in concordance with RECIST evaluation by computed tomography (CT). We demonstrated the feasibility of using this novel anatomical analysis for direct assessment of chemotherapy response in an AOC patient as an adjunct to RECIST criteria