Crossover-SGD: A gossip-based communication in distributed deep learning for alleviating large mini-batch problem and enhancing scalability

Distributed deep learning is an effective way to reduce the training time of deep learning for large datasets as well as complex models. However, the limited scalability caused by network overheads makes it difficult to synchronize the parameters of all workers. To resolve this problem, gossip-based methods that demonstrates stable scalability regardless of the number of workers have been proposed. However, to use gossip-based methods in general cases, the validation accuracy for a large mini-batch needs to be verified. To verify this, we first empirically study the characteristics of gossip methods in a large mini-batch problem and observe that the gossip methods preserve higher validation accuracy than AllReduce-SGD(Stochastic Gradient Descent) when the number of batch sizes is increased and the number of workers is fixed. However, the delayed parameter propagation of the gossip-based models decreases validation accuracy in large node scales. To cope with this problem, we propose Crossover-SGD that alleviates the delay propagation of weight parameters via segment-wise communication and load balancing random network topology. We also adapt hierarchical communication to limit the number of workers in gossip-based communication methods. To validate the effectiveness of our proposed method, we conduct empirical experiments and observe that our Crossover-SGD shows higher node scalability than SGP(Stochastic Gradient Push).Comment: Under review as a journal paper at CCP

Possible link between Arctic Sea ice and January PM10 concentrations in South Korea

In this study, we investigated the possible teleconnection between PM10 concentrations in South Korea and Arctic Sea ice concentrations at inter-annual time scales using observed PM10 data from South Korea, NCEP R2 data, and NOAA Sea Ice Concentration (SIC) data from 2001 to 2018. From the empirical orthogonal function (EOF) analysis, we found that the first mode (TC1) was a large-scale mode for PM10 in South Korea and explained about 27.4% of the total variability. Interestingly, the TC1 is more dominantly influenced by the horizontal ventilation effect than the vertical atmospheric stability effect. The pollution potential index (PPI), which is defined by the weighted average of the two ventilation effects, is highly correlated with the TC1 of PM10 at a correlation coefficient of 0.75, indicating that the PPI is a good measure for PM10 in South Korea at inter-annual time scales. Regression maps show that the decrease of SIC over the Barents Sea is significantly correlated with weakening of high pressure over the Ural mountain range region, the anomalous high pressure at 500 hPa over the Korean peninsula, and the weakening of the Siberian High and Aleutian low. Moreover, these patterns are similar to the correlation pattern with the PPI, suggesting that the variability of SIC over the Barents Sea may play an important role in modulating the variability of PM10 in South Korea through teleconnection from the Barents Sea to the Korean peninsula via Eurasia

A superconducting tensor detector for mid-frequency gravitational waves: its multi-channel nature and main astrophysical targets

Mid-frequency band gravitational-wave detectors will be complementary for the existing Earth-based detectors (sensitive above 10 Hz or so) and the future space-based detectors such as LISA, which will be sensitive below around 10 mHz. A ground-based superconducting omnidirectional gravitational radiation observatory (SOGRO) has recently been proposed along with several design variations for the frequency band of 0.1 to 10 Hz. For three conceptual designs of SOGRO (e.g., pSOGRO, SOGRO and aSOGRO), we examine their multi-channel natures, sensitivities and science cases. One of the key characteristics of the SOGRO concept is its six detection channels. The response functions of each channel are calculated for all possible gravitational wave polarizations including scalar and vector modes. Combining these response functions, we also confirm the omnidirectional nature of SOGRO. Hence, even a single SOGRO detector will be able to determine the position of a source and polarizations of gravitational waves, if detected. Taking into account SOGRO's sensitivity and technical requirements, two main targets are most plausible: gravitational waves from compact binaries and stochastic backgrounds. Based on assumptions we consider in this work, detection rates for intermediate-mass binary black holes (in the mass range of hundreds up to $10^{4}$ $M_\odot$) are expected to be $0.0014-2.5 \,\, {\rm yr}^{-1}$. In order to detect stochastic gravitational wave background, multiple detectors are required. Two aSOGRO detector networks may be able to put limits on the stochastic background beyond the indirect limit from cosmological observations.Comment: 35 pages, 8 figures, 4 table

Nondestructive Inspection of Reinforced Concrete Utility Poles with ISOMAP and Random Forest

Reinforced concrete poles are very popular in transmission lines due to their economic efficiency. However, these poles have structural safety issues in their service terms that are caused by cracks, corrosion, deterioration, and short-circuiting of internal reinforcing steel wires. Therefore, they must be periodically inspected to evaluate their structural safety. There are many methods of performing external inspection after installation at an actual site. However, on-site nondestructive safety inspection of steel reinforcement wires inside poles is very difficult. In this study, we developed an application that classifies the magnetic field signals of multiple channels, as measured from the actual poles. Initially, the signal data were gathered by inserting sensors into the poles, and these data were then used to learn the patterns of safe and damaged features. These features were then processed with the isometric feature mapping (ISOMAP) dimensionality reduction algorithm. Subsequently, the resulting reduced data were processed with a random forest classification algorithm. The proposed method could elucidate whether the internal wires of the poles were broken or not according to actual sensor data. This method can be applied for evaluating the structural integrity of concrete poles in combination with portable devices for signal measurement (under development)

Impact of Meteorological Changes on Particulate Matter and Aerosol Optical Depth in Seoul during the Months of June over Recent Decades

The effects of meteorological changes on particulate matter with a diameter of 10 microns or less (PM10, referred to as PM in this study) and aerosol optical depth (AOD) in Seoul were investigated using observational and modeling analysis. AOD satellite data were used, obtained from the Moderate Resolution Imaging Spectroradiometer (MODIS), and PM concentration data were used from in-situ observations. The Modern-Era Retrospective Analysis for Research and Applications (MERRA) and MERRA Version 2 (MERRA-2) were used for meteorological field analysis in modeling and observation data. The results from this investigation show that meteorological effects on PM and AOD were strong in the month of June, revealing a clear decreasing trend in recent decades. The investigation focused on the underlying mechanisms influencing the reduction in PM resulting from meteorological changes during the months of June. The results of this study reveal that decreases in atmospheric stability and humidity induced the aerosol change observed in recent decades. The changes in atmospheric stability and humidity are highly correlated with changes in the intensity of the East Asian summer monsoon (EASM). This suggests that the unstable and drying atmosphere by weakening of the EASM in recent decades has improved PM air quality in Seoul during the summer. The effects of atmospheric stability and humidity were also observed to vary depending on the aerosol species. Humidity only affects hydrophilic aerosols such as sulfate, nitrate, and ammonium, whereas atmospheric stability affects all species of aerosols, including carbonaceous aerosols

Gold nanorods-conjugated TiO2 nanoclusters for the synergistic combination of phototherapeutic treatments of cancer cells

Abstract Background Recently, a combination of photodynamic therapy (PDT) and photothermal therapy (PTT) to generate reactive oxygen species (ROS) and heat to kill cancer cells, respectively has attracted considerable attention because it gives synergistic effects on the cancer treatment by utilizing the radiation of nontoxic low-energy photons such as long wavelength visible light and near IR (NIR) penetrating into subcutaneous region. For the effective combination of the phototherapies, various organic photosensitizer-conjugated gold nanocomplexes have been developed, but they have still some disadvantages due to photobleaching and unnecessary energy transfer of the organic photosensitizers. Results In this study, we fabricated novel inorganic phototherapeutic nanocomplexes (Au NR–TiO2 NCs) by conjugating gold nanorods (Au NRs) with defective TiO2 nanoparticle clusters (d-TiO2 NP clusters) and characterized their optical and photothermal properties. They were observed to absorb a broad range of visible light and near IR (NIR) from 500 to 1000 nm, exhibiting the generation of ROS as well as the photothermal effect for the simultaneous application of PDT and PTT. The resultant combination of PDT and PTT treatments of HeLa cells incubated with the nanocomplexes caused a synergistic increase in the cell death compared to the single treatment. Conclusion The higher efficacy of cell death by the combination of PDT and PTT treatments with the nanocomplexes is likely attributed to the increases of ROS generation from the TiO2 NCs with the aid of local surface plasma resonance (LSPR)-induced hot electrons and heat generation from Au NRs, suggesting that Au NR–TiO2 NCs are promising nanomaterials for the in vivo combinatorial phototherapy of cancer