TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Training

Collective communications are an indispensable part of distributed training. Running a topology-aware collective algorithm is crucial for optimizing communication performance by minimizing congestion. Today such algorithms only exist for a small set of simple topologies, limiting the topologies employed in training clusters and handling irregular topologies due to network failures. In this paper, we propose TACOS, an automated topology-aware collective synthesizer for arbitrary input network topologies. TACOS synthesized 3.73x faster All-Reduce algorithm over baselines, and synthesized collective algorithms for 512-NPU system in just 6.1 minutes

Identification of Variegated Coloring in Skin Tumors: Neural Network vs. Rule-Based Induction Methods

The use of neural networks for automatic identification of variegated coloring, which is believed to be one of the most predictive features for malignant melanoma, is described. The Nestor development system (NDS) was chosen for neural network implementation. At the heart of NDS is a three-layer neural network called a restricted Coulomb energy (RCE) network. The learning scheme and the database for detection of variegated coloring are discussed. Results are reporte