3 research outputs found
Knowledge-Guided Data-Centric AI in Healthcare: Progress, Shortcomings, and Future Directions
The success of deep learning is largely due to the availability of large
amounts of training data that cover a wide range of examples of a particular
concept or meaning. In the field of medicine, having a diverse set of training
data on a particular disease can lead to the development of a model that is
able to accurately predict the disease. However, despite the potential
benefits, there have not been significant advances in image-based diagnosis due
to a lack of high-quality annotated data. This article highlights the
importance of using a data-centric approach to improve the quality of data
representations, particularly in cases where the available data is limited. To
address this "small-data" issue, we discuss four methods for generating and
aggregating training data: data augmentation, transfer learning, federated
learning, and GANs (generative adversarial networks). We also propose the use
of knowledge-guided GANs to incorporate domain knowledge in the training data
generation process. With the recent progress in large pre-trained language
models, we believe it is possible to acquire high-quality knowledge that can be
used to improve the effectiveness of knowledge-guided generative methods.Comment: 21 pages, 13 figures, 4 table
EA-CG: An Approximate Second-Order Method for Training Fully-Connected Neural Networks
For training fully-connected neural networks (FCNNs), we propose a practical
approximate second-order method including: 1) an approximation of the Hessian
matrix and 2) a conjugate gradient (CG) based method. Our proposed approximate
Hessian matrix is memory-efficient and can be applied to any FCNNs where the
activation and criterion functions are twice differentiable. We devise a
CG-based method incorporating one-rank approximation to derive Newton
directions for training FCNNs, which significantly reduces both space and time
complexity. This CG-based method can be employed to solve any linear equation
where the coefficient matrix is Kronecker-factored, symmetric and positive
definite. Empirical studies show the efficacy and efficiency of our proposed
method.Comment: Change to AAAI-19 Versio