Peer review is a widely accepted mechanism for research evaluation, playing a
pivotal role in scholarly publishing. However, criticisms have long been
leveled on this mechanism, mostly because of its inefficiency and subjectivity.
Recent years have seen the application of artificial intelligence (AI) in
assisting the peer review process. Nonetheless, with the involvement of humans,
such limitations remain inevitable. In this review paper, we propose the
concept and pipeline of automated scholarly paper review (ASPR) and review the
relevant literature and technologies of achieving a full-scale computerized
review process. On the basis of the review and discussion, we conclude that
there is already corresponding research and implementation at each stage of
ASPR. We further look into the challenges in ASPR with the existing
technologies. The major difficulties lie in imperfect document parsing and
representation, inadequate data, defective human-computer interaction and
flawed deep logical reasoning. Moreover, we discuss the possible moral &
ethical issues and point out the future directions of ASPR. In the foreseeable
future, ASPR and peer review will coexist in a reinforcing manner before ASPR
is able to fully undertake the reviewing workload from humans