Nowadays, recommendation systems play an indispensable role in

many fields, including e-commerce, finance, economy, and gaming.

There is emerging research on publication venue recommendation

systems to support researchers when submitting their scientific

work. Several publishers such as IEEE, Springer, and Elsevier have

implemented their submission recommendation systems only to

help researchers choose appropriate conferences or journals for submission. In this work, we present a demo framework to construct an

effective recommendation system for paper submission. With the

input data (the title, the abstract, and the list of possible keywords)

of a given manuscript, the system recommends the list of top relevant journals or conferences to authors. By using state-of-the-art

techniques in natural language understanding, we combine the features extracted with other useful handcrafted features. We utilize

deep learning models to build an efficient recommendation engine

for the proposed system. Finally, we present the User Interface

(UI) and the architecture of our paper submission recommendation

system for later usage by researchers

Cuong, Dinh V.

Dang-Nguyen, Duc-Tien

Dao, Minh-Son

Gurrin, Cathal

Huynh, Phong

Huynh, Son

Nguyen, Binh T.

Nguyen, Dac

Son H.

Pradhan T.

Mikolov T.

Peters M. E.

Cheng H.-T.

Crossref

A Framework for Paper Submission Recommendation System

Nowadays, recommendation systems play an indispensable role in
many fields, including e-commerce, finance, economy, and gaming.
There is emerging research on publication venue recommendation
systems to support researchers when submitting their scientific
work. Several publishers such as IEEE, Springer, and Elsevier have
implemented their submission recommendation systems only to
help researchers choose appropriate conferences or journals for submission. In this work, we present a demo framework to construct an
effective recommendation system for paper submission. With the
input data (the title, the abstract, and the list of possible keywords)
of a given manuscript, the system recommends the list of top relevant journals or conferences to authors. By using state-of-the-art
techniques in natural language understanding, we combine the features extracted with other useful handcrafted features. We utilize
deep learning models to build an efficient recommendation engine
for the proposed system. Finally, we present the User Interface
(UI) and the architecture of our paper submission recommendation
system for later usage by researchers

Irish Universities

A Framework for paper submission recommendation system

Cuong, Dinh Viet

DCU Online Research Access Service

A Framework for Paper Submission Recommendation SystemDinh V. Cuong∗Dac H. Nguyen∗Son Huynh∗AISIA Research LabHo Chi Minh, VietnamPhong HuynhAISIA Research LabVNU HCM - University of ScienceHo Chi Minh, VietnamCathal GurrinDublin City UniversityDublin, IrelandMinh-Son DaoNational Institute of Information andCommunications TechnologyTokyo, JapanDuc-Tien Dang-Nguyen†Department of Information Scienceand Media StudiesUniversity of BergenBergen, NorwayBinh T. Nguyen‡AISIA Research LabVNU HCM - University of ScienceHo Chi Minh, VietnamABSTRACTNowadays, recommendation systems play an indispensable role inmany fields, including e-commerce, finance, economy, and gaming.There is emerging research on publication venue recommendationsystems to support researchers when submitting their scientificwork. Several publishers such as IEEE, Springer, and Elsevier haveimplemented their submission recommendation systems only tohelp researchers choose appropriate conferences or journals for sub-mission. In this work, we present a demo framework to construct aneffective recommendation system for paper submission. With theinput data (the title, the abstract, and the list of possible keywords)of a given manuscript, the system recommends the list of top rele-vant journals or conferences to authors. By using state-of-the-arttechniques in natural language understanding, we combine the fea-tures extracted with other useful handcrafted features. We utilizedeep learning models to build an efficient recommendation enginefor the proposed system. Finally, we present the User Interface(UI) and the architecture of our paper submission recommendationsystem for later usage by researchers.CCS CONCEPTS•Computingmethodologies→Machine learning algorithms;Neural networks; Feature selection; • Information systems→ Recommender systems.KEYWORDSrecommendation system, paper submission, deep learning∗All authors contributed equally to this research.†Senior author‡Corresponding authorPermission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.ICMR ’20, June 8–11, 2020, Dublin, Ireland© 2020 Association for Computing Machinery.ACM ISBN 978-1-4503-7087-5/20/06. . . $15.00https://doi.org/10.1145/3372278.3391929ACM Reference Format:Dinh V. Cuong, Dac H. Nguyen, Son Huynh, Phong Huynh, Cathal Gur-rin, Minh-Son Dao, Duc-Tien Dang-Nguyen, and Binh T. Nguyen. 2020.A Framework for Paper Submission Recommendation System. In Proceed-ings of the 2020 International Conference on Multimedia Retrieval (ICMR’20), June 8–11, 2020, Dublin, Ireland. ACM, New York, NY, USA, 4 pages.https://doi.org/10.1145/3372278.33919291 INTRODUCTIONOne of the most critical responsibilities of researchers is publishingtheir latest findings to the scientific community. During the lastdecades, the number of journals and international conferences isincreasing dramatically due to the rapid growth of both scienceand technology. Up to now, there have been over 13,000 publica-tion venues in computer science [1]. Each place has its specifictopics and scopes, and sometimes, organizers or editors updateits research topics to attract more submissions from various scien-tists. For this reason, researchers have a vast array of options fortheir paper submission and must try to select the most suitable onethat is appropriate to the quality of their scientific work as wellas increasing the chance of acceptance. This scenario may evenoccur for researchers who want to explore new domains, workon interdisciplinary projects, or simply don’t have much experi-ence. Subsequently, an efficient paper submission recommendationsystem is essential to assist authors by providing useful informa-tion and suggesting the most relevant journals or conferences fora given manuscript. Recently, there exists several paper submis-sion recommendation systems from different publishers. Publisherssuch as IEEE1, Springer2, and Elsevier3 created their systems tohelp authors quickly determine conferences and/ or journals forsubmissions. Similarly, domain-specific systems PubMender4 [2]and Wang et al.5 [1] focus on recommending within a specific fieldof research.In this work, we present a new framework for building a pa-per submission recommendation system for two research domains;mathematics and computer science. We propose different types offeatures by applying Chi-square statistics, Term Frequency-Inverse1https://publication-recommender.ieee.org/home2https://journalsuggester.springer.com/3https://journalfinder.elsevier.com/4https://www.keaml.cn:8081/5http://www.keaml.cn/prs/ Session: Demonstration  ICMR ’20, October 26–29, 2020, Dublin, Ireland                    Proceedings published June 8, 2020393Document Frequency (TF-IDF), and neural networks to select themost influential vocabularies. Subsequently, we train appropriatelearning models for developing the recommendation engine. Ourcurrent model [3] achieves quite good results and outperforms thestate-of-the-art methods in the dataset used in [1]. We aim at apply-ing other contemporary deep learning techniques for continuouslyimproving the recommendation system for paper submission. Sub-sequently, we describe the architecture of our recommendationsystem in details to develop the corresponding demo for a papersubmission recommendation platform.2 RELATEDWORKThere have been recent works related to the research of buildingthe recommendation system for paper submission. Wang and co-workers extract useful information from the abstract of a givenmanuscript for suggesting top relevant conferences or journals incomputer science. Feng and co-authors [2] utilize deep learningmethods to learn suitable recommendationmodels for the paper sub-mission recommendation problem. Typically, they use pre-trainedWord2Vec embeddings and convolutional neural networks to ex-tract deep features and then make predictions. They specialize inbiomedical publication venues with a dataset of 880,165 papers from1,130 journals in PubMed Central. Pradhan and Pal [4] propose amore comprehensive scholarly venue recommendation system thatintegrates social network analysis and contextual similarity. Theyconsider the following features: centrality measure calculation, ci-tation and co-citation analysis, topic modeling based contextualsimilarity, and key-route identification based primary path analy-sis of a bibliographic citation network. The experimental resultson the Microsoft Academic Graph (MAG) dataset show that theirproposed method outperforms state-of-the-art recommendationtechniques. It is worth noting that the paper submission recommen-dation system is interesting, and there is still room for improvement.Context-based systems prioritize simpleness and speed, but with re-cent advances in computational engines, more complex and robustmethods are practical. Network-based analysis, although exploitingmore handy features, requires more personal information or morecomplex inputs.On the other hand, there are recommendation systems for papersubmission that are available for public usage. IEEE PublicationRecommender accepts keywords, key phrases, or article titles as in-put data of its search engine. This system also supports automatedkeyword extraction from articles and has an optional filter by date.Springer Nature Journal suggester requires users to insert the man-uscript title, the manuscript text, and select the subject areas tofind the top relevant Springer and BMC journals related to thatmanuscript. Similarly, Elsevier Journal Finder needs the manuscripttitle, the corresponding abstract, optional keywords, and the fieldof research to return the most relevant Elsevier journals for a givensubmission.3 OUR PROPOSED SYSTEMIn this section, we introduce our proposed method to constructa paper submission recommendation system in different fields ofresearch.Our system primarily aims to support researchers to get moreuseful information and then make the best decision when choosingan appropriate conference or journal to submit their work. To thisaim, there are several tasks during the implementation of such asystem. For each research topic (such as, e.g., computer science ormathematics), we find the list of existing journals and all papers thatare published in these journals to create the corresponding datasetfor training and testing the performance of the recommendationengine. After that, we investigate the most appropriate recommen-dation model by using all data collected and then integrate it into arecommendation server. This recommendation server can listen toAPIs (application program interfaces) from aweb application, whichhas a convenient user interface (UI) such that users (researchers)can easily insert the necessary information of their submission andget the computed results. Figure 1 depicts the proposed frameworkof out paper submission recommendation system.3.1 Data CollectionWe consolidate data from available online resources, directly frompublication venues or via third parties. Regarding papers, we col-lect all information about the title, authors, the abstract, keywords,publication dates, DOI, the publication venue, and others. We imple-ment crawling tools for gathering data from each resource. Concern-ing each publication venue, we also accumulate the correspondingname, scope, indexes, the impact factor, the venue ranking, thevenue location, important dates, and others manually. In the end,we can get two datasets for two research fields, including com-puter science and mathematics. Typically, the computer sciencedataset is provided by the work of Wang et al. [1]. This datasetcontains 14,012 research papers, which are split into 65 differentcategories (journals or conferences). The remaining dataset con-sists of 392,894 articles related to Mathematics/Quantitative datasetfrom the publisher Springer.3.2 ModelingIn the modeling procedure, we investigate different recommenda-tion models by conducting multiple experiments and comparingthe performance of various algorithms to choose the best model forthe implementation. Generally, there are two popular approachesto construct an appropriate learning model, which are handcrafted-feature-based and deep-learning-based.3.2.1 Handcrafted-feature-based modeling. The recent work of ourinvestigation into paper submission recommendation systems hasachieved excellent performance, which outperforms the state-of-the-art algorithms on the computer science dataset [3].Firstly, all input data is pre-processed in the system using the fol-lowing technique. For each title and abstract, we remove numbers,punctuation marks, and stop-words, which may carry little infor-mation or even create more noise for the problem. It is important toemphasize that English words often have several morphemes, suchas prefixes, suffixes, and base words. Typically, words that sharebase words almost have the same meaning. To get most out of this,we use Porter Stemmer library (in NLTK package) to break downwords into their word-stems. It is worth noting that for keywords,authors often come up with phrases that already exist in variousconferences or journals. Session: Demonstration  ICMR ’20, October 26–29, 2020, Dublin, Ireland                    Proceedings published June 8, 2020394Figure 1: Our proposed workflow of a user’s query in the paper submission recommendation system.Secondly, we aim to convert texts into suitable weighted vectorrepresentations. TF-IDF is an appropriate choice since it can capturethe importance of a token according to its frequency. However, sinceabstracts are often lengthy, there is an abundant amount of uniquetokens. As a result, we apply Chi-square statistics to score thosetokens and select top vocabularies, which are most valuable for eachcategory. This procedure generates a vocabularyWA for abstracttokens. At the same time, we do not apply chi-square and create avocabularyWT containing all title tokens. Using only tokens fromWT andWA, both title and abstract of the manuscript are convertedinto vectors by TF-IDF. On the contrary, keywords are entirelymeaningful and well processed so we encode keywords by one-hotvectors.Finally, all vectors computed from different inputs are concate-nated and fed into a machine learning model. In our experiments, asmall neural network is used. This network has one single hiddenlayer of 200 units. The output layer contains a number of neu-rons equal to publication venues. Importantly, this network is welltrained by using the stochastic gradient descent algorithms.3.2.2 Deep-learning-based modeling. Deep learning models havebeen proven to be ubiquitously useful in many natural languageprocessing tasks. Pre-training techniques, such as Word2Vec, Glove,Fasttext, ELMO, BERT, or XLNet [5–10], are playing a crucial rolein solding many problems. Consequently, deep-learning architec-tures can make a significant contribution to the robustness andeffectiveness of a system.In this system, we implement various techniques that are mostsuitable for each part. In specific, pre-training word-level embed-ding may be used to vectorize keywords of the manuscript, key-words of the venue’s domains or words in a full sentence for betterword presentations. Generally, word2vec or Glove are designed tocapture linguistic contexts of words but ignore the morphology ofthe words. For this reason, they rarely handle unseen words. Incontrast, Fasttext takes the advantage of sub-word information tocompute word representations that may not appear in the trainingdata. On the other hand, deeper pre-training bidirectional modelsmake use of a very large corpus on generalizable tasks to encode thewhole context and that can be reusable on downstream tasks likeours. We use these models to deal with the context of documents,like titles or abstracts. Each of these methods has its own inputnormalization, some appropriate pre-processing to the inputs maybe applied to improve the performance of the system. Overall, ourend-to-end model architecture has different components that areresponsible for different inputs. Then, a classifier can utilize all thecomponents to make a final prediction. The classifier architecturemay be a simple multi-layer feed-forward neural network, stackedrecurrent layers, DeepFM, Wide&Deep, or stacked transformers[11–13].3.3 System ArchitectureFigure 2 depicts the architecture of our paper submission recom-mendation platform. As mentioned previously, we collect variousdatasets, including journal articles and conference papers, and trainthe most suitable recommendation engine for the platform. Afterthis step, we integrate this recommendation engine into a recom-mendation server. This server can be deployed in a cloud serverproviding necessary APIs such that the web/mobile applicationcan easily get the top relevant journals or conferences by givingenough information related to the title, the abstract, and the listof keywords from a given submission. A user can access the papersubmission recommendation platform via a web/mobile app fornecessary information.3.4 User InterfaceThe User Interface (UI) of our paper submission recommendationsystem is shown in Figure 3. In this UI, one user can easily insertnecessary information into the search bar, run the query, and getthe results for our recommendation model which is a list of targetvenues. There are two main components in the recommendationsystem for paper recommendation. The web application can collectnecessary information of each manuscript and then run the queryto get the corresponding results. The recommendation server isintegrated with our proposed algorithm and returns useful resultsvia APIs.4 CONCLUSIONWe have presented a new recommendation system for paper sub-mission target venues. We have also shown how to build this systemby collecting data, processing data, investigating the most suitablealgorithm, and integrating the recommendation algorithm into arecommendation server with an appropriate architecture and UI.In the future, we plan to enhance the performance of this platformand add more useful features for users. Session: Demonstration  ICMR ’20, October 26–29, 2020, Dublin, Ireland                    Proceedings published June 8, 2020395Figure 2: The proposed architecture for our paper submission recommendation system.Figure 3: The proposed user interface for our paper submission recommendation system.ACKNOWLEDGEMENTThis research is conducted under the Collaborative Research Agree-ment between National Institute of Information and Communi-cations Technology and University of Science, Vietnam NationalUniversity at Ho Chi Minh City. We acknowledge the support ofScience Foundation Ireland under grant number SFI/13/RC/2106and L. Meltzers Høyskolefonds, UiB 2019/2259-NILSO.REFERENCES[1] D. Wang, Y. Liang, D. Xu, X. Feng, and R. Guan, “A content-based recommendersystem for computer science publications,” Knowledge-Based Systems, vol. 157,pp. 1 – 9, 2018.[2] X. Feng, H. Zhang, Y. Ren, P. Shang, Y. Zhu, Y. Liang, R. Guan, and D. Xu,“Pubmender: A deep learning based recommender system for the biomedicalpublication venue (preprint),” 11 2018.[3] H. Son, H. Phong, N. Dac, D. V. Cuong, and N. T. Binh, “S2rscs: An efficientscientific submission recommendation system for computer science,” The 33thInternational Conference on Industrial, Engineering & Other Applications of AppliedIntelligent Systems, 2020.[4] T. Pradhan and S. Pal, “A hybrid personalized scholarly venue recommendersystem integrating social network analysis and contextual similarity,” FutureGeneration Computer Systems, 11 2019.[5] T. Mikolov, K. Chen, G. S. Corrado, and J. Dean, “Efficient estimation of wordrepresentations in vector space,” CoRR, vol. abs/1301.3781, 2013.[6] J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for wordrepresentation,” in Empirical Methods in Natural Language Processing (EMNLP),pp. 1532–1543, 2014.[7] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectorswith subword information,” Transactions of the Association for ComputationalLinguistics, vol. 5, pp. 135–146, 2016.[8] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettle-moyer, “Deep contextualized word representations,” ArXiv, vol. abs/1802.05365,2018.[9] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deepbidirectional transformers for language understanding,” in NAACL-HLT, 2019.[10] Z. Yang, Z. Dai, Y. Yang, J. G. Carbonell, R. Salakhutdinov, and Q. V. Le, “Xlnet:Generalized autoregressive pretraining for language understanding,” in NeurIPS,2019.[11] H. Guo, R. Tang, Y. Ye, Z. Li, and X. He, “Deepfm: A factorization-machine basedneural network for ctr prediction,” in IJCAI, 2017.[12] H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson,G. S. Corrado, W. Chai, M. Ispir, R. Anil, Z. Haque, L. Hong, V. Jain, X. Liu, andH. Shah, “Wide & deep learning for recommender systems,” in DLRS 2016, 2016.[13] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser,and I. Polosukhin, “Attention is all you need,” in NIPS, 2017. Session: Demonstration  ICMR ’20, October 26–29, 2020, Dublin, Ireland                    Proceedings published June 8, 2020396

http://doras.dcu.ie/24635/1/p393-dinhA1.pdf

A Framework for paper submission recommendation system

Abstract

Similar works

Full text

Available Versions

Crossref

Irish Universities

DCU Online Research Access Service