Improving Cross-domain Authorship Attribution by Combining Lexical and Syntactic Features: Notebook for PAN at CLEF 2019

Authorship attribution is a problem in information retrieval and computational linguistics that involves attributing authorship of an unknown document to an author within a set of candidate authors. Because of this, PAN-CLEF 2019 organized a shared task that involves creating a computational model that can determine the author of a fanfiction story. The task is cross-domain because of the open set of fandoms to which the documents belong. Additionally, the set of candidate authors is also open since the actual author of a document may not be among the candidate authors. We extracted character-level, word-level and syntactic information from the documents in order to train a support vector machine. Our approach yields an overall macro-averaged F1 score of 0.687 on the development data of the shared task. This is an improvement of 18.7% over the character-level lexical baseline. On the test data, our model achieves an overall macro F1 score of 0.644. We compare different feature types and find that character n-grams are the most informative feature type though all tested feature types contribute to the performance of the model