A strong baseline for question relevancy ranking

Research paper by Ana V. González-Garduño, Isabelle Augenstein, Anders Søgaard

Indexed on: 27 Aug '18Published on: 27 Aug '18Published in: arXiv - Computer Science - Computation and Language


The best systems at the SemEval-16 and SemEval-17 community question answering shared tasks -- a task that amounts to question relevancy ranking -- involve complex pipelines and manual feature engineering. Despite this, many of these still fail at beating the IR baseline, i.e., the rankings provided by Google's search engine. We present a strong baseline for question relevancy ranking by training a simple multi-task feed forward network on a bag of 14 distance measures for the input question pair. This baseline model, which is fast to train and uses only language-independent features, outperforms the best shared task systems on the task of retrieving relevant previously asked questions.