RNN based question answer generation and ranking for financial documents using financial NER
- PDF / 1,161,616 Bytes
- 10 Pages / 595.276 x 790.866 pts Page_size
- 64 Downloads / 194 Views
Sådhanå (2020)45:269 https://doi.org/10.1007/s12046-020-01501-3
Sadhana(0123456789().,-volV)FT3](012345 6789().,-volV)
RNN based question answer generation and ranking for financial documents using financial NER HARIHARAN JAYAKUMAR*, MADHAV SANKAR KRISHNAKUMAR, VISHAL VEDA VYAS PEDDAGOPU and RAJESWARI SRIDHAR National Institute of Technology Tiruchirappalli, Tanjore Main Road, National Highway 67, Tiruchirappalli 620 015, India e-mail: [email protected]; [email protected]; [email protected]; [email protected] MS received 28 April 2020; revised 22 July 2020; accepted 11 September 2020 Abstract. Organizations, governments and many entities deal with an expanse of voluminous financial documents and this necessitates a need for a financial expert system which, given a financial document, extracts finance-related questions and answers from it. This expert system helps us to adequately summarize the document in the form of a question-answer report. This paper introduces the novel idea of generating finance-related questions and answers from financial documents by introducing a custom Financial Named Entity Recognizer, which can identify financial entities in a document with an accuracy of 92%. We have introduced a method of generating finance-based questions using a sample document to obtain a set of generalized questions that we can feed to any similar financial document. We also record the expected answer type during the question generation phase, which helps to develop a robust mechanism to verify that we always generate the correct answers during the answer extraction stage. Keywords. Knowledge engineering; artificial intelligence; expert systems; natural language processing; hybrid intelligent systems.
1. Introduction Question Answering (QA) systems have become increasingly smarter over the years, especially after the advent of deep learning paradigms [1–5]. Nevertheless, even as new trends and new features come along, most of them focus on the general problem of generating questions and finding relevant solutions for an open domain as explained in Bouziane et al [6]. Considering that organizations and companies collect large amounts of records on the monthly, quarterly and yearly sales, it would be of great use, if we could summarize this vast collection of datasets as a report of generalized questions and their corresponding answers. We have introduced a novel method of generating relevant financial questions given a document. We automate this process by generating questions and ranking them so that users can threshold the number of questions based on their quality. We use the information obtained during question generation in order to aid with our answer extraction module. Our core contribution is the financial Named Entity Recognizer (NER) which we have trained on our dataset,
*For correspondence
which we obtained by scraping for financial articles online. We have developed a system which generates a set of financial questions given a text by adding our NER on top of existing deep-learning
Data Loading...
