Discovering novel memory cell designs for sentiment analysis on tweets
- PDF / 2,581,101 Bytes
- 41 Pages / 439.37 x 666.142 pts Page_size
- 2 Downloads / 210 Views
Discovering novel memory cell designs for sentiment analysis on tweets Sergiu Cosmin Nistor1,2 · Mircea Moca1 · Răzvan Liviu Nistor3 Received: 2 March 2020 / Revised: 30 September 2020 / Accepted: 21 October 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Designing a Recurrent Neural Network to extract sentiment from tweets is a very hard task. When using memory cells in their design, the task becomes even harder due to the large number of design alternatives and the costly process of finding a performant design. In this paper we propose an original evolutionary algorithm to address the hard challenge of discovering novel Recurrent Neural Network memory cell designs for sentiment analysis on tweets. We used three different tasks to discover and evaluate the designs. We conducted experiments and the results show that the best obtained designs surpass the baselines—which are the most popular cells, LSTM and GRU. During the discovery process we evaluated roughly 17,000 cell designs. The selected winning candidate outperformed the others for the overall sentiment analysis problem, hence showing generality. We made the winner selection by using the cumulated accuracies on all three considered tasks. Keywords Memory cell · Evolutionary algorithm · Deep learning · Recurrent neural network · Sentiment analysis · Tweet
The handling Associate Editor "Sebastian Risi" is a Neural Systems editor for this paper. * Răzvan Liviu Nistor [email protected] Sergiu Cosmin Nistor [email protected] Mircea Moca [email protected] 1
SYNERGY CROWDS OÜ, Tallin, Estonia
2
Department of Computer Science, Babeş-Bolyai University, Cluj‑Napoca, Romania
3
Department of Management, Babeş-Bolyai University, Cluj‑Napoca, Romania
13
Vol.:(0123456789)
Genetic Programming and Evolvable Machines
1 Introduction Extensive research has been done so far on deep learning methods, with new applications being proposed and new architectures being explored. This is a very active domain and researchers always try to find new designs that are better suited to solve various problems in different domains. Much of the success of such methods is due to recurrent neural networks. Recurrent neural networks are machine learning algorithms, very well suited to process sequential data. They do not treat their inputs independently, but they remember previous computations and take them into consideration when making new predictions. At the core of modern recurrent neural networks are the memory cells, of which there are two designs that are most often used, regardless of the problem to be solved: the Long Short-Term Memory (LSTM) cell and, more recently introduced, the Gated Recurrent Unit (GRU). Extracting sentiment from tweets has many applications, but designing a recurrent neural network for this task is very challenging. When building such networks, one of the most important architectural decisions is the memory cells that compose the hidden units of the network. In this work we prop
Data Loading...
