Negative transfer detection in transductive transfer learning
- PDF / 1,677,708 Bytes
- 13 Pages / 595.276 x 790.866 pts Page_size
- 14 Downloads / 255 Views
ORIGINAL ARTICLE
Negative transfer detection in transductive transfer learning Lin Gui1 · Ruifeng Xu1 · Qin Lu2 · Jiachen Du1 · Yu Zhou1
Received: 6 September 2016 / Accepted: 28 December 2016 © Springer-Verlag Berlin Heidelberg 2017
Abstract Transfer learning method has been widely used in machine learning when training data is limited. However, class noise accumulated during learning iterations can lead to negative transfer which can adversely affect performance when more training data is used. In this paper, we propose a novel method to identify noise samples for noise reduction. More importantly, the method can detect the point where negative transfer happens such that transfer learning can terminate at the near top performance point. In this method, we use the sum of the Rademacher distribution to estimate the class noise rate of transferred data. Transferred data having high probability of being labeled wrongly is removed to reduce noise accumulation. This negative sample reduction process can be repeated several times during transfer learning until we find the point where negative transfer occurs. As we can detect the point where negative transfer occurs, our method not only has the ability to delay the point where negative transfer happens, but also the ability to stop transfer learning algorithms at the right place for top performance gain. Evaluation based on cross-lingual/ domain opinion analysis evaluation data set shows that our algorithm achieves the state-of-the-art result. Furthermore, our system shows a monotonic increase trend in performance improvement when more training data are used beating the performance degradation curse of most transfer learning methods when training data reaches certain size. * Ruifeng Xu [email protected] 1
School of Computer Science and Technology, Harbin Institute of Technology Shenzhen Graduate School, Shenzhen, China
2
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China
Keywords Transfer learning · Negative transfer · Class noise detection
1 Introduction Data mining and machine learning have achieved considerable success in many classification tasks such as text classification [18, 19], sentiment classification [27–29], and image classification [24]. Usually, these tasks are conducted using supervised methods. That is, labeled training data is used by machine learning algorithms to achieve good results. However, the cost of manual annotation for training data is very expensive and this is a limit on the size of labeled training data. The size of labeled training data as well as its quality are two important factors in supervised learning no matter which learning algorithm is used. If there is insufficient training data in the domain of a specific task, transfer learning can be used, where labeled data of other domain(s) are used to enrich the training data for the current task. In transfer learning, the domain of the specific task is called the target domain. The domain of the available labeled data from the other do
Data Loading...
