Generating Part-of-speech Focused Adversarial Examples for Sentiment Analysis

Abstract

The focus of the contents presented in this paper is on the generation of so-called black-box adversarial examples for the natural language processing (NLP) task sentiment analysis. The adversarial examples are created through synonym-substitution of words with certain Part Of Speech (POS) tags. The effectiveness of a variety of POS tags is tested, while preserving the semantical similarity of the modified text. Naive Bayes (NB) models and convolutional neural networks (CNN) are two popular models commonly used in sentiment analysis. These models are trained and tested on the IMDB movie review dataset. The effectiveness of the proposed method and the resilience of the models is evaluated by the measure of decrease in accuracy. The results of the experiment show that the selection of the Part-of-speech tags Adjective, Adverb, and Modal generates successful and effective adversarial examples with little distortion to the original text. Additionally, the proposed method uncovers vulnerabilities in both tested models.